[SNMP4J] Multithreaded UDP transport mapping?

[Ting Zhou]

Frank,

Please see my inline comment...

Frank Fock wrote:

> Hi Ting,
>
> From my point of view, the current implementation is already
> fast. As already discussed on the list, using a single port for
> incoming traps will be the bottle neck. The current implementation
> can handle 1000 traps per second, if the dispatcher can handle it
> and when the UDP buffer is big enough.

You actually brought up a very good point. UDP buffer is more often
than not, where the problem is. I've played with Joe SNMP, which is another
open-source SNMP java implementation. It doesn't have it's own application
buffer (so solely relies on UDP buffer). When the packet dispatcher
(application code that processes alerts) is not fast enough, UDP packets
starts getting dropped badly. In the end, I had to increase the UDP 
buffer size
in the Joe SNMP code, which is a hack. Usually, my preferred way to 
implement
thing like this, is to have one thread read data from incoming UDP buffer
to application buffer, and call out to one or many worker threads to process
(or dispatch) alerts. That way, single port ceases to be bottleneck and
parallel dispatcher threads will indeed improve throughput.

>
> Using multiple threads will only provide advantages on a multi-processor
> system. But it is then the question where to parallelize the processing.
> If you want to parellize very early, then you could implement a
> MessageDispatcher following the decorator design pattern that uses a
> ThreadPool internally to use multiple threads to dispatch messages on
> the "real" dispatcher concurrently.

That's exactly what I mentioned above.

>
> Since I have already a ThreadPool implementation ready, I will add
> such a multi-threaded decorator dispatcher in the next SNMP4J release.

That will be really great.

>
> If cannot wait until then, it should not be a big deal to write one 
> yourself,
> without changing existing SNMP4J code (except you will have to subclass
> Snmp).

I also have a thread pool implementation myself. This might be the route
I'll take.

Thanks a lot for the response.

Regards,
Ting


> Hope this helps.
>
> Best regards,
> Frank
>
> Ting Zhou wrote:
>
>> Hi,
>>
>> I've been looking around for a decent Java SNMP library to implement
>> a high-performance trap daemon (currently v1/v2c only) that is capable
>> of handling 1000 traps per second. SNMP4j came into the picture as
>> a promising candidate. However, upon inspecting the implementation,
>> I found the default UDPTransportMapping included in the library
>> is single-threaded (i.e. only one thread calling out to Dispatchers).
>> Is a high-performance, multi-threaded implementation in the roadmap?
>>
>> Thanks,
>> Ting
>> _______________________________________________
>> SNMP4J mailing list
>> SNMP4J at agentpp.org
>> http://p15141779.pureserver.info/mailman/listinfo/snmp4j
>>
>
>