[SNMP4J] Asynchronous SNMP requests...

[Omair-Inam Abdul-Matin]

Hi,

I'm trying to develop a program that can query (potentially) thousands of
SNMP devices.  I'm using Asynchronous SNMP calls with a single SNMP instance
and a multi-threaded dispatcher.  The dispatcher takes a response and
creates an object graph (containing a representation of the object and
submits it to a server).  The object graph creation logic is in groovy ...

I've modified the SNMP instance to create a ThrottledSnmpInstance which
throttles the SNMP request by placing a minimum delay before sending out
consecutive SNMP requests (I place a delay of 50 ms.. so no more than 20
requests are sent per second).. I'm finding that as I scale up to larger
number of devices I still get devices timing out.. currently these devices
are all simulated so the response are all coming in very quickly... the
number of devices timing out is variable by it can be as much as 10% of the
total number of devices.

I'm wondering if I can reduce this ratio  by doing the following:
- Modifying the multithreaded dispatcher to queue all received responses..
- Having a second set of threads read off the responses and do the actual
processing..

I've read in an earlier thread that a recommended value for the number of
threads in the dispatcher is the number of CPUs on the host... Is that
correct?

If so, then I think that queueing up the responses as they come in could
allow me to scale further (I acknowledge, that the total processing time
will go up.. since a growing queue will only be serviced by a finite number
of threads as we increase the scale)

Thoughts?

Omair