[AGENT++] Re: recvfrom's blocking and process grinding to a halt

[John McCaskey]

Everyone,

I'm still working on this and digging through the code.  But perhaps someone
can offer some clarity as to the architecture of the library when it comes
to handling events and doing the recvfrom calls.  It seems from my review of
the code so far that each thread I have is sometimes calling the GetEvents
and HandleEvents calls, but then these are locking a global msgqueue and
matching up the unique request id's to place the responses with the
appropriate requests even if those requests were made by a different thread
then the one that has triggered the HandleEvents.  Is this accurate?

If so for my purposes it seems like I'm wasting alot of cycles and getting
into some threading issues because of each thread trying to check for new
data and doing recvfroms the contending for the global queue.  Wouldn't it
be better in a multithreaded scenario to have a single event handler thread
that was dedicated to checking the sockets, getting data, and updating the
global queue, then just have each thread wait for a response (or error
indication) to appear in that queue?  And if that makes sense then is there
anyway to get this type of setup in SNMP++ or would it require me to make
some fairly major modications to the IO hanlding architecture of the
library.  I fear it is the latter, but any input from those more familiar
with the way the library handles this would be greatly appreciated!

On 3/22/06, John McCaskey <jmccaskey at gmail.com> wrote:
>
> Everyone,
>
> As those of you who have been following my posts know I have SNMP++
> embedded into a multi-threaded polling engine.  I've already made some
> modifications to allow multiple v3MP objects (one per thread), and I've been
> able to get the v3 performance both thread safe and fast thanks to some
> suggestions from Frank and Jochen.
>
> However, I am now sometimes experience a situation where the process
> begins thrashing my CPU's and grinds to a halt.  If I run the debugger on
> the process when this occurs I seem to find that many threads are blocked in
> recvfrom calls.  My situation when this happens is that I am trying to
> stress test the polling engine by having it poll about 3000 devices using 80
> seperate threads.  Since I don't really have 3000 devices for testing I have
> setup linux servers with a software agent and I am polling each of them
> several hundred times.  Is it possible that there is then some sort of race
> condition where after doing the select and seeing data is available from the
> remote address there is a context switch and a recvfrom call in another
> thread is then receiving the data that the other thread thought was
> available?  Then when the context switchs back the first thread trys to
> recvfrom (having already selected) but the data is gone and it blocks?  I've
> just initially started looking into this and I'm not sure if that even makes
> sense in the way the IO works in SNMP++, but any thoughts, suggestions, or
> other insight would be appreciated!
>
> Thanks,
>
> John McCaskey
>

Thanks again,

John McCaskey