[SNMP4J] problem when running snmp walk for a long time
Eugene R. Snider
gene at cvtt.net
Thu Jan 31 04:20:27 CET 2008
I tend to agree with Frank's concerns about performance. Also I think
you are running under a mistaken assumption, having 60 threads each
performing a single task is not necessarily more concurrent, and may
even be less concurrent, than fewer threads polling more devices. The
context overhead of a thread switch is smaller than a process context
switch and at best you are probably spending signficant time ( which you
will not be able to account for ) in meaningless system activity. At
worst you are thrashing and are actually impeding performance rather
than enhancing it. While linux supports 10K+ threads without blinking
(more or less and actually less) Java, all marketing crap aside, doesn't
like them in the double digit counts.
It is possible to reduce the amount of memory allocated for each thread
with the -Xss command line switch, that might improve your memory
allocation and possibly allow you to work around this specific problem.
Gene
YUAN Shuai wrote:
> Hi,
>
> Thank you for your help.
> Actually I've got no choice but create 60 threads at the same time,
> because I have to collect data at every 0 and 30 second from all the devices.
> The reason for doing this is to study the relationship of the devices according to the performance data.
> So you see, I hava to split nearly 400 requests into about 60 jobs, each one containing 6 requests.
> A distributed system will be a good choice, however I haven't gone that far :)
>
> And maybe this is important, finally I found the PROBLEM, Netbean's fault, actually.
> When I packed the project and run it under Linux from 'java -jar', the program just never goes error again.
>
>
>
>
>
> 发件人： Frank Fock
> 发送时间： 2008-01-26 04:15:15
> 收件人： YUAN Shuai
> 抄送： snmp4j
> 主题： Re: [SNMP4J] problem when running snmp walk for a long time
> Hi,
> You should not create a Snmp session for each
> request. The overhead is too big especially
> if you are sending requests concurrently.
> Running 60 threads for performing 60 concurrently
> walks is a bad idea either! A thread pool with
> as many threads as there are CPUs in the host
> will provide best performance. You need such
> a thread pool for processing the responses only.
> Thus, using a multi-threaded message dispatcher
> is all you need.
> Always have in mind, that a thread needs about
> one MByte of memory!
> Best regards,
> Frank
> YUAN Shuai wrote:
>
>> hi all, here is the situation:
>>
>> In each thread, I put a snmp walk request in a while-loop, so that the walk can be done at every 0 and 30 second.
>>
>> Nearly 60 threads are created, and after running for about 3 hours, I got the following error and my other running applications just got errors too such as 'dll error' :
>>
>> Exception in thread "Thread-34" java.lang.OutOfMemoryError: unable to create new native thread
>> at java.lang.Thread.start0(Native Method)
>> at java.lang.Thread.start(Thread.java:597)
>> at java.util.Timer.<init>(Timer.java:154)
>> at java.util.Timer.<init>(Timer.java:122)
>> at org.snmp4j.Snmp.<init>(Unknown Source)
>> at org.snmp4j.Snmp.<init>(Unknown Source)
>> at snmpacc.SNMPAccWalk.createSnmpSession(SNMPAccWalk.java:80)
>> at snmpacc.SNMPAccWalk.walkLeaf(SNMPAccWalk.java:106)
>> at scout.PERFWalker.run(PERFWalker.java:80)
>> at java.lang.Thread.run(Thread.java:619)
>>
>> and
>>
>> java.net.SocketException: No buffer space available (maximum connections reached?): Socket creation failed
>> at java.net.PlainDatagramSocketImpl.datagramSocketCreate(Native Method)
>> at java.net.PlainDatagramSocketImpl.create(PlainDatagramSocketImpl.java:73)
>> at java.net.DatagramSocket.createImpl(DatagramSocket.java:317)
>> at java.net.DatagramSocket.<init>(DatagramSocket.java:208)
>> at java.net.DatagramSocket.<init>(DatagramSocket.java:261)
>> at java.net.DatagramSocket.<init>(DatagramSocket.java:234)
>> at org.snmp4j.transport.DefaultUdpTransportMapping.<init>(Unknown Source)
>> at snmpacc.SNMPAccWalk.createSnmpSession(SNMPAccWalk.java:79)
>> at snmpacc.SNMPAccWalk.walkLeaf(SNMPAccWalk.java:106)
>> at scout.PERFWalker.run(PERFWalker.java:80)
>> at java.lang.Thread.run(Thread.java:619)
>>
>> However I am sure to invoke the close() method, and the following is my walkLeaf():
>>
>> public String walkLeaf(String oid) throws IOException {
>> // System.out.println("walk oid:" + oid);
>> Address address = new UdpAddress(IPAddress.toString() + "/161");
>> Snmp snmp = createSnmpSession();
>> Target target = createTarget();
>> target.setVersion(getVersion(Version));
>> target.setAddress(address);
>> target.setRetries(retries);
>> target.setTimeout(timeout);
>> target.setMaxSizeRequestPDU(65535);
>> snmp.listen();
>> PDU request = new PDU();
>> request.setType(PDU.GET);
>> request.add(new VariableBinding(new OID(oid)));
>>
>> PDU response = null;
>>
>> ResponseEvent responseEvent = snmp.send(request, target);
>> if (responseEvent == null) {
>> return "no response";
>> }
>>
>> // receive the result
>> response = responseEvent.getResponse();
>>
>> // close the connection
>> snmp.close();
>> snmp = null;
>>
>> if (response == null) {
>> return "no response";
>> }
>>
>> return new String(response.get(0).getVariable().toString());
>> }
>>
>> anyone get any idea dealing with this? THANK YOU!
>>
>> vmape, 2008.01.24
>> _______________________________________________
>> SNMP4J mailing list
>> SNMP4J at agentpp.org
>> http://lists.agentpp.org/mailman/listinfo/snmp4j
>>
>> ------------------------------------------------------------------------
>>
>> _______________________________________________
>> SNMP4J mailing list
>> SNMP4J at agentpp.org
>> http://lists.agentpp.org/mailman/listinfo/snmp4j
>>
More information about the SNMP4J
mailing list