[SNMP4J] Reducing temporary object allocations for SNMP4j

[Iker Almandoz]

Hi Frank, 

After struggling quite a bit with the memory issues and garbage collection,
I think I have a better idea of why my system is limited by the garbage
collector.
My application is  sending requests with a timeout of 60 seconds and 1 retry
enabled.

I've profiled the system using JProbe and figured out that I have many PDU
objects referenced from the PendingRequest objects.
When doing a cancel() on the PendingRequest (TimerTask), the run() is never
called but a reference is kept in the Timer until the time of execution.  In
java 5 the Timer class has a new method purge that will remove the
references to the cancelled object to avoid that.
So for each request I send to snmp4j, there is a PendingRequest that is
alive for 60 seconds essentially making most of the objects go to the
tenured part of the memory making eventually a full garbage collection
necessary.

I've fixed this in my local version by overriding cancel() method in
PendingRequest to remove the references to all objects as:

    public boolean cancel(){
        boolean out = super.cancel();

        key = null;
        pdu = null;
        target = null;
        transport = null;
        listener = null;
        userObject = null;

        return out;
    }

This has reduced the garbage collection time enormously.
Do you have any thoughts on this ?

Best regards, 
Iker



-----Original Message-----
From: Frank Fock [mailto:fock at agentpp.com] 
Sent: Friday, July 11, 2008 1:58 PM
To: Iker Almandoz
Cc: snmp4j at agentpp.org
Subject: Re: [SNMP4J] Reducing temporary object allocations for SNMP4j

Hi Iker,

This is a good point. I have changed the
VariableBinding constructor already to
use Null.instance instead "new Null()".

I have introduced the cloning to avoid
race conditions in multi-threaded
applications.

Introducing a immutable OID subclass
would solve the issue and the user could
choose which one to use. The same would
work for Variable, although here an immutable
wrapper could help too.

Both approaches are not first choice, because
an immutable super class/interface would be
better.

Please let me know if you need more information
on the commercial support. You can order it
online from:
https://agentpp.plimus.com/jsp/dev_store1.jsp?developerId=93158

Best regards,
Frank

Iker Almandoz wrote:
> Hi Frank, 
> 
>  
> 
> I am using SNMP4j to collect huge amounts of SNMP data (approximately 3200
> Million OIDs per day) as part of a commercial application.
> 
>  
> 
> When I create VariableBinding objects, both the OID object and the
Variable
> objects get cloned which essentially means that I'm creating 3200 Million
> additional OID objects (with their corresponding int arrays) as well as
> 3200Million new Variable objects.   I'm solving this currently by
extending
> both Variable and OID to return a reference to themselves in the clone()
> method and by having a single Null() object such that I don't get a new
one
> for each VariableBinding. 
> 
> Is there any way inside of SNMP4j we could use to avoid creating so many
> temporaries ?
> 
> We would prefer to not use workarounds outside of SNMP4j such that we can
> remain fully compatible with the library to be able to benefit from the
> latest updates.
> 
>  
> 
> The garbage collection has become our current bottleneck in our
> implementation where in terms of object creation in bytes we get:
> 
> 48.3% int[] (from OID objects)
> 
> 7.2% BER$MutableByte
> 
> 6.7% OID
> 
> 4.5% VariableBinding
> 
> 3% Integer32.
> 
> 2.2% Null
> 
>  
> 
> We would be interested in getting support from you on this and other
issues
> are we are considering on signing for the "SNMP4J annual email support
> package". 
> 
>  
> 
> Best regards, 
> Iker
> 
> _______________________________________________
> SNMP4J mailing list
> SNMP4J at agentpp.org
> http://lists.agentpp.org/mailman/listinfo/snmp4j

-- 
AGENT++
http://www.agentpp.com
http://www.mibexplorer.com
http://www.mibdesigner.com