[AGENT++] memory limitation (crashes) + log files

[Frank Fock]

Hi Jacquemin,

If you always use logging with the macros
in the following manner then the log output
should be properly synchronized:

LOG_BEGIN(x)
[LOG(x)]*
LOG_END

If you have sync problems then a LOG_BEGIN
has no matching LOG_END or vice versa.

Best regards,
Frank

Jacquemin, Jean-Philippe wrote:
> Hi Frank,
> 
> 
> I have checked my code further in details keeping in mind your
> suggestion that it might me a "race condition" and I found out that my 4
> custom threads that are running next to the process_request threads from
> the ThreadPool are all using shared objects protected by semaphores
> except one, which is the AgentLog object.
> 
> I maintain 2 log files: 1 for the framework log, the other one for
> custom log. The latter being derived from the AgentLog class.
> 
> Both logs (writing to different files) are initialized using objects of
> type AgentLogImpl, which implements the "lock" using a pthread_mutex.
> 
> both logs use macros of kind:
> LOGBEGIN  ... log->lock
> LOG       ... logEntry += newLog
> LOGEND    ... log->unlock
> 
> When compiling without the thread pool (not defining _THREADS), the
> framework is on only 1 thread, however my 4 other tasks are still
> different threads using the same log mechanism. But in this case the
> mutex is not used anymore in the AgentLogImpl (mutex compiled out in
> SnmpSynchronized::lock) and I have a crash probably due to :
> 1) thread1          : creating a logEntry1
> 2)         thread2  : using the logEntry1
> 3)         thread2  : deleting the logEntry1
> 4) thread1          : trying to use logEntry1
>     => "__pure_virtual error" since the object has already been deleted.
> 
> I understand this is a reentrancy error and the solution would be to
> force the implementation of "lock" with mutexes in a derived
> AgentLogImpl class.
> 
> However the question is, why does it also happen with a mutlithreaded
> configuration where the "lock" is really implemented with a mutex ??
> What kind of programming error with the logs could give this?
> 
> 
> Best regards,
> 
> Jean-Philipppe
> 
> 
> PS: I will do a test with #define NO_LOGGING to kick out all the
> loggings.
> 
> 
> -----Original Message-----
> From: Frank Fock [mailto:fock at agentpp.com] 
> Sent: donderdag 10 januari 2008 1:03
> To: Jacquemin, Jean-Philippe
> Cc: agentpp at agentpp.org
> Subject: Re: [AGENT++] memory limitation (crashes)
> 
> Hi Jean-Philippe,
> 
> I do not think that it is a stack overflow, but that should be easy to
> verify by a stack dump.
> 
> I would guess that it is a race condition.
> How do you synchronize the update thread with the AGENT++ thread pool
> (requests)?
> 
> More comments inline below:
> 
> Jacquemin, Jean-Philippe wrote:
>> The questions are:
>> (1) Could it really be a stack overflow ?? Has someone ever 
>> experienced it? How to get further information ?
> 
> Use core dump to verify it.
> 
>> (2) How to reduce the amount of memory used ?
> 
> Undef logging. As last resort, use virtual tables instead of MibTable
> (thus use MibComplexEntry instead of MibTable).
> 
>> (3) Would it be useful to try and adjust the parameter MAX_SNMP_PACKET
> 
>> (currently at 4096) ?
> 
> I do not think so.
> 
>> (4) Why is MAX_SNMP_PACKET defined as 2048 in SNMP++ redefined as 4096
> 
>> in AGENT++ ?
> 
> Both should be the same (AGENT++ uses SNMP++ definition).
> Please upgrade to the latest version, if you have problems with that.
> 
>> (5) Are there other parameters that may influence the memory footprint
> ?
>>  
> The default PDU size (number of allowed variables).
> 
> Best regards,
> Frank
> 

-- 
AGENT++
http://www.agentpp.com
http://www.mibexplorer.com
http://www.mibdesigner.com