[AGENT++] Process requests in multiple threads

Arnaud COMPAN compan at ipanematech.com
Fri Jan 11 13:40:00 CET 2008


Thanks for your answer.
I've done some further investigations.
I've recompiled everything (snmp++, agent++, our agent) with threads enabled in order to use AGENT++ thread pool for requests processing.
I've got a crash and gdb gives me the following stack:

Core was generated by `/opt/ipboss/server/bin/ipm_snmp_agent --language=english --domain=arnaud'.
Program terminated with signal 11, Segmentation fault.
#0  Snmpx::send (this=0x1113c20, pdu=
        {<Pdu> = {_vptr.Pdu = 0xb77480, vbs = 0x10c00c0, vbs_size = 2, vb_count = 2, error_status = 0, error_index = 0, validity = true, request_id = 67109889, pdu_type = 162, notify_timestamp = {<SnmpUInt32> = {<SnmpSyntax> = {_vptr.SnmpSyntax = 0xb7bb48, smival = {syntax = 67, value = {sNumber = 0, uNumber = 0, hNumber = {hipart = 0, lopart = 0}, string = {len = 0, ptr = 0x0}, oid = {len = 0, ptr = 0x0}, empty = 0 '\0'}}}, valid_flag = true, output_buffer = "\000\000\000\001\000\000\000\000\000\000\000\000®\016¨", m_changed = true}, output_buffer = "\000\000\006", '\0' <repeats 16 times>, "\001·\220\000\000\000\000\000\000\000"}, notify_id = {<SnmpSyntax> = {_vptr.SnmpSyntax = 0xb7b580, smival = {syntax = 6, value = {sNumber = 0, uNumber = 0, hNumber = {hipart = 0, lopart = 0}, string = {len = 0, ptr = 0x0}, oid = {len = 0, ptr = 0x0}, empty = 0 '\0'}}}, iv_str = 0x0, iv_part_str = 0x0, m_changed = true}, notify_enterprise = {<SnmpSyntax> = {_vptr.SnmpSyntax = 0xb7b580, smival = {syntax = 6, value = {sNumber = 0, uNumber = 0, hNumber = {hipart = 0, lopart = 0}, string = {len = 0, ptr = 0x0}, oid = {len = 0, ptr = 0x0}, empty = 0 '\0'}}}, iv_str = 0x0, iv_part_str = 0x0, m_changed = true}, v1_trap_address = {<Address> = {<SnmpSyntax> = {_vptr.SnmpSyntax = 0xb7b3b0, smival = {syntax = 5, value = {sNumber = 0, uNumber = 0, hNumber = {hipart = 0, lopart = 4276223142}, string = {len = 0, ptr = 0xfee1fca6 ""}, oid = {len = 0, ptr = 0xfee1fca6}, empty = 0 '\0'}}}, addr_changed = true, valid_flag = false, address_buffer = '\0' <repeats 49 times>, static syscall_mutex = {_vptr.SnmpSynchronized = 0xb7b5f8, _mutex = {__pthread_mutex_flags = {__pthread_mutex_flag1 = 4, __pthread_mutex_flag2 = 0 '\0', __pthread_mutex_ceiling = 0 '\0', __pthread_mutex_type = 0, __pthread_mutex_magic = 19800}, __pthread_mutex_lock = {__pthread_mutex_lock64 = {__pthread_mutex_pad = "\000\000\000\000\000\000\000"}, __pthread_mutex_lock32 = {__pthread_ownerpid = 0, __pthread_lockword = 0}, __pthread_mutex_owner64 = 0}, __pthread_mutex_data = 0}}}, address = 0x0, output_buffer = ""}, v1_trap_address_set = 0}, <No data fields>}, udp_address=
      {<IpAddress> = {<Address> = {<SnmpSyntax> = {_vptr.SnmpSyntax = 0xb7b338, smival = {syntax = 4, value = {sNumber = 6, uNumber = 6, hNumber = {hipart = 6, lopart = 4276222666}, string = {len = 6, ptr = 0xfee1faca "\nR\001\003\022É"}, oid = {len = 6, ptr = 0xfee1faca}, empty = 0 '\0'}}}, addr_changed = true, valid_flag = true, address_buffer = "\nR\001\003\022É", '\0' <repeats 43 times>, static syscall_mutex = {_vptr.SnmpSynchronized = 0xb7b5f8, _mutex = {__pthread_mutex_flags = {__pthread_mutex_flag1 = 4, __pthread_mutex_flag2 = 0 '\0', __pthread_mutex_ceiling = 0 '\0', __pthread_mutex_type = 0, __pthread_mutex_magic = 19800}, __pthread_mutex_lock = {__pthread_mutex_lock64 = {__pthread_mutex_pad = "\000\000\000\000\000\000\000"}, __pthread_mutex_lock32 = {__pthread_ownerpid = 0, __pthread_lockword = 0}, __pthread_mutex_owner64 = 0}, __pthread_mutex_data = 0}}}, output_buffer = "\000\000\000\000\001\aÖ\b\000\000\000\001þáýH\000\000\000\aþáü \000hè þáûP\000jZ\224þáû\200\000\000\000\001\000·\220\000þáý°\000\000\000\001\000\000\000\002þáýH\000\000\000\000\001ukÐ\000\000\000\001þáü8", iv_friendly_name = '\0' <repeats 79 times>, iv_friendly_name_status = 0, ip_version = version_ipv4, have_ipv6_scope = false}, output_buffer = "\000\000\000\001\001æ(", '\0' <repeats 12 times>, "þáü8\000\000\000\001\000·\220\000þáý°\000\000\000\001\000\000\000\002þáýH\000\000\000\000\001\aïÀ\001ukÐ\000\000\000\001\000\000\000\000\000\000\000\000\000h\213,þáüp", sep = 58 ':'}, version=version2c, community=
      {<SnmpSyntax> = {_vptr.SnmpSyntax = 0xb7b538, smival = {syntax = 4, value = {sNumber = 6, uNumber = 6, hNumber = {hipart = 6, lopart = 770039168}, string = {len = 6, ptr = 0x2de5dd80 "public"}, oid = {len = 6, ptr = 0x2de5dd80}, empty = 0 '\0'}}}, output_buffer = 0x0, output_buffer_len = 0, m_changed = true, output_last_type = 4276222856, output_last_np_char = 0 '\0', output_last_function = 6851372, validity = true, static hex_output_type = OutputHexAndClear, static nonprintable_char = 46 '.', static linefeed_chars = "\n\000"}) at ../../snmp++/include/snmp_pp/snmpmsg.h:110
110		  { return load(pdu, community, version, 0, 0, 0); };
#0  Snmpx::send (this=0x1113c20, pdu=
        {<Pdu> = {_vptr.Pdu = 0xb77480, vbs = 0x10c00c0, vbs_size = 2, vb_count = 2, error_status = 0, error_index = 0, validity = true, request_id = 67109889, pdu_type = 162, notify_timestamp = {<SnmpUInt32> = {<SnmpSyntax> = {_vptr.SnmpSyntax = 0xb7bb48, smival = {syntax = 67, value = {sNumber = 0, uNumber = 0, hNumber = {hipart = 0, lopart = 0}, string = {len = 0, ptr = 0x0}, oid = {len = 0, ptr = 0x0}, empty = 0 '\0'}}}, valid_flag = true, output_buffer = "\000\000\000\001\000\000\000\000\000\000\000\000®\016¨", m_changed = true}, output_buffer = "\000\000\006", '\0' <repeats 16 times>, "\001·\220\000\000\000\000\000\000\000"}, notify_id = {<SnmpSyntax> = {_vptr.SnmpSyntax = 0xb7b580, smival = {syntax = 6, value = {sNumber = 0, uNumber = 0, hNumber = {hipart = 0, lopart = 0}, string = {len = 0, ptr = 0x0}, oid = {len = 0, ptr = 0x0}, empty = 0 '\0'}}}, iv_str = 0x0, iv_part_str = 0x0, m_changed = true}, notify_enterprise = {<SnmpSyntax> = {_vptr.SnmpSyntax = 0xb7b580, smival = {syntax = 6, value = {sNumber = 0, uNumber = 0, hNumber = {hipart = 0, lopart = 0}, string = {len = 0, ptr = 0x0}, oid = {len = 0, ptr = 0x0}, empty = 0 '\0'}}}, iv_str = 0x0, iv_part_str = 0x0, m_changed = true}, v1_trap_address = {<Address> = {<SnmpSyntax> = {_vptr.SnmpSyntax = 0xb7b3b0, smival = {syntax = 5, value = {sNumber = 0, uNumber = 0, hNumber = {hipart = 0, lopart = 4276223142}, string = {len = 0, ptr = 0xfee1fca6 ""}, oid = {len = 0, ptr = 0xfee1fca6}, empty = 0 '\0'}}}, addr_changed = true, valid_flag = false, address_buffer = '\0' <repeats 49 times>, static syscall_mutex = {_vptr.SnmpSynchronized = 0xb7b5f8, _mutex = {__pthread_mutex_flags = {__pthread_mutex_flag1 = 4, __pthread_mutex_flag2 = 0 '\0', __pthread_mutex_ceiling = 0 '\0', __pthread_mutex_type = 0, __pthread_mutex_magic = 19800}, __pthread_mutex_lock = {__pthread_mutex_lock64 = {__pthread_mutex_pad = "\000\000\000\000\000\000\000"}, __pthread_mutex_lock32 = {__pthread_ownerpid = 0, __pthread_lockword = 0}, __pthread_mutex_owner64 = 0}, __pthread_mutex_data = 0}}}, address = 0x0, output_buffer = ""}, v1_trap_address_set = 0}, <No data fields>}, udp_address=
      {<IpAddress> = {<Address> = {<SnmpSyntax> = {_vptr.SnmpSyntax = 0xb7b338, smival = {syntax = 4, value = {sNumber = 6, uNumber = 6, hNumber = {hipart = 6, lopart = 4276222666}, string = {len = 6, ptr = 0xfee1faca "\nR\001\003\022É"}, oid = {len = 6, ptr = 0xfee1faca}, empty = 0 '\0'}}}, addr_changed = true, valid_flag = true, address_buffer = "\nR\001\003\022É", '\0' <repeats 43 times>, static syscall_mutex = {_vptr.SnmpSynchronized = 0xb7b5f8, _mutex = {__pthread_mutex_flags = {__pthread_mutex_flag1 = 4, __pthread_mutex_flag2 = 0 '\0', __pthread_mutex_ceiling = 0 '\0', __pthread_mutex_type = 0, __pthread_mutex_magic = 19800}, __pthread_mutex_lock = {__pthread_mutex_lock64 = {__pthread_mutex_pad = "\000\000\000\000\000\000\000"}, __pthread_mutex_lock32 = {__pthread_ownerpid = 0, __pthread_lockword = 0}, __pthread_mutex_owner64 = 0}, __pthread_mutex_data = 0}}}, output_buffer = "\000\000\000\000\001\aÖ\b\000\000\000\001þáýH\000\000\000\aþáü \000hè þáûP\000jZ\224þáû\200\000\000\000\001\000·\220\000þáý°\000\000\000\001\000\000\000\002þáýH\000\000\000\000\001ukÐ\000\000\000\001þáü8", iv_friendly_name = '\0' <repeats 79 times>, iv_friendly_name_status = 0, ip_version = version_ipv4, have_ipv6_scope = false}, output_buffer = "\000\000\000\001\001æ(", '\0' <repeats 12 times>, "þáü8\000\000\000\001\000·\220\000þáý°\000\000\000\001\000\000\000\002þáýH\000\000\000\000\001\aïÀ\001ukÐ\000\000\000\001\000\000\000\000\000\000\000\000\000h\213,þáüp", sep = 58 ':'}, version=version2c, community=
      {<SnmpSyntax> = {_vptr.SnmpSyntax = 0xb7b538, smival = {syntax = 4, value = {sNumber = 6, uNumber = 6, hNumber = {hipart = 6, lopart = 770039168}, string = {len = 6, ptr = 0x2de5dd80 "public"}, oid = {len = 6, ptr = 0x2de5dd80}, empty = 0 '\0'}}}, output_buffer = 0x0, output_buffer_len = 0, m_changed = true, output_last_type = 4276222856, output_last_np_char = 0 '\0', output_last_function = 6851372, validity = true, static hex_output_type = OutputHexAndClear, static nonprintable_char = 46 '.', static linefeed_chars = "\n\000"}) at ../../snmp++/include/snmp_pp/snmpmsg.h:110
#1  0x006a3f48 in RequestList::answer (this=0x1073960, req=0x1756bd0) at ../../snmp++/include/snmp_pp/target.h:358
#2  0x00690148 in Mib::finalize (this=0x1073a20, req=0x1756bd0) at mib.cpp:4067
#3  0x0068f348 in Mib::do_process_request (this=0x1073a20, req=0x1756bd0) at mib.cpp:3575
#4  0x0069b2cc in MibTask::run (this=0x10c0130) at threads.cpp:957
#5  0x0069b188 in TaskManager::run (this=0x112fb90) at threads.cpp:779
#6  0x0069b360 in thread_starter (t=0x112fbc0) at threads.cpp:488
#7  0xfef404fc in _lwp_start () from /lib/libc.so.1
#8  0xfef404fc in _lwp_start () from /lib/libc.so.1
Backtrace stopped: previous frame identical to this frame (corrupt stack?)

Do you have any idea about this problem ?

Best regards,
	Arnaud

>> -----Message d'origine-----
>> De : Frank Fock [mailto:fock at agentpp.com] 
>> Envoyé : Thursday, January 03, 2008 10:39 PM
>> À : Arnaud COMPAN
>> Cc : agentpp at agentpp.org
>> Objet : Re: [AGENT++] Process requests in multiple threads
>> 
>> Hi Arnaud,
>> 
>> There is nothing special to keep in mind, just
>> the usual synchronization of concurrent access.
>> If the agent crashes there are at least two
>> possible reasons:
>> 
>> (1) The agent has been compiled with inconsistent
>> compiler flags (recompile *everything* with a
>> consistent setting).
>> (2) Managed objects are accessed concurrently from
>> two or more threads. Since AGENT++ uses a two
>> level locking internally, you should use the same
>> mechanism if you want to update managed objects from
>> external threads.
>> 
>> Best regards,
>> Frank
>> 
>> Arnaud COMPAN wrote:
>> > Hi,
>> >  
>> > I use SNMP++ and AGENT++ for a long time now without problem.
>> > Our agent has two threads: one that process datas and 
>> update the MIB and
>> > the other that answer SNMP requests.
>> > Note that I define _NO_THREADS and use my own lock. 
>> Basically, there is
>> > a global mutex locking all the MIB.
>> > Everything works fine.
>> >  
>> > Now, we try to use multiple threads to answer SNMP 
>> requests (because of
>> > multi-core platforms) but the agent crash.
>> > I've seen that AGENT++ can use a thread pool to process 
>> requests so it
>> > should work.
>> >  
>> > Is there anything to do to process requests in multiple threads ?
>> >  
>> > Best regards,
>> >  
>> > Arnaud.
>> >  
>> > _______________________________________________
>> > AGENTPP mailing list
>> > AGENTPP at agentpp.org
>> > http://lists.agentpp.org/mailman/listinfo/agentpp
>> 
>> -- 
>> AGENT++
>> http://www.agentpp.com
>> http://www.mibexplorer.com
>> http://www.mibdesigner.com
>> 
>> 



More information about the AGENTPP mailing list