[SNMP4J] TableUtils - more efficient fetching of sparse tables

[Frank Fock]

Hi Michal,

You can test the code in the latest SNMP4J 2.4.0 SNAPSHOT that built a 
few minutes ago:
https://oosnmp.net/dist/snapshot/org/snmp4j/snmp4j/2.4.0-SNAPSHOT/

The optimized algorithm seems to be significantly faster for nearly any 
sparse table, especially
if one reduces the max-repetitions to a lower value than the average 
number of missing values in a
column.

Best regards,
Frank

Am 07.12.2015 um 14:20 schrieb Michał Steiner:
> Great! I can be first to test it.
> I agree that improvement will be visible only in specific cases.  We 
> have a tables  with  almost 200 columns where some of them are full, 
> some are empty or have just couple of cells with value. So in our case 
>  it could be huge improvement.
>
> -- 
> Michal
>
> 2015-12-07 12:35 GMT+01:00 Frank Fock <fock at agentpp.com 
> <mailto:fock at agentpp.com>>:
>
>     Hi Michal,
>     The proposed improvement can be indeed applied to any column,
>     but will have an effect only if the a lot of cells are missing and
>     the distribution is asymmetric.
>     I will try to implement the improvement for SNMP4J 2.4.0
>
>     Best regards
>     Frank
>
>     > Am 02.12.2015 um 09:19 schrieb Michał Steiner <mist at mist.pl
>     <mailto:mist at mist.pl>>:
>     >
>     > Hi,
>     >
>     > I've noticed that getTable method in org.snmp4j.util.TableUtils
>     could be
>     > more efficient for sparse tables.
>     >
>     > Consider table with 3 columns and 3 rows, but in first column
>     there is a
>     > lot of empty cells:
>     >
>     > Column1           Column2        Column3
>     > value-1-1         value-2-1     value-3-1
>     > <empty>           value-2-2     value-3-2
>     > <empty>           value-2-3     value-3-3
>     >
>     > Today implementation works more or less like this:
>     >
>     > get-next   Column1      Column2       Column3
>     > response   value-1-1    value-2-1     value-3-1
>     >
>     > get-next   value-1-1      value-2-1     value-3-1
>     > response   value-2-1      value-2-2     value-3-2
>     >
>     > get-next   value-1-1       value-2-2  value-3-2
>     > response   value-2-1       value-2-3  value-3-3
>     >
>     > get-next   value-1-1      value-2-3     value-3-3
>     > response   value-2-1      value-3-3     value from next table
>     >
>     >
>     > As you can see  request for    value-1-1  is done 3 times and
>     only first
>     > one makes sense.
>     > This is  an issue for large tables (lot of columns and rows)
>     with many
>     > empty cells.
>     > This algorithm can be improved. After first response from other
>     column we
>     > can just stop asking about it. Conversation will look like this:
>     >
>     >
>     > get-next    Column1    Column2        Column3
>     > response   value-1-1   value-2-1      value-3-1
>     >
>     > get-next   value-1-1    value-2-1     value-3-1
>     > response   value-2-1    value-2-2     value-3-2   <-- value-2-1 
>     is from
>     > different column
>     >
>     > get-next               value-2-2      value-3-2 <-- in next
>     requests we
>     > skip  first column
>     > response               value-2-3      value-3-3
>     >
>     > get-next               value-2-3      value-3-3
>     > response               value-3-3      value from next table
>     >
>     > I guess that bulk request are handled in the similar way, aren't
>     they ?
>     >
>     > Are my observations correct ?
>     > What do you think about such improvement?
>     >
>     >
>     > Regards
>     >
>     > Michal
>     > _______________________________________________
>     > SNMP4J mailing list
>     > SNMP4J at agentpp.org <mailto:SNMP4J at agentpp.org>
>     > https://oosnmp.net/mailman/listinfo/snmp4j
>
>

-- 
---
AGENT++
Maximilian-Kolbe-Str. 10
73257 Koengen, Germany
https://agentpp.com
Phone: +49 7024 8688230
Fax:   +49 7024 8688231