[Fusioninventory-devel] A better strategy for network inventory, based on SNMP models analysis

Fri Jul 26 15:21:33 UTC 2013

Hello.

Here is the result of a short variability analysis on various SNMP 
model, covering all kinf od models. I've used the current master branch 
of the server code.

It aims to analyse the following indicators:
- occurence: the number of time this variable appears in a model
- distriution: how many different OID are mapped to this variable

For instance, the following line:
- contact: occurences 473 (39.29%), distribution 472,1 (2)

indicates than the 'contact' variables appears in 472 different models 
(meaning 39,29% of total models count), with 2 different values, one 
being present in 472 models, and the other in just 1 model.

Unfortunatly, we don't have any variable appearing with a constant value 
in all models, that could easily be hardcoded in the agent. But many of 
those variables, despite not always present, are constants (uptime, for 
instance), and many others are almost constants (contact, for instance).

The agent uses the following variables for all kind of devices:
- cpu: 12% present, almost constant
- location: 95% present, almost constant
- firmware: 40% present, high variability
- contact: 40% present, almost constant
- comments: 98% present, almost constant
- uptime: 47% present, constant
- serial: 90% present, high variability
- name: 100% present, low variability
- enterprise: 40% present, low variability
- otherserial: 6% present, constant
- memory: 56% present, low variability
- ram: 13% present, low variability
- cpuuser: undefined
- cpusystem: undefined
- firmware1: 13% present, medium variablity
- firmware2: 10% present, medium variablity
- ipAdEntAddr: 42% present, constant
- ifIndex: 93% present, almost constant
- ifdescr: 46% present, almost constant
- ifName: 88% present, medium variability
- ifType: 92% present, almost constant
- ifmtu: 46% present, constant
- ifspeed: 46% present, constant
- ifstatus: 46% present, constant
- ifinternalstatus: 46% present, constant
- iflastchange: 45% present, constant
- ifinoctets: 46% present, constant
- ifoutoctets: 46% present, constant
- ifinerrors: 46% present, constant
- ifouterrors: 45% present, constant
- ifPhysAddress: 95% present, low variability
- ifaddr: 50% present, medium variability
- portDuplex: 7%, almost constant

Two of them (cpuuser and cpusystem) are never defined anywhere, I just 
dropped any reference to them in the agent code.

For others, excepted for the most variables or barely present (firmware, 
serial, portDuplex, firmware1, firmware2, otherserial), I'd suggest 
using an hardcoded default value in the agent, which may eventually get 
overrided by a model. This way:
- the agent could achieve minimal inventory without a model (for 
instance, directly in discovery phase)
- the agent could retrieve more information for devices for which the 
current model is actually incomplete

And by removing this now useless redundant information from models:
- their size would decrease
- their number would decrease
- their potential error risk would decrease

I'm gonna test this strategy against our existing SNMP walks database.

Independently of those changes, the analysis also show potentials errors:

1) the following variables have suspect value distribution patterns:
- contact: 472,1
- cartridgeblack: 19,1
- ifdescr: 557,1
- portDuplex: 82,1
2) the Networking2073 model, defined in file 996937, seems to be 
actually a printer model, according to its variable list...

-- 
Guillaume

-------------- next part --------------
models count: 1204
variables count: 110
- PortVlanIndex: occurences 52 (4.32%), distribution 35,9,7,1 (4)
- cartridgeblack: occurences 20 (1.66%), distribution 19,1 (2)
- cartridgecyan: occurences 12 (1.00%), distribution 4,4,3,1 (4)
- cartridgecyanlight: occurences 1 (0.08%), distribution 1 (1)
- cartridgemagenta: occurences 12 (1.00%), distribution 7,2,2,1 (4)
- cartridgemagentalight: occurences 1 (0.08%), distribution 1 (1)
- cartridgeyellow: occurences 13 (1.08%), distribution 5,4,2,1,1 (5)
- cdpCacheAddress: occurences 220 (18.27%), distribution 220 (1)
- cdpCacheDeviceId: occurences 213 (17.69%), distribution 213 (1)
- cdpCacheDevicePort: occurences 213 (17.69%), distribution 213 (1)
- cdpCachePlatform: occurences 214 (17.77%), distribution 214 (1)
- cdpCacheVersion: occurences 213 (17.69%), distribution 213 (1)
- comments: occurences 1188 (98.67%), distribution 1146,41,1 (3)
- contact: occurences 473 (39.29%), distribution 472,1 (2)
- cpu: occurences 139 (11.54%), distribution 137,2 (2)
- dot1dBasePortIfIndex: occurences 401 (33.31%), distribution 401 (1)
- dot1dTpFdbAddress: occurences 341 (28.32%), distribution 306,33,1,1 (4)
- dot1dTpFdbPort: occurences 299 (24.83%), distribution 262,37 (2)
- drumblack: occurences 82 (6.81%), distribution 22,17,15,14,5,3,3,3 (8)
- drumblackmax: occurences 114 (9.47%), distribution 67,34,8,5 (4)
- drumblackremaining: occurences 110 (9.14%), distribution 62,36,7,5 (4)
- drumblackused: occurences 116 (9.63%), distribution 94,22 (2)
- drumcyan: occurences 29 (2.41%), distribution 11,10,6,1,1 (5)
- drumcyanmax: occurences 6 (0.50%), distribution 3,3 (2)
- drumcyanremaining: occurences 6 (0.50%), distribution 3,3 (2)
- drummagenta: occurences 29 (2.41%), distribution 16,12,1 (3)
- drumyellow: occurences 28 (2.33%), distribution 11,9,4,3,1 (5)
- entPhysicalModelName: occurences 417 (34.63%), distribution 205,63,49,22,15,12,11,11,6,6,3,2,2,1,1,1,1,1,1,1,1,1,1 (23)
- enterprise: occurences 485 (40.28%), distribution 445,33,3,2,2 (5)
- firmware: occurences 476 (39.53%), distribution 104,59,57,41,28,24,14,14,11,11,11,11,11,11,9,8,7,7,4,4,4,3,3,2,2,2,2,1,1,1,1,1,1,1,1,1,1,1,1 (39)
- firmware1: occurences 158 (13.12%), distribution 112,17,12,11,4,1,1 (7)
- firmware2: occurences 120 (9.97%), distribution 104,11,4,1 (4)
- ifIndex: occurences 1111 (92.28%), distribution 1103,8 (2)
- ifName: occurences 1052 (87.38%), distribution 599,409,44 (3)
- ifPhysAddress: occurences 1150 (95.51%), distribution 1089,45,11,3,2 (5)
- ifType: occurences 1115 (92.61%), distribution 1108,7 (2)
- ifaddr: occurences 606 (50.33%), distribution 473,116,15,2 (4)
- ifalias: occurences 36 (2.99%), distribution 36 (1)
- ifdescr: occurences 558 (46.35%), distribution 557,1 (2)
- ifinerrors: occurences 555 (46.10%), distribution 555 (1)
- ifinoctets: occurences 557 (46.26%), distribution 557 (1)
- ifinternalstatus: occurences 557 (46.26%), distribution 557 (1)
- iflastchange: occurences 546 (45.35%), distribution 546 (1)
- ifmtu: occurences 555 (46.10%), distribution 555 (1)
- ifouterrors: occurences 550 (45.68%), distribution 550 (1)
- ifoutoctets: occurences 556 (46.18%), distribution 556 (1)
- ifspeed: occurences 556 (46.18%), distribution 556 (1)
- ifstatus: occurences 557 (46.26%), distribution 557 (1)
- informations: occurences 169 (14.04%), distribution 149,20 (2)
- ipAdEntAddr: occurences 513 (42.61%), distribution 513 (1)
- ipNetToMediaPhysAddress: occurences 463 (38.46%), distribution 463 (1)
- lldpLocChassisId: occurences 136 (11.30%), distribution 136 (1)
- lldpRemChassisId: occurences 69 (5.73%), distribution 69 (1)
- lldpRemPortDesc: occurences 61 (5.07%), distribution 61 (1)
- lldpRemPortId: occurences 69 (5.73%), distribution 69 (1)
- lldpRemSysDesc: occurences 62 (5.15%), distribution 62 (1)
- lldpRemSysName: occurences 61 (5.07%), distribution 61 (1)
- location: occurences 1145 (95.10%), distribution 1142,3 (2)
- macaddr: occurences 520 (43.19%), distribution 390,57,23,20,13,12,2,2,1 (9)
- maintenancekit: occurences 45 (3.74%), distribution 17,14,8,3,3 (5)
- maintenancekitmax: occurences 82 (6.81%), distribution 48,14,12,8 (4)
- maintenancekitremaining: occurences 53 (4.40%), distribution 26,14,8,5 (4)
- maintenancekitused: occurences 106 (8.80%), distribution 93,13 (2)
- memory: occurences 673 (55.90%), distribution 518,125,28,1,1 (5)
- model: occurences 574 (47.67%), distribution 476,53,23,9,8,3,2 (7)
- name: occurences 1219 (101.25%), distribution 1114,63,26,6,4,4,1,1 (8)
- otherserial: occurences 76 (6.31%), distribution 76 (1)
- pagecounterblackpages: occurences 129 (10.71%), distribution 28,19,11,11,11,10,10,9,9,5,2,2,1,1 (14)
- pagecounterblackpages_copy: occurences 121 (10.05%), distribution 87,11,10,9,3,1 (6)
- pagecounterblackpages_print: occurences 181 (15.03%), distribution 55,34,26,13,10,10,9,9,5,3,3,2,1,1 (14)
- pagecountercolorpages: occurences 76 (6.31%), distribution 20,19,19,7,5,2,2,1,1 (9)
- pagecountercolorpages_copy: occurences 53 (4.40%), distribution 49,2,1,1 (4)
- pagecountercolorpages_print: occurences 96 (7.97%), distribution 35,19,14,12,6,4,2,1,1,1,1 (11)
- pagecounterlargepages: occurences 65 (5.40%), distribution 44,9,8,2,2 (5)
- pagecounterrectoversopages: occurences 181 (15.03%), distribution 60,37,22,15,10,10,9,9,7,2 (10)
- pagecounterscannedpages: occurences 83 (6.89%), distribution 33,9,9,9,8,7,4,2,1,1 (10)
- pagecountertotalpages: occurences 530 (44.02%), distribution 493,19,8,3,3,2,2 (7)
- pagecountertotalpages_copy: occurences 104 (8.64%), distribution 80,10,8,3,2,1 (6)
- pagecountertotalpages_fax: occurences 55 (4.57%), distribution 28,24,2,1 (4)
- pagecountertotalpages_print: occurences 183 (15.20%), distribution 50,40,14,13,9,9,8,8,5,5,5,5,5,2,2,1,1,1 (18)
- portDuplex: occurences 83 (6.89%), distribution 82,1 (2)
- ram: occurences 150 (12.46%), distribution 138,12 (2)
- serial: occurences 1083 (89.95%), distribution 295,223,84,68,58,45,31,27,24,24,23,23,18,16,15,13,9,8,8,7,7,6,5,5,5,4,4,4,3,3,2,2,2,1,1,1,1,1,1,1,1,1,1,1,1 (45)
- tonerblack: occurences 470 (39.04%), distribution 380,81,4,2,1,1,1 (7)
- tonerblack2: occurences 9 (0.75%), distribution 9 (1)
- tonerblack2max: occurences 2 (0.17%), distribution 2 (1)
- tonerblack2remaining: occurences 112 (9.30%), distribution 110,2 (2)
- tonerblackmax: occurences 359 (29.82%), distribution 312,46,1 (3)
- tonerblackremaining: occurences 356 (29.57%), distribution 275,58,20,2,1 (5)
- tonerblackused: occurences 122 (10.13%), distribution 70,33,19 (3)
- tonercyan: occurences 273 (22.67%), distribution 100,80,73,17,2,1 (6)
- tonercyanmax: occurences 190 (15.78%), distribution 87,47,45,11 (4)
- tonercyanremaining: occurences 214 (17.77%), distribution 94,61,48,10,1 (5)
- tonercyanused: occurences 106 (8.80%), distribution 103,3 (2)
- tonermagenta: occurences 278 (23.09%), distribution 109,86,70,10,2,1 (6)
- tonermagentamax: occurences 195 (16.20%), distribution 79,71,45 (3)
- tonermagentaremaining: occurences 212 (17.61%), distribution 84,81,47 (3)
- tonermagentaused: occurences 91 (7.56%), distribution 91 (1)
- toneryellow: occurences 243 (20.18%), distribution 80,71,63,22,5,2 (6)
- toneryellowmax: occurences 179 (14.87%), distribution 81,43,37,15,3 (5)
- toneryellowremaining: occurences 199 (16.53%), distribution 71,68,42,14,3,1 (6)
- toneryellowused: occurences 87 (7.23%), distribution 87 (1)
- uptime: occurences 562 (46.68%), distribution 562 (1)
- vlanTrunkPortDynamicStatus: occurences 95 (7.89%), distribution 95 (1)
- vmvlan: occurences 158 (13.12%), distribution 102,56 (2)
- vtpVlanName: occurences 279 (23.17%), distribution 103,90,85,1 (4)
- wastetoner: occurences 263 (21.84%), distribution 139,81,8,7,6,6,4,4,3,3,2 (11)
- wastetonermax: occurences 199 (16.53%), distribution 133,59,4,2,1 (5)
- wastetonerremaining: occurences 169 (14.04%), distribution 105,57,4,2,1 (5)
- wastetonerused: occurences 82 (6.81%), distribution 82 (1)