noSuchInstance result doesn't seem to be handled well

[scotloach]

using puresnmp 1.6.2.post1

I'm doing get and multiget requests. If I ask for an OID that doesn't exist, it returns noSuchInstance in the result (type code is 0x81). This doesn't seem to be handled properly.

[exhuma]

Can you run isolate the issue in a small test script and run it with these two lines added to the script:

import logging
logging.basicConfig(level=logging.DEBUG)

this will print out each SNMP packet. I am currently having trouble to reproduce the issue. I don't have an OID which raises the "noSuchInstance" error on my test-machine.

Having a hexdump of the packet which causes the error will help me isolate and fix the issue.

I can confirm that the issue exists however. I have identified the part in the code which is responsible for the issue but would still like to have a way to reliably reproduce it. Without it I can't be totally sure that I simulated the issue correctly.

[scotloach]

Sure, just let me know what else I can do to help. Here's the output of a multi-get of two OIDs, the first one exists and the second does not. I've redacted the community.

DEBUG:puresnmp.transport:Sending packet to 10.4.6.51:161 (attempt 1/3)
30 41 02 01 01 04 06 xx  xx xx xx xx xx a0 34 02   0A.....xxxxxx.4.
04 5d 9d 59 c9 02 01 00  02 01 00 30 26 30 11 06   .].Y.......0&0..
0d 2b 06 01 04 01 81 8a  31 15 01 02 0a 00 05 00   .+......1.......
30 11 06 0d 2b 06 01 04  01 81 8a 31 15 01 03 01   0...+......1....
00 05 00                                           ...
DEBUG:puresnmp.transport:Received packet:
30 42 02 01 01 04 06 xx  xx xx xx xx xx a2 35 02   0B.....xxxxxx.5.
04 5d 9d 59 c9 02 01 00  02 01 00 30 27 30 12 06   .].Y.......0'0..
0d 2b 06 01 04 01 81 8a  31 15 01 02 0a 00 02 01   .+......1.......
01 30 11 06 0d 2b 06 01  04 01 81 8a 31 15 01 03   .0...+......1...
01 00 81 00                                        ....
Traceback (most recent call last):
  File "/usr/local/lib/python3.7/site-packages/puresnmp/pdu.py", line 247, in decode
    return super(GetResponse, cls).decode(data)
  File "/usr/local/lib/python3.7/site-packages/puresnmp/pdu.py", line 113, in decode
    values, data = pop_tlv(data)
  File "/usr/local/lib/python3.7/site-packages/puresnmp/x690/types.py", line 118, in pop_tlv
    value = cls.from_bytes(chunk)
  File "/usr/local/lib/python3.7/site-packages/puresnmp/x690/types.py", line 172, in from_bytes
    return cls.decode(data)
  File "/usr/local/lib/python3.7/site-packages/puresnmp/x690/types.py", line 408, in decode
    value, data = pop_tlv(data)
  File "/usr/local/lib/python3.7/site-packages/puresnmp/x690/types.py", line 118, in pop_tlv
    value = cls.from_bytes(chunk)
  File "/usr/local/lib/python3.7/site-packages/puresnmp/x690/types.py", line 172, in from_bytes
    return cls.decode(data)
  File "/usr/local/lib/python3.7/site-packages/puresnmp/x690/types.py", line 408, in decode
    value, data = pop_tlv(data)
  File "/usr/local/lib/python3.7/site-packages/puresnmp/x690/types.py", line 118, in pop_tlv
    value = cls.from_bytes(chunk)
  File "/usr/local/lib/python3.7/site-packages/puresnmp/x690/types.py", line 172, in from_bytes
    return cls.decode(data)
  File "/usr/local/lib/python3.7/site-packages/puresnmp/pdu.py", line 96, in decode
    raise EmptyMessage('No data to decode!')
puresnmp.exc.EmptyMessage: No data to decode!

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/tmp/test.py", line 6, in <module>
    result = multiget('10.4.6.51', 'xxxxxx', ['1.3.6.1.4.1.17713.21.1.2.10.0', '1.3.6.1.4.1.17713.21.1.3.1.0'])
  File "/usr/local/lib/python3.7/site-packages/puresnmp/api/raw.py", line 98, in multiget
    raw_response = Sequence.from_bytes(response)
  File "/usr/local/lib/python3.7/site-packages/puresnmp/x690/types.py", line 172, in from_bytes
    return cls.decode(data)
  File "/usr/local/lib/python3.7/site-packages/puresnmp/x690/types.py", line 408, in decode
    value, data = pop_tlv(data)
  File "/usr/local/lib/python3.7/site-packages/puresnmp/x690/types.py", line 118, in pop_tlv
    value = cls.from_bytes(chunk)
  File "/usr/local/lib/python3.7/site-packages/puresnmp/x690/types.py", line 172, in from_bytes
    return cls.decode(data)
  File "/usr/local/lib/python3.7/site-packages/puresnmp/pdu.py", line 251, in decode
    'Nothing found at the given OID (%s)' % exc)
puresnmp.exc.NoSuchOID: Nothing found at the given OID (No data to decode!) (status-code: 2) on OID 0

[exhuma]

the last block of binary data is enough by the way. This will be the packet returned by the device and will be the one which causes the error.

[exhuma]

Thanks for the dump. This will help a lot. I'll try to work on this in the coming days.

[exhuma]

This is related to #54 and requires a rework improvement of type-detection:

endOfMibView, noSuchInstance and noSuchObject are all returned as values from SNMP and have their own type with a "NULL" value. Currently they are all incorrectly detected as the Null SNMP-type. I was unaware of this while fixin #54 and the implemented fix is a workaround which is not maintainable.

I will fix the type-detection properly which requires a bit more work but will be beneficial in the long-run.

[exhuma]

I'm slowly moving forward on this. It takes me a bit longer than expected because I have trouble finding an authoritative information on how the special types noSuchObject, noSuchInstance and endOfMibView are encoded. I know that they are 0x80, 0x81 and 0x82 respectively. I can see them referenced on page 11 of RFC-1448 or page 9 of RFC 3416, but I have yet to find the authoritative reference for their corresponding typeclass, tag and encodings. By reverse-engineering I know that they are as follows:

bytes	type-class	encoding	tag
0x80	context	primitive	0
0x81	context	primitive	1
0x82	context	primitive	2

And, in order to avoid losing more time in that rabbit-hole I will continue with these values. I am confident that they are correct.

If someone has a link to the definitions, I am curious to see it.

[exhuma]

I'm in the process of cleaning up the code-changes and will try to finish this up this week. In case you are still watching this you can already give the code on the branch "fix/1.6.3" a go and let me know if it works for you.

[scotloach]

I've tried that branch out against some of the endpoints that didn't previously work, and it seems to work now.

[scotloach]

Actually I'm not sure it does fully fix the original issue, which was, in a multi-get request, I want to be able to get the partial result and know which ones failed.