MIB parser breaks on unexpected whitespace

[sponeil]

I encountered over a dozen Cisco MIB files that this parser fails to parse. (All Cisco MIBs can be downloaded in a zip file from their site, but I will include some examples in this post.)

Note: I had been using a slightly older version of gosmi, so I just updated to v0.4.2, but it still has the same issues.

Example 1:
CISCO-AAA-SERVER-MIB has some tabs in unexpected places, and the tab between "OBJECT" and "IDENTIFIER" breaks the parser (replace all instances of that tab with a space, and it parses successfully):

cAAAServerMIBObjects\t OBJECT\tIDENTIFIER ::= { ciscoAAAServerMIB 1 }

Example 2:
DS3-MIB has some line-feeds in unexpected places, and they break the parser (clean up the line-feeds, and it parses successfully):

             ds3Conformance OBJECT IDENTIFIER ::= { ds3 14 }

             ds3Groups      OBJECT IDENTIFIER ::= {
             ds3Conformance 1 } ds3Compliances OBJECT
             IDENTIFIER ::= { ds3Conformance 2 }

It's been a while (1991) since I've had to write my own parser/compiler, but unless the language definition prohibits it, the parser is supposed to be able to handle unusual whitespace like this.

Thanks in advance,
Sean

CISCO-AAA-SERVER-MIB.txt
DS3-MIB.txt

[sleepinggenius2]

I'm going to be honest, the syntax in the ASN.1 specification is not always the clearest thing, but reading through it again, it does look like any non-zero amount of whitespace should be allowed between those tokens. I believe that would affect OBJECT IDENTIFIER and OCTET STRING with the current parser implementation. I have tested an updated implementation that correctly parses the two attached files. If you can confirm that is where you are seeing all these whitespace issues, then I would be happy to commit the changes.

[sponeil]

Yes, all of the MIB files I spot-checked seemed to break in those places. If you commit the changes, I can re-run the test and see if any are still failing. Thanks, Sean

…

On Tue, Jun 8, 2021 at 6:28 PM sleepinggenius2 ***@***.***> wrote: I'm going to be honest, the syntax in the ASN.1 specification is not always the clearest thing, but reading through it again, it does look like any non-zero amount of whitespace should be allowed between those tokens. I believe that would affect OBJECT IDENTIFIER and OCTET STRING with the current parser implementation. I have tested an updated implementation that correctly parses the two attached files. If you can confirm that is where you are seeing all these whitespace issues, then I would be happy to commit the changes. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#24 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AE7JSCRWTTPAIIPQWAPYPZ3TR2KQJANCNFSM46KW5F2A> .

[sponeil]

FYI, I spot-checked some more MIB files that failed to poll, and here are a few that seem to be failing due to different issues. I'm not certain whether "deprecated" or "MAX" are technically legal in a MIB file in those places. I imagine the "0001" should be technically legal even if it looks bad. Sometimes a MIB file is just defined incorrectly and needs to be fixed, but of course sometimes being able to handle odd things like this more gracefully will improve the gosmi library. *Different Error Types:* CISCO-XGCP-CAPABILITY.my:141:17: unexpected "deprecated" (expected ("current" | "obsolete") ...) ADMIN-AUTH-STATS-MIB.my:106:27: unexpected ".." (expected ")") CISCO-LES-MIB.my:645:23: unexpected "0" (expected ")") *Offending section for "deprecated":* PRODUCT-RELEASE "Cisco IOS 12.3" STATUS deprecated -- superceded by ciscoXgcpCapabilityV12R03AS5000 DESCRIPTION "Cisco XGCP MIB capabilities" *Offending section for "unexpected ..":* alAdminAuthServIndex OBJECT-TYPE SYNTAX Integer32 (0..MAX) MAX-ACCESS not-accessible *Offending section for "unexpected 0":* lesClientLecid OBJECT-TYPE SYNTAX INTEGER (0001..65279) -- 0x0001 to 0xFFEF MAX-ACCESS not-accessible

…

On Tue, Jun 8, 2021 at 8:01 PM Sean O'Neil ***@***.***> wrote: Yes, all of the MIB files I spot-checked seemed to break in those places. If you commit the changes, I can re-run the test and see if any are still failing. Thanks, Sean On Tue, Jun 8, 2021 at 6:28 PM sleepinggenius2 ***@***.***> wrote: > I'm going to be honest, the syntax in the ASN.1 specification is not > always the clearest thing, but reading through it again, it does look like > any non-zero amount of whitespace should be allowed between those tokens. I > believe that would affect OBJECT IDENTIFIER and OCTET STRING with the > current parser implementation. I have tested an updated implementation that > correctly parses the two attached files. If you can confirm that is where > you are seeing all these whitespace issues, then I would be happy to commit > the changes. > > — > You are receiving this because you authored the thread. > Reply to this email directly, view it on GitHub > <#24 (comment)>, > or unsubscribe > <https://github.com/notifications/unsubscribe-auth/AE7JSCRWTTPAIIPQWAPYPZ3TR2KQJANCNFSM46KW5F2A> > . >

[sleepinggenius2]

CISCO-XGCP-CAPABILITY.my:141:17: unexpected "deprecated" (expected ("current" | "obsolete") ...)

There is an obscure reference in RFC1444 Section 5.2 to a "deprecated" value, though it is not part of the formal AGENT-CAPABILITIES macro definition. As such, I will add that as a valid option.

ADMIN-AUTH-STATS-MIB.my:106:27: unexpected ".." (expected ")")

RFC2578 Appendix A specifically forbids the use of MIN and MAX in refinements within SMI, though they are valid ASN.1. I will add a comment in the code to that effect.

CISCO-LES-MIB.my:645:23: unexpected "0" (expected ")")

Every version of the ASN.1 specification (ITU-T X.680) from 07/1994 (Section 9.8), all the way through 02/2021 (Section 12.8) states: The first digit shall not be zero unless the "number" is a single digit. I will add a comment in the code to that effect.

I will get a new release out shortly with these changes.

[sponeil]

Ok, thanks for the clarification, and for the quick responses.

…

On Wed, Jun 9, 2021 at 3:20 PM sleepinggenius2 ***@***.***> wrote: CISCO-XGCP-CAPABILITY.my:141:17: unexpected "deprecated" (expected ("current" | "obsolete") ...) There is an obscure reference in RFC1444 Section 5.2 to a "deprecated" value, though it is not part of the formal AGENT-CAPABILITIES macro definition. As such, I will add that as a valid option. ADMIN-AUTH-STATS-MIB.my:106:27: unexpected ".." (expected ")") RFC2578 Appendix A specifically forbids the use of MIN and MAX in refinements within SMI, though they are valid ASN.1. I will add a comment in the code to that effect. CISCO-LES-MIB.my:645:23: unexpected "0" (expected ")") Every version of the ASN.1 specification (ITU-T X.680) from 07/1994 (Section 9.8), all the way through 02/2021 (Section 12.8) states: The first digit shall not be zero unless the "number" is a single digit. I will add a comment in the code to that effect. I will get a new release out shortly with these changes. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#24 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AE7JSCSHI25RC6TMV3YQTZLTR65HFANCNFSM46KW5F2A> .

[sponeil]

FYI, I ran the new v0.4.3 on all Cisco MIBs, and your changes worked as expected. I fixed the remaining broken MIB files by hand, and the only new type of error it encountered was: CISCO-ATM-PVCTRAP-EXTN-CAPABILITY.my:99:9: unexpected "VARIATION" (expected "END") Not sure if I'll ever need that one. At this point, I'm just reporting all parsing errors I find in case it helps someone else. Thanks, Sean

…

On Wed, Jun 9, 2021 at 6:56 PM Sean O'Neil ***@***.***> wrote: Ok, thanks for the clarification, and for the quick responses. On Wed, Jun 9, 2021 at 3:20 PM sleepinggenius2 ***@***.***> wrote: > CISCO-XGCP-CAPABILITY.my:141:17: unexpected "deprecated" (expected > ("current" | "obsolete") ...) > > There is an obscure reference in RFC1444 Section 5.2 to a "deprecated" > value, though it is not part of the formal AGENT-CAPABILITIES macro > definition. As such, I will add that as a valid option. > > ADMIN-AUTH-STATS-MIB.my:106:27: unexpected ".." (expected ")") > > RFC2578 Appendix A specifically forbids the use of MIN and MAX in > refinements within SMI, though they are valid ASN.1. I will add a comment > in the code to that effect. > > CISCO-LES-MIB.my:645:23: unexpected "0" (expected ")") > > Every version of the ASN.1 specification (ITU-T X.680) from 07/1994 > (Section 9.8), all the way through 02/2021 (Section 12.8) states: The first > digit shall not be zero unless the "number" is a single digit. I will add a > comment in the code to that effect. > > I will get a new release out shortly with these changes. > > — > You are receiving this because you authored the thread. > Reply to this email directly, view it on GitHub > <#24 (comment)>, > or unsubscribe > <https://github.com/notifications/unsubscribe-auth/AE7JSCSHI25RC6TMV3YQTZLTR65HFANCNFSM46KW5F2A> > . >

[sleepinggenius2]

Sorry, I was out of town for a few days. In reviewing that MIB file, the error is valid. An SMI macro (AGENT-CAPABILITIES in this case) only supports a single value assignment at the end, so including one after each module variation not only makes no sense, but is syntactically incorrect. See CISCO-SONET-CAPABILITY for an example of the correct syntax. In practice, I've never actually used the agent capabilities definitions, as most equipment does not seem to populate the sysORTable, though Cisco does for a lot of their equipment.

[sponeil]

Ok, thanks. It's interesting that you mention the sysORTable. The fact that most equipment neglects the sysORTable is the reason I'm bothering to attempt to successfully parse every MIB file for certain equipment manufacturers. After I build a list of "root" OIDs for all of a manufacturer's MIBs (e.g. Cisco), I can load them in numeric OID order, and then implement an efficient skip-scan algorithm to find every MIB a device supports very quickly (on the order of 50-100ms for most devices). Of course it would be quicker and easier to poll the sysORTable, but as you pointed out, that's very hit-or-miss. While Cisco seems to populate it for their hardware routers, they don't seem to for the UC virtual machines they offer (e.g. CUCM, Unity, Contact Center, Expressway).

…

On Wed, Jun 16, 2021 at 6:22 PM sleepinggenius2 ***@***.***> wrote: Sorry, I was out of town for a few days. In reviewing that MIB file, the error is valid. An SMI macro (*AGENT-CAPABILITIES* in this case) only supports a single value assignment at the end, so including one after each module variation not only makes no sense, but is syntactically incorrect. See *CISCO-SONET-CAPABILITY* for an example of the correct syntax. In practice, I've never actually used the agent capabilities definitions, as most equipment does not seem to populate the *sysORTable*, though Cisco does for a lot of their equipment. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#24 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AE7JSCVNQACOCU7D4YLUALDTTEP2RANCNFSM46KW5F2A> .