Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Exception thrown if MIB file contains more than two modules in the same file #17

Closed
MTamPham opened this issue Oct 6, 2020 · 12 comments
Closed

Exception thrown if MIB file contains more than two modules in the same file #17

MTamPham opened this issue Oct 6, 2020 · 12 comments

Comments

@MTamPham
Copy link

MTamPham commented Oct 6, 2020 •
edited
Loading

JUNIPER-WX-MIB.zip
Hi there,

You've written such a wonderful app which saves me a lot of time if I have to do it by my own. Everything works like a charm except two following issues.

One is it throws an exception of "unexpected token" if the MIB file contains more than one module.
I tried to load module with both gosmi.LoadModule() and parser.ParseFile() functions but it returns the same exception. I'm wondering if we can have any work around to fix this issue? Attached is the MIB file I'm talking about.

The other one is it it failed to load module if the MIB file defines the default value is DEFVAL { { } } with the error "branch ()? was accepted but did not progress the lexer".

I look forward to hearing from you soon.
Kind regards,
Tam Pham
@sleepinggenius2
Copy link
Owner

Thank you for your kind words.

Unfortunately, having multiple modules in a file is not currently supported. For your example file, something like this command under Linux should get you separate files that you can work with:

csplit -f JUNIPER-WX-MIB -b '%02d.mib' JUNIPER-WX-MIB '/========================================================================/' '{*}'

For your second item, I'm thinking the fix would be to put another '?' on

gosmi/parser/object.go

Line 75 in 5a282bd

Defval *string `parser:"( \"DEFVAL\" \"{\" @( \"-\"? Int | BinString | HexString | Text | Ident | ( \"{\" ( Int+ | ( Ident ( \",\" Ident )* \",\"? )? ) \"}\" ) ) \"}\" )?"`
like this:

	Defval      *string              `parser:"( \"DEFVAL\" \"{\" @( \"-\"? Int | BinString | HexString | Text | Ident | ( \"{\" ( Int+ | ( Ident ( \",\" Ident )* \",\"? )? )? \"}\" ) ) \"}\" )?"`

Unfortunately, I don't know that I have a file with that syntax in it, so if you have one that you could test that change on, I'd be happy to update the code.

@MTamPham
Copy link
Author

MTamPham commented Oct 6, 2020 •
edited
Loading

Thanks for a quick response.

I tried to change the format like what you suggested but it is not working unfortunately. Would you mind testing it from your perspective. I've attached the file having that syntax for you.

Thanks again.
Kind regards
Tam Pham
CISCO-STACK-MIB.zip

@sleepinggenius2
Copy link
Owner

Unfortunately, I'm not seeing that syntax in the attached file. I tried parsing it though and it ran without errors. It looks like the error message you are referencing was added to the participle library recently. I would suggest using Go modules, so that it pulls in the correct v0.4.1 version that is currently supported by this library.

@MTamPham
Copy link
Author

MTamPham commented Oct 6, 2020

That's my bad. I fixed the attachment. It should have that syntax now. And the participle library is already in v0.4.1 so that wouldn't be a case I think.

@sleepinggenius2
Copy link
Owner

I see that in the new attachment. With the latest 1.15.2 Go release, I checked out a fresh copy of this repository, ran a go get and tested with the included cmd/parse/main.go example against the attached file and was unable to replicate what you're seeing. It actually looks like your example DEFVAL is being parsed properly with the current code. I'd be happy to continue trying to debug if you are able to provide more details.

@MTamPham
Copy link
Author

MTamPham commented Oct 6, 2020 •
edited
Loading

I greatly appreciate your willingness.

Initially I thought it could relate to the Go version because mine is lower than yours (go1.13.8). Then I tried with the latest one but I am still having the same issue. Below is how I issue the command to run the application. Another thing is I ran it from Windows machine but I think it shouldn't be a problem.

Line 1725 is where it contains the syntax DEFVAL { { } } we discussed.

$ go1.15.2 run main.go
panic: <project_directory>\testmibs\CISCO-STACK-MIB:1725:20: branch (<ident>)? was accepted but did not progress the lexer at <project_directory>\testmibs\CISCO-STACK-MIB:1725:20 ("}")

goroutine 1 [running]:
github.com/alecthomas/participle.(*disjunction).Parse(0xc0001d1480, 0xc00076a690, 0x8dd400, 0xc000753a40, 0x199, 0x8, 0x5, 0x2030001, 0x656, 0x2030001)

Hope it could help you to debug.
Thanks a lot.

@sleepinggenius2
Copy link
Owner

Unfortunately, I do not have access to a Windows machine to be able to test that aspect, but you should be able to use WSL to test under Linux on your side. I also double-checked the participle code at tag v0.4.1 and was unable to find the error message you are getting referenced anywhere. Please run go mod graph in your code directory and verify that you are getting something like this:

github.com/sleepinggenius2/gosmi github.com/alecthomas/[email protected]
github.com/sleepinggenius2/gosmi github.com/alecthomas/[email protected]
github.com/sleepinggenius2/gosmi github.com/pkg/[email protected]
github.com/alecthomas/[email protected] github.com/alecthomas/[email protected]
github.com/alecthomas/[email protected] github.com/alecthomas/[email protected]
github.com/alecthomas/[email protected] github.com/alecthomas/[email protected]
github.com/alecthomas/[email protected] github.com/alecthomas/[email protected]
github.com/alecthomas/[email protected] github.com/alecthomas/[email protected]
github.com/alecthomas/[email protected] github.com/stretchr/[email protected]
github.com/alecthomas/[email protected] gopkg.in/alecthomas/[email protected]
github.com/alecthomas/[email protected] github.com/stretchr/[email protected]
github.com/stretchr/[email protected] github.com/davecgh/[email protected]
github.com/stretchr/[email protected] github.com/pmezard/[email protected]
github.com/stretchr/[email protected] github.com/stretchr/[email protected]
github.com/stretchr/[email protected] gopkg.in/[email protected]
gopkg.in/[email protected] gopkg.in/[email protected]
github.com/alecthomas/[email protected] github.com/davecgh/[email protected]
github.com/alecthomas/[email protected] github.com/mitchellh/[email protected]
github.com/alecthomas/[email protected] github.com/pkg/[email protected]
github.com/alecthomas/[email protected] github.com/pmezard/[email protected]
github.com/alecthomas/[email protected] github.com/stretchr/[email protected]

@MTamPham
Copy link
Author

MTamPham commented Oct 7, 2020

I apologize for the delay in getting back to you. It works for me now for some reason although I didn't change anything. What I did is just restart the machine. Sounds like a magic happens after that haha.

Anyway, I still appreciate your support and thank you for this great tool. I will close this issue as it is resolved.

@MTamPham MTamPham closed this as completed Oct 7, 2020
@MTamPham
Copy link
Author

MTamPham commented Oct 7, 2020

Sorry I was testing the wrong MIB file and looks like it is still happening. However, let me do deeper debugging and figure it out myself then I will create a new issue if necessary.

Cheers.

@MTamPham
Copy link
Author

MTamPham commented Oct 7, 2020

And yeah, I double-checked all dependencies and got this

github.com/alecthomas/go-thrift v0.0.0-20170109061633-7914173639b2
github.com/alecthomas/kong v0.2.1
github.com/alecthomas/participle v0.4.1
github.com/alecthomas/repr v0.0.0-20181024024818-d37bc2a10ba1
github.com/alecthomas/template v0.0.0-20190718012654-fb15b899a751
github.com/alecthomas/units v0.0.0-20190924025748-f65c72e2690d
github.com/davecgh/go-spew v1.1.1
github.com/joho/godotenv v1.3.0
github.com/mitchellh/mapstructure v1.1.2
github.com/pkg/errors v0.8.1
github.com/pmezard/go-difflib v1.0.0
github.com/sirupsen/logrus v1.7.0
github.com/sleepinggenius2/gosmi v0.3.2
github.com/stretchr/objx v0.1.0
github.com/stretchr/testify v1.4.0
golang.org/x/sys v0.0.0-20191026070338-33540a1f6037
gopkg.in/alecthomas/kingpin.v2 v2.2.6
gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405
gopkg.in/yaml.v2 v2.2.2

It looks identical as yours in terms of participle library version.

@MTamPham
Copy link
Author

MTamPham commented Oct 8, 2020 •
edited
Loading

Update:
I tried to run on WSL and experienced the same issue. When I did the debugging, the method Parse() of disjunction on participate library Peek(0) on both branch and context to the same closing curly bracket. I think I will detect and replace the "DEFVAL { { } }" clause by an empty string before loading the module. I hope this action does no harm on the load.

$ go version
go version go1.15.2 linux/amd64

$ go run main.go
panic: /mnt/<project_directory>/errormibs/CISCO-STACK-MIB:1725:20: branch (<ident>)? was accepted but did not progress the lexer at /mnt/<project_directory>/errormibs/CISCO-STACK-MIB:1725:20 ("}")

goroutine 1 [running]:
github.com/alecthomas/participle.(*disjunction).Parse(0xc00017b0e0, 0xc000580690, 0x6edc40, 0xc00056ba40, 0x199, 0xb0, 0xb0, 0x69a620, 0x8, 0xc00040a9b0)
        /home/tpham/go/src/github.com/alecthomas/participle/nodes.go:231 +0xb85
github.com/alecthomas/participle.(*group).Parse(0xc00017b440, 0xc000580690, 0x6edc40, 0xc00056ba40, 0x199, 0x1, 0x0, 0x1, 0x0, 0x0)
        /home/tpham/go/src/github.com/alecthomas/participle/nodes.go:156 +0xa59
github.com/alecthomas/participle.(*sequence).Parse(0xc00017b0a0, 0xc000580690, 0x6edc40, 0xc00056ba40, 0x199, 0x8, 0x203000, 0x203000, 0x203000, 0xc00040ae28)
        /home/tpham/go/src/github.com/alecthomas/participle/nodes.go:255 +0xc9
github.com/alecthomas/participle.(*group).Parse(0xc00017b500, 0xc000580690, 0x6edc40, 0xc00056ba40, 0x199, 0x0, 0x0, 0x0, 0x0, 0x0)
        /home/tpham/go/src/github.com/alecthomas/participle/nodes.go:156 +0xa59
github.com/alecthomas/participle.(*disjunction).Parse(0xc00017ad60, 0xc000580410, 0x6edc40, 0xc00056ba40, 0x199, 0x12ac0, 0x6bd, 0x9, 0x1, 0xc00040b420)
        /home/tpham/go/src/github.com/alecthomas/participle/nodes.go:215 +0x1f6
github.com/alecthomas/participle.(*group).Parse(0xc00017b560, 0xc000580410, 0x6edc40, 0xc00056ba40, 0x199, 0xc000550520, 0x0, 0x1, 0x0, 0x0)
        /home/tpham/go/src/github.com/alecthomas/participle/nodes.go:156 +0xa59
github.com/alecthomas/participle.(*capture).Parse(0xc000156a20, 0xc000580410, 0x6edc40, 0xc00056ba40, 0x199, 0x1, 0x1, 0x2, 0x0, 0x0)
        /home/tpham/go/src/github.com/alecthomas/participle/nodes.go:289 +0x162
github.com/alecthomas/participle.(*sequence).Parse(0xc00017ace0, 0xc000580410, 0x6edc40, 0xc00056ba40, 0x199, 0x0, 0x0, 0x0, 0x0, 0x0)
        /home/tpham/go/src/github.com/alecthomas/participle/nodes.go:255 +0xc9
github.com/alecthomas/participle.(*group).Parse(0xc00017b620, 0xc000580410, 0x6edc40, 0xc00056ba40, 0x199, 0x0, 0x0, 0x0, 0x0, 0x0)
        /home/tpham/go/src/github.com/alecthomas/participle/nodes.go:156 +0xa59
github.com/alecthomas/participle.(*group).Parse(0xc00017b640, 0xc00056e7d0, 0x6edc40, 0xc00056ba40, 0x199, 0x0, 0x0, 0x0, 0x0, 0x0)
        /home/tpham/go/src/github.com/alecthomas/participle/nodes.go:169 +0x1f4
github.com/alecthomas/participle.(*sequence).Parse(0xc000149360, 0xc00056e7d0, 0x6edc40, 0xc00056ba40, 0x199, 0x49, 0x11276, 0x656, 0x9, 0x6edc40)
        /home/tpham/go/src/github.com/alecthomas/participle/nodes.go:255 +0xc9
github.com/alecthomas/participle.(*strct).Parse(0xc000149320, 0xc00056e7d0, 0x6f1700, 0xc00055a480, 0x199, 0x40f270, 0xc00040c490, 0x1, 0x1, 0x755380)
        /home/tpham/go/src/github.com/alecthomas/participle/nodes.go:105 +0x1f0
github.com/alecthomas/participle.(*capture).Parse(0xc000156ab0, 0xc00056e7d0, 0x6f1700, 0xc00055a480, 0x199, 0x1, 0x0, 0x1, 0x0, 0x0)
        /home/tpham/go/src/github.com/alecthomas/participle/nodes.go:289 +0x162
github.com/alecthomas/participle.(*sequence).Parse(0xc0001492a0, 0xc00056e7d0, 0x6f1700, 0xc00055a480, 0x199, 0x0, 0x0, 0x0, 0x0, 0x0)
        /home/tpham/go/src/github.com/alecthomas/participle/nodes.go:255 +0xc9
github.com/alecthomas/participle.(*group).Parse(0xc00017b6e0, 0xc00056e7d0, 0x6f1700, 0xc00055a480, 0x199, 0x0, 0x0, 0x0, 0x0, 0x0)
        /home/tpham/go/src/github.com/alecthomas/participle/nodes.go:156 +0xa59
github.com/alecthomas/participle.(*disjunction).Parse(0xc000115ee0, 0xc00056e690, 0x6f1700, 0xc00055a480, 0x199, 0x49, 0x11234, 0x653, 0xf, 0x0)
        /home/tpham/go/src/github.com/alecthomas/participle/nodes.go:215 +0x1f6
github.com/alecthomas/participle.(*group).Parse(0xc0001f8d20, 0xc00056e690, 0x6f1700, 0xc00055a480, 0x199, 0xc00040d288, 0x4cd2ba, 0xc00023e6c0, 0xc00027f050, 0x6)
        /home/tpham/go/src/github.com/alecthomas/participle/nodes.go:156 +0xa59
github.com/alecthomas/participle.(*sequence).Parse(0xc000115ec0, 0xc00056e690, 0x6f1700, 0xc00055a480, 0x199, 0xc0001cd980, 0xc00040d3e0, 0x47381c, 0x8b9de0, 0xc000216200)
        /home/tpham/go/src/github.com/alecthomas/participle/nodes.go:255 +0xc9
github.com/alecthomas/participle.(*group).Parse(0xc0001f8f20, 0xc00056e690, 0x6f1700, 0xc00055a480, 0x199, 0x0, 0x2000, 0x8, 0x6db6db6db6db, 0x39)
        /home/tpham/go/src/github.com/alecthomas/participle/nodes.go:156 +0xa59
github.com/alecthomas/participle.(*disjunction).Parse(0xc000115e60, 0xc00056e640, 0x6f1700, 0xc00055a480, 0x199, 0x49, 0x11246, 0x655, 0x1, 0x49280f)
        /home/tpham/go/src/github.com/alecthomas/participle/nodes.go:215 +0x1f6
github.com/alecthomas/participle.(*group).Parse(0xc0001f9ba0, 0xc00056e640, 0x6f1700, 0xc00055a480, 0x199, 0x1, 0x0, 0x1, 0x0, 0x0)
        /home/tpham/go/src/github.com/alecthomas/participle/nodes.go:156 +0xa59
github.com/alecthomas/participle.(*sequence).Parse(0xc000115d80, 0xc00056e640, 0x6f1700, 0xc00055a480, 0x199, 0x49, 0x11246, 0x655, 0x1, 0x6f1700)
        /home/tpham/go/src/github.com/alecthomas/participle/nodes.go:255 +0xc9
github.com/alecthomas/participle.(*strct).Parse(0xc000115d40, 0xc00056e640, 0x6e7a40, 0xc0000c2210, 0x199, 0xc0005503a0, 0x0, 0x1, 0x7559c0, 0xc00056e5f0)
        /home/tpham/go/src/github.com/alecthomas/participle/nodes.go:105 +0x1f0
github.com/alecthomas/participle.(*capture).Parse(0xc0001e8b40, 0xc00056e640, 0x6e7a40, 0xc0000c2210, 0x199, 0xc00056c6c0, 0x1, 0x1, 0x7559c0, 0xc00056e5f0)
        /home/tpham/go/src/github.com/alecthomas/participle/nodes.go:289 +0x162
github.com/alecthomas/participle.(*disjunction).Parse(0xc00000fd20, 0xc00056e4b0, 0x6e7a40, 0xc0000c2210, 0x199, 0xc00056c540, 0x1, 0x1, 0x0, 0x0)
        /home/tpham/go/src/github.com/alecthomas/participle/nodes.go:215 +0x1f6
github.com/alecthomas/participle.(*group).Parse(0xc0001f9fa0, 0xc00056e4b0, 0x6e7a40, 0xc0000c2210, 0x199, 0x1, 0x1, 0x1, 0x0, 0x0)
        /home/tpham/go/src/github.com/alecthomas/participle/nodes.go:156 +0xa59
github.com/alecthomas/participle.(*group).Parse(0xc0001f9fc0, 0xc0000640a0, 0x6e7a40, 0xc0000c2210, 0x199, 0x0, 0x0, 0x0, 0x0, 0x0)
        /home/tpham/go/src/github.com/alecthomas/participle/nodes.go:169 +0x1f4
github.com/alecthomas/participle.(*sequence).Parse(0xc00000f4e0, 0xc0000640a0, 0x6e7a40, 0xc0000c2210, 0x199, 0x49, 0x13a, 0xc, 0x1, 0x6e7a40)
        /home/tpham/go/src/github.com/alecthomas/participle/nodes.go:255 +0xc9
github.com/alecthomas/participle.(*strct).Parse(0xc00000f4a0, 0xc0000640a0, 0x6d2e20, 0xc0000ce1c0, 0x199, 0xc000348e58, 0x0, 0x1, 0x0, 0x0)
        /home/tpham/go/src/github.com/alecthomas/participle/nodes.go:105 +0x1f0
github.com/alecthomas/participle.(*capture).Parse(0xc0001e8d80, 0xc0000640a0, 0x6d2e20, 0xc0000ce1c0, 0x199, 0x1, 0x1, 0x1, 0x0, 0x0)
        /home/tpham/go/src/github.com/alecthomas/participle/nodes.go:289 +0x162
github.com/alecthomas/participle.(*sequence).Parse(0xc00000f2a0, 0xc0000640a0, 0x6d2e20, 0xc0000ce1c0, 0x199, 0x49, 0x111, 0xa, 0x1, 0x6d2e20)
        /home/tpham/go/src/github.com/alecthomas/participle/nodes.go:255 +0xc9
github.com/alecthomas/participle.(*strct).Parse(0xc00000f260, 0xc0000640a0, 0x6d2e20, 0xc0000ce000, 0x199, 0x199, 0x0, 0xa0043d590, 0x1, 0xa)
        /home/tpham/go/src/github.com/alecthomas/participle/nodes.go:105 +0x1f0
github.com/alecthomas/participle.(*Parser).parseInto(0xc00005a540, 0xc0000640a0, 0x69afe0, 0xc0000ce000, 0x16, 0x66, 0x8ebf98)
        /home/tpham/go/src/github.com/alecthomas/participle/parser.go:207 +0xd0
github.com/alecthomas/participle.(*Parser).parseOne(0xc00005a540, 0xc0000640a0, 0x69afe0, 0xc0000ce000, 0x16, 0x0, 0x0)
        /home/tpham/go/src/github.com/alecthomas/participle/parser.go:190 +0x77
github.com/alecthomas/participle.(*Parser).ParseFromLexer(0xc00005a540, 0xc000212180, 0x69afe0, 0xc0000ce000, 0x0, 0x0, 0x0, 0x0, 0x0)
        /home/tpham/go/src/github.com/alecthomas/participle/parser.go:155 +0x4a5
github.com/alecthomas/participle.(*Parser).Parse(0xc00005a540, 0x755500, 0xc000516018, 0x69afe0, 0xc0000ce000, 0x0, 0x0, 0x0, 0x39, 0xc0000ca99a)
        /home/tpham/go/src/github.com/alecthomas/participle/parser.go:171 +0xf3
github.com/sleepinggenius2/gosmi/parser.Parse(...)
        /home/tpham/go/src/github.com/sleepinggenius2/gosmi/parser/parser.go:55
github.com/sleepinggenius2/gosmi/parser.ParseFile(0xc0000ca050, 0x49, 0x0, 0x0, 0x0)
        /home/tpham/go/src/github.com/sleepinggenius2/gosmi/parser/parser.go:64 +0x15a
github.com/sleepinggenius2/gosmi/smi/internal.LoadModule(0xc0000ca960, 0x49, 0xc0000ca960, 0x49, 0x8f4c20)
        /home/tpham/go/src/github.com/sleepinggenius2/gosmi/smi/internal/module.go:322 +0x6e
github.com/sleepinggenius2/gosmi/smi/internal.GetModule(0xc0000ca960, 0x49, 0x0, 0x4dec1f, 0xc000202060)
        /home/tpham/go/src/github.com/sleepinggenius2/gosmi/smi/internal/module.go:312 +0x8a
github.com/sleepinggenius2/gosmi/smi.LoadModule(0xc0000ca960, 0x49, 0x1, 0x1)
        /home/tpham/go/src/github.com/sleepinggenius2/gosmi/smi/module.go:14 +0x4a
github.com/sleepinggenius2/gosmi.LoadModule(0xc0000ca960, 0x49, 0x1, 0x7fffd4029bd5, 0x9, 0xc000202060)

@sleepinggenius2
Copy link
Owner

Looking as the stack trace provided, it is showing the panic being raised at github.com/alecthomas/participle/nodes.go:231, which was only added here: alecthomas/participle@a91dcc3 and was not present in v0.4.1: https://github.com/alecthomas/participle/blob/v0.4.1/nodes.go#L231. That seems to indicate that you are still pulling in the wrong library version somewhere. Please look at your go.mod and go.sum files and double-check that you are using the correct library versions.

@sleepinggenius2 sleepinggenius2 mentioned this issue Feb 3, 2022
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@MTamPham @sleepinggenius2