Adjusting for version 3.X

[steweg]

This patch perform the following:

• refactor the code to be able to use libyang 3.X
• adds additional parsing options

[steweg]

@samuel-gauthier I have created this patch for upcoming v3.0 of libyang. Its working with my locally build libyang version 3.0.1.

In my other PRs: #103, #104, #102 I have failing check of test using the latest devel. I believe it is because the devel branch is having some parts of code from v3.0.... So can you please help me somehow to fix it (skip the test, or pull part of this commit, or point me what should be changed to make this working)?
Thanks

[rjarry]

Is there a way to support both libyang 2 and 3 with the same cffi/python code base?

[steweg]

I am afraid this can be possible only if you would keep two different cdefs definitions as v3.0 has API backward incompatible API changes in it and have some switch for user, which version should be used. But I am not that deep with cdefs, so I am not even sure if it is technically doable.... thats up to you guys to judge.

[senthilkvs]

Can you pls point me to branch in which libyang 3.0 development is happening ?

Thanks !

[steweg]

Can you pls point me to branch in which libyang 3.0 development is happening ?

Thanks !

It was in the libyang repo branch new_so, but very recently it was merged to branch devel already

[steweg]

I finally managed to get it compiled correctly. But for the proper merge, I suggest that the tox-install.sh changes will not be used, and job "CI / libyang_devel" check/task will not be used as-is. Otherwise it will not be possible to merge anything into version based on v2 (as it is now)

[smark28]

Hi @steweg
I'm not sure I put the question in the right place.
I wanted to clarify one point, is this PR contains the feature for implementing referenced yang modules?
I'll explain the details. For example, I'm getting this error when validating xml against open-config model.

yanglint openconfig-platform.yang hardware.xml

libyang.util.LibyangError: failed to parse data tree: Invalid identityref "oct:ACTIVE" value - 
identity found in non-implemented module "openconfig-platform-types".

I can avoid this error by implementing openconfig-platform-types.yang in this command

yanglint openconfig-platform-types.yang openconfig-platform.yang hardware.xml

But how to use this approach in libyang-python?

Any help is appreciated, thanks!

[steweg]

Hi @steweg I'm not sure I put the question in the right place. I wanted to clarify one point, is this PR contains the feature for implementing referenced yang modules? I'll explain the details. For example, I'm getting this error when validating xml against open-config model.

yanglint openconfig-platform.yang hardware.xml

libyang.util.LibyangError: failed to parse data tree: Invalid identityref "oct:ACTIVE" value - 
identity found in non-implemented module "openconfig-platform-types".

I can avoid this error by implementing openconfig-platform-types.yang in this command

yanglint openconfig-platform-types.yang openconfig-platform.yang hardware.xml

But how to use this approach in libyang-python?

Any help is appreciated, thanks!

I am not sure in which exact release of libyang was the feature added, but I guess was done way before v3, so this PR shall not be needed for your use-case. This PR should focuses only on changes between v2 and v3. Anyhow to emulate what you are looking for, try to simply do the following:

ctx = libyang.Context()
mod1 = ctx.load_module("openconfig-platform-types.yang")
mod2 = ctx.load_module("openconfig-platform.yang")
with open("hardware.xml", encoding="utf-8") as f:
    dnode = self.ctx.parse_data_file(f, "xml")

[smark28]

mod1 = ctx.load_module("openconfig-platform-types.yang")
mod2 = ctx.load_module("openconfig-platform.yang")

It works for me, thanks a lot!

[rjarry]

Hello,

I will put this PR on hold until libyang 3 has been released.

Thanks

[steweg]

Hello,

I will put this PR on hold until libyang 3 has been released.

Thanks

The new version was just released. SO version is 3.0.8

[samuel-gauthier]

Correct me if I'm wrong, but I have the feeling that this patch does more than just libyang 3 support, it also adds new things.
As you understood I'm sure now, we want to split things in this project. It makes the lives of the users and the maintainers easier in the long run. With that in mind, could you submit only the libyang3 change (which apparently is only a bunch of changes in cdefs.h, and a rework of the logging function)? Those new parsing options can be submitted later.
Thanks!

[steweg]

Correct me if I'm wrong, but I have the feeling that this patch does more than just libyang 3 support, it also adds new things. As you understood I'm sure now, we want to split things in this project. It makes the lives of the users and the maintainers easier in the long run. With that in mind, could you submit only the libyang3 change (which apparently is only a bunch of changes in cdefs.h, and a rework of the logging function)? Those new parsing options can be submitted later. Thanks!

I originally planned to have it separated, but it turned out that it doesn't make sense. If you look at how the libyang API was changed when it comes to lyd_new_* APIs, you will realize that the many of previous bool flags, where replaced with common option flags. So those options LYD_NEW_* were there already. When it comes to these:

LYD_NEW_PATH_WITH_OPAQ
LYD_NEW_ANY_USE_VALUE
LYD_NEW_VAL_STORE_ONLY
LYD_PARSE_STORE_ONLY

Those were added only to avoid a need of extending the same structures/APIs again and again, as those are still related to libyang API changes in v3. So that's why I have put them in... anyway I have remove those, so it should be ok.

If you still wish I can remove them, but frankly speaking