Policy on breaking changes in the pack schema

[jld00]

I note that a recent change to the PACK.xsd schema file has rendered certain previously valid PDSC files invalid:
b6a2170

For organizations that build products that process PDSC files, breakage like this is a big deal. Consumers of the specification need confidence that their PDSC files will continue to adhere to the specification.

Is there any policy in this area at present? If not, I would like to start a discussion on process improvement.

[jkrech]

@jld00 my apologies for this breaking change and the disrupt this has caused for you and others. If you browse through the history of the changes to the PACK.xsd schema you will find that we have been very careful not to make any breaking changes.
One criteria for accepting changes is that with every PR we are running a schema validation against all public packs, which obviously does not give us 100% coverage. There is also the requirement for Reviews of PRs.
The project is supposed to follow semantic versioning, meaning that breaking changes would require the major version to be incremented.
We have clearly breaking this rule in this case:
Clang got introduced in 30. Jan 2023: v1.7.16 and got changed to CLANG in 26. July 2023: v1.7.27 at the time when we added initial LLVM/CLANG support in CMSIS-Toolbox 2.0.0

Thanks for raising this and I do agree that we must absolutely avoid this from happening again.
I will raise this topic in the next Open-CMSIS-Pack Technical Project Meeting (Tuesday 2023-09-12) to discuss this further.
If you or anybody else have any suggestions, e.g. adding a list of mandatory reviewers/approvers, let us know here.

[jld00]

Thank you for your prompt response, @jkrech. A couple of thoughts:

• It is important for all reviewers to appreciate that there are consumers of this specification beyond the CMSIS-Toolbox initiative and beyond the principal CMSIS packs repository. Even minor changes such as renaming a DeviceVendorEnum can lead to problems if an organization has released product that performs PDSC file generation based on an earlier version of the schema. There are multiple examples of such clean up in the history of PACK.xsd.
• Automated detection of breaking changes would be ideal. There are tools that generate sample XML from a schema. It might be possible to use such as tool to generate a more exhaustive test case for successive revisions of the schema.

[jkrech]

Thanks for the additional feedback. I had brought this topic to yesterday's Technical Project meeting (https://linaro.atlassian.net/wiki/spaces/CMSIS/pages/29007151163/Open-CMSIS-Pack+Technical+Meeting+2023-09-12).
a) we welcome organizations using the specification to let us know and ideally participate in the evolution of the specification
b) we are very interested in understanding the use cases better, to clearly define the "compatibility". E.g. at this point we are not considering the renaming of a type or enumeration a break in compatibility, as it does not impact the validation of pdsc files. We also came across situations where we added restrictions affecting identifiers (e.g. componentIDs that we had to represent in yml).
c) we are looking forward to getting help from you or someone in your organization to develop/generate a more exhaustive test case of successive revisions. This is an open-source project and we rely on people like you with good ideas and the ability to actively contribute to make this project flourish.

[jld00]

Further on this topic, I have recently discovered a significant PDSC file validation issue with older CMSIS packs that use build metadata (a "+build" suffix) in component version strings. The component element documentation describes the Cversion attribute as having a value of type VersionType (which permits build metadata), but the schema change at 7af014d now precludes the use of build metadata in the Cversion attribute value.

Based on comments in the schema file, it appears that accommodation of build metadata in pack version strings but not in component version strings is deliberate. Would a PR to also permit build metadata in component version strings be acceptable?

[JamesD76]

Hello @jkrech , would there be any update on the above issue reported by John please?
It now causes a tremendous amount of schema validation warnings when parsing most our CMSIS packs which contains build metadata.
Thanks, James

[jkrech]

Maybe you come an join the Technical Project Meeting (https://linaro.atlassian.net/wiki/spaces/CMSIS/overview) and share your intentions and expectations in tools about using component versions with "+build" information.
Do you expect that the user specifies a component with a version that contains build information?
If no build information is provided what are the tools supposed to do if there is multiple components with the same version but different build information?
I think this adds unnecessary complexity and potential confusion for users but I am sure I am missing your use case.