Update schema JSON to use cross-referencing and force more validations

[sei-vsarvepalli]

This is starting attempt to resolve #697 - don't merge yet. I am running a number of tests locally as well with this schema change. It is so far a non-breaking change of the schema tools.

• resolves SSVC should not be allowed to be empty #697

Copilot summary

This pull request includes several updates to JSON schema files to improve descriptions, add new constraints, and enhance the clarity of the schema definitions. The changes also include updates to test files to reflect the schema modifications.

Schema improvements:

• data/schema/v1/Decision_Point-1-0-1.schema.json: Enhanced descriptions for properties, added minLength constraints, and included example values for better clarity. Updated the namespace property to include a pattern and examples. [1] [2] [3]
• data/schema/v1/Decision_Point_Group-1-0-1.schema.json: Added a description for the schema, improved property descriptions, and added minLength constraints with examples. [1] [2]
• data/schema/v1/Decision_Point_Value_Selection-1-0-1.schema.json: Added a schema description, refined property descriptions, added minLength constraints, and used $ref to reference properties from other schemas. [1] [2]

Test updates:

• src/test/test_dp_base.py: Updated the namespace value in test cases to match the schema changes. [1] [2]

[ahouseholder]

Converted to draft so we don't merge too early

[tschmidtb51]

Please see my feedback.

[tschmidtb51]

Please stick with date-time. You can enforce validation (either by specification or schema) - but most programs will do that anyway.

Suggested change

	"pattern": "^(?:[1-9]\\d{3}-[01]\\d-[0-3]\\d[Tt][0-2]\\d:[0-5]\\d:[0-5]\\d(?:\\.\\d+)?(?:[Zz]|[+-][0-2]\\d:[0-5]\\d))$"
	"format": "date-time"

[tschmidtb51]

Sorry, I missed to quote two \

Suggested change

	"pattern": "^(0|[1-9]\d*)\\.(0|[1-9]\d*)\\.(0|[1-9]\\d*)(?:-((?:0|[1-9]\\d*|\\d*[a-zA-Z-][0-9a-zA-Z-]*)(?:\\.(?:0|[1-9]\\d*|\\d*[a-zA-Z-][0-9a-zA-Z-]*))*))?(?:\\+([0-9a-zA-Z-]+(?:\\.[0-9a-zA-Z-]+)*))?$"
	"pattern": "^(0|[1-9]\\d*)\\.(0|[1-9]\\d*)\\.(0|[1-9]\\d*)(?:-((?:0|[1-9]\\d*|\\d*[a-zA-Z-][0-9a-zA-Z-]*)(?:\\.(?:0|[1-9]\\d*|\\d*[a-zA-Z-][0-9a-zA-Z-]*))*))?(?:\\+([0-9a-zA-Z-]+(?:\\.[0-9a-zA-Z-]+)*))?$"

[sei-vsarvepalli]

comma missing at the end as well that is fixed now in the 0dc08d0

[sei-vsarvepalli]

Should we stick to [0-9] instead of \d as \d matches say Arabic numerals when regex is used with unicode?

[ahouseholder]

(Commenting without diff'ing the strings) I'd be most comfortable with the second example provided by

https://semver.org/

specifically

https://regex101.com/r/vkijKf/1/

on the basis that if the semver.org thinks that's the regex, I'd take their word for it.

[sei-vsarvepalli]

What we have is exactly that with the double backslashes for usage in the JSON schema that's all.

[tschmidtb51]

Should we stick to [0-9] instead of \d as \d matches say Arabic numerals when regex is used with unicode?

I'd like that as it eliminates the ambiguousness. \d is unfortunately sometimes interpreted differently...

[tschmidtb51]

The line 98 in the decision point schema looks off. Shouldn't that be inside a keyword?

[ahouseholder]

@sei-vsarvepalli can you please update the description of this PR to say what it actually does (more than linking to the issue)?

I noticed a number of description fields in this changeset that are vague in what they're talking about ("decision point or decision point group", etc.) when in fact they are for one but not the other. I've only commented on the ones that showed up in the changeset, but without looking at the rest of the file I suspect that's an issue throughout the description fields. We should clean that up to avoid confusion.

[ahouseholder]

This appears to be inside a decision point value spec. So it's a brief name for the decision point value, right? It's not describing the decision point or the group.

[sei-vsarvepalli]

Yeh - those have been the same. I have not changed them. Bu happy to change them as well. You are only seeing it because the "comma" at the end added.

[ahouseholder]

description of the decision point group, not decision point

[tschmidtb51]

The line 98 in the decision point schema looks off. Shouldn't that be inside a keyword?

Sorry - my bad. After I looked at an instance again, I understand now what you are doing.

[sei-vsarvepalli]

@sei-vsarvepalli can you please update the description of this PR to say what it actually does (more than linking to the issue)?

I noticed a number of description fields in this changeset that are vague in what they're talking about ("decision point or decision point group", etc.) when in fact they are for one but not the other. I've only commented on the ones that showed up in the changeset, but without looking at the rest of the file I suspect that's an issue throughout the description fields. We should clean that up to avoid confusion.

All the "descriptions" are from the old JSON file nothing has changed in these. But I do see your point, we can update them too. They were left generic in the first edition, and remained the same so far.

[ahouseholder]

@sei-vsarvepalli can you please update the description of this PR to say what it actually does (more than linking to the issue)?
I noticed a number of description fields in this changeset that are vague in what they're talking about ("decision point or decision point group", etc.) when in fact they are for one but not the other. I've only commented on the ones that showed up in the changeset, but without looking at the rest of the file I suspect that's an issue throughout the description fields. We should clean that up to avoid confusion.

All the "descriptions" are from the old JSON file nothing has changed in these. But I do see your point, we can update them too. They were left generic in the first edition, and remained the same so far.

Yeah, I figured they had been that way for a while. It's up to you, if it's an easy fix this time, I think it's worth doing. If it's not, then we can merge this PR as-is and leave a breadcrumb issue behind to pick up the descriptions at another time. I don't necessarily want us to block on this PR just because some description text could be clearer.

Should the changes (outside of the descriptions) imply a versioning event for the SSVC schema? Major, minor or patch, if so?

Does modifying the descriptions imply a versioning event for the SSVC schema? And is it a big deal if we bump versions for stuff like that? The description changes would presumably be like a patch bump rather than a minor or major version I assume.

[sei-vsarvepalli]

@sei-vsarvepalli can you please update the description of this PR to say what it actually does (more than linking to the issue)?
I noticed a number of description fields in this changeset that are vague in what they're talking about ("decision point or decision point group", etc.) when in fact they are for one but not the other. I've only commented on the ones that showed up in the changeset, but without looking at the rest of the file I suspect that's an issue throughout the description fields. We should clean that up to avoid confusion.

All the "descriptions" are from the old JSON file nothing has changed in these. But I do see your point, we can update them too. They were left generic in the first edition, and remained the same so far.

Yeah, I figured they had been that way for a while. It's up to you, if it's an easy fix this time, I think it's worth doing. If it's not, then we can merge this PR as-is and leave a breadcrumb issue behind to pick up the descriptions at another time. I don't necessarily want us to block on this PR just because some description text could be clearer.

Should the changes (outside of the descriptions) imply a versioning event for the SSVC schema? Major, minor or patch, if so?

Does modifying the descriptions imply a versioning event for the SSVC schema? And is it a big deal if we bump versions for stuff like that? The description changes would presumably be like a patch bump rather than a minor or major version I assume.

I would say description change does not require a version bump in the schema (unlike when changing a Decision Point or a Decision Point Group instance).

The schema version change is really for a few events like

1. Add/remove elements or change schema in a non-breaking way (addition/patch)
2. Add/remove elements in a potentially-breaking way (revision/minor)
3. Modify the schema in a breaking way checks/formats/required etc. (model/major)

None of these is happening as I see it, so don't see a need to change version number. Adoption of CVE and CSAF program are the only external interactions we are managing, so I think version change is not needed IMO.

If I get time next week I can try to come up with better language for "description" of these fields. This patch mostly is to facilitate a startup adoption by CSAF and CVE programs.

Vijay

[sei-vsarvepalli]

This is all complete for review and merge, I added a better description for each JSON schema fields.

[ahouseholder]

These all look good. Thanks!