Modify USJ spec

[kavitharaju]

Closes #225
Implementing the changes suggested in USJ schema by the USFM/X Committee

1. Change oneOf condition to anyOf
2. Instead of the field {type: "val1:val2"} keep two fields {type:"val1", marker: "val2"}

Changes in this PR

• Update the schema definition at schemas/usj.js
• Change the USJ generation scripts in python usfm-grammar module
• Change the List generation scripts the module, that depends on the USJ output
• Change the code in filtering of USJ based on include_markers and exclude_markers
• Change the code where USFM is generated back from USJ
• Change scripts in tests where the output USJ is worked on (to check if all markers are present, unwanted markers are filteres etc)
• Update the origin-usj.json samples in USFM-Grammar testsuite, (which is a copy of the tcdocs/tests)
• Tested and fixed issues found

[joelthe1]

The changes look good. Are you planing to add the change to Book Code regex separately (basically to mirror what's followed in USX)?

[kavitharaju]

Honestly I had forgotten that change. But now I remember what we had decided was

1. To keep checking strictly with enum in usfm-grammar at our grammar level(no change needed for that)
2. In the USJ schema keep as any 3 letter string using regex.
   I had sent a pr to tcdocs repo with the changes already in this pr and it is merged. So how about I make the json schema change as next task?

About point 2, is that okay? Or should we do it just like in usx by listing all codes as enum and then opening it up for any 3 letter string? I see the following reasons to not to

• the listing of enums would have no effect programmatically as the validation will solely be based on the regex
• our schema for USJ is a very high level one, as of now. Which has its beauty and simplicity. (Mark was planning on defining one with detailed rules of which node is valid where). In the current one, if we are to add such along list and treat one field with special validations, there might be others rules also we would need which are there in USX and USFM

[joelthe1]

Agreed. I will merge this in and we can work on the 'regex for book code' change as a separate PR. Also, let's leave out the special schema handling for now- enforcing any three letter regex is probably good enough (probably using what USX specifies already).