Test vector set creation

[codalogic]

I'm thinking of writing some scripts that scan through the *_spec.rb files and extract the test cases. At the moment I'm thinking each spec file could have a corresponding test vector file that has a format something like (based on evaluate_object_rules_spec.rb line 100ish):

# should pass an object with one string against an object rule with a string member or a string member
JCR: Pass
trule { "foo":string | "bar":string }
JSON: Pass
{ "foo":"thing" }

# should fail an object with one string against an object rule with a string member and an integer member
JCR: Pass
trule { "foo":string, "bar":integer }
JSON: Fail
{ "foo":"thing" }

I figure it may involve a preliminary automated pass and then a lot of manual cleanup! It's also need a Ruby script to run the tests and check the test vectors yield the pass/fail results your reference code says they should.

This should help third-parties creating implementations, including me.

[anewton1998]

That's a good idea.
Unfortunately, I think there is a lot of overlap in the testing.

[codalogic]

There may end up being even more overlap in the resultant test vector set because they can't get into the level of detail that the Ruby tests can. In fact, we could use that as a reason to justify any dupes we have! Dupes don't bother me as long as they can be run in a reasonable amount of time.

[codalogic]

I've completed my first pass at this. The results are at: https://github.com/codalogic/jcrvalidator/tree/test-vectors/test%20vectors . Time to write some C++!

[codalogic]

The tests created previously will need updating.