Programs aux/config schema specification

[frankiebee]

It would be good to have a schema spec for aux/config data on deployment so when we pull program info from chain we have a rough idea of what we are getting. JSON schema seems like the best option but the it be comes a question of which version and is their also a compatible rust crate

@entropyxyz/core-developers we can use this issue to track the progress on it

• one we are looking at is: https://ajv.js.org
• possible rust crate: https://docs.rs/jsonschema/latest/jsonschema

[mixmix]

Hey @ameba23 I'm gonna be leading out on specifying the Schema for Aux/Config.

One thing I'm pretty keen on is validation happening at the lowest level possible. I am keen on aiming for

1. no dead code on chain
   • no wasted funds deploying things which are not valid schema
   • no wasted space storing stuff which can never be used
2. no leaky APIs
   • if there is a Schema in place, it cannot be side-stepped for signing
     • JS sdk/cli
     • Rust lib/ CLI

We don't have to put guard rails in Rust immediately, but I have the sense it would be good to choose a Schema standard which will work for JS + Rust.

A) Why do you think?
B) can you give this crate a sniff ? https://docs.rs/jsonschema/latest/jsonschema/
- does it look "healthy"?
- would it be easy to use (documented, nice API)?

[ameba23]

Hey,

i can give a fully thought out response later, but just for info, the device key program's aux data uses https://docs.rs/schemars/latest/schemars/ - a different JSON schema library. It looks to me like jsonschema is more focussed on validating json, whereas schemars is for generating schemas for rust structs (so more useful to the program developer, not the program user).

[ameba23]

@mixmix Where on the rust side would this be used?

I can see three possible places:

• In the programs or pallet, which is part of the chain runtime. That means we would need a no-std library which jsonschema is not unfortunately.
• In entropy-tss, to validate aux data against the schema before running a program. This would be possible, but i'm not sure we gain anything by checking at that point. If we don't check, the program will fail with an error when it attempts to deserialize the json, and attempting to run the program is probably no more expensive in terms of computation than validating using the schema beforehand.
• in entropy-client (the client code used by our tests and the test-cli). We could for sure do this, but it is supposed to be quite a low-level library which can work on different platforms, so i'd be wary of adding anything with a lot of dependencies.

I think it makes most sense for the validation to happen in full-featured client libraries - such as the SDK.

Also, personally i think using JSON for program aux and configuration should be a recommended guideline but not a requirement for program developers. Json is inefficient both in terms of storage space and computation needed to deserialize when compared to binary serialization, which could be a deal breaker for program developers who want to have large configurations or auxiliary data. But thats just my opinion and i wouldn't wanna block this.

[JesseAbram]

Also, personally i think using JSON for program aux and configuration should be a recommended guideline but not a requirement for program developers. Json is inefficient both in terms of storage space and computation needed to deserialize when compared to binary serialization, which could be a deal breaker for program developers who want to have large configurations or auxiliary data. But thats just my opinion and i wouldn't wanna block this.

I agree it also goes to the point of neutrality on the platform, as in, this is what you can do, to work with the sdk, out our tools, but anyone can create or write anything or anytools they want, think of an organization using entropy with their own preferred standards

[mixmix]

Good questions raised. At the heart of this is the question "What are Schema for?"

Are they:

• A) a guideline for people inputting data?
• B) a guard against invalid inputs

I have had in mind the user stories:

As a general entropy user I don't want to write my own program, I want to pick one off the shelf and add some config to tune it to my needs.

As program developer, I want to write a program which can be configured, and I want to define what is a valid configuration.

Example: petty-cash program

Entropy wants to make it possible for team leads to draw on petty-cash for small expenses.

Peg writes a program which will sign "any Eth transaction for <= $X"
Peg deploys the program with

• bytecode
• configurationSchema which says X MUST be defined in config, and MUST be a number less than 1

Tux installs this program with a config which says X = 0.1. (She also installs the device-key-proxy program and say "Frankie, Jesse, Vi can all request signatures")

Questions

1. Is config for programs expected to be used to help define valid inputs for signing?
2. What would happen if Tux installed the program with "X = 100"?
   • at install-time => nothing?
   • at sign-time => ???
3. What would happen if Tux installed the program without a config?
   • at install time => nothing?
   • at sign time => ???
4. Whose responsibility is it to honour the Schema?
   • are they a guide or a guard
   • if it's not at a low level then it feels more like a "guide" to me, and users are open to abuse/ hurt?
5. If we want un-opinionated schema, how do we track the format and know how to use them?
   • depends on the answer to (4)

I think the least worst case is "signing fails, you wasted money badly installing a program".
My experience is saying "if there is a schema then that means I expect it as a guard, and if it's not then it should probably not be on chain"

I am wearing 2 hats here:

• 👒 UX : I don't want Tux to install something and later discover the install was bad (you just wasted time + money)
• 🎩 Security : are we offering a guard rail or not?

[ameba23]

Are they:

* A) a _guideline_ for people inputting data?

* B) a **guard** against invalid inputs

Ideally they are both. I think in the best case we can come up with a way of specifying schemas for whichever serialization format people choose. Eg: a json object with one property describing which serialisation format is used (JSON, Protobuf, RLP, SCALE, BSON, whatever), and another property with the actual schema (Json schema, protobuf schema, etc).

But if that turns out to be too difficult im not against sticking with JSON for the sake of having standardisation and keeping things simple.

[JesseAbram]

no they should not be guard because then we have to decide what is and isn't allowed and we become opinionated.

They should act almost like an ABI for smart contracts on ethereum, it is how I talk to the program as a user.

Should they be on chain..........well this is a good point to be revisited, no probably not, they were originally added as an ask from @frankiebee as we don't have any public structure to place these items, but if they were to look like an ABI then ya maybe third parties should be responsible for this

[ameba23]

Re: other public place to put them, we do have the program-metadata-http-service and there is an open PR to read configuration schema and aux data schema out of the Cargo.toml file of the project source code - as it is needed for getting the hash that the program will have on-chain.

[ameba23]

at sign-time => ???

At sign time - signing will fail with an error. Which isn't so bad - requesting a signature doesn't cost anything.

[mixmix]

It does cost users I reckon @ameba23 - confusion over why signing is failing. If the system "doesn't work" people will say "it's broken"

[ameba23]

It does cost users I reckon @ameba23 - confusion over why signing is failing. If the system "doesn't work" people will say "it's broken"

I totally agree that we need guards against these things - but its a bit expensive / impractical to have those guards in the chain runtime.

i think it should be up to client software, like tooling for program developers and the SDK for users, to provide guards against giving bad inputs when deploying or registering with a program. If people choose to use the chain API directly with no client software then yes they can make mistakes and will get confusing errors.