improve task and pin models by supporting resources with units and making a lot optional

[trusch]

This tries to improve our model layer, especially for tasks and pins as those two are really end-user facing.

• I made all optional things optional, so no more env: [] to make it parse
• The whole metadata field is optional
• resource units can be raw numbers or strings with units
  • cpus: 1000mcpu works the same as cpus: 1000, but also get stuff like time: 3h
  • time parsing can be improved by using humantime crate, right now its a rather simple implementation
• Pins have CID and fallback_url optional, but parsing has now a validation step where it checks if any of this two is set
• I added some tests to illustrate and test the changes

[trusch]

Note: This is actually a good place to add more user data validation, so we catch it early and reject it. For example we could check if the image is a properly formatted URI or other stuff like the check if the CID or the fallback url ist set etc.

[tuommaki]

Initially seeing the name of this type I thought "why not use Either?" (that being familiar from Haskell).

Then looking at this implementation I'm thinking that maybe the naming should be a bit more specific like ComputeUnit perhaps? Also: Should this maybe implement FromStr ?

[trusch]

I renamed it and also added FromStr, so now "1024kB".parse::<ComputeUnit>() works :)

[trusch]

There is now ByteUnit, CoreUnit and TimeUnit

[tuommaki]

Especially with time (but possibly with storage space as well), I'd maybe test also combinations like "1h 47min 3 sec". Would "3kb 3M" parse ? What about "7min 8gpus" 🙈

[trusch]

"7min 8gpus" is probably something for the future but for time we can use humantime crate which does exactly what you describe

[tuommaki]

Would it then make sense to split different types (CPUs/GPUs, MEM, Time) and possibly have an enum including them, with some brute force parsing logic trying each one (or w/ regex match based heuristics)?

[trusch]

When removing the time cases (which I did), the rest of the units isnt that much, so I think its fine like this. We can always extend it, but I feel it does its job well.

[trusch]

@tuommaki Would you have another look at this when you find time?

[tuommaki]

I guess this is fine now. There are some semantic issues (such as mgpu) still, I think, but maybe we can ignore them for a bit as long as we don't document & advertise use of them.