Consider making Replica and DependencyMap traits

[Verneri]

As some DBMSs (like SurrealDB) might be able to represent for example dependencies straight from the DB it might be a good idea to abstract the Replica and DependencyMap to traits so that they can be given other Solutions than the way Rust implementation handles calculating dependencies, it might be a good idea to make Traits for these.

In some cases this might help as Well with bindings to other languages/platforms

[djmitche]

There are some constraints on the data format that might make such usage difficult:

• tasks have dep_... properties to represent dependencies, as part of the public API for tasks
• changes to tasks are modeled as operations, adding or removing key/value pairs

That said, I'm interested in what you have in mind. A concrete case for storing task data in a different backend would help to clarify where these interfaces need to be changed.

Note also #372 is going to happen soon, and will modify the TC API pretty substantially.

[Verneri]

I'll try to Create an example, it might take some time though

[Verneri]

I will try to implement most of the system straight into SurrealDB to showcase the possibilities. Of course it might sometimes be better to use dedicated code for this but i think creating as much funtionality as possible, straight in the db shows the ideas better.

Also was thinking of trying out using BerkleyDB or rocksDB etc for the storage layer just to see if there is difference in the performance

[djmitche]

Sounds good!

[djmitche]

It might make more sense to create a struct that implements the Storage (and StorageTxn) traits, instead of a replica.

The set of methods provided in StorageTxn probably deserves some scrutiny: lots of very common things, such as determining whether a task is BLOCKED or BLOCKING, currently require scanning all pending tasks, which is pretty expensive.

Let's see if this bug branches in either of those directions: creating a new storage backend using SurrealDB; or modifying the StorageTxn methods to improve performance.

[djmitche]

There's some discussion in #472 of making it possible to express filter expressions in a way that the storage backend could, optionally, interpret for the caller. I think that covers part of what is proposed here, but dependency maps are not mentioned. I'll add a note about that, and close this issue in favor of that discussion.