Allow for easy versioning of structures

[Ekleog]

I was thinking something like http://www.boost.org/doc/libs/1_36_0/libs/serialization/doc/tutorial.html#versioning could be expressed elegantly with attributes like (unchecked syntax):

#[derive(Serialize, Deserialize)]
#[serde(version=5, removed_field(version=4, name="old_field", type=String))]
struct Example {
    #[serde(added(version=5, default_value=None))]
    new_field: Option<String>,

    #[serde(added(version=4, default="build_example"))]
    field: String,
}

fn build_example(old_field: &String, new_field: &Option<String>) -> String {
    "OLD ".to_owned() + old_field
}

Here, there has been a field added in version 5, which defaults to None when deserializing a file generated with a previous version, and in version 4 the field old_field has been renamed to field and it's required to add "OLD " in front of every value encoded with pre-4 versions.

This would avoid the need to write custom serializer/deserializers when having to handle multiple versions of a format, and having something more transparent to the user than #745

Then this whole idea requires quite a bit of engineering, I guess, and of thought about the way to pass arguments to build_example.

What do you think about it?

[Ekleog]

Another caveat with it actually: I don't know what the versioning policy for serde is, but implementing it requires that serde adds a version number to all the serialized data (else if someone forgot the version=0 on the struct they wouldn't be able to bump the version for the next iteration of the struct), so this may require a major version bump? Or just not have version fields by default and forbid upgrading from a non-versionned struct.

[avl]

This would be immensely useful for anyone writing large amounts of data to disk in an evolving program where data structures change but old saved data still needs to be usable.

[dtolnay]

I agree this would be valuable! Lots of users want versioning of structures and the ability to evolve over time while continuing to support old data.

I am closing the issue because I believe this would be better explored as a library separate from Serde, possibly built on top of Serde. Once there is a standalone prototype and people generally agree that it meets their needs, we can reconsider whether an modifications in Serde are required to integrate better with the functionality.

[avl]

I've written something: https://github.com/avl/savefile

[ssokolow]

@avl For the record, this means that savefile will never be useful to me for any of my projects:

Features savefile does not have, and will not have:

• Support for external protocols/data formats. There'll never be json, yaml, xml or any other backends. Savefile uses the savefile format, period.

...for several reasons:

1. I don't trust my data to binary serialization formats... with one exception. (SQLite, because it's open-source, has a test suite which is literally avionics grade, and has an unmatched track record.)
2. I don't trust my data to serialization formats with only one implementation and no published standard... with one exception. (SQLite, because it's included in the standard library of every "batteries included" language and has the aforementioned test suite, track record, and opened source.)
3. In the name of data portability, I choose formats which don't single-handedly impose an additional "tools for building compiled extensions" requirement on scripting languages.)
4. My main use for Serde in the near term would be migrating to Rust with existing JSON-serialized data where I currently implement Allow integer tags for internally tagged enums #745 by using schemaless parsing and then matching the version field against a map/hash/dict/etc. of deserialization functions.

[wickedchicken]

I'm not sure if there is a "blessed" library yet, but I did run across https://github.com/fredpointzero/serde-version which at least attempts to address this issue.

[mainrs]

@wickedchicken Not sure if I totally understood the library as I find it, based on the examples, not that useful.

Would something like this even work?

# Version one
{
  "text": "some text"
}

# Version two
{
  "text": {
    "somekey": "other value"
  }
}

[saona-raimundo]

Just adding to the discussion.
There is this other crate: versionize

[ssokolow]

Probably a good idea to mention this about versionize right in the thread:

It does not aim to be a generic serialization framework and only the bincode backend is supported.