breaking changes for sync layer authors & app devs -- extra version information

[tantaman]

If you are the author of a sync layer, the latest version of crsqlite updates how versions are tracked.

The table schema for crsql_changes was:

CREATE TABLE crsql_changes(
  [table] TEXT NOT NULL,
  [pk] TEXT NOT NULL, [cid] TEXT NOT NULL,
  [val],
  [version] INTEGER NOT NULL,
  [site_id] BLOB
)

but is now:

CREATE TABLE crsql_changes(
  [table] TEXT NOT NULL,
  [pk] TEXT NOT NULL, [cid] TEXT NOT NULL,
  [val],
  [col_version] INTEGER NOT NULL,
  [db_version] INTEGER NOT NULL,
  [site_id] BLOB
)

changes should be pulled from the db by db_version:

SELECT * FROM crsql_changes WHERE db_version > ?

• The db_version is a lamport clock for the entire database, allowing peers to know when they're behind some other database.
• the col_version is a lamport clock for that specific column, allowing peers to merge without bumping the column clock to something neither peer has seen

More details here: #114

If you're an app dev

If you haven't shipped something to production, drop and recreate your tables.

Or, if you don't want to drop base tables:

For each crr:

1. drop the corresponding table__crsql_clock table
2. recreate the crr via: select crsql_as_crr('table')

If you have shipped something to production, you need to migrate all clock tables to:

1. rename version to db_version
2. add a col_version column. Set the col_version value to the db_version currently recorded for that row.

[tantaman]

These changes were published as:

@vlcn.io/crsqlite 0.6.1
@vlcn.io/wa-crsqlite 0.6.1
@vlcn.io/sync-client 0.5.1
@vlcn.io/sync-server 0.5.1
@vlcn.io/rx-tbl 0.5.1
@vlcn.io/crsqlite-allinone 0.5.1

[AlexErrant]

Just wondering - will there be a breaking change when support for RGAs (or other CRDTs) arrives? We're using lamport clocks, which as you note are good enough for LWW. Will there be a migration to vector clocks once more complex CRDTs are supported?

[tantaman]

They shouldn't create breaking changes for the LWW features (cc @ivertom).

Once we start introducing other CRDTs the plan is to be able to specify what behavior you want. So you could say "give me LWW w/ lamport clocks for this table" and "give me hybrid logical clocks for this one"

In terms of thinking relationships across tables which are different crdts... we haven't done much of that but I'm rather confident that LWWs won't have any more breaking changes.