SurrealDB
Offline Capabilities? #107
Replies: 14 comments 15 replies
-
|
Hi @AgarwalPragy , absolutely - this is a definite intention for SurrealDB. And that's the whole reason why SurrealDB is designed to run embedded in the browser, as well as in a server environment... We're not fully set on the implementation details of this at the moment. We've looked at Operational Transformation, and CRDTs, and we would ideally like to find a middle ground between them (in terms of offline support / speed / being able to operate as a full database), which is an area of research which is seeing more interest at the moment! |
Beta Was this translation helpful? Give feedback.
-
|
Offline-first and multi-user data online sync would make me dump my [p|c]ouchDB stack and jump on surrealDB immediately. |
Beta Was this translation helpful? Give feedback.
-
|
agree! |
Beta Was this translation helpful? Give feedback.
-
|
Ditto! |
Beta Was this translation helpful? Give feedback.
-
|
💯 |
Beta Was this translation helpful? Give feedback.
-
|
With tight integration with Cloudflare, the database on the edge will/can act as distributed endpoint for lower latency |
Beta Was this translation helpful? Give feedback.
-
|
Any update @tobiemh ? |
Beta Was this translation helpful? Give feedback.
-
|
any update please |
Beta Was this translation helpful? Give feedback.
-
|
Would be create to use crdt because tikv uses raft and a network partition or network lag means your view of the data is potentially old. It’s such an age old problem :) crdt is best of both worlds in that it embraces the reality that the data you have might be old. It’s a global solution for the reality that the server is in a way offline or shall we say “ lagging “. Along with this same problem domain is schema evolution. So you may have 100 versions of data type x , but the shape of those 100 versions of X could be different too. I heard that the kv store needs to support range queries ? |
Beta Was this translation helpful? Give feedback.
-
|
I was looking something like that. |
Beta Was this translation helpful? Give feedback.
-
|
Well the tikv requires an always on system. the nats kv can work when offline. That’s why I suggested ( I think ) to use NATS in leaf mode . Marmot uses NATS to do the same thing for SQLite . https://github.com/maxpert/marmot I am considering doing the same with Surreal db but can’t find an API to write to . The docs don’t explain it . I heard that range queries are needed which sounds very s3 ish . |
Beta Was this translation helpful? Give feedback.
-
|
@tobiemh is this on back-burner? |
Beta Was this translation helpful? Give feedback.
-
|
I arrived here after discovering SurrealDB. I am also looking for an "offline-first" data sync solution that syncs between server and local. SurrealDB looks amazing but I can't find anything on synchronisation between embedded and server. |
Beta Was this translation helpful? Give feedback.
-
|
https://github.com/electric-sql/electric much better for postgres -> sqlite |
Beta Was this translation helpful? Give feedback.
-
|
I went with RealmDB in the end. I need a C++ SDK. Theirs has reached preview stage, almost production ready. |
Beta Was this translation helpful? Give feedback.
-
|
There's also https://github.com/vlcn-io/cr-sqlite which is a SQLite extension and is available anywhere SQLite is. The API is available over SQL so it'll work with any language that has SQLite bindings. I'm working on a revised iteration that allows using any DB on the backend you'd like (e.g., postgres, mysql, surreal). That is many months out, however. |
Beta Was this translation helpful? Give feedback.
-
|
I need multi tenant solution I had to develop a layer on top of sqlite for that but is not native. Can subrrealdb be the solution? |
Beta Was this translation helpful? Give feedback.
-
|
Any updates on this? @tobiemh |
Beta Was this translation helpful? Give feedback.
-
|
I did ask @tobiemh in QnA, this is not on near term horizon. I can understand, its very complicated problem. |
Beta Was this translation helpful? Give feedback.
-
|
Implementing fallback (offline) support would be a valuable feature for SurrealDB. For example, if the main db responds too slowly when writing log entries, sensor data, or any kind of data that doesn't have to be synced in realtime then embedded / local database could store these entries and once the main database is back online or functioning properly, the local db could be synced with the main one. This would enhance reliability and ensure continuous operation. |
Beta Was this translation helpful? Give feedback.
-
|
As far as I can tell it can be quite easily done with live queries.. As I have thought of it, while there is no such replication native support, one can share schema for server and db and simply subscribe to live queries for all tables that put all the data to local replica respecting user's permissions. However the tricky part would be diffs syncs... For this obviously Change Feeds will be suitable, but again - then you need to import this manually and idk how permissions are handled with CF. In any case looks doable, but with some manual work |
Beta Was this translation helpful? Give feedback.
-
|
Offline capability along with a synch mechanism would be a gamechanger. I tried to achieve this using tikv by running a pd/tikv cluster on multiple nodes, each containing a surreal db instance as well. One of them was "simulating" an onprem machine that at some time could go offline, local db updates can be done and after it gets connected again it snychs with the rest of the cluster. Turns out tikv doesnt work that way and whenever my pd instance got disconnected from the rest of the cluster it failed, and with it the surreal db instance. Bummer. :-/ But would have been to easy I guess. Any idea of how to achieve this (without being to hacky) using tikv? |
Beta Was this translation helpful? Give feedback.
-
|
You should try nats Jetstream . it’s very much design for online / offline synchronicity use cases by maintaining order . Surreal db though does not currently support nats kv . I suggested it but heard nothing . I do the same scenarios as you with SQLite and nats Jetstream and it works for online and offline sync. |
Beta Was this translation helpful? Give feedback.
-
|
We just saw the release of 2.0 are there anything on the horizon for offline support? Specifically for my usecase it would be
|
Beta Was this translation helpful? Give feedback.
-
|
I will take the liberty and add you @tobiemh hope to hear back from the team |
Beta Was this translation helpful? Give feedback.
-
|
Dont do that... |
Beta Was this translation helpful? Give feedback.
-
|
Why not it's a real concern and its been over 2 years since this was mentioned as something they wanted to do, it's an easy thing to answer it can be one of three responses:
Can't see why that is such a big problem |
Beta Was this translation helpful? Give feedback.
-
|
Sorry I was saying dont tag people - especially the ceo. Have patience, they're busy with lots of more important issues right now. |
Beta Was this translation helpful? Give feedback.
-
|
But i was just tagging the person who commented on it back then, I respect peoples time and I dont expect an answer but I think its only fair to tag the person who was the respondand back then |
Beta Was this translation helpful? Give feedback.
-
Offline first apps deliver a better experience when the user goes offline. They also offer a snappier experience in general thanks to the client-side cache.
Are there any plans for supporting offline-first capabilities to surreal db, especially the features highlighted in these essays
Beta Was this translation helpful? Give feedback.
All reactions