SurealDb performance/benchmark transparency

[Snazzie]

Many have been asking for benchmarks since surrealDbs announcement, but have always been met by unwillingness to provide benchmark data. Quick search on reddit shows a few users complain about abysmal performance, worse than Sqlite. Seeing SurrealDb claiming its trusted by teams at big name companies raises my eyebrows.

[nickchomey]

It's a newish tool with a very broad and ambitious goal, so maybe cut them some slack? They only have so many hands.

In a recent live Q&A on discord, Tobie confirmed to me that v1 isn't really meant to be "production-ready", but just stable. V2, which will hopefully be released in the next few months, will come with performance improvements, benchmarks and benchmarking tools.

Have some patience!

[solmanter]

Hi surreal team. I am really happy to new announcement and features. does new features includes benchmarking tools?

[hazyone]

I've heard the same statements one year ago :)

[AlexFrid]

I'm well aware @hazyone, it's been a journey to get here to the stability and performance improvements we've done now with 2.0 😅

[carueda]

I've been excited since the announcement of v2, but just now finding some time to start running some exercises I've put in place in the past ... however, when just about to get ready for that, gave a look at https://surrealdb.com/docs/surrealdb/faqs, and what a disappointment that performance is not even mentioned there anymore :/

[PaulDotSH]

We're currently working on it and will be releasing our benchmarking results and code to replicate soon. We've been waiting on a range of stability and performance improvements we've done now with 2.0, so we will be ready with the benchmarks soon.

This was the case for over a year now, is there any ETA?

[AlexFrid]

@PaulDotSH, the ETA for the first batch of performance benchmarks is towards the end of this month.

[TheOrdinaryWow]

And as for now 2.0 has released, and I am so into the features it provided. Yet still, was stopping me is that I am concerned about the performance. I cannot even find a single piece of data on the Internet.

[AlexFrid]

Totally understand, we are releasing performance benchmarks this month 🙌

[dougg0k]

Were those benchmarks released? If yes, where?

Any comparison between other databases like sqlite, postgres, etc?? Those would be useful to have a general perspective.

[nickchomey]

Nope. Were supposed to come in November. No acknowledgement of why they weren't or update of when they might come.

[mahbubabbas]

@tobiemh a very important observation from side: I have developed one application using Surreal 1.x last year where all tables are schemaless and another one just couple of weeks back (Sdb 1.5.5) where all tables are schemafull. I can the see the speed of former is excellent where as the later one is much slower. I am using golang sdk.

Note: I am not able to use the latest SDB (2.0.4) as the Go sdk 0.3.0 is giving lots of issues which I have already reported but still waiting for fix.

[nickchomey]

When you say the performance of schemaless is excellent, what sorts of queries are you doing? And how do their performance compare to other dbs?

So far I've only found competitive performance with surreal when doing exceptionally simple "point" queries - retrieve specific record ID, as if it's a kv store. Relational is a disaster, and traversing a graph seems to be very slow as well.

It could be that I'm not doing it right, but documentation doesn't seem to go into ANY advanced details or best practices (eg how to use indexes in graph queries, or how to structure graph queries, or use query planner/explainer, etc) It's all just very basic and contrived.