Fast retreival of 8k vectors of dim 1024

[ExtReMLapin]

Hello,
We recently moved away from a H5 file to arcadedb to leverage all the DB safety features (ACID etc).

We now stores our embeddings (linked to the node it does the embedding with an edge) in Arcade.

Issue is retreival at boot of out software is quite slow.

---

Cypher query

match (vector:EMBEDDING)-[:embb]->(targetNode)
        return ID(targetNode) as rid, vector as vector

Takes 21s for 7692 entries

---

SQL query

MATCH {type: EMBEDDING, as: embb}-->{ as: target}
RETURN embb.vector, target.asRID()

Takes 21s for 7692 entries

profiling of the cypher query returns that :

Traversal Metrics
Step                                                               Count  Traversers       Time (ms)    % Dur
=============================================================================================================
ArcadeFilterByTypeStep(vertex,EMBEDDING)                            7692        7692           2,799     1,00
HasStep@[vector]                                                    7692        7692           0,994     0,35
VertexStep(OUT,[embb],vertex)@[targetNode]                          7692        7692          10,008     3,57
SelectStep(last,[targetNode, vector])                               7692        7692           2,538     0,90
NoOpBarrierStep(2500)                                               7692        7692           4,808     1,71
ProjectStep([rid, vector],[[SelectOneStep(last,...                  7692        7692         259,488    92,46
  SelectOneStep(last,targetNode,null)                               7692        7692           1,589
  IdStep                                                            7692        7692           1,269
  SelectOneStep(last,vector,null)                                   7692        7692           1,380
  ChooseStep([PropertiesStep([vector],value), H...                  7692        7692         234,318
    PropertiesStep([vector],value)                                  7692        7692         116,185
    HasNextStep                                                     7692        7692           1,100
    PropertiesStep([vector],value)                                  7692        7692         106,749
    EndStep                                                         7692        7692           1,273
    ConstantStep(null)                                                                         0,000
    EndStep                                                                                    0,000
                                            >TOTAL                     -           -         280,637        -
                                            

Returning the vector RID instead of the vector itself obviously boosts the query a lot, (takes less than 1sec) which leads me to believe it's returning the 8k*1024 dim that slows the whole query down.

vector Property was created as an ARRAY_OF_FLOATS

[lvca]

Maybe the serializer of arrays is not efficient and the cost is just in serializing the result back.

Alsomif.you store large records, it could be helpful changing the page size from 65k to 2x or 4x.

Could you please provide a test case or even a database with similar data to spin some tests locally?

[ExtReMLapin]

Now there is something else confusing, when just priting the RIDs, it's much faster with Cypher

match (vector:EMBEDDING)-->(targetNode)
        return ID(targetNode) as rid, ID(vector) as vector

Returned 6717 records in 53ms

---

MATCH {type: EMBEDDING, as: embb}-->{ as: target}
RETURN embb.asRID(), target.asRID()

Returned 6717 records in 3446ms

But I guess it should have it's own issue, I don't know

[lvca]

What's the query I can run on the larger database (fat_orano)?

[lvca]

Ok, I run MATCH {type: EMBEDDING, as: embb}-->{ as: target} RETURN embb.vector, target.asRID() and I see it takes a lot of time on serializing large arrays into JSON. I don't think it's a performance problem of the engine, it's just JSON/HTTP stuff.

In your use case, are you using a driver or a Java app?

[ExtReMLapin]

We are only using the rest API.

[ExtReMLapin]

Also, we're storing in ArcadeDB and fetching only to use the ACID features, we retreive all of it to do GPU cosine sim, clustering etc.
From my understanding using vector indexes right now from rest is a little complicated.

Also, unless I misunderstood your message, json serialization should not be that slow.
In python (no jit compiler like java), getting the data, converting to python and back to json string takes less than two seconds

Getting vectors for DB ORANO_DOC took 15.355653762817383s
Converting to numpy took 0.11538863182067871s
Converting back to json took 1.7384111881256104s