Where in the codebase are the outputs of TctColBertDocumentEncoder/TctColBertQueryEncoder combined to create the MaxSim score?

[plamb-viso]

TctColBertDocumentEncoder/TctColBertQueryEncoder make it super easy to setup document/query encoding with castorini/tct_colbert-v2-hnp-msmarco but I cannot find where the outputs from the encode functions in these classes are combined to create a MaxSim score per document.

[jacklin64]

Hi @plamb-viso, thanks for the question. Here we'd like to clarify that TCTColBERT uses dot product operation to compute scores like standard single-vector DPR instead of ColBERT using MaxSim. The main difference between TCTColBERT and DPR is training, where TCTColBERT learns knowledge from ColBERT maxim scores through all the in-batch samples but after training. See more detail in our paper https://aclanthology.org/2021.repl4nlp-1.17/

[plamb-viso]

Ah so the final output between TctColBertDocumentEncoder/TctColBertQueryEncoder would be a dot product? Curious if there is an example, say, in a test case, where this is computed for the outputs of these two classes encode method in the codebase. Basically what I'm interested in is an example of encoding a couple documents, encoding a query and computing the final score per each document.

[jacklin64]

I don't remember we have script to directly compute inner product between TCT-ColBERT encoded query and documents. But you can use the script here to compute the embedding for documents

pyserini/scripts/tct_colbert/encode_corpus_msmarco_doc.py

Line 107 in db1347c

embeddings = model.encode(text_batch)

. And also creating a TctColBertQueryEncoder class and encode query embedding.

[plamb-viso]

Ah I should've stated what i was trying to do -- I'm using ColBERT for reranking, so it gets a set of documents and a query on the fly; those documents are dynamic so you cant index/precompute them beforehand. That's why I'm interested in an example of computing the inner product (it looks like perhaps here FAISS will be computing it over previously indexed documents).

[plamb-viso]

Would you suggest just using a numpy.inner() or numpy.dot() ?

[plamb-viso]

hi @jacklin64 just wanted to try pinging you one more time on the above