fix: Fix colbert model shape mismatch #413
Merged
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
We don't truncate the input when adding the document marker in late interaction models. Sometimes, the input exceeded the maximum length the onnx model can handle, the tokenizer by default truncates with the maximum configuration found inside the tokenizer_config. But we didn't make sure that we truncate after adding the marker token. Example, input length 600, max input size 512. The tokenizer will truncate at 512. We add the marker token (513) and onnx model will raise. Thus, I've updated the truncate with maximum_length -1 to not overflow the input to onnx model.
ref: #407
ref: #410
All Submissions:
New Feature Submissions:
pre-commit
withpip3 install pre-commit
and set up hooks withpre-commit install
?New models submission: