Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: Fix colbert model shape mismatch #413

Merged
merged 2 commits into from
Nov 28, 2024

Conversation

hh-space-invader
Copy link
Contributor

@hh-space-invader hh-space-invader commented Nov 25, 2024

We don't truncate the input when adding the document marker in late interaction models. Sometimes, the input exceeded the maximum length the onnx model can handle, the tokenizer by default truncates with the maximum configuration found inside the tokenizer_config. But we didn't make sure that we truncate after adding the marker token. Example, input length 600, max input size 512. The tokenizer will truncate at 512. We add the marker token (513) and onnx model will raise. Thus, I've updated the truncate with maximum_length -1 to not overflow the input to onnx model.

ref: #407
ref: #410

All Submissions:

  • Have you followed the guidelines in our Contributing document?
  • Have you checked to ensure there aren't other open Pull Requests for the same update/change?

New Feature Submissions:

  • Does your submission pass the existing tests?
  • Have you added tests for your feature?
  • Have you installed pre-commit with pip3 install pre-commit and set up hooks with pre-commit install?

New models submission:

  • Have you added an explanation of why it's important to include this model?
  • Have you added tests for the new model? Were canonical values for tests computed via the original model?
  • Have you added the code snippet for how canonical values were computed?
  • Have you successfully ran tests with your changes locally?

@hh-space-invader hh-space-invader merged commit da30f93 into main Nov 28, 2024
14 checks passed
@hh-space-invader hh-space-invader deleted the TSK-410-fix-shape-mismatch-colbert branch November 28, 2024 11:59
@I8dNLo I8dNLo mentioned this pull request Dec 24, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants