Skip to content

Commit

Permalink
Bump tokenizers from 0.20.4 to 0.21.0 (tracel-ai#2574)
Browse files Browse the repository at this point in the history
Bumps [tokenizers](https://github.com/huggingface/tokenizers) from 0.20.4 to 0.21.0.
- [Release notes](https://github.com/huggingface/tokenizers/releases)
- [Changelog](https://github.com/huggingface/tokenizers/blob/main/RELEASE.md)
- [Commits](huggingface/tokenizers@v0.20.4...v0.21.0)

---
updated-dependencies:
- dependency-name: tokenizers
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
  • Loading branch information
dependabot[bot] authored Dec 2, 2024
1 parent e0f3814 commit c890af8
Show file tree
Hide file tree
Showing 3 changed files with 4 additions and 4 deletions.
4 changes: 2 additions & 2 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion examples/text-classification/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -34,7 +34,7 @@ burn = { path = "../../crates/burn", features = [
], default-features = false }

# Tokenizer
tokenizers = { version = "0.20.3", default-features = false, features = [
tokenizers = { version = "0.21.0", default-features = false, features = [
"onig",
"http",
] }
Expand Down
2 changes: 1 addition & 1 deletion examples/text-generation/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ f16 = []
burn = {path = "../../crates/burn", features=["train", "tch"]}

# Tokenizer
tokenizers = {version = "0.20.3", default-features = false, features = [
tokenizers = {version = "0.21.0", default-features = false, features = [
"onig",
"http",
]}
Expand Down

0 comments on commit c890af8

Please sign in to comment.