Dependency bug from chroma 0.5.23 at SentenceTransformerEmbeddingFunction

[kirankotari]

Transformer library updated to use "tokenizers>=0.21,<0.22" (released 4 days ago)
Where in chroma we are still point to 'tokenizers >= 0.13.2, <= 0.20.3',

Which creating dependency conflict, can you please update tokenizers to latest package.

Here is the I am trying to run

sentence_transformer_ef = embedding_functions.SentenceTransformerEmbeddingFunction(
    model_name="all-MiniLM-L6-v2"
)

Links:
https://github.com/huggingface/transformers/blob/34f4080ff59b1668d919a1ba9f8bc4a3a2a3f478/setup.py#L183C5-L183C30
https://github.com/chroma-core/chroma/blob/1110d07f17c75e44c372361fd2684f5b08edec0d/pyproject.toml#L31C3-L31C37

I tried to use lower version, which started creating more issues, related to torch 2.5.1 compatible not found.

Platform:
MacOS
Python 3.12.4

[kokhoor]

is it safe to use tokenizers version 0.21.0 with chromadb?