Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Missing model field in the huggingface_endpoint.py #29521

Open
ENCODERS09 opened this issue Jan 31, 2025 · 2 comments
Open

Missing model field in the huggingface_endpoint.py #29521

ENCODERS09 opened this issue Jan 31, 2025 · 2 comments
Labels
🤖:bug Related to a bug, vulnerability, unexpected error with an existing feature

Comments

@ENCODERS09
Copy link

Whatever model is being sent as a parameter it will never get called, only the default model will be called in the huggingface_endpoint.py because in the post call of the client and async_client the model is not being consumed which is why the default model is being called not the one passed

json={"inputs": texts, **_model_kwargs}, task=self.task

json={"inputs": texts, **_model_kwargs}, task=self.task

@dosubot dosubot bot added the 🤖:bug Related to a bug, vulnerability, unexpected error with an existing feature label Jan 31, 2025
@keenborder786
Copy link
Contributor

Nope the models is being passed during the initialization of InferenceClient

@ENCODERS09
Copy link
Author

ENCODERS09 commented Feb 5, 2025

@keenborder786

https://github.com/huggingface/huggingface_hub/blob/f2e1889c66607528be17253f984fa2cd687d9b30/src/huggingface_hub/inference/_client.py#L266

If you see here, model is passed but the issue lies in the fact that, assigning the self.model attribute in the validate_environment method, but there is no ensuring that this assignment is persisted when we call the aembed_documents method.
The problem is In the validate_environment method, we are assigning the self.model attribute based on the self.repo_id attribute.
However, when we call the aembed_documents method, we are not re-validating the environment, so the self.model attribute might not be up-to-date.

So either we do this in aembed_documents() function

await self.validate_environment()
responses = await self.async_client.post(
    json={"inputs": texts, "parameters": _model_kwargs}, task=self.task
)

or we pass model=self.model

responses = await self.async_client.post(
    json={"inputs": texts, "parameters": _model_kwargs}, task=self.task, model=self.model
)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
🤖:bug Related to a bug, vulnerability, unexpected error with an existing feature
Projects
None yet
Development

No branches or pull requests

2 participants