Skip to content

Update Inference specification for Hugging Face's completion and chat completion tasks #4383

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged

Conversation

Jan-Kazlouski-elastic
Copy link
Contributor

This PR is for changes to specification caused by elastic/elasticsearch#127254:

Extended Task Support:

  • Added completion and chat_completion tasks to the list of supported Hugging Face tasks.

Model Requirements for Chat Tasks:

  • Updated documentation to describe specific requirements for using chat_completion and completion tasks, including model compatibility with the OpenAI API format and usage guidelines for serverless vs. dedicated endpoints.

New Configuration Parameters:

  • Introduced optional model_id field in Hugging Face service settings, applicable to completion and chat_completion tasks.

Rate Limit Clarifications:

  • Updated rate_limit documentation to clarify default behavior and guidance for tuning based on deployment specifics.

Documentation Fixes:

  • Corrected typos in existing text_embedding request examples.

Additional actions

  • Signed the CLA

  • Executed make contrib

Copy link
Contributor

Following you can find the validation results for the APIs you have changed.

API Status Request Response
inference.chat_completion_unified Missing test Missing test
inference.completion Missing test Missing test
inference.delete Missing test Missing test
inference.get 🟢 1/1 1/1
inference.inference Missing test Missing test
inference.put_alibabacloud Missing test Missing test
inference.put_amazonbedrock Missing test Missing test
inference.put_anthropic Missing test Missing test
inference.put_azureaistudio Missing test Missing test
inference.put_azureopenai Missing test Missing test
inference.put_cohere Missing test Missing test
inference.put_elasticsearch Missing test Missing test
inference.put_elser Missing test Missing test
inference.put_googleaistudio Missing test Missing test
inference.put_googlevertexai Missing test Missing test
inference.put_hugging_face Missing test Missing test
inference.put_jinaai Missing test Missing test
inference.put_mistral Missing test Missing test
inference.put_openai Missing test Missing test
inference.put_voyageai Missing test Missing test
inference.put_watsonx Missing test Missing test
inference.put Missing test Missing test
inference.rerank Missing test Missing test
inference.sparse_embedding Missing test Missing test
inference.stream_completion Missing test Missing test
inference.text_embedding Missing test Missing test
inference.update Missing test Missing test

You can validate these APIs yourself by using the make validate target.

Copy link
Contributor

Following you can find the validation results for the APIs you have changed.

API Status Request Response
inference.chat_completion_unified Missing test Missing test
inference.completion Missing test Missing test
inference.delete Missing test Missing test
inference.get 🟢 1/1 1/1
inference.inference Missing test Missing test
inference.put_alibabacloud Missing test Missing test
inference.put_amazonbedrock Missing test Missing test
inference.put_anthropic Missing test Missing test
inference.put_azureaistudio Missing test Missing test
inference.put_azureopenai Missing test Missing test
inference.put_cohere Missing test Missing test
inference.put_elasticsearch Missing test Missing test
inference.put_elser Missing test Missing test
inference.put_googleaistudio Missing test Missing test
inference.put_googlevertexai Missing test Missing test
inference.put_hugging_face Missing test Missing test
inference.put_jinaai Missing test Missing test
inference.put_mistral Missing test Missing test
inference.put_openai Missing test Missing test
inference.put_voyageai Missing test Missing test
inference.put_watsonx Missing test Missing test
inference.put Missing test Missing test
inference.rerank Missing test Missing test
inference.sparse_embedding Missing test Missing test
inference.stream_completion Missing test Missing test
inference.text_embedding Missing test Missing test
inference.update Missing test Missing test

You can validate these APIs yourself by using the make validate target.

Copy link
Contributor

@l-trotta l-trotta left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

spec wise LGTM, let's hear from @szabosteve for the docs part!

Copy link
Contributor

@jonathan-buttner jonathan-buttner left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the PR, I left a few suggestions.

*/
rate_limit?: RateLimitSetting
/**
* The URL endpoint to use for the requests.
* For `completion` and `chat_completion` tasks, endpoint must be compatible with the OpenAI API format and include `v1/chat/completions`.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How about we expand this a little. Maybe something like:

Suggested change
* For `completion` and `chat_completion` tasks, endpoint must be compatible with the OpenAI API format and include `v1/chat/completions`.
* For `completion` and `chat_completion` tasks, the deployed model must be compatible with the Hugging Face Chat Completion interface (https://huggingface.co/docs/inference-providers/en/tasks/chat-completion#conversational-large-language-models-llms). The URL must include `v1/chat/completions`.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since OpenAI mode is the only visible way of determining whether or not OpenAI API format is supported, I propose expanding it a bit more:

Suggested change
* For `completion` and `chat_completion` tasks, endpoint must be compatible with the OpenAI API format and include `v1/chat/completions`.
* For `completion` and `chat_completion` tasks, the deployed model must be compatible with the Hugging Face Chat Completion interface (https://huggingface.co/docs/inference-providers/en/tasks/chat-completion#conversational-large-language-models-llms). OpenAI mode must be enabled, and the endpoint URL must include `v1/chat/completions`.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What is OpenAI mode? I searched hugging face to see if I couldn't find it. Are you referring to the request format that the inference API requires?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Did the change

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry, missed your question regarding OpenAI mode. OpenAI mode is what I call the toggle that needs to be enabled in order for endpoint to use OpenAI API.
Not every model has this toggle - so not every model can be used for elastic inference.
image

package.json Outdated
@@ -1,6 +1,6 @@
{
"dependencies": {
"@redocly/cli": "^1.34.1",
"@redocly/cli": "^1.34.3",
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we need this change?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This change is being made automatically by running pre commit set of tasks. This value is being incremented along with different changes committed over the time, so ignoring the change made by pre commit set of tasks would have to have a reason behind it.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah thanks!

@@ -5,7 +5,7 @@
"packages": {
"": {
"dependencies": {
"@redocly/cli": "^1.34.1",
"@redocly/cli": "^1.34.3",
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Was this file supposed to change?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As answered above change is being made automatically by running pre commit set of tasks. If we don't have reason for ignoring changes made by pre commit set of tasks - I'd keep it.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yep sounds good, thanks for explaining.

…chat-completion-integration

# Conflicts:
#	output/schema/schema-serverless.json
#	output/schema/schema.json
Copy link
Contributor

Following you can find the validation results for the APIs you have changed.

API Status Request Response
inference.chat_completion_unified Missing test Missing test
inference.completion Missing test Missing test
inference.delete Missing test Missing test
inference.get 🟢 1/1 1/1
inference.inference Missing test Missing test
inference.put_alibabacloud Missing test Missing test
inference.put_amazonbedrock Missing test Missing test
inference.put_anthropic Missing test Missing test
inference.put_azureaistudio Missing test Missing test
inference.put_azureopenai Missing test Missing test
inference.put_cohere Missing test Missing test
inference.put_elasticsearch Missing test Missing test
inference.put_elser Missing test Missing test
inference.put_googleaistudio Missing test Missing test
inference.put_googlevertexai Missing test Missing test
inference.put_hugging_face Missing test Missing test
inference.put_jinaai Missing test Missing test
inference.put_mistral Missing test Missing test
inference.put_openai Missing test Missing test
inference.put_voyageai Missing test Missing test
inference.put_watsonx Missing test Missing test
inference.put Missing test Missing test
inference.rerank Missing test Missing test
inference.sparse_embedding Missing test Missing test
inference.stream_completion Missing test Missing test
inference.text_embedding Missing test Missing test
inference.update Missing test Missing test

You can validate these APIs yourself by using the make validate target.

Copy link
Contributor

@szabosteve szabosteve left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Left a few comments and a tiny suggestion, otherwise LGTM!

@@ -29,13 +29,16 @@ import { Id } from '@_types/common'
/**
* Create a Hugging Face inference endpoint.
*
* Create an inference endpoint to perform an inference task with the `hugging_face` service.
* Creates an inference endpoint to perform an inference task with the `hugging_face` service.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I suggest changing it back to be consistent with the rest of the endpoint docs.

Suggested change
* Creates an inference endpoint to perform an inference task with the `hugging_face` service.
* Create an inference endpoint to perform an inference task with the `hugging_face` service.

Copy link
Contributor Author

@Jan-Kazlouski-elastic Jan-Kazlouski-elastic Jun 3, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

FYI Using of "Create" vs "Creates" is not consistent across the endpoint docs. For Amazon and Mistral it is "Creates" and it was taken as example. Thought it made more sense describing "what it does", but now I see that it is invitation to "do it".
Changed it since for the most of the providers it is "Create".

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@Jan-Kazlouski-elastic Thanks! I'll fix Amazon and Mistral.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you @szabosteve

*/
rate_limit?: RateLimitSetting
/**
* The URL endpoint to use for the requests.
* For `completion` and `chat_completion` tasks, the deployed model must be compatible with the Hugging Face Chat Completion interface (https://huggingface.co/docs/inference-providers/en/tasks/chat-completion#conversational-large-language-models-llms). OpenAI mode must be enabled, and the endpoint URL must include `v1/chat/completions`.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OpenAI mode must be enabled

What is OpenAI mode?

Can we remove that portion of the sentence?

Suggested change
* For `completion` and `chat_completion` tasks, the deployed model must be compatible with the Hugging Face Chat Completion interface (https://huggingface.co/docs/inference-providers/en/tasks/chat-completion#conversational-large-language-models-llms). OpenAI mode must be enabled, and the endpoint URL must include `v1/chat/completions`.
* For `completion` and `chat_completion` tasks, the deployed model must be compatible with the Hugging Face Chat Completion interface (https://huggingface.co/docs/inference-providers/en/tasks/chat-completion#conversational-large-language-models-llms). The endpoint URL must include `v1/chat/completions`.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

image
OpenAI mode is the toggle that needs to be enabled in order for endpoint to use OpenAI API.
I think that having this info is useful for the customer. Not every model has this toggle - so not every model can be used for elastic inference.
Do you still want me to remove it?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah I hadn't seen that before.

Can you try disabling it and see if a chat completion request using the inference API still works? I wonder if that only controls the UI in hugging face.

Can you give me an example of a model that has it?

Copy link
Contributor Author

@Jan-Kazlouski-elastic Jan-Kazlouski-elastic Jun 3, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you try disabling it and see if a chat completion request using the inference API still works?

After disabling it on UI - it still works, but url that is presented to the client doesn't contain v1/chat/completions. So basically switching it on and off doesn't keep endpoint from processing OpenAI payloads if they are sent to the /v1/chat/completions, but hides the /v1/chat/completions section of the URL and provides different payload example. So if OpenAI mode is turned off or not there at all for a specific model - then full URL with /v1/chat/completions is hidden - client can't see/use the URL that must be used for integration.
Client might have prior knowledge on how url should look like and we're saying that it must contain /v1/chat/completions, but it is not safe to imply that customer will understand this section must be added on top of the regular url. Specially if model doesn't support OpenAI payload and attempt to include /v1/chat/completions will be made and error will be returned because model doesn't support it.

I guess we could make it clearer by telling customer that absence of /v1/chat/completions on running model page can be caused by OpenAI mode being turned off. And enabling this mode if this toggle is present when url doesn't contain /v1/chat/completions can lead to it being shown. But since this mode always been turned ON by default for me I think this issue has really low risks. BUT! Presence of this toggle is one of the signs that model is capable of processing OpenAI type payloads and can be used for integration. So for me personally having this note is making us safer.
But you mentioning that you hadn't seen it before and obviously running successful tests in the past makes me wonder if it is something local to only some of the models.

UI when OpenAI mode is
On:
image

Off:
image

I wonder if that only controls the UI in hugging face.

Yes it just controls UI.

Can you give me an example of a model that has it?

https://huggingface.co/microsoft/Phi-3-mini-128k-instruct

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah I see. I wonder if it's be more helpful to put this information and an example picture like you have somewhere else in the docs. @szabosteve what do you think?

My suggestion is to would be to have the text as something like:

For `completion` and `chat_completion` tasks, the deployed model must be compatible with the Hugging Face Chat Completion interface (https://huggingface.co/docs/inference-providers/en/tasks/chat-completion#conversational-large-language-models-llms). The endpoint URL must include `v1/chat/completions`. To determine if the model supports the Hugging Face Chat Completion Interface and to access the correct URL follow the information here.

We'd have a link or something to either another page or a different section of the page that explains that the deployment should have a toggle for OpenAI. Then to get the correct URL they should enable the toggle and ensure the URL ends with v1/chat/completions.

Copy link
Contributor

@szabosteve szabosteve Jun 4, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the best would be to present this information somehow here. I'm afraid that linking to another docs page from the reference for such a low-level detail would be hiding information. I suggest something like this:

Suggested change
* For `completion` and `chat_completion` tasks, the deployed model must be compatible with the Hugging Face Chat Completion interface (https://huggingface.co/docs/inference-providers/en/tasks/chat-completion#conversational-large-language-models-llms). OpenAI mode must be enabled, and the endpoint URL must include `v1/chat/completions`.
* For `completion` and `chat_completion` tasks, the deployed model must be compatible with the Hugging Face Chat Completion interface (see the linked external documentation for details). The endpoint URL for the request must include `/v1/chat/completions`.
* If the model supports the OpenAI Chat Completion schema, a toggle should appear in the interface. Enabling this toggle doesn't change any model behavior, it reveals the full endpoint URL needed (which should include `/v1/chat/completions`) when configuring the inference endpoint in Elasticsearch. If the model doesn't support this schema, the toggle may not be shown.
* @ext_doc_id huggingface-chat-completion-interface

And then add the following to the table.csv:

huggingface-chat-completion-interface, https://huggingface.co/docs/inference-providers/en/tasks/chat-completion#conversational-large-language-models-llms

It will provide a link with the text External documentation at the end of the description, it makes the docs a bit more readable.

@jonathan-buttner What do you think?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yep that looks good. Thanks!

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Applied change proposed by @szabosteve

* After the endpoint is initialized (for dedicated) or ready (for serverless), ensure it supports the OpenAI API and includes `/v1/chat/completions` part in URL. Then, copy the full endpoint URL for use.
* Recommended models for `chat_completion` and `completion` tasks:
*
* * `Mistral-7B-Instruct-v0.2`
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@szabosteve should we include the full URL link to these models?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would rather not include the full URLs. Currently, we can provide only one link per description; otherwise, the generator complains.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah ok

Copy link
Contributor

github-actions bot commented Jun 3, 2025

Following you can find the validation results for the APIs you have changed.

API Status Request Response
inference.chat_completion_unified Missing test Missing test
inference.completion Missing test Missing test
inference.delete Missing test Missing test
inference.get 🟢 1/1 1/1
inference.inference Missing test Missing test
inference.put_alibabacloud Missing test Missing test
inference.put_amazonbedrock Missing test Missing test
inference.put_anthropic Missing test Missing test
inference.put_azureaistudio Missing test Missing test
inference.put_azureopenai Missing test Missing test
inference.put_cohere Missing test Missing test
inference.put_elasticsearch Missing test Missing test
inference.put_elser Missing test Missing test
inference.put_googleaistudio Missing test Missing test
inference.put_googlevertexai Missing test Missing test
inference.put_hugging_face Missing test Missing test
inference.put_jinaai Missing test Missing test
inference.put_mistral Missing test Missing test
inference.put_openai Missing test Missing test
inference.put_voyageai Missing test Missing test
inference.put_watsonx Missing test Missing test
inference.put Missing test Missing test
inference.rerank Missing test Missing test
inference.sparse_embedding Missing test Missing test
inference.stream_completion Missing test Missing test
inference.text_embedding Missing test Missing test
inference.update Missing test Missing test

You can validate these APIs yourself by using the make validate target.

…chat-completion-integration

# Conflicts:
#	output/openapi/elasticsearch-openapi.json
#	output/openapi/elasticsearch-serverless-openapi.json
#	output/schema/schema-serverless.json
#	output/schema/schema.json
Copy link
Contributor

github-actions bot commented Jun 4, 2025

Following you can find the validation results for the APIs you have changed.

API Status Request Response
inference.chat_completion_unified Missing test Missing test
inference.completion Missing test Missing test
inference.delete Missing test Missing test
inference.get 🟢 1/1 1/1
inference.inference Missing test Missing test
inference.put_alibabacloud Missing test Missing test
inference.put_amazonbedrock Missing test Missing test
inference.put_anthropic Missing test Missing test
inference.put_azureaistudio Missing test Missing test
inference.put_azureopenai Missing test Missing test
inference.put_cohere Missing test Missing test
inference.put_elasticsearch Missing test Missing test
inference.put_elser Missing test Missing test
inference.put_googleaistudio Missing test Missing test
inference.put_googlevertexai Missing test Missing test
inference.put_hugging_face Missing test Missing test
inference.put_jinaai Missing test Missing test
inference.put_mistral Missing test Missing test
inference.put_openai Missing test Missing test
inference.put_voyageai Missing test Missing test
inference.put_watsonx Missing test Missing test
inference.put Missing test Missing test
inference.rerank Missing test Missing test
inference.sparse_embedding Missing test Missing test
inference.stream_completion Missing test Missing test
inference.text_embedding Missing test Missing test
inference.update Missing test Missing test

You can validate these APIs yourself by using the make validate target.

Copy link
Contributor

github-actions bot commented Jun 4, 2025

Following you can find the validation results for the APIs you have changed.

API Status Request Response
inference.chat_completion_unified Missing test Missing test
inference.completion Missing test Missing test
inference.delete Missing test Missing test
inference.get 🟢 1/1 1/1
inference.inference Missing test Missing test
inference.put_alibabacloud Missing test Missing test
inference.put_amazonbedrock Missing test Missing test
inference.put_anthropic Missing test Missing test
inference.put_azureaistudio Missing test Missing test
inference.put_azureopenai Missing test Missing test
inference.put_cohere Missing test Missing test
inference.put_elasticsearch Missing test Missing test
inference.put_elser Missing test Missing test
inference.put_googleaistudio Missing test Missing test
inference.put_googlevertexai Missing test Missing test
inference.put_hugging_face Missing test Missing test
inference.put_jinaai Missing test Missing test
inference.put_mistral Missing test Missing test
inference.put_openai Missing test Missing test
inference.put_voyageai Missing test Missing test
inference.put_watsonx Missing test Missing test
inference.put Missing test Missing test
inference.rerank Missing test Missing test
inference.sparse_embedding Missing test Missing test
inference.stream_completion Missing test Missing test
inference.text_embedding Missing test Missing test
inference.update Missing test Missing test

You can validate these APIs yourself by using the make validate target.

…chat-completion-integration

# Conflicts:
#	output/openapi/elasticsearch-openapi.json
#	output/openapi/elasticsearch-serverless-openapi.json
#	output/schema/schema.json
Copy link
Contributor

github-actions bot commented Jun 5, 2025

Following you can find the validation results for the APIs you have changed.

API Status Request Response
inference.chat_completion_unified Missing test Missing test
inference.completion Missing test Missing test
inference.delete Missing test Missing test
inference.get 🟢 1/1 1/1
inference.inference Missing test Missing test
inference.put_alibabacloud Missing test Missing test
inference.put_amazonbedrock Missing test Missing test
inference.put_anthropic Missing test Missing test
inference.put_azureaistudio Missing test Missing test
inference.put_azureopenai Missing test Missing test
inference.put_cohere Missing test Missing test
inference.put_elasticsearch Missing test Missing test
inference.put_elser Missing test Missing test
inference.put_googleaistudio Missing test Missing test
inference.put_googlevertexai Missing test Missing test
inference.put_hugging_face Missing test Missing test
inference.put_jinaai Missing test Missing test
inference.put_mistral Missing test Missing test
inference.put_openai Missing test Missing test
inference.put_voyageai Missing test Missing test
inference.put_watsonx Missing test Missing test
inference.put Missing test Missing test
inference.rerank Missing test Missing test
inference.sparse_embedding Missing test Missing test
inference.stream_completion Missing test Missing test
inference.text_embedding Missing test Missing test
inference.update Missing test Missing test

You can validate these APIs yourself by using the make validate target.

…chat-completion-integration

# Conflicts:
#	package-lock.json
Copy link
Contributor

@jonathan-buttner jonathan-buttner left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looking good.

* For Elastic's `chat_completion` and `completion` tasks:
* The selected model must support the `Text Generation` task and expose OpenAI API. HuggingFace supports both serverless and dedicated endpoints for `Text Generation`. When creating dedicated endpoint select the `Text Generation` task.
* After the endpoint is initialized (for dedicated) or ready (for serverless), ensure it supports the OpenAI API and includes `/v1/chat/completions` part in URL. Then, copy the full endpoint URL for use.
* Recommended models for `chat_completion` and `completion` tasks:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@szabosteve should we say something like "known supported models"?

…chat-completion-integration

# Conflicts:
#	output/openapi/elasticsearch-openapi.json
#	output/openapi/elasticsearch-serverless-openapi.json
#	output/schema/schema.json
#	specification/inference/chat_completion_unified/examples/request/PostChatCompletionRequestExample2.yaml
#	specification/inference/chat_completion_unified/examples/request/PostChatCompletionRequestExample3.yaml
Copy link
Contributor

github-actions bot commented Jun 9, 2025

Following you can find the validation results for the APIs you have changed.

API Status Request Response
inference.chat_completion_unified Missing test Missing test
inference.completion Missing test Missing test
inference.delete Missing test Missing test
inference.get 🟢 1/1 1/1
inference.inference Missing test Missing test
inference.put_alibabacloud Missing test Missing test
inference.put_amazonbedrock Missing test Missing test
inference.put_anthropic Missing test Missing test
inference.put_azureaistudio Missing test Missing test
inference.put_azureopenai Missing test Missing test
inference.put_cohere Missing test Missing test
inference.put_elasticsearch Missing test Missing test
inference.put_elser Missing test Missing test
inference.put_googleaistudio Missing test Missing test
inference.put_googlevertexai Missing test Missing test
inference.put_hugging_face Missing test Missing test
inference.put_jinaai Missing test Missing test
inference.put_mistral Missing test Missing test
inference.put_openai Missing test Missing test
inference.put_voyageai Missing test Missing test
inference.put_watsonx Missing test Missing test
inference.put Missing test Missing test
inference.rerank Missing test Missing test
inference.sparse_embedding Missing test Missing test
inference.stream_completion Missing test Missing test
inference.text_embedding Missing test Missing test
inference.update Missing test Missing test

You can validate these APIs yourself by using the make validate target.

Copy link
Contributor

github-actions bot commented Jun 9, 2025

Following you can find the validation results for the APIs you have changed.

API Status Request Response
inference.chat_completion_unified Missing test Missing test
inference.completion Missing test Missing test
inference.delete Missing test Missing test
inference.get 🟢 1/1 1/1
inference.inference Missing test Missing test
inference.put_alibabacloud Missing test Missing test
inference.put_amazonbedrock Missing test Missing test
inference.put_anthropic Missing test Missing test
inference.put_azureaistudio Missing test Missing test
inference.put_azureopenai Missing test Missing test
inference.put_cohere Missing test Missing test
inference.put_elasticsearch Missing test Missing test
inference.put_elser Missing test Missing test
inference.put_googleaistudio Missing test Missing test
inference.put_googlevertexai Missing test Missing test
inference.put_hugging_face Missing test Missing test
inference.put_jinaai Missing test Missing test
inference.put_mistral Missing test Missing test
inference.put_openai Missing test Missing test
inference.put_voyageai Missing test Missing test
inference.put_watsonx Missing test Missing test
inference.put Missing test Missing test
inference.rerank Missing test Missing test
inference.sparse_embedding Missing test Missing test
inference.stream_completion Missing test Missing test
inference.text_embedding Missing test Missing test
inference.update Missing test Missing test

You can validate these APIs yourself by using the make validate target.

@Jan-Kazlouski-elastic Jan-Kazlouski-elastic merged commit b48894f into main Jun 9, 2025
8 checks passed
@Jan-Kazlouski-elastic Jan-Kazlouski-elastic deleted the feature/hugging-face-chat-completion-integration branch June 9, 2025 12:56
Copy link
Contributor

github-actions bot commented Jun 9, 2025

The backport to 8.19 failed:

The process '/usr/bin/git' failed with exit code 1

To backport manually, run these commands in your terminal:

# Fetch latest updates from GitHub
git fetch
# Create a new working tree
git worktree add .worktrees/backport-8.19 8.19
# Navigate to the new working tree
cd .worktrees/backport-8.19
# Create a new branch
git switch --create backport-4383-to-8.19
# Cherry-pick the merged commit of this pull request and resolve the conflicts
git cherry-pick -x --mainline 1 b48894fcf05fdbd121ccb20c54c2b66be1924124
# Push it to GitHub
git push --set-upstream origin backport-4383-to-8.19
# Go back to the original working tree
cd ../..
# Delete the working tree
git worktree remove .worktrees/backport-8.19

Then, create a pull request where the base branch is 8.19 and the compare/head branch is backport-4383-to-8.19.

Jan-Kazlouski-elastic added a commit that referenced this pull request Jun 9, 2025
… completion tasks (#4383)

* Update Inference specification for Hugging Face's completion and chat completion tasks

* Extend description for url parameter

* Fix model_id description for text_embedding task and update endpoint creation comment

* Enhance Hugging Face integration documentation for chat completion and completion tasks

* Add @typescript-eslint/rule-tester to devDependencies in package-lock.json

* Enhance Hugging Face integration to support chat_completion and completion tasks

* Fix description for model_id field in Hugging Face integration to clarify request failure conditions

* Update json schema

(cherry picked from commit b48894f)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants