New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Serving LLMs on Open Data Hub / RHODS with ModelMesh #46

Open

Tracked by

codificat opened this issue May 16, 2023 · 1 comment

Open

Tracked by

Serving LLMs on Open Data Hub / RHODS with ModelMesh #46

codificat opened this issue May 16, 2023 · 1 comment

Assignees

Member

codificat commented May 16, 2023 •

edited

Loading

How can we create model endpoints using Modelmesh on the cluster?

This should involve instructions on deploying modelmesh using Operator Hub and adding the model to a s3 bucket.

Create the predict function and explain how it is being called and demo a notebook that sends a payload to the model endpoint.

codificat mentioned this issue

[EPIC] Serving Foundation Models #37

Open

3 tasks

codificat changed the title How can we create model endpoints using Modelmesh on the cluster? This should involve instructions on deploying modelmesh using Operator Hub and adding the model to a s3 bucket. Create the predict function and explain how it is being called and demo a notebook that sends a payload to the model endpoint. Serving LLMs on Open Data Hub / RHODS with ModelMesh

codificat self-assigned this

Member

Shreyanand commented Mar 22, 2024

@codificat can we add the document that you created to the repo for the record?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment