Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore: retrieves the exact model running status upon message error #3976

Open
wants to merge 2 commits into
base: dev
Choose a base branch
from

Conversation

louis-jan
Copy link
Contributor

@louis-jan louis-jan commented Nov 8, 2024

Describe Your Changes

  • In previous versions, the model was marked as stopped as soon as a message error occurred. However, it could be an issue with the inference parameters, meaning the model was still running, leading to an incorrect status. This PR aims to improve this by checking the model status before marking it as not started.
  • This also updates legacy model.json settings to overwrite cortex.cpp prompt extraction from the model's metadata, which could cause issues.

Changes made

The changes in the provided diff focus on adding a feature to check the loading status of a model and ensuring better code readability. Here’s a summary of the changes:

  1. ModelExtension Class in model.ts:

    • Added a new abstract method isModelLoaded(model: string): Promise<boolean> to check if a model is loaded.
  2. CortexAPI in cortex.ts:

    • Reformatted the importModel method for better readability by spreading its parameters over multiple lines.
    • Added a new method getModelStatus(model: string): Promise<boolean> which sends a GET request to check the status of a model and returns true if successful, false otherwise.
    • Minor formatting change in the finalizeModel method to add a comma at the end of the model metadata.
  3. JanModelExtension in index.ts:

    • Implemented the isModelLoaded method which uses the newly added getModelStatus method from CortexAPI to check a model's loading status.
  4. EventHandler in EventHandler.tsx:

    • Imported ModelExtension.
    • Enhanced error handling by checking if the active model is still loaded. If not, it sets the active model to undefined and resets the state model, ensuring that the application handles model load status accurately.

These changes collectively improve the application's ability to verify and respond to the model's loading status and improve method readability by spreading parameters on multiple lines.

@github-actions github-actions bot added the type: chore Maintenance, operational label Nov 8, 2024
Copy link
Contributor

github-actions bot commented Nov 8, 2024

Barecheck - Code coverage report

Total: 69.58%

Your code coverage diff: 0.00% ▴

Uncovered files and lines
FileLines
core/src/browser/extensions/model.ts12
web/hooks/useModels.ts47, 60-61, 65

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
type: chore Maintenance, operational
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants