Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Bearer Token vs Model parameter? #53

Open
msj121 opened this issue Jun 11, 2023 · 0 comments
Open

Bearer Token vs Model parameter? #53

msj121 opened this issue Jun 11, 2023 · 0 comments

Comments

@msj121
Copy link

msj121 commented Jun 11, 2023

Just curious why bearer token is used to determine model location, why not just a parameter in the json like "model" which is used to select model in openai? This to me seems more intuitive but perhaps you have a reason for bearer token?

It would allow a more "seamless" approach to changing the model between requests for example. (Although currently the code doesn't check model path is the same (ie: to restart model or not, just last messages)).

I think this IS the point of the "model" parameter for openai, just they know the location and it is by name, and here it is the location itself, although you could assume location (/llama.cpp/models) and just use the filename as model name. This seems more to me to be a 1:1 behaviour, no?

Happy to submit a PR to alter the api key check with a fall back to req checking the json.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant