Can I use my OPENAI API?

[Cossackx]

Rather than using Llama?

[Swaggeroo]

It wasn't planned but when there is interest I would consider implementing it.

I had some arguments against it:

• every picture would cost around 0.005$ to analyse
  • This could be very expensive to analyse a lot of pictures
• Privacy is also a concern
• Also ollama is super easy to install

[hermanmak]

Adding a +1, I took a look at the plugin settings, can we do openai because we can spin up our own openai compatible endpoint.

This would enable offhost compute

[Swaggeroo]

This would enable offhost compute

Off host is also possible with Ollama at the moment. Could you provide an example for an Openai compatibility endpoint?

[hermanmak]

@Swaggeroo Hmm can you share me the ollama off host?

So I am running Open WebUI which offers both below with required bearer token auth in the HEADER:
Ollama proxy at DOMAIN/ollama
OpenAI compatible proxy at DOMAIN/openai

References:
https://docs.openwebui.com/getting-started/advanced-topics/api-endpoints/
OpenAI API (which would be available at DOMAIN/openai/
https://litellm-api.up.railway.app/

Thanks!

[Swaggeroo]

I will have a look

[Swaggeroo]

@hermanmak
Good news I added it with #89 🎉

Have a look in the new readme and I would be happy if you test it.

[Swaggeroo]

@hermanmak
Did you had time to look at it?
Is it what you requested?