AI: all about AI

[bangank36]

Summary

• This issue logs all development steps for AI model usage

• Gpt4All - Require higher macOS

• FlexGPT VScode

• Llama.cpp used to invoke AI server in local host, llama server notes

• 7B model as example with llama example

[bangank36]

FlexGPT to use

• First need to install Llama.cpp
• Download model
• Init llama server
• Put the localhost URL to flexGPT extension

lmstudio

• https://lmstudio.ai/

• Download AI model in interface

• Explore the Server panel to check for the local Ai server URL

  • This feature is not working as expected because the flexiGPT send request to /completion endpoint while the app expecting /completions with s
  • Looks like openAPI expected /completions https://platform.openai.com/docs/guides/gpt
  • While llama.cpp server expected /completion https://github.com/ggerganov/llama.cpp/tree/master/examples/server#testing-with-curl

• GOT IT WORKS

  • First remove the default Flexigpt › Llamacpp: Default Origin to neutralize it
  • Grab an API key from https://platform.openai.com/account/api-keys, it won't be used
  • Add the URL Flexigpt › Openai: Default Origin to http://localhost:1234 for kicking off the server in local