README.md

LLMFarm_core.swift

LLMFarm_core swift library to work with large language models (LLM). It allows you to load different LLMs with certain parameters.
Based on ggml and llama.cpp by Georgi Gerganov.

Features

• MacOS (13+)
• iOS (16+)
• Various inferences
• Various sampling methods
• Metal (dont work on intel Mac)
• Model setting templates
• LoRA adapters support (read more)
• LoRA train support
• LoRA export as model support
• Restore context state (now only chat history)

Inferences

• LLaMA
• GPTNeoX
• Replit
• GPT2 + Cerebras
• Starcoder(Santacoder)
• RWKV (20B tokenizer)
• Falcon
• MPT
• Bloom
• StableLM-3b-4e1t
• Qwen
• Yi models
• Deepseek models
• Mixtral MoE
• PLaMo-13B

Note: For Falcon, Alpaca, GPT4All, Chinese LLaMA / Alpaca and Chinese LLaMA-2 / Alpaca-2, Vigogne (French), Vicuna, Koala, OpenBuddy (Multilingual), Pygmalion/Metharme, WizardLM, Baichuan 1 & 2 + derivations, Aquila 1 & 2, Mistral AI v0.1, Refact, Persimmon 8B, MPT, Bloom select llama inferece in model settings.

Sampling methods

• Temperature (temp, tok-k, top-p)
• Tail Free Sampling (TFS)
• Locally Typical Sampling
• Mirostat
• Greedy
• Grammar (dont work for GPTNeoX, GPT-2, RWKV)
• Classifier-Free Guidance

Installation

git clone https://github.com/guinmoon/llmfarm_core.swift

Swift Package Manager

Add llmfarm_core to your project using Xcode (File > Add Packages...) or by adding it to your project's Package.swift file:

dependencies: [
  .package(url: "https://github.com/guinmoon/llmfarm_core.swift")
]

Build and Debug

To Debug llmfarm_core package, do not forget to comment .unsafeFlags(["-Ofast"]) in Package.swift. Don't forget that the debug version is slower than the release version.

To build with QKK_64 support uncomment .unsafeFlags(["-DGGML_QKK_64"]) in Package.swift.

Usage

See examples in the Demo Project

Also used sources from:

• rwkv.cpp by saharNooby.
• Mia by byroneverson.
• swift-markdown-ui by gonzalezreal

Projects based on this library

• LLM Farm

App to run LLaMA and other large language models locally on iOS and MacOS.