Skynet is an API server for AI services wrapping several apps and models.
It is comprised of specialized modules which can be enabled or disabled as needed.
- Summary and Action Items with vllm (or Ollama)
- Live Transcriptions with Faster Whisper via websockets
- RAG Assistant
- 🚧 More to follow
- Poetry
- Redis
# disable authorization
export BYPASS_AUTHORIZATION=1
# start Redis
docker run -d --rm -p 6379:6379 redis
# If using vLLM (running on NVIDIA GPU)
export LLAMA_PATH="$HOME/models/Llama-3.1-8B-Instruct"
poetry install --with vllm
# If using Ollama
export LLAMA_PATH="llama.3.1"
poetry install
./run.shVisit http://127.0.0.1:8000
Note: Make sure to have ffmpeg < 7 installed and to update the
DYLD_LIBRARY_PATHwith the path to the ffmpeg libraries, e.g.export DYLD_LIBRARY_PATH=/Users/MyUser/ffmpeg/6.1.2/lib:$DYLD_LIBRARY_PATH.
mkdir -p "$HOME/models/streaming-whisper"
export WHISPER_MODEL_NAME="tiny.en"
export BYPASS_AUTHORIZATION="true"
export ENABLED_MODULES="streaming_whisper"
export WHISPER_MODEL_PATH="$HOME/models/streaming-whisper"
poetry install
./run.shdocker compose -f compose-dev.yaml up --build
docker cp $HOME/models/Llama-3.1-8B-Instruct-Q8_0.gguf skynet-web-1:/models
docker restart skynet-web-1
# localhost:8000 for Skynet APIs
# localhost:8001/metrics for Prometheus metricsGo to Streaming Whisper Demo to test your deployment from a browser
OR
Go to demos/streaming-whisper/ and start a Python http server.
python3 -m http.server 8080Open http://127.0.0.1:8080.
Detailed documentation on how to configure, run, build and monitor Skynet and can be found in the docs.
If you want to contribute, make sure to install the pre-commit hook for linting.
poetry run githooks setupSkynet is distributed under the Apache 2.0 License.
