Roadmap (Sept 2024)

[tjtanaa]

Action Items

1. Convert more Intel Model and push to EmbeddedLLM Huggingface Model Space to speed up auto-download of model. Users do not need to convert on the fly. (On-going)
2. Separate the model into another process.
3. Support auto relaunch of ONNX DirectML Engine when kernel crashes.
4. Intel NPU LLM model support

v0.3

1. OpenVINO LLM Support
2. Support loading and unloading of OpenVINO and IPEX-LLM model using ModelUI.
3. Support OpenVINO Vision Model. Phi-3.5-Vision
4. AMD iGPU ellm_api_server
5. Intel OpenVINO ellm_api_server
6. Intel IPEX-LLM ellm_api_server