forked from art-from-the-machine/Mantella
-
Notifications
You must be signed in to change notification settings - Fork 0
Roadmap texts
Leidtier edited this page Apr 8, 2024
·
4 revisions
- Eliminate all issues in setup and usage that comes from having to handle files
- Setup issues
- Encoding issues
- Root Builder
- Having to cast the spell to make it work
- etc.
- Being able to easily send any kind of data from and to the game that we like without having to introduce more communication files
- Use HTTP instead of file read/write to communicate between game and MantellaSoftware
- Have SKSE and F4SE plugins that allow the games to communicate with an HTTP server
- Change the interaction logic to a server-client communication in which the game is the client that calls the MantellaSoftware to get information
- MantellaSpell and MantellaGun use a Quest to track the current conversation, updates are received as events
- MantellaSoftware generates and provides packages of information for the game containing information about what happens next
- Have a feature complete PR for integration into MantellaSoftware
- Have a fallback for games that can not (yet) use HTTP communication
- Have support for debugging
- Have a feature complete PR for integration into MantellaSpell
- Have support for debugging
- Have a feature complete PR for integration into MantellaGun
- Have support for debugging
- SKSE_HTTP: A plugin that allows Skyrim to communicate strongly typed data over HTTP
- Skyrim SE
- Skyrim AE
- Skyrim VR
- F4SE_HTTP: A plugin that allows Fallout 4 to communicate strongly typed data over HTTP
- Fallout 4
- Fallout 4 VR
- Create an interface that provides the conversation logic with a unified simple access to the different things we want an LLM to generate
- Spoken sentences
- Switching Character
- Narrations
- Actions
- Vision analysis
- etc.
- Create different implementations to this interface that then in turn provide the requested things (or not if unable)
- By separating creation and consumption clearly we can develop them independently (like creating the support in the game, even if the LLMs are still triggering it unreliable)
- Different implementation of these can make us of
- Specific implementations for different LLM services (OpenAI, OpenRouter, text-gen-webui, koboldcpp, other)
- Different implementations for ways to get structured output out of LLMs (JSON schema, Tools, function calls, text parsing)
- Different implementations for text parsing
- None
- Allow MantellaSoftware to load plugins on start that extend the existing functionality
- Creating a plugin would allow to add and develop new implementations without constantly having to change *MantellaSoftware
- Create dedicated interfaces in MantellaSoftware that plugins can be written against
- TTS implementations
- STT implementations
- LLM implementations
- Text parsings
- Actions
- etc.
- Tests for a good clean way of dynamically loading and managing modules
- Pretty simple and straightforward refactoring
- Create a common interface for TTS that then is used by MantellaSoftware to generate voicelines
- Separate implementations for xVASynth and xTTS into different classes that both implement the new TTS interface
- Maybe create a common abstract base class for both of them if they have code in common
- This would also allow to more easily create other TTS implementations
- None
- Requested feature
- Original origin maybe too NSFW ?
- Individual text files make for good support for version control like Git
- Extendable JSON format
- Some somewhere
- Have a UI for MantellaSoftware that should provide two main features
- A guided and structured way to set the
config.inivalues correctly - A chat function that allows to chat with the characters without having to start the game. This would also replace the debug mode
- A guided and structured way to set the
- As we now have an HTTP server in MantellaSoftware a UI running in the browser would be nice
- None