Cycle through suggestions

[snobu]

Is there a way to cycle through code completion suggestions? You know, similar to what GitHub Copilot can do with Alt (Option) + ] and Alt (Option) + [, and then Tab to accept it.

[gergap]

No, this is a copilot feature. Regular LLMs don't provide multiple solutions.
You can cancel the completion and try again. Based on the configured tempreature setting the response is more "creative" or more determinisitc (reproducible). I prefer the later, which is why I normally set temperature=0.

In theory we could generate multiple solutions as well with prompt engineering, but I have not tried id yet. What the plugin does is using FIM tokens to complete the current code fragment only. Nothing more.

[gergap]

Another idea could be to use multiple different LLMs simultanously, so we could cycle through the different solutions of multiple models. The question will be how many LLMs can you load into VRAM on consumer hardware, though. Completion models are smaller than the chat models normally, so this could work.

[snobu]

I think prompt engineering would be easier to implement while still maintaining sane requirements (running a single model instance).
We could have a key binding that asks the model for another code block than the one already suggested - "[code context] + provide another suggestion, that either improves on the current or uses a different approach: [current code suggestion block]".

We could use a list to store subsequent suggestions so we could mimic the GitHub Copilot behavior.

Don't get me wrong, i absolutely enjoy this project and am grateful it exists, but this seems like a feature that is necessary in a real development environment.