chore: conditionally disable modal state tools #256
Conversation
| @@ -42,8 +42,10 @@ export function createServerWithTools(options: Options): Server { | |||
| }); | |||
|
|
|||
| server.setRequestHandler(ListToolsRequestSchema, async () => { | |||
| const modalStates = context.modalStates().map(state => state.type); | |||
| const activeTools = tools.filter(tool => !tool.clearsModalState || modalStates.includes(tool.clearsModalState)); | |||
There was a problem hiding this comment.
I am not a firm believer that the dynamic tool list is a good idea. I think it will throw LLM off. Here is my thinking:
- If this mechanism worked, we would only list the tools that clear currently engaged states. The client would then re-build the tool prologue for the LLM and re-compile the chat. Now LLM will be presented with a chat history where it calls non-existing tools
- An alternative, more heavyweight approach would be inlining the messages about the tools being turned off and on dynamically
Maybe there are other ways to implement this, but none of the ways I come up with improve the quality of the prompt, only adds unnecessary noise. I think dynamic tooling is not a great idea when it comes to keeping LLM sharp.
There was a problem hiding this comment.
I'm not convinced. If dynamic tools threw off the LLM, then MCP wouldn't have been designed with explicit support for dynamic tool lists, or at least it'd allow the client to announce compatibility. My guess is they explicitly added it for this usecase, also as a measure to limit the number of tools.
I think we can do two things to figure this out:
- play around with it
- ask for guidance in the MCP issue tracker: Should conditional tools be hidden from the tool list? modelcontextprotocol/modelcontextprotocol#409
I'll do both of that and report back.
If this mechanism worked, we would only list the tools that clear currently engaged states.
Good idea!
An alternative, more heavyweight approach would be inlining the messages about the tools being turned off and on dynamically
As far as I know, LLMs are already trained specifically with tool usage in mind, and i'd imagine that turning tools on or off is part of that training. So the tool list feels like the natural place to communicate this, not the message.
Helps to keep our number of tools low.