LLM Agent-directed Toolset

[rickbraddy-pharma]

Hi,

As context, I started with LangChain, so some definite preconceptions.

My need is like many others… the need for intelligent agents to perform increasingly complex tasks, many of which involve accessing other IT systems via APIs or CLI commands. LangChain calls these Tools 🛠️
(I think everyone calls them that lol). Tools provide a means to bridge an LLM with everything else in the world that’s needed.

Tools become useful when an LLM “program” (like a Guidance program) can use Tools to access the world outside the agent runtime environment. Common tools include Search, Browser, Math/Calculator, and Filesystems.

With that as a backdrop, how should us LangChain-brains be thinking about Tools and their direction for use with Guidance?

It would also seem to beg the question about Chain of Thought (CoT), ReAct and other techniques mentioned in the opening statements about Guidance but hard to find documentation or code snippets across so many diverse notebooks.

Are these concepts truly in scope for Guidance or should I be looking at other options?

[chrisfentiman]

The most straightforward answer, now that we've got several guidance programs up and running, is that you've got to build it. Guidance is excellent at control, and that's precisely why we left Langchain for it. But with Guidance, you don't get the fancy bells and whistles that Langchain offers - you have to construct all those yourself. And that's another reason why we're really into Guidance. It's all about control.

Don't get me wrong, Langchain is solid, but Guidance fits our production use case way better. If you need more control over the LLM and how it interacts with your system, Guidance is simpler to tweak and comes with less overhead.

[rickbraddy-pharma]

That's great! That being the case, clearly more examples will help.

[rickbraddy-pharma]

Meanwhile, I have figured out how to use Guidance to build a search tool.

I created my own Tool base class then subclassed a SerperSearchTool using Serper GoogleSearch. It’s implemented using a Guidance program.

Best of all, runs fast as it uses GPT3.5-TURBO, which is more than capable of reliable tool handling.

If there’s interest, I can probably create a notebook example that can be used to quickly create tools powered by Guidance.

[slundberg]

Coming up with a set of best practices for tool use with Guidance is on a (long) todo list. So if you have a start in that direction I would be happy to take a look. I expect that many of the main tool use stratagies can be implemented with a relatively simple Guidance program, and sharing a collection of those with each other would be nice. Note that the await_missing parameter and the await command can be very useful for reusable guidance programs and tool use (as can stop_regex).

[rickbraddy-pharma]

Would love to see a thumbnail sketch (or actual program) to fully exploit those features with Tools. My search program is pretty basic at this stage. Will brush up on those and how to potentially apply them - thanks for the guidance.