Merge pull request #105 from brainlid/me-update-api-to-tools

update api to "tools" concept

MODEL_BEHAVIORS.md

@@ -0,0 +1,11 @@
+# Model Behaviors
+
+This is to document some of the differences between model behaviors. This is specifically about how one LLM responds differently than another. This information is likely to get out of date quickly as models change. Still, it may be helpful as a reference or starting point.
+
+## Pre-filling the assistant's response
+
+- ChatGPT 3.5 and 4 do not complete a pre-filled assistant response as we might hope. Giving it instructions to provide an answer in an `<answer>{{ANSWER}}</answer>` template, then starting an assistant message with `<answer>` to have it complete it, it will not include the closing `</answer>` tag.
+  - 2024-04-17
+
+- Anthropic's Claude 3 does respond well to pre-filled assistant responses and it is officially encouraged.
+  - 2024-04-17

README.md

@@ -205,3 +205,18 @@ Executing a specific test, whether it is a `live_call` or not, will execute it c
 When doing local development on the `LangChain` library itself, rename the `.envrc_template` to `.envrc` and populate it with your private API values. This is only used when running live test when explicitly requested.
 
 Use a tool like [Direnv](https://direnv.net/) or [Dotenv](https://github.com/motdotla/dotenv) to load the API values into the ENV when using the library locally.
+
+
+TODO: explicitly return JSON formatted data only to a request. Use in an example? Use for data extraction chain? (assuming only OpenAI?)
+- https://platform.openai.com/docs/guides/text-generation/json-mode
+
+TODO: For data extraction with OpenAI, can set the "tool_choice" to extra force that function use.
+
+TODO: Multiple tool calls must mean that multiple tool messages can be returned in a sequence.
+
+TODO: Pull the "before" OpenAI module for legacy API support? Allow for using it against OTHER APIs that have OpenAI compatibility and may not be updated yet.
+- still need to be updated to use "Tool" related messages. Assistant "tool_calls" would be different too.
+
+TODO: add a `version` to the structs. The version is set by the model. So it using an OpenAI assistant message with tool_calls is either a string date or "v2" or something. It can be pattern matched on when writing to a newer model.
+
+- Perhaps the ChatOpenAI struct has the version number on it. Then it converts a `for_api` call differently for a tool to look like a function when targeting an older version.

lib/chains/data_extraction_chain.ex

@@ -72,6 +72,7 @@ defmodule LangChain.Chains.DataExtractionChain do
   require Logger
   alias LangChain.PromptTemplate
   alias LangChain.Message
+  alias LangChain.Message.ToolCall
   alias LangChain.Chains.LLMChain
 
   @function_name "information_extraction"
@@ -101,15 +102,19 @@ Passage:
       {:ok, chain} = LLMChain.new(%{llm: llm, verbose: verbose})
 
       chain
-      |> LLMChain.add_functions(build_extract_function(json_schema))
+      |> LLMChain.add_tools(build_extract_function(json_schema))
       |> LLMChain.add_messages(messages)
       |> LLMChain.run()
       |> case do
         {:ok, _updated_chain,
          %Message{
            role: :assistant,
-           function_name: @function_name,
-           arguments: %{"info" => info}
+           tool_calls: [
+             %ToolCall{
+               name: @function_name,
+               arguments: %{"info" => info}
+             }
+           ]
          }}
         when is_list(info) ->
           {:ok, info}
@@ -136,6 +141,12 @@ Passage:
     LangChain.Function.new!(%{
       name: @function_name,
       description: "Extracts the relevant information from the passage.",
+      function: fn args, _context ->
+        # NOTE: The function is not executed here because we won't be returning
+        # this to the LLM. The LLMChain does not run the function, but stops at
+        # the request for it.
+        {:ok, args}
+      end,
       parameters_schema: %{
         type: "object",
         properties: %{