Extract basic building blocks from chat_completion/2 to make a more composable interface

[martosaur]

Hi 👋 and thank you for the amazing library! This PR proposes a little code reshuffling in order to hopefully make main interface more flexible.

Problem

I want to use Instructor's flow with OpenAI's discounted Batch API. However, Instructor.chat_completion/2 tightly encapsulates the entire workflow, from preparing a prompt to making an API call, evaluating the result, and potentially issuing retries.

Solution

If we could expose basic building blocks of Instructor.chat_completion/2, the client code would be able to compose them in order to accommodate whatever specific needs it has.

After playing with the code for a while I came up with the following new public functions:

1. Instructor.prepare_prompt/2. This is a near-pure function that accepts the same parameters and optional config as Instructor.chat_completions/2 and returns a prompt in a form of a map, ready to be passed to an HTTP client. The prompt is adapter-specific, this is why we need to introduce new Instructor.Adapter.prompt/1 callback. And to remove code duplication, it makes sense to change Instructor.chat_completion/2 callback to Instructor.chat_completion/3 with prompt as the new first argument.
2. Instructor.consume_response/2. This is a pure function, that accepts a response from Instructor.chat_completion/3 and the same parameters that were supplied to Instructor.prepare_prompt. It validates the response, attempts to cast it to the response_model and either returns an {:ok, response_model} or {:error, changeset, params} tuple, with updated params that can be used for the next attempt.

With this two functions the client can manage how they want to call the api, alter prompt at any stage, add logging or telemetry, etc – essentially, rebuild Instructor.chat_completion/2 to their liking.

The changes to adapter behaviour are unfortunately not backwards compatible, but this should be ok at this stage I think?

Example

Here's a shortened example of how it works with batch API:

prompt =
  Instructor.prepare_prompt(response_model: MyResponseModel, model: "gpt-4o", messages: messages)

jsonl =
  [prompt]
  |> Enum.map(
    &Jason.encode!(%{custom_id: "foo_id", method: "POST", url: "/v1/chat/completions", body: &1})
  )
  |> Enum.join("\n")

multipart =
  Multipart.new()
  |> Multipart.add_part(Multipart.Part.text_field("batch", "purpose"))
  |> Multipart.add_part(
    Multipart.Part.file_content_field("test.jsonl", jsonl, :file, filename: "test.jsonl")
  )

headers = [
  {"Authorization", "Bearer #{Application.compile_env(:instructor, [:openai, :api_key])}"},
  {"Content-Type", Multipart.content_type(multipart, "multipart/form-data")}
]

{:ok, %{body: %{"id" => file_id}}} =
  Req.post("https://api.openai.com/v1/files",
    headers: headers,
    body: Multipart.body_binary(multipart)
  )

{:ok, %{body: %{"id" => batch_id}}} =
  Req.post("https://api.openai.com/v1/batches",
    json: %{input_file_id: file_id, endpoint: "/v1/chat/completions", completion_window: "24h"},
    headers: auth_headers
  )

{:ok, %{body: %{"output_file_id" => output_file_id}}} =
  Req.get("https://api.openai.com/v1/batches/#{batch_id}", headers: auth_headers)

{:ok, %{body: body}} =
  Req.get("https://api.openai.com/v1/files/#{output_file_id}/content", headers: auth_headers)

result =
  Jason.decode!(body)["response"]["body"]
  |> Instructor.consume_response(response_model: MyResponseModel, messages: messages)

# {:ok, %MyResponseModel{...}}

Let me know what you think! If it goes through in any version, I'll add some documentation as well.

[TwistingTwists]

This feels elegant to me @martosaur !