Generating structure with GPT4 / partial completions

[crystalneth]

The doc says: "Since partial completions are not allowed, you can't really use output structure inside an assistant block, but you can still set up a structure outside of it."

I guess this is because "insert" is not supported for newer OpenAI models.

Is it possible to generate a reliable json object parsed by guidance by using structure outside the assistant block or in some other way? Is this less reliable because the model doesn't see the entire json object before completion?

Also I guess it must be far less efficient to generate each json value with a separate assistant block? How is conversational memory handled? I have only worked with the GPT4 playground so far and it seems like the entire conversation is processed with each completion call and must be preserved by the user - is this also the case with guidance? Does it send the entire conversation with each gen call racking up usage?

I know there's lots of discussion about how to generate JSON with GPT elsewhere, however wondering about any tips or tricks here.