Render streamed input incrementally

[david-crespo]

I wrote a simple CLI for talking to LLM APIs and I pipe markdown-formatted output into glow for rendering.

function ai() {
  ~/repos/llm-cli/main.ts "$@" | glow
}

I would like to be able to use streaming responses from LLM APIs and write the result to stdout chunk by chunk and have glow render it incrementally. But it seems glow always reads the full input before rendering:

glow/main.go

Lines 246 to 247 in 2430b0a

	func executeCLI(cmd *cobra.Command, src *source, w io.Writer) error {
	b, err := io.ReadAll(src.reader)

I understand why it is that way, I'm sure it's dramatically simpler. It's certainly what I would have done.

Alternatives

• I could simply not pipe to glow when I have a streaming response (or stream raw markdown to stdout and then re-render with glow once it's done)
• Use your lovely mods CLI instead, which does support streaming but does not support Claude

[david-crespo]

Update for anyone interested in this: I'm able to solve this outside of glow by accumulating the input and clearing and re-rendering the whole thing on each chunk.

import $ from "jsr:@david/[email protected]"

let inputBuffer = ""

const decoder = new TextDecoder()

for await (const chunk of Deno.stdin.readable) {
  inputBuffer += decoder.decode(chunk)
  // --style auto is there to force it to output styled
  // https://github.com/charmbracelet/glow/blob/2430b0a/main.go#L158
  const output = await $`glow --style auto`.stdinText(inputBuffer).text()
  console.clear()
  console.log(output)
}

2024-05-08-stream-to-glow-no-flash.mp4