Response Streaming with Effect

25 Aug, 2026

A user submits a question, and somewhere in our program LanguageModel.generateText sets off to fetch the answer. The errand takes ten, twenty, maybe thirty seconds.

By second twenty, the user has mentally drafted a one-star review.

The answer, when it lands, is superb. But nobody cares anymore.

The Problem with Standard Responses

LanguageModel.generateText returns an effect that succeeds with the complete response. One value, delivered once, at the end. Until the final token is generated, our program holds nothing at all.

What we want instead is many small deliveries, starting immediately.

Enter the Stream

Where an Effect produces one value, a Stream produces any number of them, as they become available.

LanguageModel.streamText takes the same options as generateText but returns Stream.Stream<Response.StreamPart, ...>.

Claude begins responding immediately, and each piece of the response arrives as a separate part.

Understanding Stream Parts

Each part announces its role in a type field. The cast, in order of appearance:

• response-metadata — a new response is underway
• text-start — a chunk of text content is beginning
• text-delta — an increment of the actual generated text
• text-end — that chunk of text is complete
• finish — generation is done, with a finish reason and token usage

Of these, text-delta carries the treasure: its delta field holds the actual generated text our users are waiting for.

Watching the Raw Stream

Stream.runForEach consumes a stream, running an effect for each element:

const program = Effect.gen(function* () {
  yield* LanguageModel.streamText({ prompt }).pipe(
    Stream.runForEach((part) => Effect.log(part)),
  );
});

Run it, and a cataract of part objects. Metadata, starts, deltas, ends, a finish with its usage accounting. Our one requested sentence is in there, diced into fragments.

We only want the deltas.

Filtering Down to Text

With Stream.filter, it's the same as filtering an array. We keep only the text-delta parts, then use Stream.tap to print each one as it passes:

LanguageModel.streamText({ prompt }).pipe(
  Stream.filter((part) => part.type === "text-delta"),
  Stream.tap((part) => Effect.sync(() => process.stdout.write(part.delta))),
);

Past the filter, it knows every part is a TextDeltaPart. So part.delta typechecks in the tap without a single cast on our part.

The sentence now assembles itself on screen, word by word.

Getting the Complete Message

One loose end.

Streaming the chunks delights the user, but our application usually needs the complete text afterwards. For database storage, maybe for conversation history, or just for further processing.

Stream.runFold consumes the stream, threading an accumulator through every element. Since our tap sits upstream, each delta is printed on its way into the fold:

const program = Effect.gen(function* () {
  const fullText = yield* LanguageModel.streamText({ prompt }).pipe(
    Stream.filter((part) => part.type === "text-delta"),
    Stream.tap((part) => Effect.sync(() => process.stdout.write(part.delta))),
    Stream.runFold("", (text, part) => text + part.delta),
  );

  // fullText: the complete response, ready for storage
});

One pipeline, consumed once: the user watches the text arrive in real time, and the program ends up holding the assembled whole.

No one waits. Nothing is lost.

Repo

Code exercises set up here