Yuzhe's Blog

yuzhes

typed-pipeline v2 — A Type-Safe Asynchronous Pipeline Library

typed-pipeline v2.0 is not trying to make the API flashier; it is trying to place three often conflicting goals into one design: chainable readability, asynchronous composability, and TypeScript-level type safety.

In v1, I had already validated that the “pipeline + previous-step type inference” route is viable. The issue was the old implementation relied on too many clever structures, making both runtime and types difficult to follow. The v2 design decisions were therefore clear: runtime should read like a normal library, and the type system should only express semantics that truly exist, not add layer upon layer of utility types just to patch gaps.


Why return to a Pipeline class

The most important v2 decision was abandoning a purely functional assembler and returning to a class-based API: new Pipeline<TInput>().

The reason is pragmatic. A class aggregates three pieces of state in one instance—“current output type,” “saved snapshots,” and “pending steps”—and TypeScript can continue inference through the generic this during chain calls. By contrast, a pure functional pipe(a, b, c) is short, but once you need operations that change context like .saveAs(), .parallel(), or .tap(), type signatures quickly become unmaintainable.

The three-generic model Pipeline<TIn, TOut, TSaved> maps directly to actual runtime behavior:

class Pipeline<TInput, TOutput = TInput, TSaved = {}> {
  pipe<TNext>(step: Step<TOutput, TNext, TSaved>): Pipeline<TInput, TNext, TSaved>
  saveAs<TKey extends string>(key: TKey): Pipeline<TInput, TOutput, TSaved & Record<TKey, TOutput>>
  run(input: TInput): Promise<TOutput>
}

TInput is the external seed type, TOutput is the value at the chain tail, and TSaved is the named snapshots formed by all .saveAs() calls. The advantage is that every generic users see corresponds to something real at runtime; there are no “ghost” types existing only for type gymnastics.


AsyncPipeline is not a separate DSL

The second design decision is treating async as a default capability rather than an optional add-on. Real data-processing steps regularly mix sync computation, database queries, HTTP requests, and side effects. If sync and async pipelines were two separate APIs, users would eventually hit boundary friction.

So in v2, Pipeline steps return MaybePromise<T>, and AsyncPipeline is more like a semantic alias: it states “this chain is explicitly async-first,” while the underlying scheduling model is consistent with Pipeline, awaiting each step in sequence.

const p = new AsyncPipeline<UserId>()
  .pipe(id => loadUser(id))
  .pipe(async user => ({
    ...user,
    posts: await loadPosts(user.id),
  }))
  .pipe(user => user.posts.length)

const count = await p.run("u_42")

The key is not merely “supporting Promise.” It is to guarantee that each step sees Awaited<previous-step output> as its input type. In other words, async changes scheduling, not the mental model. Someone writing the third step does not need to care whether the second step is synchronous or async; they only need to know the final produced value.


The design of parallel

Many pipeline libraries support parallelism, but a common approach is reducing parallel results to unknown[] or broad array types. That keeps runtime working but loses information in the type layer. v2’s parallel takes a different path: it treats “parallel” as “run multiple substeps against the same input while preserving tuple shape of results.”

const p = new Pipeline<number>()
  .parallel(
    n => n + 1,
    async n => n * 2,
    n => `v=${n}`,
  )

const result = await p.run(5)
// [6, 10, "v=5"]

Here the return is not (number | string)[], but the precise tuple [number, number, string]. In implementation, parallel uses the current TOutput as unified input for each branch, then obtains outputs through variadic tuple mapping:

type ParallelResult<TIn, TSteps extends readonly AnyStep[]> = {
  [K in keyof TSteps]: StepOutput<TSteps[K], TIn>
}

At runtime it is just one Promise.all. This matches v2’s overall style: contain complexity in places that directly affect user value, not where unnecessary. Concurrency scheduling itself does not need to be complicated; what matters is preserving exact branch result shapes, because later steps often destructure that tuple immediately.


saveAs semantics: saving snapshots, not aliases

saveAs is the most misunderstood and the most important API in v2. Its meaning is not “giving the current value an alias,” but “adding the current step output snapshot to saved context so later steps can read it explicitly.”

const p = new Pipeline<number>()
  .pipe(n => n * 2)
  .saveAs("doubled")
  .pipe(n => n + 3)
  .pipe((current, $) => current + $.doubled)

Here $.doubled is always the saved value 10, not “some reference that changes with current.” If saveAs were aliasing, optimizations, parallel steps, or repeated runs could easily create semantic ambiguity: is the read from the current chain position or the historical node?

After adopting snapshot semantics, the behavior becomes explicit:

Then TSaved naturally follows. After saveAs("doubled"), $ in later steps has type { doubled: number }; save another one as "formatted", and the type extends to { doubled: number; formatted: string }. The type system is describing exactly the snapshot set of history.


A complete example

When these designs are assembled, v2 usage looks like this:

const pipeline = new AsyncPipeline<string>()
  .pipe(id => loadUser(id))
  .saveAs("user")
  .parallel(
    user => loadProfile(user.id),
    user => loadTeams(user.id),
  )
  .pipe(async ([profile, teams], $) => ({
    id: $.user.id,
    profile,
    teams,
  }))

In this code, the class carries context; async steps chain naturally; parallel preserves [Profile, Team[]] tuple shape; and saveAs gives stable historical snapshots. The four features are not bolted together awkwardly, but all unfold around a single state model.


Conclusion

What I like most about v2 is not “more features,” but that each feature now has stable boundaries. Pipeline owns chain state; AsyncPipeline owns the async mental model; parallel handles multi-branch with one input; saveAs provides stable historical snapshots. Together, they behave like a library rather than a set of clever but loosely coupled tricks.

For TypeScript libraries, this matters more than adding flashy APIs: when runtime semantics are clear, the type system can stay honest more easily.