Make z.preprocess defer optionality to inner schema#5929
Conversation
`z.preprocess(fn, schema)` desugared to `pipe(transform(fn), schema)`, so the resulting pipe's `optin` was inherited from `ZodTransform` (undefined) instead of from the inner schema. When the inner schema was itself optional and the preprocess sat as an object property, the object compiler took the `!isOptionalIn` branch and synthesized a `nonoptional` issue for missing keys — making the position of `.optional()` change presence semantics. Introduce `ZodPreprocess` as a structural subtype of `ZodPipe` (same `def.type === "pipe"`, `instanceof ZodPipe` still true). It pins `def.in` to a permissive `z.unknown()` and re-installs the four lazy metadata props so `optin`, `optout`, `values`, and `propValues` all defer to `def.out` (the inner schema). Backward direction throws, matching today's behavior. The two JSON-schema sites that special-cased the old `pipe(transform, ...)` shape (`pipeProcessor` and `isTransforming`) now also detect preprocess via `_zod.traits`.
|
TL;DR — Key changes
Summary ｜ 9 files ｜ 8 commits ｜ base: Presence vs. input-set delegation
The core constructor calls
JSON Schema awareness via traits
Documentation and type-level assignability tests
|
![pullfrog[bot]](https://ghub.bbstale.com/avatars/in/1768019?s=60&v=4){kind=small}
There was a problem hiding this comment.
Reviewed — no issues found.
Claude Opus ｜ 𝕏
`values` and `propValues` describe the *input* set a schema accepts.
Preprocess opens that set to anything `fn` can map to a B-accepted
value, so deferring these to B was unsound:
- discriminated unions use `propValues` as a fast-path disc map; with
B's propValues exposed, an option claims to match only B's literal
inputs and silently routes wrong (e.g. preprocess(toUpperCase,
z.literal("A")) would fail to match input "a"). Now reverts to
throwing at construction, matching pre-PR behavior.
- `z.record()` uses `values` to enumerate expected keys; with B's
values exposed it short-circuits before the preprocess fn ever runs
on input keys.
Presence semantics (`optin`/`optout`) describe whether the *outer*
container can omit this slot; preprocess is transparent to those, so
they continue to defer to B (the original #5917 fix).
There was a problem hiding this comment.
Important
Minor comment/code mismatch introduced by the follow-up commit.
- Stopped deferring
values/propValuesfrom B in the core$ZodPreprocessinit — onlyoptin/optoutare re-installed now - Added three tests covering discriminated-union, record-key, and direct
values/propValuesnon-propagation - Updated the block comment and interface to remove the
values/propValuestype overrides
Claude Opus ｜ 𝕏
| } | ||
|
|
||
| // ZodPreprocess | ||
| // Subtype of ZodPipe whose `optin`/`optout`/`values`/`propValues` defer to B |
There was a problem hiding this comment.
This comment still mentions values/propValues deferring to B, but the second commit deliberately stopped doing that. Should read something like:
// Subtype of ZodPipe whose `optin`/`optout` defer to B
Document the invariant that ZodPreprocess<B> is structurally assignable to ZodPipe<$ZodType, B>, and that the optin/optout override surfaces B's declared type (narrowed for schemas like ZodOptional that hardcode it, open `"optional" | undefined` for schemas like ZodString that inherit it).
There was a problem hiding this comment.
No new issues. Reviewed the following changes:
- Added type-level tests verifying
ZodPreprocess<B>assignability toZodPipe,optin/optoutnarrowing, and output/input inference
Prior review feedback:
- Comment on line 2365 of
classic/schemas.tsstill mentionsvalues/propValuesdeferring to B — not addressed
Claude Opus ｜ 𝕏
`pipeProcessor` was mixing detection styles: `traits.has("$ZodPreprocess")`
for the new subtype and `def.in._zod.def.type === "transform"` for the
legacy pipe(transform, ...) form. Both check "input contributes no
validation, use B for input-side JSON schema." Converge on traits.
There was a problem hiding this comment.
No new issues. Reviewed the following changes:
- Refactored
pipeProcessorto detect the legacypipe(transform, ...)form viadef.in._zod.traits.has("$ZodTransform")instead of inspectingdef.typedirectly — consistent with how$ZodPreprocessis already detected; added an explanatory comment
Prior review feedback:
- Comment on line 2365 of
classic/schemas.tsstill mentionsvalues/propValuesdeferring to B — not addressed
Claude Opus ｜ 𝕏
Same diagnosis as the preprocess case: `isTransforming` recurses into
`def.in` and `def.out` looking for `def.type === "transform"`, but
`ZodCodec` embeds an implicit transform fn (its `decode`/`encode`)
while `def.in` and `def.out` are validating schemas. The recursion
finds no transform and returns false even though the codec absolutely
is transforming, causing output-side examples/defaults to leak into
the input JSON schema.
Detect via `_zod.traits.has("$ZodCodec")` alongside the preprocess
check.
There was a problem hiding this comment.
No new issues. Reviewed the following changes:
- Added
$ZodCodectrait detection toisTransforming()so codecs are recognized as transforming — output-sideexamples/defaultmetadata is now correctly stripped from input JSON schemas - Added test verifying codec example stripping behavior for both
io: "input"andio: "output"
Prior review feedback:
- Comment on line 2365 of
classic/schemas.tsstill mentionsvalues/propValuesdeferring to B — not addressed
Claude Opus ｜ 𝕏
Surface ZodPreprocess everywhere the codebase enumerates first-party
types as a structural sibling of ZodCodec under ZodPipe:
- core.mdx: $ZodTypes union comment ("$ZodCodec and $ZodPreprocess
extend this") and the inheritance diagram
- assignability.test.ts: explicit `satisfies` checks confirming the
type-level relationship to $ZodPreprocess and $ZodPipe (with
generics specified — the bare-generic form bites variance because
inherited reverseTransform is contravariant in B)
The $ZodTypes union itself doesn't need a new entry — preprocess is
covered transitively via $ZodPipe, same as $ZodCodec — but documenting
the relationship in prose and tests makes the hierarchy discoverable.
There was a problem hiding this comment.
No new issues. Reviewed the following changes:
- Documented
$ZodPreprocessin the core.mdx inheritance hierarchy (comment update and tree entry) - Added type-level assignability tests verifying
ZodPreprocesssatisfies both$ZodPipeandZodPipewith the expected type parameters
Prior review feedback:
- Comment on line 2365 of
classic/schemas.tsstill mentionsvalues/propValuesdeferring to B — not addressed
Claude Opus ｜ 𝕏
The bug is purely a static-metadata problem: $ZodPipeInternals.optin
was inheriting from A (the leading transform, which has no optin)
instead of from B (the inner schema). The runtime parse path for
pipe(transform(fn), schema) was already correct.
So the simpler design is: keep the runtime structure exactly as the
legacy form (def.in is a real ZodTransform), and just override the
optin/optout lazies in a thin subtype.
What this eliminates from the previous design:
- Custom parse function in $ZodPreprocess.init (forward direction with
embedded transform fn)
- Custom parse function in classic ZodPreprocess.init (addIssue
injection — the inner ZodTransform's classic parse override already
does this)
- The synthetic z.unknown() input slot
- The required `transform` field on the def
- traits.has("$ZodPreprocess") check in pipeProcessor and
isTransforming — def.in is a real ZodTransform, so the existing
detection paths fire correctly
The traits.has("$ZodCodec") check in isTransforming stays — codec
embeds its transform fn directly on the def (different shape) and
needs its own detection.
There was a problem hiding this comment.
No new issues. Reviewed the following changes:
- Simplified
ZodPreprocessfrom a custom-parse-function subtype to a pure metadata-override subtype —def.inis now a realZodTransform(viatransform(fn)), and the pipe's inherited parse handles the flow - Removed the
def.transformfield andhandlePreprocessResulthelper since the transform logic now lives insidedef.in - Narrowed the
Atype parameter from$ZodTypeto$ZodTransformacross interfaces and the factory - Simplified
isTransformingto drop the$ZodPreprocesstrait check — recursion intodef.in(a real transform) detects it naturally - Updated
pipeProcessorto usedef.in._zod.traits.has("$ZodTransform")uniformly for both preprocess and legacypipe(transform, ...)forms - Updated type-level assignability tests and comments to reflect
$ZodTransformas the A parameter
Prior review feedback:
- Comment on line 2365 of
classic/schemas.tsstill mentionsvalues/propValuesdeferring to B — addressed (comment now correctly states onlyoptin/optout)
Claude Opus ｜ 𝕏
`z.object({ a: z.preprocess(fn, T) }).parse({})` worked in 4.3 (the fn
ran with `undefined`, produced a value, the inner schema validated it)
but started failing on absent keys after #5661 tightened the object
parser. Users commonly use preprocess to inject pre-parse defaults for
fields that may be missing — that pattern broke silently in 4.4.
Restore by marking $ZodPreprocess as `optin === "optional"`, telling
`$ZodObject` that absent keys are legal here. The fn then runs with
`undefined` exactly as it did in 4.3.
To preserve the long-stable behavior of `preprocess(fn, T).optional()
.parse(undefined)` returning `undefined` (true in both 4.3 and 4.4 for
multi-year compatibility), have `$ZodTransform` set the `fallback`
payload flag on every invocation. `$ZodOptional` already clobbers a
result with `fallback === true` when its input was `undefined`, so the
outer optional keeps short-circuiting to `undefined` even though the
transform now runs underneath.
z.object({ a: z.preprocess(v => v ?? "X", z.string()) }).parse({})
// 4.3: { a: "X" }
// 4.4: FAIL (regression)
// now: { a: "X" }
z.preprocess(v => v ?? "X", z.string()).optional().parse(undefined)
// 4.3 + 4.4: undefined
// now: undefined (preserved)
Drops the `optin` defer-to-inner from #5929, but the same outcome
holds: when inner is `.optional()`, preprocess still accepts absent
keys (`optin === "optional"` either way).
…n absent keys (#5941) * fix(v4): propagate fallback flag through pipe boundaries `$ZodCatch` sets a payload flag when its `catchValue` substitutes so an outer `$ZodOptional` can clobber the recovery value with `undefined` (per #5939). But `handlePipeResult` was building a fresh payload for the right side of the pipe without copying the flag, so any chain like `catch().transform()...optional()` lost it — `optional` couldn't tell the inner had recovered, and surfaced the catch value instead of clobbering. Propagate the flag through pipe handoffs, alongside `value`/`issues`. Also rename `caught` to `fallback`: a slightly broader name that describes the consumer contract ("override me if you have a better value when input was undefined") rather than the producer ("catch fired me"). Internal-only; no public API surface. * fix(v4): restore preprocess handling for absent object keys `z.object({ a: z.preprocess(fn, T) }).parse({})` worked in 4.3 (the fn ran with `undefined`, produced a value, the inner schema validated it) but started failing on absent keys after #5661 tightened the object parser. Users commonly use preprocess to inject pre-parse defaults for fields that may be missing — that pattern broke silently in 4.4. Restore by marking $ZodPreprocess as `optin === "optional"`, telling `$ZodObject` that absent keys are legal here. The fn then runs with `undefined` exactly as it did in 4.3. To preserve the long-stable behavior of `preprocess(fn, T).optional() .parse(undefined)` returning `undefined` (true in both 4.3 and 4.4 for multi-year compatibility), have `$ZodTransform` set the `fallback` payload flag on every invocation. `$ZodOptional` already clobbers a result with `fallback === true` when its input was `undefined`, so the outer optional keeps short-circuiting to `undefined` even though the transform now runs underneath. z.object({ a: z.preprocess(v => v ?? "X", z.string()) }).parse({}) // 4.3: { a: "X" } // 4.4: FAIL (regression) // now: { a: "X" } z.preprocess(v => v ?? "X", z.string()).optional().parse(undefined) // 4.3 + 4.4: undefined // now: undefined (preserved) Drops the `optin` defer-to-inner from #5929, but the same outcome holds: when inner is `.optional()`, preprocess still accepts absent keys (`optin === "optional"` either way). * fix(v4): generalize optin=optional from preprocess to transform Promotes the "user-written input handler accepts absence" signal from $ZodPreprocess to $ZodTransform. Any schema with a transform fn at its input boundary (preprocess, standalone z.transform) now declares optin="optional" at runtime. Effects: - preprocess inherits optin="optional" via pipe.optin = transform.optin (same outcome as the previous commit's explicit override; preprocess loses both its optin and optout overrides since pipe already does the optout defer) - standalone z.transform(fn) now accepts absent object keys - z.string().transform(fn): unchanged (pipe.optin = string.optin = undefined; transform on the OUT side doesn't drive optin) - z.unknown().transform(fn).pipe(A): unchanged (pipe.optin = unknown. optin = undefined) The static type stays unchanged — transform's interface doesn't declare optin, so this only sets the runtime value, mirroring the catch pattern. Captures the "flexible inputs, strict outputs" design principle: schemas with a user-written escape hatch (catch's recovery, transform's fn) accept undefined at runtime even when the static type declares the input as required. After this, $ZodPreprocess is a near-empty marker subtype — the constructor body is just $ZodPipe.init(inst, def), kept for type narrowing and traits identity. * docs(wiki): add internal reference for v4 optionality semantics Captures the current state of optin/optout/fallback, who sets each, who reads each, the static/runtime divergence pattern, and walked- through cases for the gnarly interactions (catch+optional, default vs catch vs preprocess vs transform under optional, etc.). Also documents the "flexible inputs, strict outputs" design principle that motivates the runtime/static optin divergence on $ZodCatch and $ZodTransform: schemas with a user-written escape hatch accept undefined at runtime while keeping their declared input type strict. Internal-only doc; not published. * docs(wiki): explain why unknown.transform.pipe stays strict Adds the explicit contrast between z.preprocess(fn, T) (= pipe(transform, T), accepts absent) and z.unknown().transform(fn).pipe(T) (= pipe(pipe( unknown, transform), T), rejects absent). The two look structurally similar but only the leading position drives optin, and z.unknown() isn't input-optional. Also drops the stale "prototype only" caveat from the standalone z.transform(fn) walkthrough — the runtime optin=optional move from preprocess to transform is now a real part of this branch, not a prototype.
1 participant
Fixes #5917.
z.preprocess(fn, schema)desugared topipe(transform(fn), schema), so the resulting pipe inheritedoptinfromZodTransform(undefined) instead of from the inner schema. When the inner schema was itself optional and the preprocess sat as an object property, the object compiler took the!isOptionalInbranch and synthesized anonoptionalissue for missing keys — making the position of.optional()change presence semantics:Introduce
ZodPreprocessas a structural subtype ofZodPipe(samedef.type === "pipe",instanceof ZodPipestill true). It pinsdef.into a permissivez.unknown()and re-installs the four lazy metadata props sooptin,optout,values, andpropValuesall defer todef.out(the inner schema). Forward direction skips the no-opdef.in.run()and embeds the user fn as the codec's decode; backward throws, matching today's behavior.The two JSON-schema sites that special-cased the old
pipe(transform, ...)shape (pipeProcessorandisTransforming) now also detect preprocess via_zod.traits.