Expand description
Python expression parsing with ast.unparse-normalized output.
Sphinx renders parameter defaults and (pieces of) annotations by running
them through ast.parse + ast.unparse (sphinx/util/inspect.py routes
them through ast_unparse). parse_py_expr + unparse reproduce that
round trip byte-for-byte for the expression subset the py domain needs;
anything outside the subset — lambda, comprehensions, f-strings,
walrus, await, yield, conditional expressions, comparisons, slices,
**kwargs in calls, complex literals — is a PyExprError, which
routes callers into the same fallback paths Sphinx takes when
ast.parse raises SyntaxError.
Sphinx does not use one parse mode everywhere, so neither do we:
parse_py_exprisast.parse(s, mode='eval')— the mode reached throughsignature_from_str’sdef func(...)wrapper, where a default value sits in an expression slot. A top-level*ais aSyntaxErrorthere.parse_py_expr_stmtis plainast.parse(s)(exec), the mode_parse_annotationuses (_annotations.py:232,type_comments=True). A top-levelStarredis a legalExprstatement there, so PEP 646*Ts/*tuple[int, ...]annotations parse, and an empty (or blank) source isModule(body=[])rather than an error.parse_py_star_annotationis CPython’sstar_annotationproduction ('*' bitwise_or | expression), the only grammar slot where*args: *Tsis legal — i.e. the annotation of a var-positional parameter, whichsignature_from_strhands to Sphinx as'*Ts'.
Every normalization rule implemented here is pinned by the unit-test
oracle battery below, generated with the REAL pinned toolchain
(uv run --python 3.12 --with 'sphinx==9.1.0' --with 'docutils==0.22.4'),
never from memory. The paren-placement logic is a faithful port of
CPython 3.12 ast._Unparser (Lib/ast.py): its _Precedence table,
require_parens, items_view, and the per-node visitors for the subset.
Known, documented divergences (all vanishingly rare in signatures, and
all conservative — we return Err and the caller falls back, we never
print something different from ast.unparse):
\N{...}escapes need the Unicode name database (no new deps) →Err.- Lone-surrogate escapes (
'\ud800') cannot live in a RustString→Err. - Identifier characters are approximated as
char::is_alphabetic+_(start) and additionally Nd digits via the wave-3 [crate::rst::digits] tables (continue), instead of exactXID_Start/XID_Continue; identifiers are NFKC-normalized like CPython’s tokenizer. repr()’s “printable” test for exotic non-ASCII characters inside string constants useschar::is_controlplus a curated Zs/Zl/Zp/Cf/Co table rather than full Unicode category data. A full-codespace sweep against the pinned interpreter shows the table never over-escapes, and the only remaining under-escape class is the unassigned (Cn) code points, which render unescaped where CPython would escape them — a sanctioned, documented divergence.- Exec mode only ever yields ONE statement here: a source holding
several (
int;,int\nstr—Module(body=[Expr, Expr]), which Sphinx’sfunctools.reduceovernode.bodyrenders as the concatenation of both) isErr, so the caller falls back to a single whole-text xref where Sphinx renders each statement. - [
MAX_DEPTH] is a whole-expression complexity budget, not a nesting depth: trailers (a.b.c,f(x)(y),m[i][j]) and binop folds each charge it too, so a FLAT chain of roughly 200 operations isErrwhere CPython parses it. Err-side and conservative like the rest — the caller falls back — and no realistic signature comes near it.
Structs§
- PyExpr
Error - Opaque parse error; callers only branch on
Err(the message exists for diagnostics and tests, not for dispatch).
Enums§
- PyBool
Op ast.boolop.- PyConst
ast.Constantvalues. Numeric payloads are kept as normalized strings (whatrepr(value)prints), so arbitrary-precision ints survive and floats carry Python’s shortest-repr text.- PyExpr
- A parsed Python expression — the subset of
ast.exprthe py domain walks (task-3 brief). Negative literals areUnaryOp(USub, Constant), exactly as inast; dotted names areAttributechains. - PyOp
ast.BinOpoperators, i.e. every Python binary operator that is anast.operator(comparisons are a different node kind and stay out of scope;and/orarePyBoolOp).- PyUnary
Op ast.unaryop.
Functions§
- parse_
py_ expr - Parse a whole string as one Python expression (
ast.parse(s, mode='eval')for the supported subset). Trailing garbage is an error; so is anything outside the subset. Never panics. - parse_
py_ expr_ stmt - Parse a whole string the way
_parse_annotationdoes — plainast.parse(s), i.e. exec mode (_annotations.py:232). - parse_
py_ star_ annotation - Parse the annotation of a var-positional parameter — CPython’s
star_annotationproduction ('*' bitwise_or | expression), the only place a PEP 646 unpack may appear in a signature. - unparse
- Render an expression exactly as CPython 3.12
ast.unparsewould render the equivalentasttree (a port ofast._Unparserfor the subset).