Skip to main content

Module expr

Module expr 

Source
Expand description

Python expression parsing with ast.unparse-normalized output.

Sphinx renders parameter defaults and (pieces of) annotations by running them through ast.parse + ast.unparse (sphinx/util/inspect.py routes them through ast_unparse). parse_py_expr + unparse reproduce that round trip byte-for-byte for the expression subset the py domain needs; anything outside the subset — lambda, comprehensions, f-strings, walrus, await, yield, conditional expressions, comparisons, slices, **kwargs in calls, complex literals — is a PyExprError, which routes callers into the same fallback paths Sphinx takes when ast.parse raises SyntaxError.

Sphinx does not use one parse mode everywhere, so neither do we:

  • parse_py_expr is ast.parse(s, mode='eval') — the mode reached through signature_from_str’s def func(...) wrapper, where a default value sits in an expression slot. A top-level *a is a SyntaxError there.
  • parse_py_expr_stmt is plain ast.parse(s) (exec), the mode _parse_annotation uses (_annotations.py:232, type_comments=True). A top-level Starred is a legal Expr statement there, so PEP 646 *Ts / *tuple[int, ...] annotations parse, and an empty (or blank) source is Module(body=[]) rather than an error.
  • parse_py_star_annotation is CPython’s star_annotation production ('*' bitwise_or | expression), the only grammar slot where *args: *Ts is legal — i.e. the annotation of a var-positional parameter, which signature_from_str hands to Sphinx as '*Ts'.

Every normalization rule implemented here is pinned by the unit-test oracle battery below, generated with the REAL pinned toolchain (uv run --python 3.12 --with 'sphinx==9.1.0' --with 'docutils==0.22.4'), never from memory. The paren-placement logic is a faithful port of CPython 3.12 ast._Unparser (Lib/ast.py): its _Precedence table, require_parens, items_view, and the per-node visitors for the subset.

Known, documented divergences (all vanishingly rare in signatures, and all conservative — we return Err and the caller falls back, we never print something different from ast.unparse):

  • \N{...} escapes need the Unicode name database (no new deps) → Err.
  • Lone-surrogate escapes ('\ud800') cannot live in a Rust String → Err.
  • Identifier characters are approximated as char::is_alphabetic + _ (start) and additionally Nd digits via the wave-3 [crate::rst::digits] tables (continue), instead of exact XID_Start/XID_Continue; identifiers are NFKC-normalized like CPython’s tokenizer.
  • repr()’s “printable” test for exotic non-ASCII characters inside string constants uses char::is_control plus a curated Zs/Zl/Zp/Cf/Co table rather than full Unicode category data. A full-codespace sweep against the pinned interpreter shows the table never over-escapes, and the only remaining under-escape class is the unassigned (Cn) code points, which render unescaped where CPython would escape them — a sanctioned, documented divergence.
  • Exec mode only ever yields ONE statement here: a source holding several (int;, int\nstr — Module(body=[Expr, Expr]), which Sphinx’s functools.reduce over node.body renders as the concatenation of both) is Err, so the caller falls back to a single whole-text xref where Sphinx renders each statement.
  • [MAX_DEPTH] is a whole-expression complexity budget, not a nesting depth: trailers (a.b.c, f(x)(y), m[i][j]) and binop folds each charge it too, so a FLAT chain of roughly 200 operations is Err where CPython parses it. Err-side and conservative like the rest — the caller falls back — and no realistic signature comes near it.

Structs§

PyExprError
Opaque parse error; callers only branch on Err (the message exists for diagnostics and tests, not for dispatch).

Enums§

PyBoolOp
ast.boolop.
PyConst
ast.Constant values. Numeric payloads are kept as normalized strings (what repr(value) prints), so arbitrary-precision ints survive and floats carry Python’s shortest-repr text.
PyExpr
A parsed Python expression — the subset of ast.expr the py domain walks (task-3 brief). Negative literals are UnaryOp(USub, Constant), exactly as in ast; dotted names are Attribute chains.
PyOp
ast.BinOp operators, i.e. every Python binary operator that is an ast.operator (comparisons are a different node kind and stay out of scope; and/or are PyBoolOp).
PyUnaryOp
ast.unaryop.

Functions§

parse_py_expr
Parse a whole string as one Python expression (ast.parse(s, mode='eval') for the supported subset). Trailing garbage is an error; so is anything outside the subset. Never panics.
parse_py_expr_stmt
Parse a whole string the way _parse_annotation does — plain ast.parse(s), i.e. exec mode (_annotations.py:232).
parse_py_star_annotation
Parse the annotation of a var-positional parameter — CPython’s star_annotation production ('*' bitwise_or | expression), the only place a PEP 646 unpack may appear in a signature.
unparse
Render an expression exactly as CPython 3.12 ast.unparse would render the equivalent ast tree (a port of ast._Unparser for the subset).