The parser: tokens to expressions
Recursive descent with one function per precedence level (Esm.Document.Math.Print
lists the levels), each taking fuel. The +, *, and and or chains are read
into one n-ary node; binary -, the comparisons and the Typst fraction / group to
the left; unary - takes a whole product (-a b is -(a b), spec §4.1.1); ^ takes
a postfix-level base and one exponent.
Accepted variants are read here too (the printer never writes them), each with a
lemma in Esm.Document.Math.Variants
.
Small pieces
def close (o : String) : List Json → Json
Close an n-ary chain: one operand is itself, more make a node.
def atomStart : Tok → Bool
Tokens that can start an operand of a product: juxtaposition continues a product when the next token is one of these.
def isLp : Tok → Bool
An opening parenthesis: ( or \left(; the parser reads them alike.
Names
def decodeFormula (raw : List Char) : List Char
The name a formula token spells, if it is formula-shaped.
def subOrPlain (sf : Sf) (b : List Char) (r : List Tok) : List Char × List Tok
A single letter or Greek base, then an optional upright name subscript.
def pNameLoop (sf : Sf) : Nat → List Char → List Tok → Option (List Char × List Tok)
A name: segments joined by the scope dot.
JSON inside LaTeX escape nodes
Negative literals after an operand
def negTail : List Tok → Option (List Tok)
A negative literal right after an operand (k-1, k -1) is a binary minus and a
positive number: the tokens with the sign moved off the number, when the number without
its sign has a canonical form (the negation of -2^63 has none, so that spelling is
refused after an operand).
Expressions
def mkNodeP (o : String) (as : List Json) (fs : List (String × Json)) : Json
Rebuild a node with fields: op and args go back to their sorted places.
inductive CallArg where
| pos (v : Json)
| named (k : String) (v : Json)
What pCallArgs
read: positional arguments, or named ones.
def sortKV : List (String × Json) → List (String × Json)
Sort bindings by key (insertion).
def unitsOfRaw (raw : String) : Option (List Esm.Units.Factor)
The unit a LaTeX \mathrm{…} after a number spells.
def tNode (n : String) (bs : List (String × Json)) : Json
A template application node.
def constNode (v : Json) : Option String → Json
A constant node: the value, and the units when there are some.
def pCells (sep : Kw) : List Tok → Option (List JNum × List Tok)
Numbers separated by sep, up to the first other token.
def pGrid (csep rsep close : Kw) : Nat → List Tok → Option (List (List JNum) × List Tok)
Rows of numbers: cells separated by csep, rows by rsep, closed by close
.
def gridJson (rows : List (List JNum)) : Json
A LaTeX bmatrix: one cell per row is a column (rank 1).
def constTail (v : Json) (r : List Tok) : Option (Json × List Tok)
The unit after an array, if any.
def jsonStart : List Tok → Bool
A LaTeX field value that is JSON starts with {, [, a string or a word.
def keyOf (c : Ctx) (ts : List Tok) : Option (String × List Tok)
A field key: LaTeX \mathrm{key} =, Typst upright("key") =.
def dCallNode (w : String) (xs : List Json) : Json
A spatial derivative with trailing operands.
def eulerHead (c : Ctx) : List Tok → Bool
The input starts with Euler’s number raised to a power: an upright e, or an
italic e when no variable e is declared.
def pNamePrim (c : Ctx) (ts : List Tok) : Option (Json × List Tok)
A name, or an initial condition N(t_{0}).
def pWord (c : Ctx) (w : String) (r : List Tok) : Option (Json × List Tok)
A bare run of letters (RT): the declared name when only it is declared, the
product of its letters when only they are, refused otherwise (ambiguous or unknown).
def dName (c : Ctx) (k : Kw) : List Tok → Option (String × List Tok)
d then a name, up to a closing token k: dN} (a run of letters that is not
itself a declared name) or d N}.
def pItalicD (c : Ctx) (r : List Tok) : Option (Json × List Tok)
The italic-d derivative variant, when no variable d is declared and neither
dN nor dt written as one word is a declared name (a declared whole word wins, as in
pWord
: \frac{dx}{dt} with dx and dt declared is their quotient): LaTeX
\frac{dN}{dt} after \frac{, Typst (d N)/(d t) after the first (.
inductive Row where
| rng (k : String) (v : Json)
| filt (x : Json)
A row of a reduction’s subscript, or an output entry: a range, or an expression (the filter, or a bare output entry).
def pRedHead (sf : Sf) : List Tok → Option (RedK × Bool × List Tok)
The operator of a reduction and whether it is marked (args listed): LaTeX
\sum\limits, Typst limits(sum).
def pNames (sf : Sf) : Nat → List Tok → Option (List String × List Tok)
Names separated by commas.
def rowParts : List Row → Option (List (String × Json) × Option Json)
The ranges and the filter of a list of rows: ranges, then at most one filter, last.
def entryParts : List Row → Option (List Json × List (String × Json))
The output entries of a list of entry rows, with their ranges.
def withOut (F : FaqForm) (outs : List Json) (ors : List (String × Json)) : FaqForm
The node an output form reads: the reduction’s parts with the output entries.
def withSup (F : FaqForm) (st : List String) : FaqForm
The superscript set read after the output entries (the reduction inside the bracket
carries none, bracketSetOK
).
def bracketSetOK (F : FaqForm) (st : List String) : Bool
The sets of an output form as the printer writes them: none on the reduction inside
the bracket (the printer writes the set on the bracket), and none at all when the
operator is marked (args lists the body’s arrays).
def pSup (sf : Sf) (r : List Tok) : Option (List String × List Tok)
The superscript set of a reduction or of an output form, if there is one: LaTeX
^{\{f, g\}}, Typst ^({f, g}).
def mkTail (sf : Sf) (rows : List (Json × List (Json × Json))) (r : List Tok) : Option (Json × List Tok)
The makearray node of a table read back, with its subscript of output indices if
one follows.
def pAnd (c : Ctx) : Nat → List Tok → Option (Json × List Tok)
Level 2: an and chain.
def pCmp (c : Ctx) : Nat → List Tok → Option (Json × List Tok)
Level 3: one comparison (comparisons do not chain: a \le k \le b is a range row of a
reduction, never an expression).
def pAdd (c : Ctx) : Nat → List Tok → Option (Json × List Tok)
Level 4: + chains and binary -, including a negative literal right after an
operand (negTail
). x :: acc is the pending + chain.
def pMul (c : Ctx) : Nat → List Tok → Option (Json × List Tok)
Level 5: unary minus, and * chains (juxtaposition, \cdot, \times; LaTeX inline
/ groups to the left). A negative literal never continues a product by juxtaposition
(k -1 is k - 1); after \cdot it is a factor (k \cdot -1).
def pFrac (c : Ctx) : Nat → List Tok → Option (Json × List Tok)
Level 7: the Typst fraction, grouping to the left.
def pExpo (c : Ctx) : Nat → List Tok → Option (Json × List Tok)
An exponent: LaTeX {…}; Typst (…) or one simple token.
def pPost (c : Ctx) : Nat → List Tok → Option (Json × List Tok)
Level 9: postfix index, _{…} or […].
def pList (c : Ctx) : Nat → Nat → List Tok → Option (List Json × List Tok)
A comma-separated list of expressions at level L (1 or 4), possibly empty when
the next token closes it.
def pLv (c : Ctx) : Nat → Nat → List Tok → Option (Json × List Tok)
Parse at level 1 or level 4.
def pCallArgs (c : Ctx) : Nat → List Tok → Option (List CallArg × List Tok)
Call arguments at level 4: each positional, or name = value.
def pFields (c : Ctx) : Nat → List Tok → Option (List (String × Json) × List Tok)
Open-operator fields after ;: key = value, the value as math or as JSON.
def pVal (c : Ctx) : Nat → List Tok → Option (Json × List Tok)
A field value: JSON (LaTeX JSON tokens, Typst #value) or an expression.
def pOpn (c : Ctx) : Nat → String → List Tok → Option (Json × List Tok)
\operatorname{n}(…): a call-form core operator, a deprecated one (rewritten), a
template (by the symbol table, positional or named), or an open operator.
def pDCall (c : Ctx) : Nat → List Tok → Option (Json × List Tok)
\operatorname{D}_{x}(u, a, b) after the _.
def pEulerExp (c : Ctx) : Nat → List Tok → Option (Json × List Tok)
\exp written as a power of e.
def pRange (c : Ctx) : Nat → List Tok → Option (Row × List Tok)
A range row: k \in S, k \in S(i, j), a \le k \le b.
def pRow (c : Ctx) : Nat → (Tok → Bool) → List Tok → Option (Row × List Tok)
A row ending at a token isEnd accepts: an expression read whole first (it never
reads a range row whole: a range row has \in or a second \le), else a range.
def pRows (c : Ctx) : Nat → Kw → Kw → List Tok → Option (List Row × List Tok)
Rows separated by sep, up to close
(consumed).
def pRedBody (c : Ctx) : Nat → RedK → Bool → List Tok → Option (FaqForm × List Tok)
A reduction after its operator: the subscript rows (LaTeX: one row, a lone dense
range k=a with ^{b}, or \substack; Typst: rows in _( … )), then the body.
def pRed (c : Ctx) : Nat → List Tok → Option (Json × List Tok)
A rank-0 reduction, or a reduction wrapped with its output entries.
def pFn1 (c : Ctx) : Nat → String → List Tok → Option (Json × List Tok)
f(x) after the opening parenthesis.
def pFracL (c : Ctx) : Nat → List Tok → Option (Json × List Tok)
After \frac{: a time or spatial derivative, the italic-d variant, or a quotient.
def pDT (c : Ctx) : Nat → Kw → List Tok → Option (Json × List Tok)
(dif N)/(dif t) and (partial u)/(partial x) after the first (dif / (partial.
def pDApply (c : Ctx) : Nat → String → List Tok → Option (Json × List Tok)
A derivative operator applied to a parenthesized operand.
def pCases (c : Ctx) : Nat → List Tok → Option (Json × List Tok)
cases rows: value & if cond then a row break, ending with value & otherwise;
a chain re-nests to the right.
def pPair (c : Ctx) : Nat → List Tok → Option ((Json × Json) × List Tok)
A bound pair after its [: a, b].
def pRegTail (c : Ctx) : Nat → List Tok → Option (List (Json × Json) × List Tok)
The further bound pairs of a region, each after \times [.
def pMkRows (c : Ctx) : Nat → Kw → Kw → Kw → List Tok → Option (List (Json × List (Json × Json)) × List Tok)
The rows of a makearray table: a value, the cell separator csep, a region; rows
separated by rsep, up to close
(consumed).