^h8SrSSKJr SSKJrJrJrJrJrJ r SSK J r J r \"S5r "SS\\ 5r"SS 5r"S S \\ 5r"S S 5rSrSrSrSrS\S\4SjrSrSrSrg)a This module defines the data structures used to represent a grammar. Specifying grammars in pgen is possible with this grammar:: grammar: (NEWLINE | rule)* ENDMARKER rule: NAME ':' rhs NEWLINE rhs: items ('|' items)* items: item+ item: '[' rhs ']' | atom ['+' | '*'] atom: '(' rhs ')' | NAME | STRING This grammar is self-referencing. This parser generator (pgen2) was created by Guido Rossum and used for lib2to3. Most of the code has been refactored to make it more Pythonic. Since this was a "copy" of the CPython Parser parser "pgen", there was some work needed to make it more readable. It should also be slightly faster than the original pgen2, because we made some optimizations. ) literal_eval)TypeVarGenericMappingSequenceSetUnion) GrammarParserNFAState _TokenTypeTcH\rSrSrSrS\S\\\S4S\\S44SjrS r g ) Grammar%z Once initialized, this class supplies the grammar tables for the parsing engine implemented by parse.py. The parsing engine accesses the instance variables directly. The only important part in this parsers are dfas and transitions between dfas. start_nonterminal rule_to_dfaszDFAState[_TokenTypeT]reserved_syntax_stringsReservedStringc(X lX0lXlgNnonterminal_to_dfasrr)selfrrrs O/home/james-whalen/.local/lib/python3.13/site-packages/parso/pgen2/generator.py__init__Grammar.__init__/s$0 '>$!2rN) __name__ __module__ __qualname____firstlineno____doc__strrrr__static_attributes__rrrr%sA3$'3&sH5L,M'MN3+2#7G2G*H3rrc<\rSrSrSr/4SSS\S4SjjrSrSrg ) DFAPlan8z^ Plans are used for the parser to create stack nodes and do the proper DFA state transitions. next_dfaDFAState dfa_pushescXlX lgr)r(r*)rr(r*s rrDFAPlan.__init__=s  $rcnURR<SUR<SUR<S3$)N(z, )) __class__rr(r*rs r__repr__DFAPlan.__repr__As!#~~66 tWWr)r*r(N) rrrr r!rrr2r#r$rrr&r&8s,QS%%*9M%Xrr&cL\rSrSrSrS\S\\S\4SjrSr Sr S r S r S r g ) r)EaE The DFAState object is the core class for pretty much anything. DFAState are the vertices of an ordered graph while arcs and transitions are the edges. Arcs are the initial edges, where most DFAStates are not connected and transitions are then calculated to connect the DFA state machines that have different nonterminals. from_rulenfa_setfinalc[U[5(de[[[U55[5(de[U[5(deXlX l0Ul0Ul0Ul X2;Ul gr) isinstancesetnextiterr r6r7arcsnonterminal_arcs transitionsis_final)rr6r7r8s rrDFAState.__init__Osu'3''''$tG}-x8888%****" ,. 9;RT( rc[U[5(deX R;de[U[5(deXRU'gr)r:r"r>r))rnext_labels radd_arcDFAState.add_arc`sD%%%%%II%%%%**** %rcrURR5Hup4XALdM X RU'M gr)r>items)roldnewrErDs r unifystateDFAState.unifystatefs* IIOO-LE|#& % .rcF[U[5(deURUR:wag[UR5[UR5:wagURR 5H$up#X1RR U5LdM$ g g)NFT)r:r)rAlenr>rIget)rotherrErDs r__eq__DFAState.__eq__ksw%**** ==ENN * tyy>S_ , IIOO-LEJJNN511.rcpSURR<SUR<SUR<S3$)N)r0rr6rAr1s rr2DFAState.__repr__ys% NN # #T^^T]]  r)r>r6rAr7r?r@N)rrrr r!r"rr rrFrLrRr2r#r$rrr)r)Es9)#)H )h)"! '  rr)c,\rSrSrSrS\4SjrSrSrg)rz Most grammars will have certain keywords and operators that are mentioned in the grammar as strings (e.g. "if") and not token types (e.g. NUMBER). This class basically is the former. valuecXlgrrZ)rrZs rrReservedString.__init__s rcRURR<SUR<S3$)Nr.r/)r0rrZr1s rr2ReservedString.__repr__s>>22DJJ??rr\N) rrrr r!r"rr2r#r$rrrrs c@rrcSnU(aiSn[U5HNup#[US-[U55H-nXnX5:XdMX UHnURXS5 M Sn ML MP U(aMhgg)z This is not theoretically optimal, but works well enough. Algorithm: repeatedly look for two states that have the same set of arcs (same labels pointing to the same nodes) and unify them, until things stop changing. dfas is a list of DFAState instances TFN) enumeraterangerOrL)dfaschangesistate_ijstate_jstates r_simplify_dfasrkssG #D/JA1q5#d),'%!%((:"&"G-* 'rc^ [U[5(de[U[5(deU 4Sjm [5nT "X5 [URX!5/nUHn0nUR H\nUR HInURcMURUR[55nT "URU5 MK M^ UR5HXupUHn U R U:XdM O) [URX5n URU 5 URX5 MZ M U$)z Uses the powerset construction algorithm to create DFA states from sets of NFA states. Also does state reduction if some states are not needed. c>[U[5(deX;agURU5 URH%nURbMT"UR U5 M' gr)r:r addr>nonterminal_or_stringr<) nfa_state base_nfa_setnfa_arc addclosures rrs_make_dfas..addclosuresU)X....  $ # ~~G,,47<<6&r) r:r r;r)r6r7r>ro setdefaultr<rIappendrF) startfinishrqstatesrjr>rprrr7ro nested_staterss @r _make_dfasr{s eX & && & fh ' '' '75Lu#u = >FI$>>00<"oog.K.KSUSGw||W5*'/3jjl * ! & ''72!' (I  l+ MM, >/;, Mrc[SUR5 U/n[U5Hup4[SX4UL=(a S=(d S5 URHonURUR pvXr;aUR U5nO[U5nURU5 Uc[SU-5 M`[SXh4-5 Mq M g)NzDump of NFA for State(final)z -> %d %s -> %d) printr6rbr>ror<indexrOrv) rwrxtodorfrjarcrErDrhs r _dump_nfars U__- 7DdO iVO9 ?R@::C44chh5}JJu%I E"}kAo&nz12$rc 0[SUSR5 [U5Hnup[SXR=(a S=(d S5 URR 5H#up4[SX0R U54-5 M% Mp g)NzDump of DFA forrr}r~rr)rr6rbrAr>rIr)rdrfrj nonterminalrDs r _dump_dfasrsr T!W../dO iNN8y>B?"'**"2"2"4 K .KE1B#CC D#5$r bnf_grammarreturnc0nSn[U5R5H:upE[XE5n[U5 XbUR'UbM.URnM< 0nUR 5HkupUH`n U R R 5H?upX;aXRU 'M[UUU 5n [U 5U RU 'MA Mb Mm [U5 [X2U5$)z ``bnf_text`` is a grammar in extended BNF (using * for repetition, + for at-least-once repetition, [] for optional parts, | for alternatives and () for grouping). It's not EBNF according to ISO/IEC 14977. It's a dialect Python uses in its own parser. N) r parser{rkr6rIr>r?_make_transitionr&r@_calculate_tree_traversalr) rtoken_namespacerrnfa_anfa_zrdreserved_stringsr dfa_stateterminal_or_nonterminalr( transitions rgenerate_grammarrsL%k288: %' t(,U__%  $ % ;68)//1 I5>^^5I5I5K1'*:JR../FG!1'(/"J 9@8II))*56L2l+ $4D EErc&USR5(a [X5$USS;dU5eURS5(dURS5(ae[U5nX$![a [ U5=oAU'Us$f=f)z Creates a reserved string ("if", "for", "*", ...) or returns the token type (NUMBER, STRING, ...) for a given grammar terminal. r)"'z"""z''')isalphagetattr startswithrKeyErrorr)rrrErZrs rrrs  Qx..Qx:%,u,%##E**53C3CE3J3JJJU# *1 1 1?1F FA.H s.A22BBc 0n[UR55nUR5 UHnX1;dM [XU5 M UR 5HnUHnUR nUR R5Hup7XR5HupX;aXhn [U R(aU RSROU RRU (aU SRO UR/5n [SURU4[U 5--5e[Xy5Xh'M M M M g)z By this point we know how dfas can move around within a stack node, but we don't know how we can add a new stack node (nonterminal transitions). rzZRule %s is ambiguous; given a %s token, we can't determine if we should evaluate %s or %s.N)listkeyssort_calculate_first_plansvaluesr@r?rIsortedr*r6r( ValueErrortupler&) r first_plans nonterminalsrrdrr@r(rpushes prev_planchoicess rrr.s[ K+0023L#  ) "#6[ Q$$**,I#//K)2)C)C)I)I)K% *5*B*H*H*J&J!0$/$; #)$-#7#7!* 4 4Q 7 A A%.%7%7%A%A$*!'q 3 3/7/A/A * #)N%.$7$7$.!"%*'N!3  /6h.GK+5+K*L-rc~Xn0nSX'USnURR5HupgUR/XF'M URR5H=upXn U c[ SU-5eU R5H upU/U -XJ'M M? XAU'U$![ a [ XU5n NCf=f)z Calculates the first plan in the first_plans dictionary for every given nonterminal. This is going to be used to know when to create stack nodes. Nrzleft recursion for rule %r)r@rIr(r?rrr) rrrrdnew_first_plansrjrrD nonterminal2 first_plans2trs rrr`s  +DO#K GE"..446 ',~~&6#7 %55;;=  M&4L# != !KLL$&++-IA"'6!1O . > /   b12ET`aL bs"B##B<;B<N)r!astrtypingrrrrrr parso.pgen2.grammar_parserr r r rr&r)rrkr{rrr"rrrrr$rrrs*BB>m$ 3gk"3& X X7 w{#7 t @ @./d3$E'F#'F7'FT(/Hdr