hA SSKrSSKrSSKrSSKrSSKrSSKrSSKJr SSKJ r /\R"S\RR5S9P\R"S\RR5S9P\R"S\RR5S9P\R"S\RR5S9PS P\R"S \RR5S9P\R"S \RR5S9P\R"S \RR5S9P\R"S \RR5S9P\R"S\RR5S9PSPSP\R"S\RR5S9P\R"S\RR5S9P\R"S\RR5S9P\R"S\RR5S9PSP\R"S\RR5S9P\R"S\RR5S9P\R"S\RR5S9P\R"S\RR5S9PSP\R"S\RR5S9P\R"S\RR5S9P\R"S\RR5S9P\R"S\RR5S9P\R"S\RR5S9P\R"S\RR5S9P\R"S \RR5S9P\R"S!\RR5S9PS"P\R"S#\RR5S9P\R"S$\RR5S9P\R"S%\RR5S9P\R"S&\RR5S9P\R"S'\RR5S9P\R"S(\RR5S9P\R"S)\RR5S9P\R"S*\RR5S9P\R"S+\RR5S9P\R"S,\RR5S9P\R"S-\RR5S9P\R"S.\RR5S9P\R"S/\RR5S9P\R"S0\RR5S9P\R"S1\RR5S9Pr \RRS2\ 5S35rS4rS5r\R$R&S=S6\R$R(S7\S8\4S9jj5r\RR0\RRS2\ 5\R2"\"5S:9S2\S;\S8S4S<j555rg)>N) Tokenizer)get_lang_classfr)marksafarbgbnbocacsdadeelenesetfafigahehihrhuiditknlbltlvnbnlplptrosiskslsqsrsvtatetltrtturkmrlangcb[U5"5Rn[R"SUS35nURHanU"U5Vs/sH!oDR (aMUR PM# nnURU5Vs/sHoDSPM nnXV:XaMae gs snfs snf)Nz spacy.lang.z .examples)r tokenizerpytest importorskip sentencesis_spacetextexplain)r3r6examplessentencettokens debug_tokenss \/home/james-whalen/.local/lib/python3.13/site-packages/spacy/tests/tokenizer/test_explain.pytest_tokenizer_explainrCAst$&00I""[i#@AH&&"+H"5H"5QZZ&!&&"5H&/&7&7&AB&A!&A B%%%'HBsB'&B' B,cT[R"S5n[R"S5nSSS0/0n[UUURURS9nU"S5Vs/sHoUR PM nnUR S5Vs/sHoUSPM nnXg:Xdegs snfs snf)Nz[\.]$z[/]za.ORTH)rules suffix_searchinfix_finditerza/a.r5)recompilersearchfinditerr;r<)en_vocab suffix_reinfix_rerFr6r?r@explain_tokenss rB&test_tokenizer_explain_special_matcherrQKs 8$Izz&!H VTN# $E&&(( I (/ 0/ff/F 0$-$5$5f$=>$=qd$=N>  ## #1>s B B%cSSS0/0n[UUS9nSnU"U5Vs/sHoDRPM nnURU5Vs/sHoDSPM nnXV:Xdegs snfs snf)Nz:]rE)rFz: ]r5)rr;r<)rMrFr6r;r?r@rPs rB1test_tokenizer_explain_special_matcher_whitespacerSZs{ VTN# $EI D'o .offoF .$-$5$5d$;<$;qd$;N<  ## #/ token_pairtokens rBsentence_strategyrlfs#&((<!3!3 4!3A299Qr>cT[R"U5Rn[R"SSU5R 5nU"U5Vs/sHo3R PM nnURU5Vs/sHo3SPM nnXE:XdUSUSU35egs snfs snf)z Tests whether output of tokenizer.explain() matches tokenizer output. Input generated by hypothesis. lang (str): Language to test. text (str): Fuzzily generated sentence to tokenize. z\s+r]r5z, N)spacyblankr6rIsubstripr;r<)r3r>r6r?r@rAs rBtest_tokenizer_explain_fuzzyrrs!;;t,66Ivvfc8,224H'1 21ff1F 2"+"3"3H"=>"=QaD"=L>  !JfXR ~Rz#JJ !3>s B ;B%))rIr_rchypothesis.strategiesr7rnspacy.tokenizerr spacy.utilrparammarkslow LANGUAGES parametrizerCrQrSrd compositeDrawFnintstrrlxfailgivenrrrBrs  %% /  LLV[[--/0/  LLV[[--/0/  LLV[[--/0/  LLV[[--/0 /   /   LLV[[--/0 /  LLV[[--/0/  LLV[[--/0/  LLV[[--/0/  LLV[[--/0/  /  /  LLV[[--/0/  LLV[[--/0/  LLV[[--/0/   LLV[[--/0!/ " #/ $ LLV[[--/0%/ & LLV[[--/0'/ ( LLV[[--/0)/ * LLV[[--/0+/ , -/ . LLV[[--/0// 0 LLV[[--/01/ 2 LLV[[--/03/ 4 LLV[[--/05/ 6 LLV[[--/07/ 8 LLV[[--/09/ : LLV[[--/0;/ < LLV[[--/0=/ > ?/ @ LLV[[--/0A/ B LLV[[--/0C/ D LLV[[--/0E/ F LLV[[--/0G/ H LLV[[--/0I/ J LLV[[--/0K/ L LLV[[--/0M/ N LLV[[--/0O/ P LLV[[--/0Q/ R LLV[[--/0S/ T LLV[[--/0U/ V LLV[[--/0W/ X LLV[[--/0Y/ Z LLV[[--/0[/ \ LLfkk..01]/  d+&,& $ $   PJ1188PsPSVP!P4+ ,./ Ks Kc Kd K0, Kr