h2zSSKr\RRS5S5r\RRS5S5r\RRS5\RR SSS /5S 55r\RRS 5\RR S S S/5S55r\RRS5\RR S SS/5S55r\RRS5\RR S/SQ5S55r \RRS5\RR S /SQ5S55r \RRS5\RR S SS/5S55r \RRS5\RR S SS/5S 55r \RRS!5\RR S S"S#/5S$55r \RRS%5\RR S S&/5S'55r\RRS(5\RR S S)/5S*55r\RRS+5\RR S,55r\RR S S-S./5\RRS/5S055r\RRS15S25r\RRS35S45r\RRS55S65r\RR S7/S8Q5\RRS95S:55r\RRS;5\RR S S55rg)?Ni_cU"S5nUSRS:Xde[US5S:XdeUSRS:Xdeg)Nz This is a cat.r)idxlen en_tokenizerdocs \/home/james-whalen/.local/lib/python3.13/site-packages/spacy/tests/lang/en/test_tokenizer.py test_issue351r sK * +C q6::?? s1v;!   q6::??ihc6U"S5n[U5S:deg)z!Test tokenization of big ellipsisz$45...............AskingNrr tokenss r test_issue360r s4 5F v;??r iz text,number)7am7)z11p.m.11c`U"U5n[U5S:XdeUSRU:Xdeg)z\Test that times like "7am" are tokenized correctly and that numbers are converted to string.rrNrtext)r rnumberrs r test_issue736rs6 $ F v;!   !9>>V ## #r irz3/4/2012z 01/12/1900c6U"U5n[U5S:Xdeg)zTest that dates are not split and kept as one token. This behaviour is currently inconsistent, since dates separated by hyphens are still split. This will be hard to prevent without causing clashes with numeric ranges.rNrr rrs r test_issue740rs $ F v;!  r izWe were scaredzWe Were Scaredc|U"U5n[U5S:XdeUSRR5S:Xdeg)zmTest that 'were' and 'Were' are excluded from the contractions generated by the English tokenizer exceptions.rrwereN)rrlowerrs r test_issue744r"'s? $ F v;!   !9>>   !V ++ +r iz text,is_num))oneT)tenT) tenelevenFc>U"U5nUSRU:Xdeg)Nr)like_num)r ris_numrs r test_issue759r)1s& $ F !9   '' 'r i)ShellshellShedshedc`U"U5n[U5S:XdeUSRU:Xdeg)zoTest that 'Shell' and 'shell' are excluded from the contractions generated by the English tokenizer exceptions.rrNrrs r test_issue775r/:s6 $ F v;!   !9>>T !! !r izThis is a string cU"U5nSRUVs/sHo3RPM sn5U:Xdegs snf)zGTest for Issue #792: Trailing whitespace is removed after tokenization.Njoin text_with_wsr rr tokens r test_issue792r7D; t C 77C8C5&&C8 9T AA A8;zThis is a stringzThis is a string cU"U5nSRUVs/sHo3RPM sn5U:Xdegs snf)z6Test base case for Issue #792: Non-trailing whitespacer1Nr2r5s r test_control_issue792r;Lr8r9i[zaaabbb@ccc.com Thank you!zaaabbb@ccc.com Thank you!c8U"U5nURU:Xdeg)z5Test that no extra space is added in doc.text method.N)rr rr s r test_issue859r>Ts t C 88t  r ivzDatum:2014-06-02 Dokument:76467cU"U5nUHRn[UR5[UR5:XdeXRURS:XaMRe g)zLTest that token.idx matches the original text index for texts with newlines.rN)rrr4rr5s r test_issue886r@^sU t C5::#e&8&8"9999II%**Q-///r i{z want/needc`U"U5n[U5S:XdeUSRS:Xdeg)z(Test that / infixes are split correctly.rr/Nrrs r test_issue891rChs6$ F v;!   !9>>S  r ic[R"S5 SH9nSn[SS5HnX![U5-- nM U"U5nU(aM9e g)zTest that spaCy doesn't hang on many punctuation characters. If this test hangs, check (new) regular expressions for conflicting greedy operators pytest_timeout) .,'":?!;-0rdN)pytest importorskiprangestr)r punctstringir s r test_issue957rXqsU ()>q#A c!fn $F6"s s ?r ztest@example.comzjohn.doe@example.co.ukicbU"U5n[U5S:XdeUSR(aeg)z6Test that doc doesn't identify email-addresses as URLsrrN)rlike_urlr=s r test_issue1698r[s1 t C s8q==1v r ic6U"S5n[U5S:Xdeg)zDTest that "would've" is handled by the English tokenizer exceptions.zwould'verNrrs r test_issue1758r]s * %F v;!  r icfU"S5nUSRS:XaUSRS:wdegg)zuTest that spaces don't receive a POS but no TAG. This is the root cause of the serialization issue reported in #1773. rSPACEr1N)pos_tag_rs r test_issue1773rcs9 t C 1v{{g1v{{b   r i cU"S5n[U5S:XdeUSRS:XdeUSRS:XdeUSRS:Xdeg) z2Test that hyphens are split correctly as prefixes.uA—Yo me llamo... –murmuró el niño– Emilio Sánchez Pérez.ru—u– Nr) es_tokenizerr s r test_issue3277risa Z [C s8r>> q6;;( "" " q6;;( "" " q6;;( "" "r word)zdon'tudon’tzI'duI’di c@U"U5SnUR(deg)Nr)is_stop)r rjtoks r test_issue3521rns t Q C ;;;r i)thesesthisrec6U"U5n[U5S:Xdeg)zqTest that 'theses' and 'thisre' are excluded from the contractions generated by the English tokenizer exceptions.rNrrs r test_issue10699rrs $ F v;!  r )rQmarkissuer r parametrizerrr"r)r/r7r;r>r@rCslowrXr[r]rcrirnrrr r rxsF 33 37G(HI$J$3*l!;<=3"24D!EF,G,3G(( 3!CD"E"3"57O!PQBRB 3"46J!KLBMB 3 )+H I 3"D!EF0G03+/!0!3  "46N!OP4Q4 4!!4##!EF4G 5(H!567r