h>)hSSKrSSKJr SSKrSSKJr SSKJr SSKJ r J r J r SSK J r Jr SSKJrJrJr SS KJr \SS j5r\R,S 5r\R,S 5r\R2R5S 5S5rSr\R2R5S 5S5rSrg)N)contextmanagerEnglish)contains_cycle)DocDocBinSpan)CorpusExample)create_lower_casing_augmentercreate_orth_variants_augmentermake_whitespace_variant) make_tempdirc## [5nX!- n[US9RU5 Uv SSS5 g!,(df  g=f7f)N)docs)rrto_disk)rnametmpdir output_files ^/home/james-whalen/.local/lib/python3.13/site-packages/spacy/tests/training/test_augmenters.py make_docbinrs5 6m D!!+. s A!7 A AAc[5$)Nrrnlprs 9rcf/SQn/SQn/SQn/SQnSSS.n[URXX4S9nXVlU$) N) Sarahz'ssisterflewtoSiliconValleyviaLondon.) NNPPOSNNVBDINr'r'r+r'r&) PROPNPARTNOUNVERBADPr,r,r0r,PUNCT) zB-PERSONzI-PERSONOr2zB-LOCzI-LOCr2B-GPEr2?)TRAVELBAKING)wordstagsposents)rvocabcats)rr9r:r;r<r>docs rr?r? s? _E LD ]C VDS )D ciiuS DCH Jrzignore::UserWarningcS/SS/S.S//SQS./n/SQn/SQnS /[U5-nS US 'S US '[URX$US 9n[SSSU0S9n[ U/S-5n[ XvS9n[ U"U55 SSS5 [SSSU0S9n[ U/S-5n[ XvS9nU"U5H@n U RH-n U RU RR5:XaM-e MB SSS5 [URUS /[U5-S9n[SSSU0S9n[ U/S-5n[ XvS9nU"U5HLn [U RU5H/upU RU RR5:XaM/e MN SSS5 [URUS /[U5-S9n[SSSU0S9n[ U/S-5n[ XvS9nU"U5H>n [U RU5H!upU RU R:XaM!e M@ SSS5 g!,(df  GN=f!,(df  GNd=f!,(df  N=f!,(df  g=f)NNFP…...)r:variants:)-—–-----——)z A BabrBrCrFrGrHrIrJrK)_SPr)rMr)r)r)rArArErErErErErETFrr)r9spacesr:g??single)levellower orth_variants  augmenterr5)r9rRr6) lenrr=r rr list referencetextrVzip) rrTr9r:rRr?rZrreaderexampletokenex_token doc_tokens rtest_make_orth_variantsre.sxuen5$NOF gE \DVc%j FF1IF1I ciiu$ ?C.Xv,>I cURZ K 9 VC[ ! /Xv,>I cURZ K 9c{G **zzUZZ%5%5%7777+# ! ciiudVc%j-@ AC.Xv,>I cURZ K 9c{G'*7+<+>>>(C# ! ciiudVc%j-@ AC.Xv,>I cURZ K 9c{G'*7+<+AI'I'&AI82I8 I I$' I58 Jc[SS9n[U/5n[X2S9n[U"U55nSSS5 WSnURR UR R 5:XdeURR UR R 5:XdeURVs/sH%owRURUR4PM' nnURRVs/sH%owRURUR4PM' snU:Xde[URRUR5H/upU R U R R 5:XaM/e UV s/sHoRPM sn URV s/sHoRPM sn :XdeURV s/sHoRPM sn UV s/sHoRPM sn :Xde/SQn [UR U S9n[U/5n[X2S9n[U"U55nSSS5 USnURR UR R 5:XdeURR UR R 5:XdeURV s/sHoR PM sn U V s/sHoR 5PM sn :XdeURV s/sHoR PM sn UR#UR R 55V s/sHoR PM sn :Xdeg!,(df  GNZ=fs snfs snfs sn fs sn fs sn fs sn f!,(df  GNd=fs sn fs sn fs sn fs sn f)Nr5)rUrYr)rLrNzCCC.)r9)r rr r\r]r^rV predictedr<startendlabelr_ent_iobpos_rr=make_doc) rr?rZrr`corpuseger<ref_entorig_enttr9s rtest_lowercase_augmenterrtes-C8I cU { 9fSk"  B <<   0 00 0 <<   0 00 0/2xx 8x!WWaeeQWW %xD 8/1||/@/@ A/@!WWaeeQWW %/@ AT II I !2!2CHH=||x}}224444>" #s!IIs #2<<'H# URnT (aJUVs/sH<n[R"5S:aUR5OUR5PM> nnOG[ U5VVs/sH/upSUS-(aUR5OUR5PM1 nnnUR 5nUR SRU55nUVs/sHoRPM snUSS'Uv URXv5v gs snfs snnfs snf7f)NrSrr3token_annotationORTH) r^randomrVupper enumerateto_dictrmjoin from_dict) rrar^cchi example_dictr?rsrvs raugmentRtest_custom_data_augmentation..create_spongebob_augmenter..augments<.create_spongebob_augmenters 7rrYz4Sarah 's sister flew to Silicon Valley via London . z4SaRaH 's sIsTeR FlEw tO SiLiCoN VaLlEy vIa lOnDoN . r)F) boolrr r\r^r]rgr<rhrirj) rr?rrr`rn orig_text augmentedrpr<s rtest_custom_data_augmentationrs d  cU { /I/KLfSk" GIFI !9>>Y && & !9   # #y 00 0 !9   # #y 00 0 !9>>Y && & !9   # #y 00 0 !9   # #y 00 0/2xx 8x!WWaeeQWW %xD 8/5ay/B/B/G/G H/G!WWaeeQWW %/G HD PP P/5ay/B/B/G/G H/G!WWaeeQWW %/G HD PP P   9 H Hs G :,G,G ,G% Gc n Sn/SQn/SQn/SQn/SQn/SQn/SQn/SQn[URUUUUUUUS 9n U RU:Xde[UR U5U 5n [ X S S 5n U R RS RS :Xde[ X S S5n U R RS RS:Xde[ X S S5n U R RS RS:Xde[ X S S5n U R RS RS :Xde[[U 5S-5GH'n [ X S U 5n U R U R(deU R V s/sHoRPM sn USU S/-XLS-:XdeU R V s/sHoRPM sn USU S /-X\S-:XdeU R V s/sHoRPM sn USU S/-X|S-:XdeU R RS5(aeU R RS5(ae[U R V s/sHoR R"PM sn 5(ae[[%U R&55S:XdeU S :Xa*U R U R R"S:XdeO,U R U R R"U S- :XdeSHn[ X SU5n[UR V s/sHoR R"PM sn 5(ae[[%U R&55S:XdeUR UR R"US- :XaMe [U R5[U R R5:Xde[)SU R 55(deU R RH/nUS R(aeUSR(dM/e GM* SU R S l [ X S S5n U RU R R:XdeSU R S l U R S S/U R R*S'[ X S S5n U RU R R:XdeU R R*S [-U S SS S!S"9/U R l[ X S S5n U RU R R:Xdegs sn fs sn fs sn fs sn fs sn f)#Nz?They flew to New York City. Then they drove to Washington, D.C.)Theyr r!NewYorkCityr& Thentheydrover! Washington,D.C.)TTTTTFFFTTTTFTF)PRPr*r+r'r'r'r&rQRBrr*r+r'rr')rflyr!rrrr&rthenrdriver!rrr)rrrrrrXrXrXrXrX r)nsubjROOTprepcompoundrpobjpunctdepadvmodrrrrrappos)r2r3r2r4I-GPErr2r2r2r2r2r2r4r2r4)r9rRr:lemmasheadsdepsr< rz New York CityrzNew York CityrzNew York CityrrQrr(MORPHr)rrXz c3># UHoRS:Hv M g7f)rN)rk).0rss r /test_make_whitespace_variant..s<+;a99>+;sr3rspansENTQ123)rjkb_id)rr=r^r rmrr]r<ranger[is_spacetag_lemma_dep_has_annotationrheadrr\sentsanyrr )rr^r9rRr:rrrr<r?ramod_exrrsjmod_ex2ents rtest_make_whitespace_variantrsG MD ~E lF mD ~F AE RD iD      C 88t  cll4(#.G $S3 :F     # ( (O ;; ; $S3 :F     # ( (,< << < $S3 :F     # ( (,< << < $S3 :F     # ( (O ;; ; 3s8a< (sA>"++++ & 0 01 01 01T"1X5G$r(5RRRR"("2"23"2Q"23vbqzSE7IFSUJ7VVVV & 0 01 01 01T"1X5G$r(5RRRR##2259999##227;;;;!V5E5E"F5E66885E"FGGGG4 ?#q((( 6##A&++--2 22##A&++--Q6 66A-c8QGG%9J9J&K9JAvvxx9J&KLL LLtCII'1, ,,$$Q',,..!a%7 77  388}F$4$4$9$9 ::::<6+;+;<<<<<##((C1v &&2w''' '')7!B!#Ga $S3 :F ;;'++00 00 0 'Ga)0(9(9!A(>'?GG$ $S3 :F ;;'++00 00 0(#31EHIG $S3 :F ;;'++00 00 0W231 #G'LsV V#V(V- V2 )zroundtrip.spacy)r{ contextlibrpytest spacy.lang.enr(spacy.pipeline._parser_internals.nonprojr spacy.tokensrrr spacy.trainingr r spacy.training.augmentr r rutilrrfixturerr?markfilterwarningsrertrrrrrrs % !C***    1237337l>12Q3Q@P1r