h$&SSKrSSKJr SSKJrJrJrJrJr SSK r SSK J r SSK J r SSKJrJr SSKJr SS KJr SS KJr S S KJrJrJrJrJrJr \R<"S 5\R<"S5\"SSS9\"SSSS9\"SSSSSS9\"SSSSS9\"SSSS S9\"SS!S"S#S9\"SS$S%S&SSS'9\"S(S)S*S+S9\"SS,S-S.S9\"S/S0S1S2S94 S3\S4\S5\\S6\\S7\ S8\!S9\\S:\ S;\!S<\4S=jj55r"SRS3\S4\S5\\S7\ S8\!S9\\S:\ S>\!S<\S;\!S?\\\44S@jjr#04S/SSA.SB\\\4SC\\\4S<\S>\!S?\\\44 SDjjjr$SSSE\\SF\SG\SH\ SI\!SJ\!SK\!4SLjjr%SM\ SB\\\\\&44SN\SO\S?S4 SPjr'SM\ SB\\\\\&44S?S4SQjr(g)TN)Path)AnyDictListOptionalUnion)fix_random_seed)Printer)displacyutil)Scorer)Doc)Corpus)ArgOptapp benchmark_cli import_code setup_gpuaccuracyevaluate.zModel name or path)helpz3Location of binary evaluation data in .spacy formatT)rexistsz--outputz-ozOutput JSON file for metricsF)rdir_okayz--codez-czNPath to Python file with additional code (registered functions) to be importedz--gpu-idz-gzGPU ID or -1 for CPUz--gold-preprocz-GzUse gold preprocessingz--displacy-pathz-dpz+Directory to output rendered parses as HTML)rr file_okayz--displacy-limitz-dlz!Limit of parses to render as HTMLz--per-componentz-PzSReturn scores per component, only applicable when an output JSON file is specified.scz --spans-keyz-skz*Spans key to use when evaluating Doc.spansmodel data_pathoutput code_pathuse_gpu gold_preproc displacy_pathdisplacy_limit per_component spans_keyc >[U5 [UUUUUUUUSU S9 g)aE Evaluate a trained pipeline. Expects a loadable spaCy pipeline and evaluation data in the binary .spacy format. The --gold-preproc option sets up the evaluation examples with gold-standard sentences and tokens for the predictions. Gold preprocessing helps the annotations align to the tokenization, and may result in sequences of more consistent length. However, it may reduce runtime accuracy due to train/test skew. To render a sample of dependency parses in a HTML file, set as output directory as the displacy_path argument. DOCS: https://spacy.io/api/cli#benchmark-accuracy F)r#r%r&r'r(r)silentr*N)rr) r!r"r#r$r%r&r'r(r)r*s L/home/james-whalen/.local/lib/python3.13/site-packages/spacy/cli/evaluate.py evaluate_clir.s2<   !#%# r,returnc  [Xw(+S9n [5 [X7S9 [R"U5n[R"U5n [R"U5nUR 5(dU R SUSS9 U(a&UR 5(dU R SUSS9 [XS9n [R"U5n [U "U 55nU RXS9nU (a+UnUcU RS 5 GO(U RS 5 GO0S S _S S_SS_SS_SS_SS_SS_SS_SS_SS_SS _S!S"_S#S$_S%S&_S'S(US)3_S*S(US+3_S,S(US-3_S.S/0En0n0nUR5HunnUU;dMUS :XaUS0-URS1S25-S3-n[UU[ ["45(aUS/:Xa UUS4UU'OUUS5-S6UU'OS7UU'UUU[$R&"S8S9UR)55'M U R+US:S;9 [-UUXS<9nU(aU R.Vs/sHnU R1U5R2PM nn[U R5S=USU555nS>U;nS?U;nS@U;n[7UUUUUUUSA9 U R9SBUSC3U5 U b+[:R<"U U5 U R9SDU 35 U$s snf)ENno_printpretty)r,zEvaluation data not foundr)exitsz(Visualization output directory not found)r&)r)zdThe per-component option is enabled but there is no output JSON file provided to save the scores to.z7Per-component scores will be saved to output JSON file.TOK token_accTAGtag_accPOSpos_accMORPH morph_accLEMMA lemma_accUASdep_uasLASdep_laszNER Pents_pzNER Rents_rzNER Fents_fTEXTCAT cats_scorezSENT Psents_pzSENT Rsents_rzSENT Fsents_fzSPAN Pspans__pzSPAN R_rzSPAN F_fSPEEDspeedz (cats_score_descunk)z.0fd.2f-z[\s/]_Results)titler*r,c3L# UHoRRv M g7f)N) referencetext).0exs r- evaluate..sV9U2\\..9Us"$parsernerspancat) model_namelimitdepsentsspansz Generated z parses as HTMLzSaved results to )r r rr ensure_pathrfailr load_modellistrwarninfoitemsget isinstanceintfloatresublowertablehandle_scores_per_type pipe_names get_pipe_metafactorypipe render_parsesgoodsrsly write_json)r!r"r#r%r&r'r(r,r*r)msg output_pathcorpusnlp dev_datasetscoresdatametricsresultsmetrickeyr~ factory_namesdocs render_deps render_ents render_spanss r-rr=s 6* 5C g%  +I""6*K$$]3M      ,iqA]1133 ;]RST I 9F //% Cvc{#K \\+\ CF > HHv  HHN O ; 9  9  [  [  9   9  X  X  X  |  i  i  i  yk, yk,! " yk,# $ W% ("==?KFCf},&#d]VZZ8I5-QQTWWFfSkC<88g~-3C[,=-3C[_S,A&)GFO;A#;RVVHc399;78+ ' +%fdiWEH^^T^T**4088^ TCHHV_n9UVVW-/ },  M1      :n-_=}M d+ $[M23 K)Us%Lr[rrc&[X3(+S9nSU;a"US(a[X@SSS5 USUS'SU;a"US(a[X@SSS5 USUS'SU;a"US(a[X@SS S5 USUS'S US 3U;a2US US 3(a$[X@S US 3S S5 US US 3US US 3'S U;a"US (a[X@S SS5 US US 'SU;a US(a[X@S5 USUS'U$)Nr2morph_per_featr<featdep_las_per_typerBtype ents_per_typeNERrL _per_typeSPANScats_f_per_typez Textcat Flabelcats_auc_per_type)r print_prf_per_typeprint_textcats_auc_per_cat)rrr*r,rs r-rzrzss 6* 5C6! " # s+;$v N'-.@'AD# $& / " s?$;UF K$*?$;D !  {)$. F9+Y/ 0 fYKy9:GV 396)I9V2WD6)I. /F" # $ s+<$={G T&,->&?D" #f$ % & &s3F,G H(./B(CD$ % Mr/rrrfrgrhrirjcbX SRS'U(aF[R"USUSSS9nUS- RSSS 9nUR U5 SSS5 U(aI[R"USUS SS S0S 9nUS - RSSS 9nUR U5 SSS5 U(aG[R"USUSSS9nUS- RSSS 9nUR U5 SSS5 gg!,(df  N=f!,(df  Nn=f!,(df  g=f)NrrZentT)stylepagez entities.htmlwutf8)encodingdepcompact)rroptionsz parses.htmlspanz spans.html) user_datar renderopenwrite) rrrfrgrhrirjhtmlfile_s r-rrs",Gg tFU|5tDO+ 1 1# 1 G5 KK H  %LD9d:K M) / /f / E KK F tFU|6EL( . .sV . D KK E D H G F E E Ds$C>D"D > D  D D.rnamerc /nUR5HYupVU/nSH:nXhn UR[U [[45(aU S-SOU 5 M< URU5 M[ UR USSUSUS3S9 g) N)prfrUrV)PRF)lrrrz (per rTheaderalignsrZ)rqappendrsrtrury) rrrrrrvaluerowkvs r-rrs Dlln e AA JJ:a#u+F+F!c'#A N! C % II "#fTF!$ r/c URUR5VVs/sH(up#U[U[[45(aUSOU4PM* snnSSSS9 gs snnf)NrV)rzROC AUC)rrzTextcat ROC AUC (per label)r)ryrqrsrurt)rrrrs r-rrseII  &jUCL991S'q A& + s/A )NrFNrTr F)rTTT))rvpathlibrtypingrrrrrr thinc.apir wasabir rr r scorerrtokensrtrainingr_utilrrrrrrcommandstrrtboolr.rrzrrurrr/r-rsY 33 %GGZS34#$Ybfg z4>\glm #D(D@P!Qr:t2HIU$4dAYZ$'.?MzDHTY%Zb"4eBefe%6DYZ}e:fg& && TN & ~ &  &&D>&&&&&X"$(^ ^^ TN^ ^  ^ D> ^^ ^^^ #s(^^F" " cN" sCx." "  "  #s(^ "P s)       6 sDe$445=@HK $   sDe$445   r/