h$$SSKrSSKrSSKrSSKJr SSKJrJrJrJ r J r SSK r SSK r SSKJr SSKJr SSKJrJr SSKJr S S KJrJrJrJrJr S S S S.r\R<"SS SS.S9\"SSS9\"SSSS9\"SSS9\"SSS9\"SSS9\"\SSSSS9\"SSS S!S9\"\S"S#S$S%S9\"\S&S'S(S)S9\"S S*S+S,S-S94 S.\S/\S0\S1\S2\S\ S3\ \S"\ S&\!S4\!4S5jj5r"\S\S"\S&SS6.S.\S/\S0\S1\S2\S\ S"\ S&\!S7\!S8\ \#\#\\#\#444S9jjr$g):N)Path)AnyDictListOptionalTuple)util)Errors)MultiLabel_TextCategorizerTextCategorizer)Corpus)ArgOptapp import_code setup_gpu F)n_trialsuse_gpu gold_preproczfind-thresholdT)allow_extra_argsignore_unknown_options)context_settings.zModel name or path)helpz3Location of binary evaluation data in .spacy format)rexistsz&Name of pipe to examine thresholds forz7Key of threshold attribute in component's configurationzMetric to optimizerz --n_trialsz-nz0Number of trials to determine optimal thresholdsz--codez-czNPath to Python file with additional code (registered functions) to be importedrz--gpu-idz-gzGPU ID or -1 for CPUrz--gold-preprocz-GzUse gold preprocessingz --verbosez-Vz-VVz/Display more information for debugging purposesmodel data_path pipe_name threshold_key scores_key code_pathverbosec U (a-[RR[R5 [ U5 [ UUUUUUUUSS9 g)au Runs prediction trials for a trained model with varying thresholds to maximize the specified metric. The search space for the threshold is traversed linearly from 0 to 1 in `n_trials` steps. Results are displayed in a table on `stdout` (the corresponding API call to `spacy.cli.find_threshold.find_threshold()` returns all results). This is applicable only for components whose predictions are influenced by thresholds - e.g. `textcat_multilabel` and `spancat`, but not `textcat`. Note that the full path to the corresponding threshold attribute in the config has to be provided. DOCS: https://spacy.io/api/cli#find-threshold F) rr r!r"r#rrrsilentN)r loggersetLevelloggingDEBUGrfind_threshold) rr r!r"r#rr$rrr%s R/home/james-whalen/.local/lib/python3.13/site-packages/spacy/cli/find_threshold.pyfind_threshold_clir.sI@ W]]+ #! )rrrr'r'returnc ^^[XhS9 [R"U5nUR5(d[R R SUSS9 [R"U5n X)R;a0[[RRX)RS95eU RU5n [U S5(d[[R5e[!U 5[":Xa[R R%S5 U(d'[R R'SUS US US 3S 9 [)XS 9n [+U "U 55n UR-S5n S[.[0[24S[4[0S[6S[.[0[244SjnS[.[0[24S[4[0S[0S[.[0[244U4Sjjm0mSU/U QnSn[8R:"SSU5n[=[R>RASU/US95 UGHbn[R"UU"T"U RBUSREU55RG5UU5S9n [U S5(a.[IU RU5SU"[KU S5U U55 U RMU 5nUU;a=[R R SUS3S[+URO553SS 9 UUTU'[QTU[6[R45(d"[R R S!US"3SS9 [=[R@"[UUS#5[UTUS#5/US95 GMe [WTRO5U4S$jS%9n[Y[[TR]555S:Xa<[R R%S&[QU [^5(aUS';aS(OS)S*9 O'U(d [=S+[UUS,S-9S.US/TUS35 UTUT4$)0a Runs prediction trials for models with varying thresholds to maximize the specified metric. model (Union[str, Path]): Pipeline to evaluate. Can be a package or a path to a data directory. data_path (Path): Path to file with DocBin with docs to use for threshold search. pipe_name (str): Name of pipe to examine thresholds for. threshold_key (str): Key of threshold attribute in component's configuration. scores_key (str): Name of score to metric to optimize. n_trials (int): Number of trials to determine optimal thresholds. use_gpu (int): GPU ID or -1 for CPU. gold_preproc (bool): Whether to use gold preprocessing. Gold preprocessing helps the annotations align to the tokenization, and may result in sequences of more consistent length. However, it may reduce runtime accuracy due to train/test skew. silent (bool): Whether to print non-error-related output to stdout. RETURNS (Tuple[float, float, Dict[float, float]]): Best found threshold, the corresponding score, scores for all evaluated thresholds. )r'zEvaluation data not foundr)exits)nameoptsscorerzThe `textcat` component doesn't use a threshold as it's not applicable to the concept of exclusive classes. All thresholds will yield the same results.zOptimizing for z for component 'z' with z trials.)title)r.configkeysvaluer0cdU[R"[RUSSU5US'U$)zSet item in nested dictionary. Adapted from https://stackoverflow.com/a/54138200. config (Dict[str, Any]): Configuration dictionary. keys (List[Any]): Path to value to set. value (float): Value to set. RETURNS (Dict[str, Any]): Updated dictionary. Nr) functoolsreduceoperatorgetitem)r8r9r:s r-set_nested_item'find_threshold..set_nested_items4KP ))49f=d2hG r/full_keyc >USU;aK[RRSUSUS/S3SUS/S[UR 553SS9 US[ U5S:aT"XSUSS U50$XS0$) aFilters provided config dictionary so that only the specified keys path remains. config (Dict[str, Any]): Configuration dictionary. keys (List[Any]): Path to value to set. full_key (str): Full user-specified key. RETURNS (Dict[str, Any]): Filtered dictionary. rzFailed to look up `z` in config: sub-key z not found.zMake sure you specified z: correctly. The following sub-keys are available instead: rr6textr2N)wasabimsgfaillistr9len)r8r9rB filter_configs r-rK%find_threshold..filter_configs 7& JJOO+H:5JDQRG9+U`a/a {:t &')   G4y1}#6q'?DHhG  Q  r/ components) rNr Threshold)widths)r8cfgzFailed to look up score `z` in evaluation results.z`Make sure you specified the correct value for `scores_key`. The following scores are available: rDzReturned score for key 'zG' is not numeric. Threshold optimization only works for numeric scores.c>TU$)N)keyscoress r- find_threshold..s r/)rUz?All scores are identical. Verify that all settings are correct.) cats_macro_f cats_micro_fz\Use `cats_macro_f` or `cats_micro_f` when optimizing the threshold for `textcat_multilabel`.)r6rEz Best threshold: )ndigitsz with z value of )0rr ensure_pathrrFrGrH load_modelcomponent_namesAttributeErrorr E001formatget_pipehasattrE1045typer warninforrIsplitrstrrrfloatnumpylinspaceprinttablesrowr8joincopysetattrgetattrevaluater9 isinstanceintroundmaxrJsetvaluesr )rr r!r"r#rrrr'nlppipecorpus dev_dataset config_keysr@config_keys_fulltable_col_widths thresholds threshold eval_scoresbest_thresholdrKrVs @@r-r,r,Gs:g%  +I      3YaH //% C+++ KK  I4G4G  H   << "D 4 " "V\\** Dz_$  M  #J c3h ."$F$i>+>1h/J &--  L:CS  TU oo "JJ 0#((;K2L$&     4    Y'e 4k9M  ll;/ [ ( JJOO1*=UV";#3#3#5679   ( 3y&+eS\:: JJOO*:,7    JJy!$eF9,=q&AB'  G T-DFN 3v}}  A% St%?@@!AA p   $U>1%E$FfZLXbcijxcybzz{|  6.16 99r/)%r<r*r>pathlibrtypingrrrrrrm wasabi.tablesrFr[r errorsr pipeliner r trainingr_utilrrrrr _DEFAULTScommandrkrxboolr.rlr,rTr/r-rs33 B88  */4P S34#$Ybfg#KLS'`a#$89 *-|THz{ #D(D@P!Qy+ZDZ[Y~68H$Umn{D%>op) )) )  )  ))~)))) )fj)Y'">2b: b:b:b: b:  b:b:b:b: b: 5%eUl+ +,b:r/