hȲHSSKJr SSKJrJrJrJrJrJrJ r J r J r SSK r SSKJr SSKJr SSKJrJrJr SSKJr SS KJrJr \(aSS KJr S r\"/S Q5r"S S5r "SS5r!"SS5r"S\\S\\#\44Sjr$SSjr%Sr&Sr'Sr(SSjr)Sr*g)) defaultdict) TYPE_CHECKINGAnyCallableDictIterableListOptionalSetTupleN)Errors) Morphology)DocSpanToken)Example)SimpleFrozenListget_lang_class)Language)sentertagger morphologizerparsernertextcat)Nrc \rSrSrSrSSSS.S\S\S\SS 4S jjrS\4S jrS rS r S\ S\ SS 4Sjr \ S\ 4Sj5r\ S\ 4Sj5r\ S\ 4Sj5rS\\\ 44SjrSrg )PRFScorezA precision / recall / F score.rtpfpfnr"r#r$returnNc(XlX lX0lgNr!)selfr"r#r$s F/home/james-whalen/.local/lib/python3.13/site-packages/spacy/scorer.py__init__PRFScore.__init__"scNURUR-UR-$r'r!r(s r)__len__PRFScore.__len__-sww 477**r,cU=RUR- slU=RUR- slU=RUR- slU$r'r!r(others r)__iadd__PRFScore.__iadd__0s= 588 588 588 r,c[URUR-URUR-URUR-S9$)Nr!)rr"r#r$r2s r)__add__PRFScore.__add__6s=ww!dgg&8TWWuxx=O  r,candgoldcU=R[URU55- slU=R[X- 5- slU=R[X!- 5- slgr')r"len intersectionr#r$r(r9r:s r) score_setPRFScore.score_set;sJ 3t((.// 3t{## 3t{##r,cTURURUR-S-- $N0.++)r"r#r.s r) precisionPRFScore.precision@#ww$''DGG+f455r,cTURURUR-S-- $rB)r"r$r.s r)recallPRFScore.recallDrFr,cPURnURnSX-X-S-- -$)NrC)rDrH)r(prs r)fscorePRFScore.fscoreHs, NN KKQUquv~.//r,cJURURURS.$)N)rLrMf)rDrHrNr.s r)to_dictPRFScore.to_dictNs^^$++DKKHHr,)r$r#r")__name__ __module__ __qualname____firstlineno____doc__intr*r/r4r7setr?propertyfloatrDrHrNrstrrR__static_attributes__r,r)rrs)          ++  $c$$$ 6566666000 Ic5j)Ir,rcB\rSrSrSrS SjrS SjrSr\S5r Sr g) ROCAUCScoreRzAn AUC ROC score. This is only defined for binary classification. Use the method is_binary before calculating the score, otherwise it may throw an error.Nc</Ul/UlSUlSUlg)Nr)goldscands saved_scoresaved_score_at_lenr.s r)r*ROCAUCScore.__init__Ws " " "#r,cpURRU5 URRU5 gr')rfappendrer>s r)r?ROCAUCScore.score_set]s& $ $r,cZ[[R"UR55S:H$NrK)r<npuniquerer.s r) is_binaryROCAUCScore.is_binaryas299TZZ()Q..r,cUR5(d9[[RR [ UR 5S95e[UR 5UR:Xa UR$[UR UR5Ul [UR 5UlUR$)Nlabel) rq ValueErrorrE165formatrZrer<rhrg_roc_auc_scorerfr.s r)scoreROCAUCScore.scoreds~~V[[//c$**o/FG G tzz?d55 5## #)$**djjA"%djj/r,)rfrergrh)r%N) rTrUrVrWrXr*r?rqr[rzr^r_r,r)raraRs*$  /  r,rac\rSrSrSrSS\4S\SS\S\\S S4S jjr S S .S \\ S\ S \ \\ 44Sjjr\S \\ S \ \\ 44Sj5r\\\S.S \\ S\S\\\/\ 4S\\ S \ \\ 44 Sjj5r\\\S.S \\ S\S\\\/\ 4S\\ S \ \\ 44 Sjj5r\\SSS S.S \\ S\S\\\/\\4S\\\/\ 4S\ S\ S \ \\ 44Sjj5r\\\"5SSSS.S \\ S\S\\\/\ 4S\\S\ S \\S!\\S \ \\ 44S"jj5r\S \\ S#\\S \ \\ 44S$j5r\\S%\\"5\S&.S \\ S\S\\\/\ 4S'\S(\\\/\4S)\\S\\ S \ \\ 44S*jj5rS+r g),ScorerozCompute evaluation scores.Nxxnlpr default_langdefault_pipeliner%c X@lU(aXlg[U5"5nUHnURU5 M Xlg)z?Initialize the Scorer. DOCS: https://spacy.io/api/scorer#init N)cfgrradd_pipe)r(rrrrpipes r)r*Scorer.__init__rs9 H .0C( T")Hr,F) per_componentexamplesrcX0n[URRS5(a|U(a5URRR"U40URD6US'O@UR URRR"U40URD65 URR HlupE[US5(dMU(a!UR"U40URD6X4'M@UR UR"U40URD65 Mn U$)a!Evaluate a list of Examples. examples (Iterable[Example]): The predicted annotations + correct annotations. per_component (bool): Whether to return the scores keyed by component name. Defaults to False. RETURNS (Dict): A dictionary of scores. DOCS: https://spacy.io/api/scorer#score rz tokenizer)hasattrrrrzrupdatepipeline)r(rrscoresname components r)rz Scorer.scores 488%%w / /&*hh&8&8&>&>x&T488&T{# dhh0066xL488LM#xx00ODy'** #,??8#Htxx#HFLMM)//("Gdhh"GH 1  r,c [5n[5nUGHmnURnURnUR(aM/URn[ 5n[ 5n UHWn U R R5(aM$URU RU R[U 5-45 MY UHn U R R5(aM$U RU RU R[U 5-45 URRU RS:waU=RS- slMU=RS- slM UR!X5 GMp [U5S:a/UR"UR"UR$UR&S.$SSSSS.$)aFReturns accuracy and PRF scores for tokenization. * token_acc: # correct tokens / # gold tokens * token_p/r/f: PRF for token character spans examples (Iterable[Example]): Examples to score RETURNS (Dict[str, Any]): A dictionary containing the scores token_acc/p/r/f. DOCS: https://spacy.io/api/scorer#score_tokenization r r) token_acctoken_ptoken_rtoken_fN)r reference predictedhas_unknown_spaces alignmentrZorth_isspaceaddidxr<x2ylengthsir#r"r?rDrHrN) rr acc_score prf_scoreexamplegold_docpred_docalign gold_spans pred_spanstokens r)score_tokenizationScorer.score_tokenizationsxJ J G((H((H**%%EJJ!;;&&(( 599s5z+ABC"";;&&(( 599s5z+ABC99$$UWW-2LLA%LLLA%L"    7) * y>A &00$..$++$++  "  r,)gettermissing_valuesattrrrc [5nUGH1nURnURnURn [ 5n [ 5n [ U5H=upU"X5nX;aU R X"X545 M,U R U 5 M? [ 5nUHn U RR5(aM$U RRU RS:XdMMU RU RSn X;dMpUR X"X545 M URX5 GM4 US3n[U5S:XaUS0$UUR0$)a3Returns an accuracy score for a token-level attribute. examples (Iterable[Example]): Examples to score attr (str): The attribute to score. getter (Callable[[Token, str], Any]): Defaults to getattr. If provided, getter(token, attr) should return the value of the attribute for an individual token. missing_values (Set[Any]): Attribute values to treat as missing annotation in the reference annotation. RETURNS (Dict[str, Any]): A dictionary containing the accuracy score under the key attr_acc. DOCS: https://spacy.io/api/scorer#score_token_attr r r_accN)rrrrrZ enumeraterrrrrrr?r<rN)rrrrr tag_scorerrrr gold_tagsmissing_indicesgold_irvalue pred_tags score_keys r)score_token_attrScorer.score_token_attrsE.J G((H((H%%EI!eO!*8!4 u+.MM66%+>"?@#''/ "5 I!;;&&((99$$UWW-2"YYuww/2F4! vve/B&CD "    5) *fDM y>Q t$ $y//0 0r,c J[5n0nUGHnURnURn URn 0n [ 5n [ U 5HupU"X5nU R RUnX;aU[R:waUR[R5HanUR[R5unnUU;a [5UU'UU ;a [ 5U U'U URU U45 Mc MU RU 5 M 0nUGH3nURR5(aM%U R R"UR$S:XdMNU R UR$Sn X;dMqU"X5nU R RUnX;dMU[R:wdMUR[R5HanUR[R5unnUU;a [5UU'UU;a [ 5UU'UURU U45 Mc GM6 UHnUR'UR)U[ 55U R)U[ 555 UUR'UR)U[ 55U R)U[ 555 M GM 0n[+U5S:asUR,UUS3'UR.UUS3'UR0UUS3'UR35VVs0sHunnUUR55_M snnUUS3'U$SUUS3'SUUS3'SUUS3'SUUS3'U$s snnf)aMReturn micro PRF and PRF scores per feat for a token attribute in UFEATS format. examples (Iterable[Example]): Examples to score attr (str): The attribute to score. getter (Callable[[Token, str], Any]): Defaults to getattr. If provided, getter(token, attr) should return the value of the attribute for an individual token. missing_values (Set[Any]): Attribute values to treat as missing annotation in the reference annotation. RETURNS (dict): A dictionary containing the micro PRF scores under the key attr_micro_p/r/f and the per-feat PRF scores under attr_per_feat. r r_micro_p_micro_r_micro_f _per_featN)rrrrrZrvocabstringsr EMPTY_MORPHsplit FEATURE_SEP FIELD_SEPrrrrrrr?getr<rDrHrNitemsrR)rrrrr micro_scoreper_featrrrr gold_per_featrrrrmorphfeatfieldvalues pred_per_featresultkvs r)score_token_attr_per_feat Scorer.score_token_attr_per_featsh.j G((H((H%%E,.M!eO!*8!4 u+ ..u5.5JjHUO#( #=;>5M%$8 -e 4 8 8&$ H )L"&"%%!%%eSU3]5F5Fuce5T))!%%eSU3]5F5Fuce5T "O \"$ { a (3(=(=FdV8$ %(3(:(:FdV8$ %(3(:(:FdV8$ %EM^^EU)VEUTQ!QYY[.EU)VFdV9% &  )-FdV8$ %(,FdV8$ %(,FdV8$ %)-FdV9% & *WsNT)rhas_annotationlabeled allow_overlaprrrc [5n[5nUGHn U Rn U Rn UbU"U 5(dM.[ U"X5V s/sHoR PM sn 5n UbUb;U"U 5(a.U [ U"X5V s/sHoR PM sn 5-n U Vs0sH o[ 5_M nnU HnX;dM [5X'M [ 5n[ 5nU"X5H{nU(a'UR UR URS- 4nOUR URS- 4nURU5 UUR RU5 M} U Vs0sH o[ 5_M nnUbUbU"U 5(aU RU"X5U5H{nU(a'UR UR URS- 4nOUR URS- 4nURU5 UUR RU5 M} U(a9UR5H%un nU U;dMURUU X5 M' URUU5 GM US3SUS3SUS3S0nU(aSUUS3'[U5S:axURUUS3'URUUS3'URUUS3'U(a;UR5V Vs0sHun nU UR!5_M snn UUS3'U$s sn fs sn fs snfs snfs snn f)aReturns PRF scores for labeled spans. examples (Iterable[Example]): Examples to score attr (str): The attribute to score. getter (Callable[[Doc, str], Iterable[Span]]): Defaults to getattr. If provided, getter(doc, attr) should return the spans for the individual doc. has_annotation (Optional[Callable[[Doc], bool]]) should return whether a `Doc` has annotation for this `attr`. Docs without annotation are skipped for scoring purposes. labeled (bool): Whether or not to include label information in the evaluation. If set to 'False', two spans will be considered equal if their start and end match, irrespective of their label. allow_overlap (bool): Whether or not to allow overlapping spans. If set to 'False', the alignment will automatically resolve conflicts. RETURNS (Dict[str, Any]): A dictionary containing the PRF scores under the keys attr_p/r/f and the per-type PRF scores under attr_per_type. DOCS: https://spacy.io/api/scorer#score_spans Nr _p_r_f _per_typer)rdictrrrZlabel_startendrget_aligned_spans_x2yrr?r<rDrHrNrR)rrrrrrrrzscore_per_typerrrrlabelsru gold_per_typerrspan gold_span pred_per_type pred_spanr final_scoress r) score_spansScorer.score_spansYs/> G((H((H).2J2JF8,BC,Bq((,BCDF%*~h/G/G#1GH1GAxx1GHIIGM,NveCE\vM,N.,4JN) JJx.!%djj$((Q, GI!%TXX\ :Iy)dkk*..y9/HN,NveCE\vM,N%*~h/G/G#998*MD%)[[$**dhhl$K %)ZZA$> NN9-!$++.229=*002DAqM) M!$4m6FG3 OOJ 3c hfBKfBKfBK( /3LD6+ , u:>(-LD6 %(- LD6 %(- LD6 %/=/C/C/E4/Etq!Aqyy{N/E4 vY/0wD I,N-OD4sM M 5MMM)rr multi_labelpositive_label thresholdrrrrc Uc U(aSOSnU(dSnUVs0sH o[5_M n nUVs0sH o[5_M n n[U5nUGHn U"U RU5n U R 5V Vs0sHupX;dM X_M n n nU"U R U5nUR 5V Vs0sHupX;dM X_M nn nUHOnU R US5nUR U5nU(d U(dSnUcM;XRUU5 MQ U(aUHnU R US5nUR U5nUcM+UU:aUS:aX=RS- slMPUU:aUS:XaX=RS- sl MuUU:dM}US:dMX=RS- sl M GMU (aU(a[U R 5SS9unn[UR 5SS9unnUU:XaU U=RS- slGMU U=RS- sl U U=RS- sl GMU(a2[US S9unnUS:aU U=RS- sl GMQGMTU (dGM^[U R 5S S9unnU U=RS- sl GM [5nU R5H`nU=RUR- slU=RUR- sl U=RUR- sl Mb [U 5S -n[S U R555U- n[S U R555U- n[SU R555U- n[SU R555U- nUS3SUS3SUS3URUS3UR US3UR"US3UUS3UUS3UUS3UUS3U R 5V Vs0sHupXR%5_M snn US3U R 5V Vs0sH(upXR'5(a UR(OS_M* snn 0 n[U5S:Xa2U(d+U(a$UUS3USnUUUS3'SUS3UUS3'U$U(dUUS3UUS3'SUUS3'U$UUS3UUS3'S UUS3'U$s snfs snfs snn fs snn fs snn fs snn f)!aReturns PRF and ROC AUC scores for a doc-level attribute with a dict with scores for each label like Doc.cats. The reported overall score depends on the scorer settings. examples (Iterable[Example]): Examples to score attr (str): The attribute to score. getter (Callable[[Doc, str], Any]): Defaults to getattr. If provided, getter(doc, attr) should return the values for the individual doc. labels (Iterable[str]): The set of possible labels. Defaults to []. multi_label (bool): Whether the attribute allows multiple labels. Defaults to True. When set to False (exclusive labels), missing gold labels are interpreted as 0.0 and the threshold is set to 0.0. positive_label (str): The positive label for a binary task with exclusive classes. Defaults to None. threshold (float): Cutoff to consider a prediction "positive". Defaults to 0.5 for multi-label, and 0.0 (i.e. whatever's highest scoring) otherwise. RETURNS (Dict[str, Any]): A dictionary containing the scores, with inapplicable scores as None: for all: attr_score (one of attr_micro_f / attr_macro_f / attr_macro_auc), attr_score_desc (text description of the overall score), attr_micro_p, attr_micro_r, attr_micro_f, attr_macro_p, attr_macro_r, attr_macro_f, attr_macro_auc, attr_f_per_type, attr_auc_per_type DOCS: https://spacy.io/api/scorer#score_cats Ng?rdrr c US$Nr r_its r)#Scorer.score_cats.. rRSur,)keyc US$rr_rs r)rrrr,c US$rr_rs r)rrsr!ur,c US$rr_rs r)rrrr,rCc38# UHoRv M g7fr'rD.0prfs r) $Scorer.score_cats.."C/Bmm/Bc38# UHoRv M g7fr'rHrs r)rr#@,?Sjj,?rc38# UHoRv M g7fr'rNrs r)rr$rrc3f# UH'oR5(a UROSv M) g7f)rdN)rqrz)raucs r)rr(s#WAV#]]__ #5AVs/1_score _score_descrrr_macro_p_macro_r_macro_f _macro_auc _f_per_type _auc_per_typerKrQzF ()zmacro Fz macro AUC)rrarZrrrrr?r"r#r$maxrr<sumrDrHrNrRrqrz)rrrrrrrrru f_per_type auc_per_typer pred_catsrr gold_cats pred_score gold_score pred_label gold_label micro_prf label_prfn_catsmacro_pmacro_rmacro_f macro_aucresultspositive_label_fs r) score_catsScorer.score_catssv\  *II5;DT)U& J),Y__->DT)U& J+z*--2--z*--2-z*--2--),Yz*--2--"),Y__->DT)U& J:&))Q.))Q RJ #**,I LLILL (L LLILL (L LLILL (L-Z6)Cz/@/@/BCCfL@J,=,=,?@@6I@J,=,=,?@@6I WATATAVW W   fFOTfK $fH y22fH y//fH y//fH wfH wfH wfJ fK j>N>N>P"Q>Pda1iik>>P"QfM "BNBTBTBV%BV$!kkmm1775BV%#  v;! KN&${';<^LSQ '7GtfFO $.1.1A,CGtfK( ) '.$x/@'AGtfFO $,5GtfK( )(/$z/B'CGtfFO $,7GtfK( )i=A LKz#R%s.U U UU UU*U", /U(0negative_labelsc 0nUGHVn0nURRHnXeURUR4'M URRHnUR URUR4S5nUcM0UR n X;a [5X9'URn U cM^URn X;aX;aMvX:XaX9=RS- sl MX;aX9=RS- sl MX;aX9=RS- sl MX9=RS- sl X9=RS- sl GM GMY [5n UR5H`n U =RU R- sl U =RU R- sl U =RU R- sl Mb [U5S-n[SUR555U- n[SUR555U- n[SUR555U- nSU RSS S U R S U R"S U RS USUSUSUR%5VVs0sHunnUUR'5_M snn0 nU$s snnf)aReturns PRF for predicted links on the entity level. To disentangle the performance of the NEL from the NER, this method only evaluates NEL links for entities that overlap between the gold reference and the predictions. examples (Iterable[Example]): Examples to score negative_labels (Iterable[str]): The string values that refer to no annotation (e.g. "NIL") RETURNS (Dict[str, Any]): A dictionary containing the scores. DOCS: https://spacy.io/api/scorer#score_links Nr rCc38# UHoRv M g7fr'rrs r)r%Scorer.score_links..{rrc38# UHoRv M g7fr'rrs r)rr.|rrc38# UHoRv M g7fr'r rs r)rr.}rr nel_scorenel_score_desczmicro F nel_micro_p nel_micro_r nel_micro_f nel_macro_p nel_macro_r nel_macro_fnel_f_per_type)rents start_charend_charrrrrkb_id_r"r#r$rr<rrNrDrHrrR)rr+rrrgold_ent_by_offsetgold_entpred_entrrur:predr r!n_labelsr#r$r%rrr's r) score_linksScorer.score_linksFs G!# #--22OWH$7$79J9J#KL3$--22.22(((*;*;J #**,I LLILL (L LLILL (L LLILL (L-z?V+Cz/@/@/BCChN@J,=,=,?@@8K@J,=,=,?@@8K)**yI//I,,I,,GGG:;K;K;MN;M41a199;;MN   Os!K(head)r head_attr head_getter ignore_labelsrrFrGrHc X[5n[5n [5n [5n UGHn U Rn U RnU R n[5n0n[ U 5HunnU"UU5nU"UU5nUU;aoUU;agURUURU45 UU ;a [5U U'UU;a [5UU'UURUURU45 MMU RU5 M [5n0nUGHxnURR5(aM%URRURS:waSnOURURSnUU ;dMsU"UU5nU"UU5nUU;dMURR5(dMURRURS:XaURURSnOSnUbUc-U=RS- slU =RS- slGM'URUUU45 UU ;a [5U U'UU;a [5UU'UURUUU45 GM{ U RUU5 U HHnU URUR!U[55UR!U[555 MJ UR[SU55[SU555 GM [#U5S:aXUS3UR$US3U R$US3U R'5VVs0sHunnUUR)5_M snn0$US3SUS3SUS3S0$s snnf) aReturns the UAS, LAS, and LAS per type scores for dependency parses. examples (Iterable[Example]): Examples to score attr (str): The attribute containing the dependency label. getter (Callable[[Token, str], Any]): Defaults to getattr. If provided, getter(token, attr) should return the value of the attribute for an individual token. head_attr (str): The attribute containing the head token. Defaults to 'head'. head_getter (Callable[[Token, str], Token]): Defaults to getattr. If provided, head_getter(token, attr) should return the value of the head for an individual token. ignore_labels (Tuple): Labels to ignore while scoring (e.g., punct). missing_values (Set[Any]): Attribute values to treat as missing annotation in the reference annotation. RETURNS (Dict[str, Any]): A dictionary containing the scores: attr_uas, attr_las, and attr_las_per_type. DOCS: https://spacy.io/api/scorer#score_deps r Nrc3*# UH oSSv M g7frnr_ritems r)r$Scorer.score_deps..s3!Hc3*# UH oSSv M g7frnr_rKs r)rrMs8XidbqirN_uas_las _las_per_type)rrrZrrrrrrrrrrstripr#r?rr<rNrrR)rrrrFrGrHrr unlabelledlabelledlabelled_per_deprrrrr gold_depsgold_deps_per_deprrdeprE pred_depspred_deps_per_dep gold_headrrs r) score_depsScorer.score_depssBZ :6%G((H((H%%EI02 !*8!4 UD)"5)4n,-/! vtvvs&;<&664Y->&MMQ.M$KK1,KK%MM69c*BC"*::8@ 0 5"*;;9< 1# 6-c266 37OP5"6   y) 4' %//%))#su57H7L7LSRURW7X(  333S8Xi8X5X k p z?Q & z00& x& &/?/E/E/G)/Gtq!Aqyy{N/G) & t& t& &  )s2N&)rr)!rTrUrVrWrXDEFAULT_PIPELINEr r]rr*rboolrrrz staticmethodrgetattrMISSING_VALUESrrr rrrrrrr\r)rCr]r^r_r,r)r}r}os$%) *:  j !#3-   *EJ )=A c3h4/Xg%6/$sCx.//b /6#1 017#0101%s*+ 01 C 01 c3h0101d /6#1 Q7#QQ%s*+ Q C Q c3hQQf 8?:>#b7#bb#sXd^34 b !3%+!67 b  bb c3hbbH -4 0 2 (,%)E7#EE#sS) E  E  E! EE?E c3hEENB7#B9A#B c3hBBH /65<'7'9#1i7#ii%s*+ i  i uclE12 i }iCi c3hiir,r}rr%c [[5nUGHnURRS5(dM&URRVs1sH%oDR UR UR4iM' nnURRnURRHnUR U;a[5X'R 'XgR URn[U5(dMSURUSUSS-n [SU 55(dMUR USUSS-4n X;a4X'R =RS- sl URU 5 MX'R =RS- slM UHupn X+=R S- slM GM [5nUR#5HnX- nM [U5S:aXUR$UR&UR(UR+5VVs0sHunnUUR-5_M snnS.$SSSSS.$s snfs snnf)zGCompute micro-PRF and per-entity PRF scores for a sequence of examples.ENT_IOBrr c3># UHoRS:gv M g7f)rN)ent_iob)rrs r)rget_ner_prf.. s>ve}})vs)ents_pents_rents_f ents_per_typeN)rryrr:rrrrrxr<allr"remover#r$rrDrHrNrrR)rkwargsregere align_x2yr@indicesg_spanrrurrtotalsrrrs r) get_ner_prfrys *Ntt""9-- 57TTYY?Y((AGGQUU+Y?LL$$  Hn42:*/>G7||gaj72;?;>v>>>#??GAJ aHC|&7::a?: S)&7::a?:" "' E#  ! $ $ ) $"'+.ZF$$& ' 6{Q&&mmmm9G9M9M9OP9OAan9OP   !   ?@8Qs ,I4I$ c .[R"U5nUcUnO[R"U5nURS:XaI[R"U5nS/UR-nURSXS'UR U5nO[R"XS9nURn[ S5/U-n[ S5/U-n[ SS5Xs'[ SS5X'X@[U5U[U5--S- RU5n U $![ap [R"U5n[R"U5n[RRX@[U5U[U5--S- U5n U $f=f)a[ Integrate along the given axis using the composite trapezoidal rule. If `x` is provided, the integration happens in sequence along its elements - they are not sorted. Integrate `y` (`x`) along each 1d slice on the given axis, compute :math:`\int y(x) dx`. When `x` is specified, this integrates along the parametric curve, computing :math:`\int_t y(t) dt = \int_t y(t) \left.\frac{dx}{dt}\right|_{x=x(t)} dt`. Parameters ---------- y : array_like Input array to integrate. x : array_like, optional The sample points corresponding to the `y` values. If `x` is None, the sample points are assumed to be evenly spaced `dx` apart. The default is None. dx : scalar, optional The spacing between sample points when `x` is None. The default is 1. axis : int, optional The axis along which to integrate. Returns ------- trapezoid : float or ndarray Definite integral of `y` = n-dimensional array as approximated along a single axis by the trapezoidal rule. If `y` is a 1-dimensional array, then the result is a float. If `n` is greater than 1, then the result is an `n`-1 dimensional array. See Also -------- cumulative_trapezoid, simpson, romb Notes ----- Image [2]_ illustrates trapezoidal rule -- y-axis locations of points will be taken from `y` array, by default x-axis distances between points will be 1.0, alternatively they can be provided with `x` array or with `dx` scalar. Return value will be equal to combined area under the red lines. References ---------- .. [1] Wikipedia page: https://en.wikipedia.org/wiki/Trapezoidal_rule .. [2] Illustration image: https://en.wikipedia.org/wiki/File:Composite_trapezoidal_rule_illustration.png Examples -------- Use the trapezoidal rule on evenly spaced points: >>> import numpy as np >>> from scipy import integrate >>> integrate.trapezoid([1, 2, 3]) 4.0 The spacing between sample points can be selected by either the ``x`` or ``dx`` arguments: >>> integrate.trapezoid([1, 2, 3], x=[4, 6, 8]) 8.0 >>> integrate.trapezoid([1, 2, 3], dx=2) 8.0 Using a decreasing ``x`` corresponds to integrating in reverse: >>> integrate.trapezoid([1, 2, 3], x=[8, 6, 4]) -8.0 More generally ``x`` is used to integrate along a parametric curve. We can estimate the integral :math:`\int_0^1 x^2 = 1/3` using: >>> x = np.linspace(0, 1, num=50) >>> y = x**2 >>> integrate.trapezoid(y, x) 0.33340274885464394 Or estimate the area of a circle, noting we repeat the sample which closes the curve: >>> theta = np.linspace(0, 2 * np.pi, num=1000, endpoint=True) >>> integrate.trapezoid(np.cos(theta), x=np.sin(theta)) 3.141571941375841 ``trapezoid`` can be applied along a specified axis to do multiple computations in one call: >>> a = np.arange(6).reshape(2, 3) >>> a array([[0, 1, 2], [3, 4, 5]]) >>> integrate.trapezoid(a, axis=0) array([1.5, 2.5, 3.5]) >>> integrate.trapezoid(a, axis=1) array([2., 8.]) Nr raxisrfg@) ro asanyarrayndimdiffshapereshapeslicetuplerrvasarrayrreduce) rnrodxr|drndslice1slice2rets r) trapezoidr)saL aAy  MM!  66Q; AC!&&LE''!*EK % A%A BDk]R FDk]R FD>FLr?FLSeFm$qv'7783>CCDI J S JJqM JJqMffmmA5=!1AeFm4D!DEKTR J Ss'1DA6FFc[[R"U55S:wa:[[R R [R"U5S95e[X5up#n[X#5$)a#Compute Area Under the Receiver Operating Characteristic Curve (ROC AUC) from prediction scores. Note: this implementation is restricted to the binary classification task Parameters ---------- y_true : array, shape = [n_samples] or [n_samples, n_classes] True binary labels or binary label indicators. The multiclass case expects shape = [n_samples] and labels with values in ``range(n_classes)``. y_score : array, shape = [n_samples] or [n_samples, n_classes] Target scores, can either be probability estimates of the positive class, confidence values, or non-thresholded measure of decisions (as returned by "decision_function" on some classifiers). For binary y_true, y_score is supposed to be the score of the class with greater label. The multiclass case expects shape = [n_samples, n_classes] where the scores correspond to probability estimates. Returns ------- auc : float References ---------- .. [1] `Wikipedia entry for the Receiver operating characteristic `_ .. [2] Fawcett T. An introduction to ROC analysis[J]. Pattern Recognition Letters, 2006, 27(8):861-874. .. [3] `Analyzing a portion of the ROC curve. McClish, 1989 `_ rKrt) r<rorprvrrwrx _roc_curve_auc)y_truey_scorefprtpr_s r)ryrysYH 299V "++"))F2C+DEEV-KCa >r,c[X5up#n[RSU4n[RSU4n[RUSS-U4nUSS::a0[R"[RUR 5nOX"S- nUSS::a0[R"[RUR 5nOX3S- nXVU4$)aECompute Receiver operating characteristic (ROC) Note: this implementation is restricted to the binary classification task. Parameters ---------- y_true : array, shape = [n_samples] True binary labels. If labels are not either {-1, 1} or {0, 1}, then pos_label should be explicitly given. y_score : array, shape = [n_samples] Target scores, can either be probability estimates of the positive class, confidence values, or non-thresholded measure of decisions (as returned by "decision_function" on some classifiers). Returns ------- fpr : array, shape = [>2] Increasing false positive rates such that element i is the false positive rate of predictions with score >= thresholds[i]. tpr : array, shape = [>2] Increasing true positive rates such that element i is the true positive rate of predictions with score >= thresholds[i]. thresholds : array, shape = [n_thresholds] Decreasing thresholds on the decision function used to compute fpr and tpr. `thresholds[0]` represents no instances being predicted and is arbitrarily set to `max(y_score) + 1`. Notes ----- Since the thresholds are sorted from low to high values, they are reversed upon returning them to ensure they correspond to both ``fpr`` and ``tpr``, which are sorted in reversed order during their calculation. References ---------- .. [1] `Wikipedia entry for the Receiver operating characteristic `_ .. [2] Fawcett T. An introduction to ROC analysis[J]. Pattern Recognition Letters, 2006, 27(8):861-874. rr rf)_binary_clf_curveror_repeatnanr)rrfpstps thresholdsrrs r)rrs\-V=Cj %%3-C %%3-Cz!}q(*45J 2w!|ii *Gm 2w!|ii *Gm Z r,cSn[R"U5n[R"U5nX:Hn[R"USS9SSS2nXnXnSn[R"[R"U55Sn[R XPR S- 4n[X-5UnSU-U- nXX4$)aCalculate true and false positives per binary classification threshold. Parameters ---------- y_true : array, shape = [n_samples] True targets of binary classification y_score : array, shape = [n_samples] Estimated probabilities or decision function Returns ------- fps : array, shape = [n_thresholds] A count of false positives, at index i being the number of negative samples assigned a score >= thresholds[i]. The total number of negative samples is equal to fps[-1] (thus true negatives are given by fps[-1] - fps). tps : array, shape = [n_thresholds <= len(np.unique(y_score))] An increasing count of true positives, at index i being the number of positive samples assigned a score >= thresholds[i]. The total number of positive samples is equal to tps[-1] (thus false negatives are given by tps[-1] - tps). thresholds : array, shape = [n_thresholds] Decreasing score values. ? mergesort)kindNrfrr )roravelargsortwhererrsize_stable_cumsum) rr pos_labeldesc_score_indicesweightdistinct_value_indicesthreshold_idxsrrs r)rrs8I XXf FhhwG FG+>ttD)G  'F F  XXbggg&67:UU1;;?BCN  ). 9C n s "C W, ,,r,c @[R"X[RS9n[R"X[RS9n[R"[R "UR SUS9XRUSS95(d[[R5eU$)aUse high precision for cumsum and check that final value matches sum Parameters ---------- arr : array-like To be cumulatively summed as flat axis : int, optional Axis along which the cumulative sum is computed. The default (None) is to compute the cumsum over the flattened array. rtol : float Relative tolerance, see ``np.allclose`` atol : float Absolute tolerance, see ``np.allclose`` )r|dtyperfr{T)rtolatol equal_nan) rocumsumfloat64rrpisclosetakervrE163)arrr|rroutexpecteds r)rrRsu ))C"** 5CvvcBJJ7H 66 HHRdH #Xtt    %% Jr,c[R"U5n[R"U5nSn[R"U5n[R"US:5(aG[R"US:*5(aSnO&[ [ RRUS95eU[X5-n[U[R5(aURRU5nU$)aCompute Area Under the Curve (AUC) using the trapezoidal rule This is a general function, given points on a curve. For computing the area under the ROC-curve, see :func:`roc_auc_score`. Parameters ---------- x : array, shape = [n] x coordinates. These must be either monotonic increasing or monotonic decreasing. y : array, shape = [n] y coordinates. Returns ------- auc : float r rrf)ro)rorranyrprvrE164rxr isinstancememmaprtype)rorn directionrareas r)rrls$  A  AI B vvb1f~~ 66"'??IV[[//!/45 5 y &D$ ""zzt$ Kr,)Nrrf)Ngh㈵>g:0yE>)+ collectionsrtypingrrrrrr r r r numpyroerrorsr morphologyrtokensrrrtrainingrutilrrlanguagerr_ frozensetrcrrar}r]ryrryrrrrr_r,r)rs#   "$$2"U=)0I0If  :F F R* (7+* $sCx.* bL'T@ F3-l4#r,