h 4SSKrSSKrSSKrSSKrSSKrSSKrSSKrSSKrSSKrSSK r SSK r SSK r SSK r SSK r SSKrSSKrSSKrSSKJr SSKJr SSKJr SSKJr SSKJrJrJrJrJrJrJ r J!r!J"r"J#r#J$r$J%r%J&r&J'r'J(r(J)r)J*r* SSK+r+SSK,r,SSK-r-SSK.r.SSK/r/SSK+J0r0J1r1 SSK2J3r3 SS K4J5r5J6r6 SS K7J8r8J9r9 SS K:J;r;Jr>J?r?J@r@JArA SSKBrCSS K:JErEJFrFJGrG S SKHJIrI S SKJJKrKJCrCJLrLJMrM S SKNJOrOJPrPJQrQ S SKRJSrS \(aS SKTJUrUJVrV S SKWJXrXJYrY S SKZJ[r[ \-R"\-R5Rr_Sr`/SQra/SQrb\R"S5rd\R"5rf\fR\R"S55 \dR\f5 "SS5rj"SS\/R5rk"SS\l5rm"S S!\n5roS"\pS#\q4S$jrrS"\pS#\$\p4S%jrsS"\pS#\(S&4S'jrtS(\pS)\(S&S#S4S*jruS+\S#\4S,jrvS+\)\p\4S#\)\l\n44S-jrwS.\S#\4S/jrx\o"5ryS0\y\y\y\m"5S1.S(\)\p\4S2\)S3\q4S4\)\p\\p4S5\)\p\\p4S6\)\p\\p4S7\)\\p\4\<4S#S&4S8jjrzS0\y\y\y\m"5S1.S(\pS2\)S3\q4S4\)\p\\p4S5\)\p\\p4S6\)\p\\p4S7\)\\p\4\<4S#S&4S9jjr{SS0\y\y\y\m"5S:.S;\S<\$\\p\4S2\)S3\q4S4\)\p\\p4S5\)\p\\p4S6\)\p\\p4S7\)\\p\4\<4S#S&4S=jjr|\m"5S0\y\y\yS>S0S?.S7\)\\p\4\<4S<\\p\4S2\)S3\q4S4\)\p\\p4S5\)\p\\p4S6\)\p\\p4S@\qSA\qS#S&4SBjjr}S7\)\\p\4\<4S#\\p\\p\444SCjr~S7\4S+\)\p\4SI\\p\4SJ\qS#\<4SKjjr\m"5S>4SL\pSI\\p\4SJ\q4SMjjrS#\!\p4SNjrS(\pS#\$\p4SOjrSSP\pSQ\pSR\qS#\$\q4SSjjrSSQ\pSR\qS#\$\q4STjjrSU\pS#\'\p\p44SVjrSP\pS#\p4SWjrSQ\pS#\$\p4SXjrSP\pS#\q4SYjrSP\pS#\p4SZjrSP\pS#\$\p4S[jrS\\pS]\pS#\q4S^jrS+\)\p\4S#\\p\44S_jrS+\)\p\4S#\\p\44S`jrS(\pS#\q4SajrS(\pS#\4SbjrSc\>Sd\>Se\>S#S4SfjrSg\pS#\!\p4ShjrSS>Si.Sg\)\p\!\p4Sj\$\Sk\qS#\ GR(4Sljjr\S+\)\p\4S#\ \4Smj5r\S#\\SS44Snj5rS#\q4SojrS#\q4SpjrSq\S#\p4SrjrSs\St\S#\q4SujrSSv\qSw\qS#\$\K4SxjjrSyrS+\)\p\4S#\%4SzjrS{\\)\p\%4S#\%4S|jrS{\\)\p\%4S#\%4S}jrS{\\)\p\%4S#\%4S~jrS\\p/\4S#\\p/\44SjrS\\p/\4S\\p\4S\pS#\4SjrS\\p\!\l4S#\\p\!\l44SjrS\\p\!\l4S\pS\pS#\\p\!\l44SjrSS\S\S\S\$\S#\'\\44 SjjrS\SS#\!S4SjrS\SS#\!S4SjrSrS\\p\/\44S6\\pS#\4SjrS\S\\p\\/\44S6\\pS#S4SjrS\\p\/\44S6\\pS#\\p\44SjrS\\p\4S\\p\\/\44S6\\pS#\\p\44SjrS+\)\p\4S\\p\\/S44S6\\pS#\4SjrS+\)\p\4S\\p\\/S44S6\\pS#\4SjrS(\pS\)\p\4S#\4SjrS\pS#\p4SjrSL\pS#\p4SjrS\\pSL\pS#\'\!\p\!\q44SjrS7\)\\p\4\<4S#\<4SjrS\\p\4S#\\p\l44SjrS>S.Sq\\p\l4S\qS#\\p\44SjjrS7\S.S\\p\4S\!\pS\qS#\ \'\!\p\44SjjjrS\S#\!\p4Sjr\m"54S\!\\p\$\4SI\\p\$\4S#\\p\$\44Sjjr"SS5rS#\@4SjrSrS\S#\q4SjrS\pS#\q4SjrS\SSSS(\pS\\pS\!S\/\#4S\"\p\4S#\ S4 SjrSrSrSrSrS#\4SjrS#\\p\!\p44SjrSrSS\S\pS#\q4SjjrSS\S\pS\qS#\4Sjjrg!\Da SrCGNf=f)N) defaultdict)contextmanager)Path) ModuleType) TYPE_CHECKINGAnyCallableDict GeneratorIterableIteratorListMappingNoReturnOptionalPatternSetTupleTypeUnioncast)Registry RegistryError) Requirement)InvalidSpecifier SpecifierSet)InvalidVersionVersion)AdamConfigConfigValidationErrorModelNumpyOps Optimizerget_current_ops) compoundingdecayingfix_random_seed)about) CudaStreamcupyimportlib_metadata is_windows)OLD_MODEL_SHORTCUTSErrorsWarningsORTH)Language PipeCallable)DocSpan)Vocabi)csdadeelengrcidlbmkptrusrtath) paths variablessystemnlp componentscorporatraining pretraining initializespacyz)[%(asctime)s] [%(levelname)s] %(message)sc\rSrSrSrSrg)ENV_VARSfSPACY_CONFIG_OVERRIDESN)__name__ __module__ __qualname____firstlineno__CONFIG_OVERRIDES__static_attributes__rUD/home/james-whalen/.local/lib/python3.13/site-packages/spacy/util.pyrRrRfs/r\rRc J\rSrSr\R "SSSS9r\R "SSSS9r\R "SSSS9r\R "SSSS9r \R "SS SS9r \R "SS SS9r \R "SS SS9r \R "SS SS9r \R "SS SS9r\R "SSSS9r\R "SSSS9r\R "SSSS9r\R "SSSS9r\R "SSSS9r\R "SSSS9r\R "SS5r\R "SSSS9r\R "SSSS9r\S!Sj5r\S\\4Sj5r\S\S\S\4Sj5r\S\S\S\\\ \!\\"444Sj5r#\S\S\S\$4Sj5r%S r&g)"registryjrP languagesT) entry_points architectures tokenizers lemmatizerslookupsdisplacy_colorsmisc callbacksbatchersreaders augmentersloggersscorersvectors factoriesinternal_factoriesmodelsclireturnNc2SSKJnJn U(dU"5 gg)z?Ensure the registry is populated with all necessary components.r))REGISTRY_POPULATEDpopulate_registryN) registrationsrvrw)clsrvrws r]ensure_populatedregistry.ensure_populateds I!  "r\cUR5 /n[R"U5HEup#URS5(aM[ U[ 5(dM4UR U5 MG [U5$)zList all available registries._)rzinspect getmembers startswith isinstancerappendsorted)rynamesnamevalues r]get_registry_namesregistry.get_registry_namess_ "--c2KD??3''Juh,G,G T"3e}r\ registry_name func_namec UR5 [X5(dNSRUR55=(d Sn[ [ R RXS95e[X5nURU5nU$![a URS5(a@URSS5nURU5s$![Ra Of=fSR[UR5R555=(d Sn[ [ R RX!US95Sef=f)z,Get a registered function from the registry., noner availablespacy. spacy-legacy.rreg_namerN)rzhasattrjoinrrr0E892formatgetattrgetrreplace cataloguerget_allkeysE893)ryrrrregfunc legacy_namers r]r registry.gets/ s**IIc4467A6E 2 2 2 WX Xc) 779%D  ##H--'///J 77;// .. &););)=">?I6I """i#   1;B2ECEC+(E*C++A+Ec UR5 [X5(dNSRUR55=(d Sn[ [ R RXS95e[X5nURU5nU$![a URS5(a@URSS5nURU5s$![Ra Of=fSR[UR5R555=(d Sn[ [ R RX!US95Sef=f)aEFind information about a registered function, including the module and path to the file it's defined in, the line number and the docstring, if available. registry_name (str): Name of the catalogue registry. func_name (str): Name of the registered function. RETURNS (Dict[str, Optional[Union[str, int]]]): The function info. rrrrrrN)rzrrrrr0rrrfindrrrrrrr)ryrrrr func_inforrs r]r registry.finds0 s**IIc4467A6E 2 2 2 WX Xc) +I ##H--'///J 88K00 .. &););)=">?I6I """i#   rcUR5 [X5(dg[X5nURS5(a!UR SS5nX#;=(d XC;$X#;$)z4Check whether a function is available in a registry.Frr)rzrrrr)ryrrrrs r]has registry.hasse s**c)    ) )#++HoFK#9{'9 9r\rU)rtN)'rVrWrXrYrcreaterarcrdrerfrgrhrirjrkrlrmrnro_entry_point_factoriesrprrrs classmethodrzrstrrr rr rrintrboolrr[rUr\r]r_r_js8  +DII$$WoDQM!!'?I  gxd CF   7E =C  494!!,/! c8E#s(O,, -!!F          r\r_cj^\rSrSrSr\R S.S\SS4U4SjjjrSr S S jr S r Sr U=r $) SimpleFrozenDictzSimplified implementation of a frozen dict, mainly used as default function or method argument (for arguments that should default to empty dictionary). Will raise an error if user or spaCy attempts to add to dict. errorrrtNc2>[TU]"U0UD6 Xlg)zInitialize the frozen dict. Can be initialized with pre-defined values. error (str): The error message when user tries to assign to dict. N)super__init__r)selfrargskwargs __class__s r]rSimpleFrozenDict.__init__s $)&) r\c,[UR5eNNotImplementedErrorr)rkeyrs r] __setitem__SimpleFrozenDict.__setitem__!$**--r\c,[UR5err)rrdefaults r]popSimpleFrozenDict.poprr\c,[UR5err)rothers r]updateSimpleFrozenDict.updaterr\r)rVrWrXrY__doc__r0E095rrrrrr[ __classcell__rs@r]rrs= ,2;;ST....r\rc^\rSrSrSr\R S.S\SS4U4SjjjrSr S r S r S r S r S rSrSrSrU=r$)SimpleFrozenListanWrapper class around a list that lets us raise custom errors if certain attributes/methods are accessed. Mostly used for properties like Language.pipeline that return an immutable list (and that we don't want to convert to a tuple to not break too much backwards compatibility). If a user accidentally calls nlp.pipeline.append(), we can raise a more helpful error. rrrtNc,>Xl[TU]"U6 g)z`Initialize the frozen list. error (str): The error message when user tries to mutate the list. N)rrr)rrrrs r]rSimpleFrozenList.__init__s   $r\c,[UR5errrrrs r]rSimpleFrozenList.appendrr\c,[UR5errrs r]clearSimpleFrozenList.clearrr\c,[UR5errrs r]extendSimpleFrozenList.extendrr\c,[UR5errrs r]insertSimpleFrozenList.insertrr\c,[UR5errrs r]rSimpleFrozenList.poprr\c,[UR5errrs r]removeSimpleFrozenList.removerr\c,[UR5errrs r]reverseSimpleFrozenList.reverse rr\c,[UR5errrs r]sortSimpleFrozenList.sort#rr\)rVrWrXrYrr0E927rrrrrrrrrrr[rrs@r]rrsV,2;; S 4  .........r\rlangrtc(U[R;$)aCheck whether a Language class is already loaded. Language classes are loaded lazily, to avoid expensive setup code associated with the language data. lang (str): Two-letter language code, e.g. 'en'. RETURNS (bool): Whether a Language class has been loaded. )r_ra)rs r]lang_class_is_loadedr's 8%% %%r\c`SSKnUS:Xag/n[R"URR5HVnUR nUS:XaUR S5 M([R"U5(dMEUR U5 MX [R"XSS9nUS:XagU$)a& Given an IETF language code, find a supported spaCy language that is a close match for it (according to Unicode CLDR language-matching rules). This allows for language aliases, ISO 639-2 codes, more detailed language tags, and close matches. Returns the language code if a matching language is available, or None if there is no matching language. >>> find_matching_language('en') 'en' >>> find_matching_language('pt-BR') # Brazilian Portuguese 'pt' >>> find_matching_language('fra') # an ISO 639-2 code for French 'fr' >>> find_matching_language('iw') # obsolete alias for Hebrew 'he' >>> find_matching_language('no') # Norwegian 'nb' >>> find_matching_language('mo') # old code for ro-MD 'ro' >>> find_matching_language('zh-Hans') # Simplified Chinese 'zh' >>> find_matching_language('zxx') None rNxxmul ) max_distance) spacy.langpkgutil iter_modulesr__path__rr langcodes tag_is_validclosest_supported_match)rrPpossible_languagesmodinfocodematchs r]find_matching_languager2s6 t|'' (;(;<|| 4<  % %e ,  # #D ) )  % %d + =  - -dUV WE ~ r\r4cHU[R;a[RRU5$[R"SU3S5n[U[XRS55 [RRU5$![ an[ U5nO&![RRa SnOf=fU(a!Un[R"SU3S5nSnAN[ [RRXS95UeSnAff=f)zuImport and load a Language class. lang (str): IETF language code, such as 'en'. RETURNS (Language): Language class. z.lang.rPN)rerrr)r_rar importlib import_module ImportErrorrr tag_parserLanguageTagErrorr0E048rset_lang_classr__all__)rmodulerrs r]get_lang_classris x!!!!!%%d++ S,,vdV_gFF tWV^^A->?@    ! !$ '' S .t4''88  "006$'J!&++"4"4$"4"HIsR Ss;B D! B'&D' C D C  &D5'DD!rryc>[RRXS9 g)zSet a custom Language class name that can be loaded via get_lang_class. name (str): Name of Language class. cls (Language): Language class. )rN)r_raregister)rrys r]r r s  /r\pathcF[U[5(a [U5$U$)zEnsure string is converted to a Path. path (Any): Anything. If string, it's converted to Path. RETURNS: Path or original argument. )rrrrs r] ensure_pathrs $Dz r\cL[U5nUR5(a[R"U5$UR UR S-5nUR5(a[R "U5$[[RRUS95e)zLoad JSON language data using the given path as a base. If the provided path isn't present, will attempt to load a gzipped version before giving up. path (str / Path): The data to load. RETURNS: The loaded data. z.gzr) rexistssrsly read_json with_suffixsuffixread_gzip_json ValueErrorr0E160rrs r]load_language_datarsz t D {{}}t$$   DKK%/ 0D {{}}##D)) V[[''T'2 33r\r c*[US5(d/[[RR [ U5S95e[ [[R[RURR55nUR$)zdGet the path of a Python module. module (ModuleType): The Python module. RETURNS (Path): The path. rW)r )rrr0E169rreprrrosPathLikesysmodulesrW__file__parent)r  file_paths r]get_module_pathr)sh 6< ( (++4<+@AAT"++s{{63D3D'E'N'NOPI   r\Tvocabdisableenableexcludeconfigr+r8r,r-r.r/c>UUUUUS.n[U[5(aURS5(a [UR SS55"5$[ U5(a [ U40UD6$[U5R5(a[[U540UD6$O[US5(a [U40UD6$U[;a.[[RRU[US95e[[R RUS95e)a{Load a model from a package or data path. name (str): Package name or model path. vocab (Vocab / True): Optional vocab to pass in on initialization. If True, a new Vocab object will be created. disable (Union[str, Iterable[str]]): Name(s) of pipeline component(s) to disable. enable (Union[str, Iterable[str]]): Name(s) of pipeline component(s) to enable. All others will be disabled. exclude (Union[str, Iterable[str]]): Name(s) of pipeline component(s) to exclude. config (Dict[str, Any] / Config): Config overrides as nested dict or dict keyed by section values in dot notation. RETURNS (Language): The loaded nlp object. r*zblank:r)rfullr)rrrrr is_packageload_model_from_packagerrload_model_from_pathrr/IOErrorr0E941rE050)rr+r,r-r.r/rs r] load_modelr:s, F$ ??8 $ $!$,,x"<=? ? d  *4:6: : :    'T =f= =  x #D3F33 ""fkk((d9LT9R(STT &++$$$$/ 00r\cP[R"U5nURXX4US9$)aGLoad a model from an installed package. name (str): The package name. vocab (Vocab / True): Optional vocab to pass in on initialization. If True, a new Vocab object will be created. disable (Union[str, Iterable[str]]): Name(s) of pipeline component(s) to disable. Disabled pipes will be loaded but they won't be run unless you explicitly enable them by calling nlp.enable_pipe. enable (Union[str, Iterable[str]]): Name(s) of pipeline component(s) to enable. All other pipes will be disabled (and can be enabled using `nlp.enable_pipe`). exclude (Union[str, Iterable[str]]): Name(s) of pipeline component(s) to exclude. Excluded components won't be loaded. config (Dict[str, Any] / Config): Config overrides as nested dict or dict keyed by section values in dot notation. RETURNS (Language): The loaded nlp object. r*)rrload)rr+r,r-r.r/rys r]r5r5s*2  ! !$ 'C 88%Y_8 ``r\)metar+r,r-r.r/ model_pathr=c UR5(d&[[RR US95eU(d [ U5nUS- n[ USS9n[XxS9n[UUUUUUS9n U RXUS9$)aLoad a model from a data directory path. Creates Language class with pipeline from config.cfg and then calls from_disk() with path. model_path (Path): Model path. meta (Dict[str, Any]): Optional model meta. vocab (Vocab / True): Optional vocab to pass in on initialization. If True, a new Vocab object will be created. disable (Union[str, Iterable[str]]): Name(s) of pipeline component(s) to disable. Disabled pipes will be loaded but they won't be run unless you explicitly enable them by calling nlp.enable_pipe. enable (Union[str, Iterable[str]]): Name(s) of pipeline component(s) to enable. All other pipes will be disabled (and can be enabled using `nlp.enable_pipe`). exclude (Union[str, Iterable[str]]): Name(s) of pipeline component(s) to exclude. Excluded components won't be loaded. config (Dict[str, Any] / Config): Config overrides as nested dict or dict keyed by section values in dot notation. RETURNS (Language): The loaded nlp object. rz config.cfgT for_overrides) overrides)r+r,r-r.r=)r.rB) rr7r0E052rget_model_meta dict_to_dot load_configload_model_from_config from_disk) r>r=r+r,r-r.r/ config_pathrBrJs r]r6r6s8     fkk((j(9:: j)|+KF$7I  :F    C == = JJr\F)r=r+r,r-r. auto_fillvalidaterJrKc SU;a&[[RRUS95eUSnSU;dUSc&[[RRUS95e[ US5n U R UUUUUUUUS9n U $)aCreate an nlp object from a config. Expects the full config file including a section "nlp" containing the settings for the nlp object. name (str): Package name or model path. meta (Dict[str, Any]): Optional model meta. vocab (Vocab / True): Optional vocab to pass in on initialization. If True, a new Vocab object will be created. disable (Union[str, Iterable[str]]): Name(s) of pipeline component(s) to disable. Disabled pipes will be loaded but they won't be run unless you explicitly enable them by calling nlp.enable_pipe. enable (Union[str, Iterable[str]]): Name(s) of pipeline component(s) to enable. All other pipes will be disabled (and can be enabled using `nlp.enable_pipe`). exclude (Union[str, Iterable[str]]): Name(s) of pipeline component(s) to exclude. Excluded components won't be loaded. auto_fill (bool): Whether to auto-fill config with missing defaults. validate (bool): Whether to show config validation errors. RETURNS (Language): The loaded nlp object. rJr/r)r+r,r-r.rJrKr=)rr0E985rE993r from_config) r/r=r+r,r-r.rJrK nlp_configlang_clsrJs r]rGrG2s: F++6+:;;J Z:f#5#=++:+>??j01H      C Jr\cURS05R5VVs0sHupSU;dM SU;dMX_M snn$s snnf)zRETURNS (List[str]): All sourced components in the original config, e.g. {"source": "en_core_web_sm"}. If the config contains a key "factory", we assume it refers to a component factory. rKfactorysource)ritems)r/rcfgs r]get_sourced_componentsrXdsW L"5;;= =ID C  $,O  =  s AAA dot_names.c0n/n/nUHnUcURU5 MURS5SnXb;aT[RX5(a[R SX05SnO[R X5nXrU'UR[ X%55 M U(a [XS9e[U5$![ a, SU3nURURS5US.5 Mf=f)a&Resolve one or more "dot notation" names, e.g. corpora.train. The paths could point anywhere into the config, so we don't know which top-level section we'll be looking within. We resolve the whole top-level section, although we could resolve less -- we could find the lowest part of the tree. .rr/znot a valid section reference: )locmsg)r/errors) rsplitr_ is_promiseresolve dot_to_objectKeyErrorr!tuple) r/rYresolvedoutputr^rsectionresultr]s r]resolve_dot_namesrirsHF F < MM$ jjoa(G&&&v77%--x.IJ8TF%--fo>F$*! D mH;<$#6AA =  D7v> djjocBC DsC  2C?>C? init_filec [U5Rn[U5nUSSUSSUS3nXh- n UR5(d&[ [ R RU S95e[U UUUUUUS9$)aQHelper function to use in the `load()` method of a model package's __init__.py. vocab (Vocab / True): Optional vocab to pass in on initialization. If True, a new Vocab object will be created. disable (Union[str, Iterable[str]]): Name(s) of pipeline component(s) to disable. Disabled pipes will be loaded but they won't be run unless you explicitly enable them by calling nlp.enable_pipe. enable (Union[str, Iterable[str]]): Name(s) of pipeline component(s) to enable. All other pipes will be disabled (and can be enabled using `nlp.enable_pipe`). exclude (Union[str, Iterable[str]]): Name(s) of pipeline component(s) to exclude. Excluded components won't be loaded. config (Dict[str, Any] / Config): Config overrides as nested dict or dict keyed by section values in dot notation. RETURNS (Language): The loaded nlp object. rr}r-versionr)r+r=r,r-r.r/) rr'rDrr7r0rCrr6) rjr+r,r-r.r/r>r=data_dir data_paths r]load_model_from_init_pyrps2i''J * %Dv,qfaY/@AH%I     fkk((i(899   r\rB interpolatecN[U5n[[S9n[U5S:Xa,UR [ R R5XS9$U(aUR5(d'[[RRUSS95eURX1US9$)akLoad a config file. Takes care of path validation and section order. path (Union[str, Path]): Path to the config file or "-" to read from stdin. overrides: (Dict[str, Any]): Config overrides as nested dict or dict keyed by section values in dot notation. interpolate (bool): Whether to interpolate and resolve variables. RETURNS (Config): The loaded config.  section_orderrlrBrqz config filerr)rr CONFIG_SECTION_ORDERrfrom_strr$stdinreadis_filer7r0E053rrH)rrBrqrIr/s r]rFrFsd#K "6 7F ;3 IINN    +"5"5"7"7&++,,+M,RS S +   r\textc8[[S9RXUS9$)zLoad a full config from a string. Wrapper around Thinc's Config.from_str. text (str): The string config to load. interpolate (bool): Whether to interpolate and resolve variables. RETURNS (Config): The loaded config. rsru)r rwrx)r}rBrqs r]load_config_from_strrs'  4 5 > > { ? r\cl[[RR5R 55$)zvList all model packages currently installed in the environment. RETURNS (List[str]): The string names of the models. )listr_rrrrrUr\r]get_installed_modelsrs% '')..0 11r\cd[R"U5$![Ra gf=f)zGet the version of an installed package. Typically used to get model package versions. name (str): The name of the installed Python package. RETURNS (str / None): The version or None if package not installed. N)r-rmPackageNotFoundErrorr3s r]get_package_versionrs0!))$//  2 2s //rm constraint prereleasescUSR5(aSU3n[U5n[U5nX#lX;$![[4a gf=f)aCheck if a version (e.g. "2.0.0") is compatible given a version constraint (e.g. ">=1.9.0,<2.2.1"). If the constraint is a specific version, it's interpreted as =={version}. version (str): The version to check. constraint (str): The constraint string. prereleases (bool): Whether to allow prereleases. If set to False, prerelease versions will be considered incompatible. RETURNS (bool / None): Whether the version is compatible, or None if the version or constraint are invalid. r==N)isdigitrrrrr)rmrrspecs r]is_compatible_versionrse!}*& J''"# ? n -s?AAcUSR5(ag[U5nXlUVs/sHo3PM nn[ U5S:XaUSR S;ag[ SU55(ag[ SU55n[ SU55nU(aU(agg![a gf=fs snf) NrFr)>>=Tc3># UHoRS;v M g7f)rNoperator.0sps r] +is_unconstrained_version..)s 15R;;4 5c3># UHoRS;v M g7f))# UHoRS;v M g7f)rNrrs r]rr,rr)rrrrlenrany)rrrrspecs has_upper has_lowers r]is_unconstrained_versionrs!}J'# $BR$E  5zQ58,, ; 15 111???I???IY !  s B, B<, B98B9 requirementcZ[U5nUR[UR54$)z@Split a requirement like spacy>=1.2.3 into ("spacy", ">=1.2.3").)rrr specifier)rreqs r]split_requirementr4s$ k "C HHc#--( ))r\cV[U5RnSUSUSSUSS-S3$)zWGenerate a version range like >=1.2.3,<1.3.0 based on a given version (e.g. of spaCy). rz,From a version range like >=1.2.3,<1.3.0 return the lower pin.)rrz~=N)rrrm Exception)rspecsetrs r]get_model_lower_versionrBsN z*D}} 22||#     s5 55 AAc,[U5R$)zCheck whether a version is a prerelease version. version (str): The version, e.g. "3.0.0.dev1". RETURNS (bool): Whether the version is a prerelease version. )r is_prereleaserms r]is_prerelease_versionrNs 7  ) ))r\c,[U5R$)zGenerate the base version without any prerelease identifiers. version (str): The version, e.g. "3.0.0.dev1". RETURNS (str): The base version, e.g. "3.0.0". )r base_versionrs r]get_base_versionrWs 7  ( ((r\c|[U5nURSUR3$![[4a gf=f)zGet the major + minor version (without patch or prerelease identifiers). version (str): The version. RETURNS (str): The major + minor version or None if version is invalid. Nr[)r TypeErrorrmajorminor)rmvs r]get_minor_versionr`sE  G ggYay !! ~ &s (;; version_a version_bcb[U5n[U5nUSL=(a USL=(a X#:H$)a6Compare two versions and check if they match in major and minor, without patch or prerelease identifiers. Used internally for compatibility checks that should be insensitive to patch releases. version_a (str): The first version version_b (str): The second version. RETURNS (bool): Whether the versions match. N)r)rrabs r]is_minor_version_matchrms2 )$A)$A D= 5Qd] 5qv5r\cT[U5nURR5(d0[[R R URS95eUR5(aUR5(d1[[RR URSS95e[R"U5nSH7nX!;d X(aM[[RR US95e SU;Ga[[RUS5(d[!US5n[#U5nUbSU-nOSU;a S US-nOS n[$R&R US S US 3USU[RS9n[(R*"U5 [-US5(a^[$R.R US S US 3USUS[1[R5S9n[(R*"U5 U$)zLoad a model meta.json from a path and validate its contents. path (Union[str, Path]): Path to meta.json. RETURNS (Dict[str, Any]): The loaded meta. r meta.jsonrv)rrrm)setting spacy_versionrspacy_git_versionz git commit zversion unknownrr}rrm)model model_versionrmcurrent)rrrmexample)rr'rr7r0rCrr{r|rrrE054rr* __version__rrr1W095warningswarnrW094r)rr=r lower_versionwarn_msgs r] load_metar{s t D ;;    fkk((dkk(:;; ;;== fkk((dkk (LMM ??4 D.  dmmV[[///@A A/$$U%6%6_8MNN3D4IJM,];M( #m 3 $, -5H0I I 1 }}++faV ~6"9o%)) ,H MM( # #D$9 : :}}++faV ~6"9o_-/0A0AB ,H MM( # Kr\c4[U5n[US- 5$)zGet model meta.json from a directory path and validate its contents. path (str / Path): Path to model directory. RETURNS (Dict[str, Any]): The model's meta data. r)rr)rr>s r]rDrDs T"J Z+- ..r\c@[R"U5 g! g=f)zCheck if string maps to a package installed via pip. name (str): Name of package. RETURNS (bool): True if installed package, False if not. TF)r- distributionr3s r]r4r4s" ''-sc[R"U5n[[[[ [ R4UR55R$)zlGet the path to an installed package. name (str): Package name. RETURNS (Path): Path to installed package. ) rrrrrrr"r#r&r')rpkgs r]get_package_pathrs>  ! !$ 'C U3 +,cll; < C CCr\rtarget replacementcDUR5H;nXR;dMX#RURRU5'M= UR5H=nURH*nUR U5ULdMUR XB5 M, M? g)zReplace a node within a model with a new one, updating refs. model (Model): The parent model. target (Model): The target node. replacement (Model): The node to replace the target with. N)walklayersindex ref_names maybe_get_refset_ref)rrrnoderef_names r]replace_model_nodersx  [[ 5@KK ))&1 2 H!!(+v5 X3'r\commandc>[R"U[(+S9$)zSplit a string command using shlex. Handles platform compatibility. command (str) : The command to split RETURNS (List[str]): The split command. )posix)shlexr_r.)rs r] split_commandrs ;;w*n 55r\)rycaptureryrc [U[5(a[U5nUnOUnSRU5n[R "U[ RR5USSU(a[ROSU(a[ROSS9nURS:wacU(a\SUS 3nUS UR3- nUR bUS - nXeR - n[R""U5nXWlXGlUeURS:wa [(R*"UR5 U$![a+ [[RRXCSS95Sef=f) aRun a command on the command line as a subprocess. If the subprocess returns a non-zero exit code, a system exit is performed. command (str / List[str]): The command. If provided as a string, the string will be split using shlex.split. stdin (Optional[Any]): stdin to read from or None. capture (bool): Whether to capture the output and errors. If False, the stdout and stderr will not be redirected, and if there's an error, sys.exit will be called with the return code. You should use capture=False when you want to turn over execution to the command, and capture=True when you want to run the command more like a function. RETURNS (Optional[CompletedProcess]): The process object.  utf8FN)envinputencodingcheckstdoutstderrr) str_commandtoolzError running command: z zSubprocess exited with status z$ Process log (stdout and stderr): )rrrr subprocessrunr"environcopyPIPESTDOUTFileNotFoundErrorr0E970r returncoderSubprocessErrorretrr$exit)rryrcmd_listcmd_strrmessagers r] run_commandrsJ$'3 )((7#nn  !&-:??4(/:$$T  ~~w.wit<3CNN3CDD :: ! A BG zz !G**73   1   J%  KK  7!  E  s A$E5E6c#6# [R"5n[U5R5n[R"[ U55 Uv [R"[ U55 g![R"[ U55 f=f7f)aAChange current working directory and returns to previous on exit. path (str / Path): The directory to navigate to. YIELDS (Path): The absolute path to the current working directory. This should be used if the block needs to perform actions within the working directory, to prevent mismatches with relative paths. N)rcwdrar"chdirr)rprev_cwdrs r] working_dirr s^xxzH4j  "GHHS\   XXsABA5 B5!BBc## [[R"55nUv Sn[RS:a[ R "[U5US9 g[ R "[U5US9 g![a;n[R"[RRXS95 SnAgSnAff=f7f)zExecute a block in a temporary directory and remove the directory and its contents at the end of the with block. YIELDS (Path): The path of the temp directory. c^[R"U[R5 U"U5 gr)r"chmodstatS_IWRITE)rmfuncrexs r] force_remove"make_tempdir..force_remove6s t}}%t r\) )onexc)onerror)dirr]N)rtempfilemkdtempr$ version_infoshutilrmtreerPermissionErrorrrr1W091r)dres r] make_tempdirr$+s X    A G:   w & MM#a& 5 MM#a&, 7 : hmm**q*899:s:&C2A;CA;:C; C1B;6C;CCc[5RRS:Xag[5RRS:XagSSKng![a Nf=f![ a gf=f)zCheck if user is running spaCy from a Jupyter or Colab notebook by detecting the IPython kernel. Mainly used for the displaCy visualizer. RETURNS (bool): True if in Jupyter/Colab, False if not. ZMQInteractiveShellTzgoogle.colab._shellrNF) get_ipythonrrVrW NameError google.colabr)googles r] is_in_jupyterr+Csv = " " + +/D D = " " - -1F F G          s("A"A A AA A,+A,cP[[S5=(d [[S5$)zCheck if user is running spaCy from an interactive Python shell. Will return True in Jupyter notebooks too. RETURNS (bool): True if in interactive mode, False if not. ps1ps2)rr$rUr\r]is_in_interactiver/[s 3  5'#u"55r\objc*[US5(aURb UR$[US5(a UR$[US5(a1[URS5(aURR$[ U5$)zGet a human-readable name of a Python object, e.g. a pipeline component. obj (Any): The Python object, typically a function or class. RETURNS (str): A human-readable name. rrVr)rrrVrr!)r0s r]get_object_namer2dsq sF 4xxsJ||sK  WS]]J%G%G}}%%% 9r\func1func2c[U5(a[U5(dg[US5(a[US5(dgURUR:Hn[R"U5[R"U5:Hn[R "U5[R "U5:HnU=(a U=(a U$)aApproximately decide whether two functions are the same, even if their identity is different (e.g. after they have been live reloaded). Mostly used in the @Language.component and @Language.factory decorators to decide whether to raise if a factory already exists. Allows decorator to run multiple times with the same function. func1 (Callable): The first function. func2 (Callable): The second function. RETURNS (bool): Whether it's the same function (most likely). FrX)callablerrXr~getfilegetsourcelines)r3r4 same_name same_file same_codes r] is_same_funcr<ss E??(5// 5. ) )1O1O""e&8&88I&'//%*@@I&&u-1G1G1NNI  0 0y0r\require non_blockingcd[5n[cg[U[5(ag[US9$)N)r>)r%r+rr#)r=r>opss r]get_cuda_streamrAs0  C C " "|44r\c[cU$[R"URSURS9nUR XS9 U$)NC)orderdtype)stream)r,ndarrayshaperEset)rF numpy_arrayarrays r] get_asyncrLs> | [..cARARS + - r\c |[U5nURSS9nUR5RS5nSSS5 SR WVs/sH2o3R 5(dMS[ R"U5-PM4 sn5n[ R"U5$!,(df  Ns=fs snf)Nr)r |^) ropenrzr_rstripreescapecompile)rfile_entriespiece expressions r] read_regexrZs t D F #u**,$$T* $-4FWE ryy WFJ ::j !! $ # Gs B(B9.B9( B6rWcSRUVs/sHoR5(dMSU-PM sn5n[R"U5$s snf)zCompile a sequence of prefix rules into a regex object. entries (Iterable[Union[str, Pattern]]): The prefix rules, e.g. spacy.lang.punctuation.TOKENIZER_PREFIXES. RETURNS (Pattern): The regex object. to be used for Tokenizer.prefix_search. rOrPrrRrSrUrWrXrYs r]compile_prefix_regexr^sBGMG5{{};3;GMNJ ::j !!N A AcSRUVs/sHoR5(dMUS-PM sn5n[R"U5$s snf)zCompile a sequence of suffix rules into a regex object. entries (Iterable[Union[str, Pattern]]): The suffix rules, e.g. spacy.lang.punctuation.TOKENIZER_SUFFIXES. RETURNS (Pattern): The regex object. to be used for Tokenizer.suffix_search. rO$r\r]s r]compile_suffix_regexrbsBGMG5{{};53;GMNJ ::j !!Nr_cSRUVs/sHoR5(dMUPM sn5n[R"U5$s snf)zCompile a sequence of infix rules into a regex object. entries (Iterable[Union[str, Pattern]]): The infix rules, e.g. spacy.lang.punctuation.TOKENIZER_INFIXES. RETURNS (regex object): The regex object. to be used for Tokenizer.infix_finditer. rOr\r]s r]compile_infix_regexrds<gGgU5gGHJ ::j !!Hs AA default_funcc8[R"[X5$)a=Extend an attribute function with special cases. If a word is in the lookups, the value is returned. Otherwise the previous function is used. default_func (callable): The default function to execute. *lookups (dict): Lookup dictionary mapping string to attribute value. RETURNS (callable): Lexical attribute getter. ) functoolspartial_get_attr_unless_lookup)rerfs r] add_lookupsrjs   4l LLr\rfstringc:UHnX#;dM X2s $ U"U5$rrU)rerfrklookups r]riris)  > !  r\base_exceptionsc [U5nUHnUR5HupE[SU55(d&[[R R XES95eSRSU55nXF:wdMa[[RR XFS95e URU5 M [USS5nU$)aUpdate and validate tokenizer exceptions. Will overwrite exceptions. base_exceptions (Dict[str, List[dict]]): Base exceptions. *addition_dicts (Dict[str, List[dict]]): Exceptions to add to the base dict, in order. RETURNS (Dict[str, List[dict]]): Combined tokenizer exceptions. c3P# UHn[U[[5v M g7fr)rr3rrattrs r]rupdate_exc..sK{tz$t*c22{s$&)rorthsr1c32# UH o[v M g7frr2rqs r]rrss$HKD$ZK'u’) dictrVallrr0E055rrE056r expand_exc)rnaddition_dictsexc additionsorth token_attrsdescribed_orths r] update_excrs  C# !*!2 DK{KKK !3!3!3!PQQWW$HK$HHN% !3!3!3!STT "3 9$ S#u %C Jr\excssearchrc Sn[U5nUR5H9upVX;dM URX5nUVs/sH o"XU5PM n nXU'M; U$s snf)aJFind string in tokenizer exceptions, duplicate entry and replace string. For example, to add additional versions with typographic apostrophes. excs (Dict[str, List[dict]]): Tokenizer exceptions. search (str): String to find and replace. replace (str): Replacement. RETURNS (Dict[str, List[dict]]): Combined tokenizer exceptions. cZ[U5nU[RX5U['U$r)rxr3r)tokenrrfixeds r] _fix_tokenexpand_exc.._fix_tokens(U Dk))&:d  r\)rxrVr) rrrrnew_excs token_stringtokensnew_keyt new_values r]r|r|sk DzH $   !"**6;GAGHAAw7IH )W  !- OIsAlengthstartstopstepcUbUS:Xd[[R5eUcSnO US:aX- n[U[ SU55nUcUnO US:aX - n[U[ X55nX4$)Nr)r)rr0E057minmax)rrrrs r]normalize_slicersy LDAI%% }   Au &E |   vs5' (D ;r\spansr7c.Sn[XSS9n/n[5nUHjnURU;dMURS- U;dM*UR U5 UR [ URUR55 Ml [USS9nU$)ahFilter a sequence of spans and remove duplicates or overlaps. Useful for creating named entities (where one token can only be part of one entity) or when merging spans with `Retokenizer.merge`. When spans overlap, the (first) longest span is preferred over shorter spans. spans (Iterable[Span]): The spans to filter. RETURNS (List[Span]): The filtered spans. cNURUR- UR*4$r)endrspans r]filter_spans..)sDJJ!6 Dr\T)rrr)cUR$r)rrs r]rr2sTZZr\)r)rrIrrrrrange)r get_sort_key sorted_spansrh seen_tokensrs r] filter_spansr sEL%4@L FEK ::[ (TXX\-L MM$    uTZZ: ;  F 7 8F Mr\c:[[R"U65$r)r itertoolschain)rs r]filter_chain_spansr6s  / 00r\c[$r)rrUr\r]make_first_longest_spans_filterr:s r\gettersc@[R"[X55$r)r msgpack_dumpsto_dict)rr.s r]to_bytesr>s   ww8 99r\ bytes_datasetterscB[[R"U5X5$r) from_dictr msgpack_loads)rrr.s r] from_bytesrBs U((4g GGr\c0nUR5H(up4URS5SU;dMU"5X#'M* U$Nr[rrVr_)rr. serializedrgetters r]rrJsAJ}}  99S>! G +$hJO' r\r]cUR5H0up4URS5SU;dMX0;dM&U"X5 M2 U$rr)r]rr.rsetters r]rrUs@ }}  99S>! G + 38 ' Jr\writersc[U5nUR5(dUR5 UR5H)up4UR S5SU;dMU"X- 5 M+ U$r)rrmkdirrVr_)rrr.rwriters r]to_diskrasZ t D ;;== }}  99S>! G + 4: ' Kr\rkc[U5nUR5H)up4URS5SU;dMU"X- 5 M+ U$r)rrVr_)rrkr.rreaders r]rHrHpsF t D}}  99S>! G + 4: ' Kr\r\c[RRU[U55n[RR U5nUR R U5 U$)zImport module from a file. Used to load models from a directory. name (str): Name of module to load. loc (str / Path): Path to the file. RETURNS: The loaded module. )rutilspec_from_file_locationrmodule_from_specloader exec_module)rr\rr s r] import_filer}sI >> 1 1$C AD ^^ , ,T 2FKKF# Mr\htmlcbUR5RSS5RSS5$)zPerform a template-specific, rudimentary HTML minification for displaCy. Disclaimer: NOT a general-purpose solution, only removes indentation and newlines. html (str): Markup to minify. RETURNS (str): "Minified" HTML. z r1rN)rRr)rs r] minify_htmlrs* ::<   + 3 3D" ==r\cURSS5nURSS5nURSS5nURSS5nU$) zReplace <, >, &, " with their HTML encoded representation. Intended to prevent HTML errors in rendered displaCy markup. text (str): The original text. RETURNS (str): Equivalent text to be safely used within HTML. &z&rz<rz>"z")r)r}s r] escape_htmlrsI <<W %D <<V $D <<V $D <<X &D Kr\wordsc lSRSRU5R55SRUR55:wa&[[RR XS95e/n/nSnUVs/sHoUR 5(aMUPM nnUHnXSRU5nUS:a*URXXG-5 URS5 XG- nURU5 URS5 U[U5- nU[U5:dMXS:XdMSUS'US - nM U[U5:a$URXS5 URS5 X#4$s snf![a( [[RR XS95Sef=f) ajGiven a list of words and a text, reconstruct the original tokens and return a list of words and spaces that can be used to create a Doc. This can help recover destructive tokenization that didn't preserve any whitespace information. words (Iterable[str]): The words. text (str): The original text. RETURNS (Tuple[List[str], List[bool]]): The words and spaces. r1)r}rrNFrTr)) rr_rr0E194risspacerrr)rr} text_words text_spacestext_posword norm_words word_starts r]get_words_and_spacesrs wwrwwu~##%&"''$**,*??+++CDDJKH#(?54 $5J? Si..t4J >   dh.CD E   u %  "H$5!CI c$i DNc$9"KO MH#d)$y/*5!  $$)@  SV[[//T/GHd R SsE<E<*F2F3c[U5R5$![a( [[RR US95Sef=f)zDeep copy a Config. Will raise an error if the config contents are not JSON-serializable. config (Config): The config to copy. RETURNS (Config): The copied config. rMN)r rrr0E961rrMs r] copy_configrsKFf~""$$ F++6+:;EFs 2A valuesc0nUR5Heup#UnUR5RS5n[U5H0upgU[ U5S- :HnUR Xx(aUO05nM2 Mg U$)zConvert dot notation to a dict. For example: {"token.pos": True, "token._.xyz": True} becomes {"token": {"pos": True, "_": {"xyz": True }}}. values (Dict[str, Any]): The key/value pairs to convert. RETURNS (Dict[str, dict]): The converted values. r[r))rVlowerr_ enumerater setdefault) rrhrrrpartsiitemis_lasts r] dot_to_dictrst!Flln  !!#& 'GA3u:>)G??4'rBD(% Mr\r@rAcj[XS9VVs0sHup#SRU5U_M snn$s snnf)aMConvert dot notation to a dict. For example: {"token": {"pos": True, "_": {"xyz": True }}} becomes {"token.pos": True, "token._.xyz": True}. obj (Dict[str, dict]): The dict to convert. for_overrides (bool): Whether to enable special handling for registered functions in overrides. RETURNS (Dict[str, Any]): The key/value pairs. r@r[) walk_dictr)r0rArrs r]rErEs=$CE EJC  uE  s/rgcUnURS5nUHnX$nM U$![[4a( [[RR US95Sef=f)aHConvert dot notation of a "section" to a specific part of the Config. e.g. "training.optimizer" would return the Optimizer object. Throws an error if the section is not defined in this config. config (Config): The config. section (str): The dot notation of the section in the config. RETURNS: The object denoted by the section r[r3N)r_rcrr0E952r)r/rg componentrrs r]rbrbsmI MM# E G!I )$ G6;;--7-;<$ F Gs $8ArcUnURS5n[U5H"upVU[U5S- :XaX#U'MX6nM$ g![[4a( [[ R RUS95Sef=f)zUpdate a config at a given position from a dot notation. config (Config): The config. section (str): The dot notation of the section in the config. value (Any): The value to set in the config. r[r)r3N)r_rrrcrr0rr)r/rgrrrrrs r]set_dot_to_objectr sI MM# EU# GCJN""'$%O $ )$ G6;;--7-;<$ F GsAA8A>rr'c## UR5HWup4/UQUPn[U[5(a2U(a[SU55(d[ XEUS9ShvN MRXT4v MY gN7f)zWalk a dict and yield the path and values of the leaves. for_overrides (bool): Whether to treat registered functions that start with @ as final values rather than dicts to traverse. c3B# UHoRS5v M g7f)@N)r)r value_keys r]rwalk_dict..+sH%Y++C00%sr@N)rVrrxrr)rr'rArr key_parents r]rrsfjjl #v^s^ eT " "H%HHH -P P P% %# QsAA1A/A1rc[R"U5n[[R /UR QUR Q55$)zGet a list of all named arguments of a function (regular, keyword-only). func (Callable): The function RETURNS (List[str]): The argument names. )r~getfullargspecrrxfromkeysr kwonlyargs)rargspecs r] get_arg_namesr2s=$$T*G  B Bw/A/ABC DDr\weightscvUVVVs0sHo"R5Hup4X4_M M nnnnURU5 [UR5Vs/sH of(aUOSPM sn5nUR5H&up4U(dMUS:dM[ XG- S5XS'M( U$s snnnfs snf)anCombine and normalize score weights defined by components, e.g. {"ents_r": 0.2, "ents_p": 0.3, "ents_f": 0.5} and {"some_other_score": 1.0}. weights (List[dict]): The weights defined by the components. overrides (Dict[str, Optional[Union[float, int]]]): Existing scores that should be preserved. RETURNS (Dict[str, float]): The combined and normalized weights. gr)rVrsumrround)rrBw_dictrrrhr weight_sums r]combine_score_weightsr =s ")*!(v\\^\c ^ * MM)v}}?!1a#o?@Jlln  5Z!^ 2A6FK% M*@s %B/B6cp\rSrSrSrSrSrS\SS4SjrS\ \ \ 4SS 4S jr S\ \ \ 4SS4S jr S rg ) DummyTokenizeriWc[er)r)rr}s r]__call__DummyTokenizer.__call__Xs!!r\c+2# UH nU"U5v M g7frrU)rtextsrr}s r]pipeDummyTokenizer.pipe[sDt* rvc g)Nr\rU)rrs r]rDummyTokenizer.to_bytesasr\datartc U$rrU)rrrs r]rDummyTokenizer.from_bytesd r\rNc grrUrrrs r]rDummyTokenizer.to_diskgsr\c U$rrUrs r]rHDummyTokenizer.from_diskjrr\rU)rVrWrXrYr rrbytesrrrrrrHr[rUr\r]r r Ws^" u3CE#t),4eCI.=Mr\r c[5$r)rrUr\r]create_default_optimizerr ns 6Mr\c#$# [U[5(a[R"U5nOUn[ U5n[ U5n[ [R"U[U555n[U5S:Xag[ U5v MS7f)zdIterate over batches of items. `size` may be an iterator, so that batch-size can vary on each step. rN) rrrrepeatiternextrislicer)rVsizesize_ batch_sizebatchs r] minibatchr*rsv$  & KE %[ Y%%eS_=> u:? 5k sBBcJSn[X5(ag[US5(a[US5(anUR[R;aP[ [RUR5UR R S5Sn[X!5$g)aSlightly hacky check for whether a callable is implemented in Cython. Can be used to implement slightly different behaviors, especially around inspecting and parameter annotations. Note that this will only return True for actual cdef functions and methods, not regular Python functions defined in Python modules. func (Callable): The callable to check. RETURNS (bool): Whether the callable is Cython (probably). __pyx_vtable__TrXrWr[rF)rrWr$r%varsrXr_)rrrcls_funcs r]is_cython_funcr/s Dt n%% D, ' ' OOs{{ * DOO45d6G6G6M6Mc6RST6UVx&& r\env_varcf[RRUS5nUS:Xag[U5$)zConvert the value of an environment variable to a boolean. Add special check for "0" (falsy) and consider everything else truthy, except unset. env_var (str): The name of the environment variable to check. RETURNS (bool): Its boolean value. F0)r"rrr)r0rs r]check_bool_env_varr3s, JJNN7E *E | ;r\docsr6procr5default_error_handlerrc#|# [US5(aUR"U40UD6ShvN g[U5nUn[US5(aUR5nSHnXd;dM UR U5 M UHnU"U40UD6nUv M gNl![ anU"X!U/U5 SnAM3SnAff=f7f)Nrget_error_handler)r()rrrxr8rr) r4r5rr6r error_handlerargdocr#s r]_piper<stV99T,V,,,f- 4, - - 224M!C} 3"C 43)&)  - 4d3%33 4s?(B<B;B<*B< BB< B9# B4.B<4B99B<cUerrU proc_namer5r4r#s r] raise_errorr@s Gr\cgrrUr>s r] ignore_errorrBsr\c[5(aLSSKJn URb8SSKJn U"5(d%[ R"[R5 gggg)ziWarn about require_gpu if a jupyter notebook + cupy + mismatched contextvars vs. thread ops are detected r)CupyOpsN)contextvars_eq_thread_ops) r+thinc.backends.cupy_opsrDxpthinc.backendsrErrr1W111)rDrEs r]warn_if_jupyter_cupyrJsC3 :: ! @,.. hmm,/ "r\cURRS05n[U5S:Xa[UR[;aFSR [5n[ R[RRXS95 ggg)N lexeme_normrr)rlangs) rf get_tablerrLEXEME_NORM_LANGSrloggerdebugr1W033r)r+component_name lexeme_normsrMs r]check_lexeme_normsrUsg==**="=L <A%**0A"A +, X]])))LM#Br\c@USLagUcgUSLagUS:XagUS:Xagg)zConvert a value to the ternary 1/0/-1 int used for True/None/False in attributes such as SENT_START: True/1/1.0 is 1 (True), None/0/0.0 is 0 (None), any other values are -1 (False). Tr)rFrrU)vals r]to_ternary_intrXs7  d{    r\c [[5n[R"5HRnUR S5=(d SR 5H#nXR URS5 M% MT [U5$)zReturn a mapping of top-level packages to their distributions. We're inlining this helper from the importlib_metadata "backport" here, since it's not available in the builtin importlib.metadata. z top_level.txtr1Name) rrr- distributions read_textr_rmetadatarx) pkg_to_distdistrs r]packages_distributionsr`sj d#K"002NN?39r@@BC   # #DMM&$9 :C3  r\cv[R"U5n[US5=(a [US5(+$)zpReturn True if all the elements are equal to each other (or if the input is an empty sequence), False otherwise.TF)rgroupbyr$)iterablegs r] all_equalres. (#A 4= /a//r\porthostc0[R"[R[R5nURX45 UR 5 g![R a UR 5 gf=f!UR 5 f=f)zCheck if 'host:port' is in use. Return True if it is, False otherwise. port (int): the port to check host (str): the host to check (default "localhost") RETURNS (bool): Whether 'host:port' is in use. FT)socketAF_INET SOCK_STREAMbindcloser)rfrgss r]_is_port_in_usero sl  fnnf&8&89A |   <<    s#AB-B?BBB auto_selectc[X5(dU$UnU(d&[[RR US95e[X15(a#US:aUS- n[X15(aUS:aMUS:Xa6[X15(a&[[R R US95e[ R"[RR XUS95 U$)aGiven a starting port and a host, handle finding a port. If `auto_select` is False, a busy port will raise an error. If `auto_select` is True, the next free higher port will be used. start (int): the port to start looking from host (str): the host to find a port on auto_select (bool): whether to automatically select a new port if the given port is busy (default False) RETURNS (int): The port to use. )rfir))rg)rgrf serve_port) rorr0E1050rE1049rrr1W124)rrgrprfs r]find_available_portrvs 5 ' ' D ,,$,788 $ % %$,   $ % %$, u}44,,$,788 MM(--&&D&NO Kr\)T)FTr) localhost)F)rgrimportlib.utilr~rloggingr"rrSrrrirrr$rr collectionsr contextlibrpathlibrtypesrtypingrrr r r r r rrrrrrrrrrrrnumpyrthincrrpackaging.requirementsrpackaging.specifiersrrpackaging.versionrr thinc.apirr r!r"r#r$r% cupy.randomr,rr&r'r(r1r*compatr+r-r.r^r/r0r1symbolsr3languager4r5rr6r7r+r8iinfouint64rOOV_RANKDEFAULT_OOV_PROBrOrw getLoggerrP StreamHandlerlogger_stream_handler setFormatter Formatter addHandlerrRr_rxrrrrrrrrr rrr)_DEFAULT_EMPTY_PIPESr:r5r6rGrXrirprFrrrrrrrrrrrrrrDr4rrrCompletedProcessrr r$r+r/r2r<rArLrZr^rbrdrjrirr|rrrrrrrrrrrrHrrrrrrrErbrrrfloatr r r r*r/r3rr<r@rBrJrUrXr`rerorvrUr\r]rs   #%( -.?5=<DD990! ;;u|| $ ( (iA   7 #--/"" AB'(00x u~~x v.t.2&.t&.R&s&t&44#4n((j!1(>004 #300 c c  4U39- 4%d 2C 4 J 4 () #')=(<)=,<,>'1 T '1 $ '13 % & '1 #x}$ % '1 3 % & '1 $sCx.&( )'1'1Z#')=(<)=,<,>a a $ a3 % & a #x}$ % a 3 % & a $sCx.&( )aa@&*"&)=(<)=,<,>+K+K 4S> "+K $  +K 3 % & +K #x}$ % +K3 % &+K $sCx.&( )+K+Kb,-"&)=(<)=/ $sCx.&( )/ sCx./ $  / 3 % & / #x}$ % /3 % &////d  $sCx.&( )  #tCH~  " "#HSM2" 38_"P#')=(<)=,<,>'T3Y' $ '3 % & ' #x}$ % ' 3 % & ' $sCx.&( )''X!1 2 T  CH~    8,<+=SX   sCx. LP 2d3i2 c hsm 8< !04 d^6*."& d^6*3*5c?* ;S;S;   *3*4*)c)c) "s "x} " 6c 6c 6d 6(E#t)$(c3h(V/sDy)/d38n/ S T  D3 D4 D4e4U4444$636496  4 3S > "4 C=4 4   4n  eCI&  8D>    :idD 01::.t0646   111d1,15 5  5)- 5 j 5"U39%"'""(5g+>"?"G""(5g+>"?"G""%W *=!>"7" Mhucz2 M3%QT*AU M C5#:& 15c3h IL  #tDz/* #tDz/, sDJ ),7: #tDz/6?C #&.6sm 38_$(T&\,1x/1DL1:d3U 334:x}:QV:HH #x -- .Hc]H H #xC(( )4c>c> c c &% C=&% #&% 49d4j !&%R Fd38nf45 F& FS#X4T ?"@E T#t)_  cSVh &3&GfGsG3G4G(/1&KP& sCx.&"&s)&DH& eDIsN#$&&EET#YE-=,> $sHUO+, -C%() #x 4.)"d0   4 5/4 4 4$ nd5k95x? 4 CH 4e_48   -N3. S$s)^ 4 0#S4"s#DS{6 Dsb bb