Viw SrSSKrSSKrSSKrSSKJr SSKJr SSK J r \ "SS 9r \ "S S 9r \ \"S 5urrrrS r"S S\5r"SS\5r"SS\5rS\ \ "\!\\ \""S5/54Sjr#\#r$"SS\ 5r%"SS\ 5r&S!Sjr'S!Sjr("SS\ 5r)"SS\ 5r*"SS \ 5r+g!\a "SS\ 5rNf=f!\a \ "5r \ "5r Nf=f!\a \r\\\\\44urrrrNf=f)"a``cacheutils`` contains consistent implementations of fundamental cache types. Currently there are two to choose from: * :class:`LRI` - Least-recently inserted * :class:`LRU` - Least-recently used Both caches are :class:`dict` subtypes, designed to be as interchangeable as possible, to facilitate experimentation. A key practice with performance enhancement with caching is ensuring that the caching strategy is working. If the cache is constantly missing, it is just adding more overhead and code complexity. The standard statistics are: * ``hit_count`` - the number of times the queried key has been in the cache * ``miss_count`` - the number of times a key has been absent and/or fetched by the cache * ``soft_miss_count`` - the number of times a key has been absent, but a default has been provided by the caller, as with :meth:`dict.get` and :meth:`dict.setdefault`. Soft misses are a subset of misses, so this number is always less than or equal to ``miss_count``. Additionally, ``cacheutils`` provides :class:`ThresholdCounter`, a cache-like bounded counter useful for online statistics collection. Learn more about `caching algorithms on Wikipedia `_. N) attrgetter)RLockc$\rSrSrSrSrSrSrg)rLz/Dummy reentrant lock for builds without threadscgNselfs e/home/james-whalen/.local/share/pipx/venvs/semgrep/lib/python3.13/site-packages/boltons/cacheutils.py __enter__RLock.__enter__N cgrr )r exctypeexcinstexctbs r __exit__RLock.__exit__Qrrr N)__name__ __module__ __qualname____firstlineno____doc__r r__static_attributes__r rr rrLs9  rr) make_sentinel_MISSING)var_name _KWARG_MARKc^\rSrSrSr\SS4SjrSrSrSr Sr S r S r S r U4S jrS rSSjrU4Sjr\4U4SjjrU4SjrU4SjrSrSSjrSrU4SjrSrU4SjrSrU=r$)LRIgaThe ``LRI`` implements the basic *Least Recently Inserted* strategy to caching. One could also think of this as a ``SizeLimitedDefaultDict``. *on_miss* is a callable that accepts the missing key (as opposed to :class:`collections.defaultdict`'s "default_factory", which accepts no arguments.) Also note that, like the :class:`LRI`, the ``LRI`` is instrumented with statistics tracking. >>> cap_cache = LRI(max_size=2) >>> cap_cache['a'], cap_cache['b'] = 'A', 'B' >>> from pprint import pprint as pp >>> pp(dict(cap_cache)) {'a': 'A', 'b': 'B'} >>> [cap_cache['b'] for i in range(3)][0] 'B' >>> cap_cache['c'] = 'C' >>> print(cap_cache.get('a')) None >>> cap_cache.hit_count, cap_cache.miss_count, cap_cache.soft_miss_count (3, 1, 1) Nc US::a[SU-5eS=Ul=UlUlXl[ 5UlUR5 Ub[U5(d[SU-5eX0l U(aURU5 gg)Nrzexpected max_size > 0, not %rz3expected on_miss to be a callable (or None), not %r) ValueError hit_count miss_countsoft_miss_countmax_sizer_lock_init_llcallable TypeErroron_missupdate)r r+valuesr0s r __init__ LRI.__init__}s q=  $$S):T 4:T 4rcj>UR URU5nX#['SSS5 g![ae [ U5UR :aUR X5 O$URX5n[[U]+U5 [[U]/X5 Nwf=f!,(df  g=fr) r,rMrDKeyErrorlenr+rQrVsuperr$ __delitem__ __setitem__)r r=rPrGrU __class__s r raLRI.__setitem__s ZZ $==cB$U Z 9t9t}},88D"@@LG#t09c4,S8  9Zs,B$2B$A,B!B$ B!!B$$ B2cZUR URUnU=R S- slU[ sSSS5 $![aJ U=RS- slUR(deUR U5=o0U'UssSSS5 $f=f!,(df  g=fN)r,r6r]r)r0r(rDr r=rGrets r __getitem__LRI.__getitem__s ZZ ((- NNa N;Z 1$||"&,,s"333i Z Zs.BABAB BBB B*cZX$![a U=RS- slUs$f=fre)r]r*r r=defaults r getLRI.gets4 9    A % N s !**c>UR [[U]U5 UR U5 SSS5 g!,(df  g=fr)r,r_r$r`rZ)r r=rbs r r`LRI.__delitem__s1 ZZ #t ( -   %ZZs %< A c>UR [[U]U5nUR U5 UsSSS5 $![ a U[ LaeUnN#f=f!,(df  g=fr)r,r_r$rYrZr]r)r r=rmrhrbs r rYLRI.pop sc ZZ *C*3/ $$S)Z h& Zs+A?AAAAA A)c>UR [[U]5nUR US5 UsSSS5 $!,(df  g=fNr)r,r_r$popitemrZ)r itemrbs r rv LRI.popitems8 ZZd+-D  a )ZZs (A Ac>UR [[U]5 UR 5 SSS5 g!,(df  g=fr)r,r_r$clearr-)r rbs r rz LRI.clears* ZZ #t " $ MMOZZs #: Ac6URURUS9$)N)r+r2)rbr+r s r copyLRI.copy$s~~t}}T~BBrcUR XsSSS5 $![a' U=RS- slX U'UssSSS5 $f=f!,(df  g=fre)r,r]r*rls r setdefaultLRI.setdefault'sW ZZ yZ $$)$#S  Z Zs&A%A A A  A Ac PUR XLa SSS5 gURn[[USS55(a#UR 5HnU"XAU5 M OUH upEU"XE5 M UHnU"XBU5 M SSS5 g!,(df  g=f)Nkeys)r,rar.getattrr)r EFsetitemkvs r r1 LRI.update0s ZZyZ&&G64011AAt$"DAAMQ4  ZZsBA2B B%c>UR XLa SSS5 g[U5[U5:wa SSS5 g[U[5(d X:HsSSS5 $[[U]U5sSSS5 $!,(df  g=f)NTF)r,r^ isinstancer$r___eq__)r otherrbs r r LRI.__eq__@sk ZZ}Z5zSY& Z eS))} Zd*51ZZsA=A=A=!A== B c X:X+$rr )r rs r __ne__ LRI.__ne__Js ""rc>URRn[[U]5nU<SUR <SUR <SU<S3$)Nz (max_size=z , on_miss=z , values=))rbrr_r$__repr__r+r0)r cnval_maprbs r r LRI.__repr__Ms< ^^ $ $T+-t}}dllG= >r)r7r6r,r(r+r)r0r*r)rrrrrDEFAULT_MAX_SIZEr3r-r?r<rMrQrVrZrarirnr`rrYrvrzr}rr1rrrr __classcell__)rbs@r r$r$gs*!1 4$($& $ & (   C 2#>>rr$c\rSrSrSrSrSrg)LRUiTaThe ``LRU`` is :class:`dict` subtype implementation of the *Least-Recently Used* caching strategy. Args: max_size (int): Max number of items to cache. Defaults to ``128``. values (iterable): Initial values for the cache. Defaults to ``None``. on_miss (callable): a callable which accepts a single argument, the key not present in the cache, and returns the value to be cached. >>> cap_cache = LRU(max_size=2) >>> cap_cache['a'], cap_cache['b'] = 'A', 'B' >>> from pprint import pprint as pp >>> pp(dict(cap_cache)) {'a': 'A', 'b': 'B'} >>> [cap_cache['b'] for i in range(3)][0] 'B' >>> cap_cache['c'] = 'C' >>> print(cap_cache.get('a')) None This cache is also instrumented with statistics collection. ``hit_count``, ``miss_count``, and ``soft_miss_count`` are all integer members that can be used to introspect the performance of the cache. ("Soft" misses are misses that did not raise :exc:`KeyError`, e.g., ``LRU.get()`` or ``on_miss`` was used to cache a default. >>> cap_cache.hit_count, cap_cache.miss_count, cap_cache.soft_miss_count (3, 1, 1) Other than the size-limiting caching behavior and statistics, ``LRU`` acts like its parent class, the built-in Python :class:`dict`. c^UR URU5nU=R S- slU[ sSSS5 $![aJ U=RS- slUR(deUR U5=o0U'UssSSS5 $f=f!,(df  g=fre)r,rMr]r)r0r(rDrgs r riLRU.__getitem__vs ZZ ==cB NNa N;Z 1$||"&,,s"333i Z Zs.BABABBBB B,r N)rrrrrrirr rr rrTs  B rrc.\rSrSrSrSrSrSrSrSr g) _HashedKeyizgThe _HashedKey guarantees that hash() will be called no more than once per cached function invocation. hash_valuec>XSS&[[U55Ulgr)hashtuplerr r=s r r3_HashedKey.__init__sQuSz*rcUR$rrr s r __hash___HashedKey.__hash__s rcdURR<S[RU5<S3$)N(r)rbrlistrr s r r_HashedKey.__repr__s >>22DMM$4GHHrrN) rrrrr __slots__r3rrrr rr rrsI+IrrFc [U5nU(a;[UR55nURU5 UR U5 U(abUR UVs/sHn[ U5PM sn5 U(a/UR WVVs/sHup[ U5PM snn5 O&[ U5S:Xa[ US5U;aUS$[U5$s snfs snnf)aMake a generic key from a function's positional and keyword arguments, suitable for use in caches. Arguments within *args* and *kwargs* must be `hashable`_. If *typed* is ``True``, ``3`` and ``3.0`` will be treated as separate keys. The key is constructed in a way that is flat as possible rather than as a nested structure that would take more memory. If there is only a single argument and its data type is known to cache its hash value, then that argument is returned without a wrapper. This saves space and improves lookup speed. >>> tuple(make_cache_key(('a', 'b'), {'c': ('d')})) ('a', 'b', _KWARG_MARK, ('c', 'd')) .. _hashable: https://docs.python.org/2/glossary.html#term-hashable rfr)rsorteditemsrBextendtyper^r) argskwargstyped kwarg_mark fasttypesr= sorted_itemsrrs r make_cache_keyrs. t*C flln-  : <   T*TDGT*+  JJL9LDAQL9 : SQ4A<941v c? +9s $C(C- c.\rSrSrSrSSjrSrSrSrg) CachedFunctionizqThis type is used by :func:`cached`, below. Instances of this class are used to wrap functions in caching logic. Nc"^Xl[T5(aTUlOP[[TSS55(a[[TSS55(d[ ST-5eU4SjnX`lX0lX@lU=(d [Ulg)NrirazYexpected cache to be a dict-like object, or callable returning a dict-like object, not %rc>T$rr )cachesr _get_cache+CachedFunction.__init__.._get_cache r) funcr. get_cacherr/scopedrrkey_funcr rrrrr=rs ` r r3CachedFunction.__init__s{ E??"DN75->??wumTBCCP#$% % 'N  -~ rcUR5nURXURS9nX4nU$![a UR"U0UD6=oSU'U$f=f)Nr)rrrr]r)r rrrr=rhs r __call__CachedFunction.__call__sg mmD m; :*C  :#yy$9&9 9C*  :s2 AAcURRnUR(dUR(d0U<SUR<SUR<SUR<S3$U<SUR<S3$)Nz(func=z , scoped=z, typed=r)rbrrrrr rs r rCachedFunction.__repr__sM ^^ $ $ ::T[[499dkk4::? @ "DII..r)rrrrrTFN) rrrrrr3rrrr rr rrs."/rrc8\rSrSrSrS SjrS SjrSrSrSr g) CachedMethodizpSimilar to :class:`CachedFunction`, this type is used by :func:`cachedmethod` to wrap methods in caching logic. Nc^Xl[USS5Ul[T[5(a[ T5UlOh[T5(aTUlOP[[TSS55(a[[TSS55(d[ST-5eU4SjnX`lX0l X@l U=(d [Ul SUl g)N__isabstractmethod__Frirazjexpected cache to be an attribute name, dict-like object, or callable returning a dict-like object, not %rc>T$rr )objrs r r)CachedMethod.__init__.._get_cacherr)rrrr basestringrrr.r/rrrrbound_tors ` r r3CachedMethod.__init__s $+D2H%$P! eZ ( ('.DN e__"DN75->??wumTBCC:z %s(func=%r, scoped=%r, typed=%r))rbrrrrr)r rrs r rCachedMethod.__repr__sX ^^ $ $IIt{{DJJ7 == $ ]]$ $DADH I2T9:r)rrrrrrrrr) rrrrrr3rrrrr rr rrs* ;rrc ^^^^UUUU4SjnU$)aCache any function with the cache object of your choosing. Note that the function wrapped should take only `hashable`_ arguments. Args: cache (Mapping): Any :class:`dict`-like object suitable for use as a cache. Instances of the :class:`LRU` and :class:`LRI` are good choices, but a plain :class:`dict` can work in some cases, as well. This argument can also be a callable which accepts no arguments and returns a mapping. scoped (bool): Whether the function itself is part of the cache key. ``True`` by default, different functions will not read one another's cache entries, but can evict one another's results. ``False`` can be useful for certain shared cache use cases. More advanced behavior can be produced through the *key* argument. typed (bool): Whether to factor argument types into the cache check. Default ``False``, setting to ``True`` causes the cache keys for ``3`` and ``3.0`` to be considered unequal. >>> my_cache = LRU() >>> @cached(my_cache) ... def cached_lower(x): ... return x.lower() ... >>> cached_lower("CaChInG's FuN AgAiN!") "caching's fun again!" >>> len(my_cache) 1 .. _hashable: https://docs.python.org/2/glossary.html#term-hashable c>[UTTTTS9$N)rrr=)rrrr=rrs r cached_func_decorator%cached..cached_func_decorator@sdE&3OOrr )rrrr=rs```` r cachedrsBPP  rc ^^^^UUUU4SjnU$)a{Similar to :func:`cached`, ``cachedmethod`` is used to cache methods based on their arguments, using any :class:`dict`-like *cache* object. Args: cache (str/Mapping/callable): Can be the name of an attribute on the instance, any Mapping/:class:`dict`-like object, or a callable which returns a Mapping. scoped (bool): Whether the method itself and the object it is bound to are part of the cache keys. ``True`` by default, different methods will not read one another's cache results. ``False`` can be useful for certain shared cache use cases. More advanced behavior can be produced through the *key* arguments. typed (bool): Whether to factor argument types into the cache check. Default ``False``, setting to ``True`` causes the cache keys for ``3`` and ``3.0`` to be considered unequal. key (callable): A callable with a signature that matches :func:`make_cache_key` that returns a tuple of hashable values to be used as the key in the cache. >>> class Lowerer(object): ... def __init__(self): ... self.cache = LRI() ... ... @cachedmethod('cache') ... def lower(self, text): ... return text.lower() ... >>> lowerer = Lowerer() >>> lowerer.lower('WOW WHO COULD GUESS CACHING COULD BE SO NEAT') 'wow who could guess caching could be so neat' >>> len(lowerer.cache) 1 c>[UTTTTS9$r)rrs r cached_method_decorator-cachedmethod..cached_method_decoratorjsD%eMMrr )rrrr=rs```` r cachedmethodrEsJNN ""rc.\rSrSrSrSrSSjrSrSrg) cachedpropertyioaThe ``cachedproperty`` is used similar to :class:`property`, except that the wrapped method is only called once. This is commonly used to implement lazy attributes. After the property has been accessed, the value is stored on the instance itself, using the same name as the cachedproperty. This allows the cache to be cleared with :func:`delattr`, or through manipulating the object's ``__dict__``. cV[US5Ul[USS5UlXlg)NrrF)rrrr)r rs r r3cachedproperty.__init__ys'tY/ $+D2H%$P! rNcvUcU$URU5=o1RURR'U$r)r__dict__r)r rrrPs r rcachedproperty.__get__~s4 ;K3799S>A TYY//0 rcXURRnSU<SUR<S3$)N)rbrrrs r rcachedproperty.__repr__s ^^ $ $!#TYY//r)rrrr) rrrrrr3rrrr rr rros  0rrc\rSrSrSrSSjr\S5rSrSr SSjr S r S r S r S rS rSrSrSrSrSrSrSrSSjrSrSrg)ThresholdCounteria$A **bounded** dict-like Mapping from keys to counts. The ThresholdCounter automatically compacts after every (1 / *threshold*) additions, maintaining exact counts for any keys whose count represents at least a *threshold* ratio of the total data. In other words, if a particular key is not present in the ThresholdCounter, its count represents less than *threshold* of the total data. >>> tc = ThresholdCounter(threshold=0.1) >>> tc.add(1) >>> tc.items() [(1, 1)] >>> tc.update([2] * 10) >>> tc.get(1) 0 >>> tc.add(5) >>> 5 in tc True >>> len(list(tc.elements())) 11 As you can see above, the API is kept similar to :class:`collections.Counter`. The most notable feature omissions being that counted items cannot be set directly, uncounted, or removed, as this would disrupt the math. Use the ThresholdCounter when you need best-effort long-lived counts for dynamically-keyed data. Without a bounded datastructure such as this one, the dynamic keys often represent a memory leak and can impact application reliability. The ThresholdCounter's item replacement strategy is fully deterministic and can be thought of as *Amortized Least Relevant*. The absolute upper bound of keys it will store is *(2/threshold)*, but realistically *(1/threshold)* is expected for uniformly random datastreams, and one or two orders of magnitude better for real-world data. This algorithm is an implementation of the Lossy Counting algorithm described in "Approximate Frequency Counts over Data Streams" by Manku & Motwani. Hat tip to Kurt Rose for discovery and initial implementation. cSUs=:aS:dO [SU-5eSUl0UlXl[ SU- 5UlSUlg)Nrrfz+expected threshold between 0 and 1, not: %r)r'total _count_map _thresholdint _thresh_count _cur_bucket)r thresholds r r3ThresholdCounter.__init__sU9 q J()* * # Y/rcUR$r)rr s r rThresholdCounter.thresholds rc U=RS- slURUS==S- ss'URUR-S:Xan[ URR 5VVs/sH#up#[U5UR:dM X#4PM% snn5UlU=RS- slg![a! SURS- /URU'Nf=fs snnf)zIncrement the count of *key* by 1, automatically adding it if it does not exist. Cache compaction is triggered every *1/threshold* additions. rfrN)rrr]rrdictrsum)r r=rrs r addThresholdCounter.adds a  = OOC  #q ( # ::** *a /"t7L7L7N$B7Ntq'*1v0@0@'@%+QF7N$BCDO    !  =$%t'7'7!';#.ThresholdCounter.most_common..sQqTrT)r=reverseN)rrr^)r nrhs r most_commonThresholdCounter.most_commonsD 6IT^^%>4H 9SX J2Awrc~[URR5VVs/sHupUPM snn5$s snnf)zHGet the sum of counts for keys exceeding the configured data threshold. )r rr2)r count_s r get_common_count!ThresholdCounter.get_common_counts2$//*@*@*BC*BheE*BCDDCs9 c<URUR5- $)zGet the sum of counts for keys that were culled because the associated counts represented less than the configured threshold. The long-tail counts. )rr%r s r get_uncommon_count#ThresholdCounter.get_uncommon_counts zzD11333rcN[UR55UR- $)aGet a float representation of the effective count accuracy. The higher the number, the less uniform the keys being added, and the higher accuracy and efficiency of the ThresholdCounter. If a stronger measure of data cardinality is required, consider using hyperloglog. )floatr%rr s r get_commonality ThresholdCounter.get_commonalitys!T**,- ::rc&URUS$rurrs r riThresholdCounter.__getitem__ss#A&&rc,[UR5$r)r^rr s r __len__ThresholdCounter.__len__s4??##rcXR;$rr/rs r __contains__ThresholdCounter.__contains__soo%%rc,[UR5$r)iterrr s r iterkeysThresholdCounter.iterkeyssDOO$$rc4[UR55$r)rr9r s r rThresholdCounter.keys sDMMO$$rc#H# URnUH nXSv M g7frur/r count_maprs r itervaluesThresholdCounter.itervaluess$OO A,q/ !s "c4[UR55$r)rr@r s r r2ThresholdCounter.valuessDOO%&&rc#L# URnUHnX!US4v M g7frur/r>s r rThresholdCounter.iteritemss)OO Al1o& &s"$c4[UR55$r)rrr s r rThresholdCounter.itemssDNN$%%rc0X$![a Us$f=f)z%Get count for *key*, defaulting to 0.)r]rls r rnThresholdCounter.gets# 9  N s  c Ubr[[USS55(a=UR5H(up4[U5HnUR U5 M M* OUHnUR U5 M U(aUR U5 gg)zLike dict.update() but add counts instead of replacing them, used to add multiple items in one call. Source can be an iterable of keys to add, or a mapping of keys to integer counts. Nr)r.rrxranger r1)r iterablerr=r#is r r1ThresholdCounter.update%sw  +t<=="*"4"4"6JC#E] +#7$CHHSM$  KK  r)rrrrr)gMbP?r)r)rrrrrr3propertyrr rr r%r(r,rir2r5r9rr@r2rrrnr1rr rr rrsw)V $8 E 4;'$&%%" '' & rrcH\rSrSrSrSrSrSrSrSr Sr S r S r S r g ) MinIDMapi8z Assigns arbitrary weakref-able objects the smallest possible unique integer IDs, such that no two objects have the same ID at the same time. Maps arbitrary hashable objects to IDs. Based on https://gist.github.com/kurtbrose/25b48114de216a5e55df cT[R"5Ul0Ul/Ulgr)weakrefWeakKeyDictionarymappingref_mapfreer s r r3MinIDMap.__init__Bs 002   rcVURUS$![a Of=fUR(a![R"UR5nO[ UR5n[ R"XR5nX#4URU'X RU'U$ru) rUr]rWheapqheappopr^rSref_cleanrV)r anxtr\s r rn MinIDMap.getGs <<?1% %    99-- *Cdll#Ckk![[)* Q S s  !!cURUup#URU URU [R"URU5 gr)rUrVrZheappushrW)r r^freedr\s r drop MinIDMap.dropVs:\\!_  LLO LL  tyy%(rc[URU5 [R"URURU5 URU gr)r;rVrZrbrW)r r\s r r]MinIDMap._clean\s: dll3  tyy$,,s"34 LL rcXR;$rrU)r r^s r r5MinIDMap.__contains__asLL  rc,[UR5$rr8rUr s r __iter__MinIDMap.__iter__dsDLL!!rc6URR5$r)rUr2r s r r2MinIDMap.__len__gs||##%%rcT^[U4Sj[TR555$)Nc3J># UHoTRUS4v M g7f)rNri).0rr s r %MinIDMap.iteritems..ks#H5G Q*+5Gs #rlr s`r rMinIDMap.iteritemsjsHT$,,5GHHHr)rWrUrVN)rrrrrr3rnrdr]r5rmr2rrr rr rQrQ8s0 )  !"&IrrQr),rrZrSroperatorr threadingr Exceptionobjectboltons.typeutilsrrr ImportErrorrK NameErrorrangestrbytesunicoderrJrErCrDrr r$rrr frozensetrrr_make_cache_keyrrrrrrrQr rr rsrBF  /j1H7K F qdCj>$j>Z.#.hII"(-)&S)T$Z'HI"J!#/V#/L5;65;p#!L'#T0V04l vl ^3Iv3I[xH(K F F&)5%#u&E#GS%Fs3B>CC/>CCC,+C,/DD