^hY:bSrSSKrSSKrSSKrSSKJr SSKrSSKJr SSK J r "SS\ 5r g) z Qdrant Semantic Cache implementation Has 4 methods: - set_cache - get_cache - async_set_cache - async_get_cache N)Any) print_verbose) BaseCachec^\rSrSrS SjrS\4SjrSrSrSr S r S r S r S r g)QdrantSemanticCacheNcSSKnSSKJn Jn Jn SSKJn Uc [S5eX0l[SUR35 Uc [S5eX@l X`l 0n U(a3[U[5(aURS5(aU "U5nU(a3[U[5(aURS5(aU "U5nU=(d) URS5=(d URS 5nU=(d URS 5nS S 0n U(aX-S 'Uc [!S5eXlX l[SUR"35 XlU "5UlU "U R*S9UlUc [S5 UR(R/UR"SURS3UR&S9nUR0S:wa[!SUR235eUR55SS(akUR(R/UR"SUR3UR&S9nUR55Ul[SUR635 gUbUS:XaSSS00nO(US:Xa SSSSS .0nOUS!:XaS!S"SS#.0nO [S$5eUR(R9UR"SUR3S%S&S'.US(.UR&S)9nUR55S(akUR(R/UR"SUR3UR&S9nUR55Ul[S*UR635 g[S+5e),Nr)_get_httpx_clientget_async_httpx_clienthttpxSpecialProvider)get_secret_strz-collection_name must be provided, passed Nonez0qdrant semantic-cache initializing COLLECTION - z2similarity_threshold must be provided, passed Nonez os.environ/ QDRANT_URLQDRANT_API_BASEQDRANT_API_KEYz Content-Typezapplication/jsonzapi-keyzQdrant url must be providedz'qdrant semantic-cache qdrant_api_base: ) llm_providerzNQuantization config is not provided. Default binary quantization will be used. /collections/z/exists)urlheadersz1Error from qdrant checking if /collections exist resultexistsz.Collection already exists. Collection details:binary always_ramFscalarint8gGz?)typequantilerproductx16) compressionrzBQuantization config must be one of 'scalar', 'binary' or 'product'iCosine)sizedistance)vectorsquantization_config)rjsonrz+New collection created. Collection details:z#Error while creating new collection)os&litellm.llms.custom_httpx.http_handlerr r r litellm.secret_managers.mainr Exceptioncollection_namersimilarity_thresholdembedding_model isinstancestr startswithgetenv ValueErrorqdrant_api_baseqdrant_api_keyr sync_clientCaching async_clientget status_codetextr'collection_infoput)selfr4r5r,r-r&r. host_typer(r r r rrcollection_existscollection_detailsquantization_paramsnew_collection_statuss _/home/james-whalen/.local/lib/python3.13/site-packages/litellm/caching/qdrant_semantic_cache.py__init__QdrantSemanticCache.__init__s   @  "KL L.>t?S?S>T U  'PQ Q$8!. /3//O4N4N55#1"A .#..>3L3L44"0!?  Vryy6 V"))DU:V (F2995E+F!#56 !/I   ":; ;.,?@T@T?UVW ,.2-55   & ` !,,00''( d6J6J5K7SLL1   ( (C /CDUDZDZC[\   ! ! #H -h 7!%!1!1!5!5++,M$:N:N9OP "6" $6#:#:# %9% !%))+H5%)%5%5%9%9//0 d>R>R=ST LL&:&"(:'>'>'@$B4CWCWBXY EFFcached_responsecUcU$[R"U5nU$![a [R"U5nU$f=fN)r'loadsr+ast literal_eval)r>rHs rD_get_cache_logic$QdrantSemanticCache._get_cache_logicsV  "" " @"jjO  @!..?O @s AAc [SU35 SSKnUSnSnUH nXgS- nM [R"URUSSS.S9nUS SS n [ U5n[ U[ 5(deS [ UR55U UUS .S ./0n URRURSURS3URU S9 g)Nz)qdrant semantic-cache set_cache, kwargs: rmessagescontentTzno-storezno-cachemodelinputcachedata embeddingpointsr;responseidvectorpayloadr/pointsrrr') ruuidlitellmrZr.r0r/uuid4r6r=r4r,r) r>keyvaluekwargsrdrQpromptmessageembedding_responserZrYs rD set_cacheQdrantSemanticCache.set_cachesA&JK*%G i( (F %..&&#6 'v.q1+> E %%%%% djjl+' &$)     ''( d6J6J5K7SLL  rGc [SU35 USnSnUH nXES- nM [R"URUSSS.S9nUSS S nUS S SS S.0SSS.nURR UR SURS3URUS9n U R5Sn U cg[U [5(a[U 5S :XagU S Sn U S SSn [SURSU SUSU 35 XR:a.U S SSn [SU SUSU 35 URU S 9$g)!Nz.sync qdrant semantic-cache get_cache, kwargs: rQrRrSTrTrUrYrrZ quantizationF@ignorerescore oversamplingrr`paramslimit with_payloadr/points/searchrcrscorerar;&semantic cache: similarity threshold: , similarity: , prompt: , closest_cached_prompt: r]got a cache hit, similarity: , Current prompt: , cached_prompt: rH)rrerZr.r6postr4r,rr'r/listlenr-rN)r>rgrirQrjrkrlrZrYsearch_responseresults similarity cached_prompt cached_values rD get_cacheQdrantSemanticCache.get_cachesFvhOP*%G i( (F %..&&#6 'v.q1+>  ##$'!   **//''( d6J6J5K>ZLL0  "&&(2 ? gt $ $7|q QZ(  9-f5  4T5N5N4O~^h]iistzs{|UVcUd e  22 2"1:i0# SSKnSSKJnJn [ SU35 USnSnUH n XS- nM UbUV s/sHoSPM sn O/n Ub~UR U ;anUR S05R S S5n URUR US S S .U S UR S05R S S5S .S9IShvN n O+[R"UR US S S .S9IShvN n U SSSn[U5n[U[5(deS[UR55UUUS.S./0nURRURSURS3UR US9IShvN gs sn fNNN7f)Nrllm_model_list llm_routerz/async qdrant semantic-cache set_cache, kwargs: rQrRrS model_namemetadata user_api_keyTrTtrace_idrzsemantic-cache-embeddingrrVrWrXrrUrYrZr[r\r^rrbrc)rdlitellm.proxy.proxy_serverrrrr.r9 aembeddingrer0r/rfr8r=r4r,r)r>rgrhrirdrrrQrjrkmrouter_model_namesrrlrZrYs rDasync_set_cache#QdrantSemanticCache.async_set_cache sIGxPQ*%G i( (F )'5 5n|_n 5   !d&:&:>P&P!::j"599."ML'1'<'<**#'T:$004 & :r : > >z4 P (= ( " (/'9'9**#'T:(" 'v.q1+> E %%%%% djjl+' &$)    ##''( d6J6J5K7SLL$   a 6 ""2 sB9FF BF F +F5F6BF F FFFc # [SU35 SSKJnJn USnSnUH nXgS- nM UbUVs/sHoSPM snO/n Ub~URU ;anUR S05R S S5n UR URUS S S .U S UR S05R S S5S .S9IShvN n O+[R "URUS S S .S9IShvN n U SSSn U SSS SS.0SS S.n URRURSURS3URU S9IShvN nUR5SnUcSURS05S'g[U[ 5(a%[#U5S:XaSURS05S'gUSSnUSSS n[S!UR$S"US#US$U35 UURS05S'UUR$:a.USSS%n[S&US'US(U35 UR'US)9$gs snfGNGNZN7f)*Nz/async qdrant semantic-cache get_cache, kwargs: rrrQrRrSrrrTrTrrrrUrYrZrpFrqrrrrvrrzrcrgzsemantic-similarityr{rar;r|r}r~rr]rrrr)rrrrr.r9rrer8rr4r,rr' setdefaultr/rrr-rN)r>rgrirrrQrjrkrrrrlrZrYrrrrrs rDasync_get_cache#QdrantSemanticCache.async_get_cacheMsGxPQI*%G i( (F )'5 5n|_n 5   !d&:&:>P&P!::j"599."ML'1'<'<**#'T:$004 & :r : > >z4 P (= ( " (/'9'9**#'T:(" 'v.q1+>  ##$'!   !% 1 1 6 6''( d6J6J5K>ZLL!7!   "&&(2 ?GJF  j" -.C D gt $ $7|q KN!!*b12GHQZ(  9-f5  4T5N5N4O~^h]iistzs{|UVcUd e DN*b)*?@ 22 2"1:i0s rD_collection_info$QdrantSemanticCache._collection_infos###s c# /nUH,nURUR"USUS40UD65 M. [R"U6IShvN gN7f)Nrr)appendrasynciogather)r> cache_listritasksvals rDasync_set_cache_pipeline,QdrantSemanticCache.async_set_cache_pipelinesLC LL--c!fc!fGG Hnne$$$sA A AA) r8r<r,r.rr4r5r-r6)NNNNNztext-embedding-ada-002N)__name__ __module__ __qualname____firstlineno__rErrNrmrrrrr__static_attributes__rGrDrrsP! 0GB  (T> @>@X t$%rGr) __doc__rLrr'typingrrelitellm._loggingr base_cacherrrrGrDrs/  *!X%)X%rG