^h#SSKrSSKJrJrJrJrJrJr SSKJ r SSK r SSK r SSK J r SSK Jr SSK Jr SSK Jr SSKJr SSKJr SSKJr SS KJr SS KJr SS K Jr! SSK"Jr# SSK$Jr% SS K&J'r( SSK&Jr) SS K*J+r+ SS K,Jr- SS K.J/r/ SSK0J1r1 SSK2J3r3J4r4J5r5 SSK6J7r7J8r8J9r9J:r:J;r;Jr>J?r? S?S\@S\@S\\@S\\8S\\@S\\\@\@44 SjjrAS@S\BS\CS\CS\\@S\\BS\\CS\\CS\\CS\\CS\\8S\\@S \\CS!\\5S"\3S#\@S\\@\@44 S$jjrSAS%\D4S&jjrES'\S\F4S(jrGSBS\\BS\\BS\\B4S)jjrHSCS\\BS%\\S*\\BS+\\FS\\BS\\B4 S,jjrIS%\S\\54S-jrJS"\\3S%\S\\34S.jrKSSS/SSSSSSSSSSSSS4S\\BS/\S0\\@S"\\3S1\\BS2\\BS3\\CS\\8S\\@S4\\DS+\\FS*\\BS\@4S5jjrLSDS6\\;\9\:\=\<\/\14S\BS\\BS"\S7S4\DS8\\FS*\\BS+\\FS9\BS\\@4S:jjrMS\BS\\BS\\@\@44S;jrNS\BS\\BS<\@S\\@\@44S=jrOSES\BS\\BS2\\BS3\\CS1\\BS4\\DS\@4S>jjrPg)FN)AnyListLiteralOptionalTupleUnion) BaseModel)verbose_logger)cost_per_character)cost_per_token) cost_router)_generic_cost_per_character)cost_per_query)cost_calculator)cost_per_second)get_model_params_and_category)HttpxBinaryResponseContent)RerankResponse)CallTypesLiteralPassthroughCallTypesUsage) CallTypes CostPerTokenEmbeddingResponse ImageResponse ModelResponseTextCompletionResponseTranscriptionResponse print_verbose token_counter prompt_tokenscompletion_tokensresponse_time_mscustom_cost_per_tokencustom_cost_per_secondreturncZUcUcgUbUSU-nUSU-nXV4$Ub XB-S- nSU4$g)zFInternal helper function for calculating cost, if custom pricing givenNinput_cost_per_tokenoutput_cost_per_tokenr)r"r#r$r%r& input_cost output_costs Q/home/james-whalen/.local/lib/python3.13/site-packages/litellm/cost_calculator.py%_cost_per_token_custom_pricing_helperr0Bsc$)?)G(*+AB]R +,CDGXX &&  +,?$F +~ modelcustom_llm_providerprompt_characterscompletion_characterscache_creation_input_tokenscache_read_input_tokensnumber_of_queries usage_object call_type!audio_transcription_file_durationc Uc [S5eU bU nO[UUX-UU S9n[UUUU U S9nUb USUS4$SnSn[RnUnUbUS-U-nUbUSUSU3nUU;aUnO[R "US9unn nUnUR SS5n[U5S:aUSnOUnUU;aUnOUU;aUnOUU;aUnUS :XdUS :XaZUc[S RUUUU55e[UUUSSSS 9unnUbUc[S RUUUUUU55eUU4$US:XdUS:Xa [UUS9$US:XdUS:Xa [UUUS9$US:Xa2[UUUS9nUS:Xa[UUUUUUS9$US:Xa [UUUUS9$gUS:Xa [!UUS9$US:Xa [#UUS9$US:Xa [%UUS9$US:Xa ['UUS9$US:Xa [)UUUS 9$US!:Xa [UUUUS9$[R*"XS9nUS"S:a US"U-nODUR-S#S5b1Ub.[/S$US%UR-S#5S&U35 US#U-S'- nUS(S:a US(U-nODUR-S)S5b1Ub.[/S$US*UR-S)5S&U35 US)U-S'- n[/S+US,US-U35 UU4$).a6 Calculates the cost per token for a given model, prompt tokens, and completion tokens. Parameters: model (str): The name of the model to use. Default is "" prompt_tokens (int): The number of tokens in the prompt. completion_tokens (int): The number of tokens in the completion. response_time (float): The amount of time, in milliseconds, it took the call to complete. prompt_characters (float): The number of characters in the prompt. Used for vertex ai cost calculation. completion_characters (float): The number of characters in the completion response. Used for vertex ai cost calculation. custom_llm_provider (str): The llm provider to whom the call was made (see init.py for full list) custom_cost_per_token: Optional[CostPerToken]: the cost per input + output token for the llm api call. custom_cost_per_second: Optional[float]: the cost per second for the llm api call. call_type: Optional[str]: the call type Returns: tuple: A tuple containing the cost in USD dollars for prompt tokens and completion tokens, respectively. Nz"Invalid arg. Model cannot be none.)r"r# total_tokensr7r8)r"r#r$r&r%r/r3speechaspeechzvprompt_characters must be provided for tts calls. prompt_characters={}, model={}, custom_llm_provider={}, call_type={})r3r4r5r6custom_prompt_costcustom_completion_costzcost for tts call is None. prompt_cost={}, completion_cost={}, model={}, custom_llm_provider={}, prompt_characters={}, completion_characters={}arerankrerankr3r4atranscription transcriptionr3r4duration vertex_ai)r3r4r;r )r3r4r5r6r"r#r )r3r4r"r# anthropic)r3usageopenai databricks fireworks_aiazure)r3rOr$geminir)input_cost_per_secondz For model=z - input_cost_per_second: z; response time: r+r*output_cost_per_secondz - output_cost_per_second: zReturned custom cost for model=z" - prompt_tokens_cost_usd_dollar: z%, completion_tokens_cost_usd_dollar: ) Exceptionrr0litellm model_costget_llm_providersplitlen ValueErrorformatr rerank_costopenai_cost_per_secondgoogle_cost_routergoogle_cost_per_charactergoogle_cost_per_tokenanthropic_cost_per_tokenopenai_cost_per_tokendatabricks_cost_per_tokenfireworks_ai_cost_per_tokenazure_openai_cost_per_tokenget_model_infogetr)r3r"r#r$r4 region_namer5r6r7r8r%r&r9r:r;r< usage_block response_costprompt_tokens_cost_usd_dollar!completion_tokens_cost_usd_dollarmodel_cost_refmodel_with_providermodel_with_provider_and_region_model_without_prefix model_parts prompt_costcompletion_costr model_infos r/r r YsT }<==" '/&:(C$;  :#+)53 M Qq!111,-!/0%''N&1C7%?  "&'q Qug> +/.@&D#'.'?'?e'L$ 1 ++c1%K ;!*1~$ ~-# . .$H Y 6  $IPP%'  (C& 3/"###$ ( $ _  /"9bii#('%)   O++ i 9#8 3   & &)*F% 36   +(& 3 . .,*$7"3&;+"3  , ,(*$7+"3  -  +'e;GG  ($5 DD  ,(uKHH  .*kJJ  '*{=M    ($& 3'/   ++  , - 112]B * NN2D 9 E , UG#=jnnMd>e=ffwyIxJK  236FFM * - . 2236GG . NN3T : F , UG#>z~~Nf?g>hhy{KzLM  347GG$N . -eW4VWtVuv[\}[~  -.OOOr1completion_responsecSnUS:XaIURS[R"55n[US[R"55nXC- nX!-S- $)NgǺV?r!createdendedr+)rjtimegetattr)ry total_time!a100_80gb_price_per_second_public start_timeend_times r/ get_replicate_completion_pricingrEsY &S(,,Y D .E* , 9D @@r1objc[US5$)N_hidden_params)hasattr)rs r/has_hidden_paramsrSs 3( ))r1cUbU$Ucg[R"US9up! nU$![a,n[R"S[ U535 SnAgSnAff=f)NrAzblitellm.cost_calculator.py::_get_provider_for_cost_calc() - Error inferring custom_llm_provider - )rXrZrWr debugstr)r3r4rses r/_get_provider_for_cost_calcrWss&"" }'.'?'?e'L$1  pqtuvqwpx y  s% A"AA base_modelcustom_pricingcDSnSn[XS9nUSLaUnUbUnSnUb[U[5(a[USS5n[USS5nUc:Ub7UR SS5b$[ US5S:aUR SU5nUb%UR SS5bUR SS5nO*Ub'[U[ 5(aUR SS5nUcUbUnUcUbUnUb/Ub,URU5(dUb USUSU3nU$USU3nU$) z 1. If custom pricing is true, return received model name 2. If base_model is set (e.g. for azure models), return that 3. If completion response has model set return that 4. Check if model is passed in return that NrHTr3rrrkr@)r isinstancer r~rjr\dict startswith) r3ryrrr4 return_modelrkcompletion_response_model hidden_paramss r/ _select_model_name_for_cost_calcrjs#'L!%K5 ! /3&:6I9+U+U$+,?$$O! 35EtL $ ,1J!!'40< g./!3,00%@  %!!-6B'++M4@K  (Z8KT-R-R$7$;$;GT$J! 9 E0  1    +''(;<<  "12!K=,PL 22!L>BL r1c\SnUb&[U[5(aURS5nU$)NrO)rrrj)ry usage_objs r/_get_usage_objectrs8"&I&:],,(++G4 r1cLUbU$Ucg[U[5(ag[U[5(ag[U[5(ag[U[5(ag[U[ 5(ag[U[ 5(ag[U[5(agU$)N completion embeddingrJrBrGimage_generationtext_completion)rrrrrrrr)r;rys r/_infer_call_typers"%}55 '): ; ; ')> ? ? ')C D D ' 8 8 ' 7 7! ')? @ @ r1messagesrsizequalitynoptional_paramsch [X`5=(d SnUS:XdUS:Xa/Ub,[U[5(a[U5S:XaUS:XaSnSnSnSnSnSnSnSn[ US 9n[ UUUUUS 9nUGb)[U[ 5(d[U[5(GaURS 05n[U[ 5(aN[U[R5(d/[US [R"S70UR5D65 Uc0nO([U[ 5(aUR5nOUnURS S5nURS S5nURSS5nURSS5nSU;a7US0:wa.US(a$URS05nURSS5n[USS5n[R"S[USS5S35 [USS5nUbURSU=(d S5nURSU5nURS05RSS5n URS05RSS5n URS05RSS5n ONUc[!SUS U35e[U5S:a [#XS!9nO[U5S:a [#XS"9n[#XS"9nUc[!SUS U35e[R$"US#9up nU[*R,R.:Xd<U[*R0R.:XdU[2R4R.:XagUS%:Xa![U[65(a [9UUS&9$GOUS':Xa,[U[65(a [;UU UUS(9$[=S)5e[?UU UU U US*9$U[*R@R.:XdU[*RBR.:Xa[RDRGUS+9nOU[*RHR.:XdU[*RJR.:Xa[US,S5nOU[*RLR.:XdU[*RNR.:XaaUb^[U[P5(aIURRnUbURS-05=(d 0n O0n U RS.5=(d Sn!U!nS/U;d S0U;dUS0:Xa[UU[+U5S19nO9U[RV;dS2U;aU[RX;a [[X5$Uc[!SUS U35eUbUS%:Xa[U5S:a=[RDR]S3U0SS49n"[RDRGU"S+9nUbO[U[^5(a:[RDRaUS59n#[RDRGU#S+9n[cUUUUUUU U UUUUUUUS69un$n%U$U%-n&U&$![&a9n[R"S$R)[U555 SnAGN{SnAff=f![&anUeSnAff=f)8aW Calculate the cost of a given completion call fot GPT-3.5-turbo, llama2, any litellm supported llm. Parameters: completion_response (litellm.ModelResponses): [Required] The response received from a LiteLLM completion request. [OPTIONAL PARAMS] model (str): Optional. The name of the language model used in the completion calls prompt (str): Optional. The input prompt passed to the llm completion (str): Optional. The output completion text from the llm total_time (float, int): Optional. (Only used for Replicate LLMs) The total time used for the request in seconds custom_cost_per_token: Optional[CostPerToken]: the cost per input + output token for the llm api call. custom_cost_per_second: Optional[float]: the cost per second for the llm api call. Returns: float: The cost in USD dollars for the completion based on the provided parameters. Exceptions: Raises exception if model not in the litellm model cost map. Register model, via custom pricing or PR - https://github.com/BerriAI/litellm/blob/main/model_prices_and_context_window.json Note: - If completion_response is provided, the function extracts token information and the model name from it. - If completion_response is not provided, the function calculates token counts based on the model and input text. - The cost is calculated based on the model, prompt tokens, and completion tokens. - For certain models containing "togethercomputer" in the name, prices are based on the model size. - For un-mapped Replicate models, the cost is calculated based on the total time used for the request. raimage_generationrNrrSzdall-e-2r!)ry)r3ryr4rrrOr"r#r7r8prompt_tokens_details cached_tokens _response_msz!completion_response response ms:  rr4rkrr 1024-x-1024rstandardrr?z[Model is None and does not exist in passed completion_response. Passed completion_response=z, model=)r3r)r3textrAzXlitellm.cost_calculator.py::completion_cost() - Error inferring custom_llm_provider - {}rM)r3image_responsebedrock)r3rrrzTcompletion_response must be of type ImageResponse for bedrock image cost calculation)r3rr4rrr)rrL billed_units search_unitstogethercomputer together_ai)r; replicater)datar;) response_obj)r3r"r#r4r$rkr&r%r5r6r7r8r:r;r<r,)2rrrr\rrr rrjrXrsetattr model_dumpr~r rr]r rZrWr^rrvaluerrpassthrough_image_generationrvertex_ai_image_cost_calculatorbedrock_image_cost_calculator TypeErrordefault_image_cost_calculatorrBrCutils_count_charactersrIrJrGrFrmetarreplicate_modelsrYrget_formatted_promptrget_response_stringr )'ryr3promptrrrr;r4rkrrrr%r&rrrr"r5r#r6r7r8r<cost_per_token_usage_objectr_usagerrrsrmeta_objrr prompt_stringcompletion_stringrnro _final_costs' r/rwrwsxdh$YDT - ->P1P!5#&&E a#w.E +//359#1536)7H 38 #1 3 3)!    * *I 6 6-t446I6M6M7I)Y// 7==99'MM;I$8$8$:;  Iy11"--/""JJ:M & +> B *0**5RTU*V '&,jj1JA&N #'6123r923(. 3JB(O%*?*C*COUV*W' !4naHJ  3G+F$'#,// {K $(():B?CCM(++,=rBFFz"%%&7<@@} qsFrGGOPUOVW8}q -E M Vq -E G -E K  =moBnCCKLQKRS  /6/G/G0 ,E1 3399 9I77===0MMSSS#k11=AA:#':B % 11=AA8#!':(7   j5#(;$3  ))// /9 @Q@Q@W@W3W ' ? ?V ? L  1177 7I3399907#Z1 - ))// /9 @Q@Q@W@W3W".:#^44/33'#+<<#C#IrL#%L!$$^49%1! % '%"m3 2%9YCWXE W-- -1E7---34GT T =moBnCCKLQKRS   */Bk/Q8}q ' B B$h/<!C! %,MM$C$C$C$W!".:#]44%,MM$E$E!4%F%!)0 (G(G*)H)% '/ 3'##9"7/"7(C$;4.O  ) -$46WW C   jqqF   D s]LZ YA>Z (Z Z FZ >CZ  Z$.ZZ ZZ Z1*Z,,Z1response_object)r aembeddingr acompletionatext_completionrrr moderation amoderationrIrJrCrBrGrF cache_hitrc Sn Ub USLaSn U $[U[5(aX@RS'[UUUUUUUUS9n U $![an U eSn A ff=f)z+ Returns - float or None: cost of response r!NTr)ryr3r;r4rrrr)rr rrwrW) rr3r4r;rrrrrrmrs r/response_cost_calculatorrsP"  Y$%6M/955DS../@A+$3#$7 /-% M s A4A AAAcSn[R"XS9up1 nUS:Xa [XS9$US:Xa [XS9$[ SUSU35e![ anUeSnAff=f) z= Returns - float or None: cost of response OR none if error. r?rHcohere)r3 num_queriesazure_aiz.invalid custom_llm_provider for rerank model: z, custom_llm_provider: N)rXrZcohere_rerank_cost_per_queryazure_ai_rerank_cost_per_queryr]rW)r3r4default_num_queriesrsrs r/r_r_*s#*#;#;$ AAq  ( */ !J .1 QSU0V/W X +27)1_%& &'>&?r/ARS '"4"44&&'>? G.. .&&7 12I1J%P_O` a   + ,v 5 = AAr1)rrr!NN)r2rrr!NNNNrrNNNNrr!)r!)N)NNN)NNNr2)NNr?rN)Qr}typingrrrrrrpydanticr rXlitellm._loggingr /litellm.litellm_core_utils.llm_cost_calc.googler rbr rcr ra.litellm.litellm_core_utils.llm_cost_calc.utilsr'litellm.llms.anthropic.cost_calculationrd#litellm.llms.azure.cost_calculationrh%litellm.llms.azure_ai.cost_calculatorrr*litellm.llms.bedrock.image.cost_calculatorrr#litellm.llms.cohere.cost_calculatorr'litellm.llms.databricks.cost_calculatorrf)litellm.llms.fireworks_ai.cost_calculatorrg$litellm.llms.openai.cost_calculationrr`re(litellm.llms.together_ai.cost_calculatorr7litellm.llms.vertex_ai.image_generation.cost_calculatorrlitellm.types.llms.openairlitellm.types.rerankrlitellm.types.utilsrrr litellm.utilsrrrrrrrrr floatr0rrrrboolrrrrrrwrr_rrr,r1r/rs =="WYRA/MM    (+48.2 uo $L1  %UO eE5L!"0(+)-'++/12-.48.2'+$("./2-iP iPiPiPuo iP "# iP }iP$C=iP"*#iP&c]iP$L1iP%UOiP" }#iP&5/'iP* +iP,(--iP. 5%</iPX A$ A*3*4* *. C=!#c],!%%))- ; C=;!#; ;TN ; "# ; c] ;|   e_ ()@C 8 "%,0!48.2&*%) $)Z C=Z Z  Z()Z 3-Zc]Z}Z $L1!Z"%UO#Z$d^%Z&TN'Z( )Z* +Zx !% $%)E;"  ; ;"#; ;<=;>~?;@ A;BTNC;D E;Fe_G;| !# 5%<: %-c]>C 5%<*.!'&* ;B ;B!#;Bc];B} ;B 3- ;B d^ ;B ;Br1