^h\SrSSKrSSKrSSKJr SSKJrJrJr SSK r SSK r SSK J r "SS\ 5r"SS 5r"S S 5r"S S 5r"SS5r"SS5r"SS5r"SS\5r"SS5r"SS5rSrS+S\\S\\S\\S\\S\\S \\S!\\S"\\S#\\S$\\\\ R844S%jjr"S&S'5rS(\S)\4S*jrg),zF Common utilities used across bedrock chat/embedding/image generation N)Enum)ListOptionalUnion) get_secretc(^\rSrSrU4SjrSrU=r$) BedrockErrorc>XlX l[R"SSS9Ul[R "XRS9Ul[TU]!UR5 g)NPOSTz0https://us-west-2.console.aws.amazon.com/bedrock)methodurl) status_coderequest) rmessagehttpxRequestrResponseresponsesuper__init__)selfrr __class__s [/home/james-whalen/.local/lib/python3.13/site-packages/litellm/llms/bedrock/common_utils.pyrBedrockError.__init__sN& }}Q  ; U   LL )rrrr)__name__ __module__ __qualname____firstlineno__r__static_attributes__ __classcell__)rs@rr r s    rr cb\rSrSrSrS\4SjrS\S\4SjrS\\ 4Sjr S\\ 4S jr S r g ) AmazonBedrockGlobalConfigcgNrs rr"AmazonBedrockGlobalConfig.__init__s rreturnc SS0$)zC Mapping of common auth params across bedrock/vertex/azure/watsonx region_nameaws_region_namer(r)s rget_mapped_special_auth_params8AmazonBedrockGlobalConfig.get_mapped_special_auth_params!s011rnon_default_paramsoptional_paramscrUR5nUR5HupEXC;dM XRX4'M U$r')r/items)rr1r2 mapped_paramsparamvalues rmap_special_auth_params1AmazonBedrockGlobalConfig.map_special_auth_params's>;;= .446LE%8= 457rc /SQ$)4 Source: https://www.aws-services.info/bedrock.html )z eu-west-1z eu-west-3z eu-central-1r(r)s rget_eu_regions(AmazonBedrockGlobalConfig.get_eu_regions.s   rc /SQ$)r;)z us-east-2z us-east-1z us-west-2z us-gov-west-1r(r)s rget_us_regions(AmazonBedrockGlobalConfig.get_us_regions8s   rr(N) rrrr rdictr/r8rstrr<r?r!r(rrr$r$sG 22 $QU S    S  rr$c \rSrSr%SrSr\\\S'Sr \\ \S'Sr \\ \S'Sr \\\S'S S\\S\\ S\\ S\\SS4 S jjr\S 5rS rg) AmazonTitanConfigDa[ Reference: https://us-west-2.console.aws.amazon.com/bedrock/home?region=us-west-2#/providers?model=titan-text-express-v1 Supported Params for the Amazon Titan models: - `maxTokenCount` (integer) max tokens, - `stopSequences` (string[]) list of stop sequence strings - `temperature` (float) temperature for model, - `topP` (int) top p for model N maxTokenCount stopSequences temperaturetopPr+c[5nUR5H(upgUS:wdM UcM[URXg5 M* gNrlocalsr4setattrr)rrFrGrHrIlocals_keyr7s rrAmazonTitanConfig.__init__Us7(!--/JCf}!23*rc URR5VVs0sH_upURS5(aM[U[R [R [[45(aMXUcM]X_Ma snn$s snnfN__ __dict__r4 startswith isinstancetypes FunctionTypeBuiltinFunctionType classmethod staticmethodclskvs r get_configAmazonTitanConfig.get_configa **, ,<<% &&--   AD,   B7B9B>Br()NNNN)rrrr __doc__rFrint__annotations__rGlistrHfloatrIrr\rbr!r(rrrDrDDs $(M8C='$(M8D>(#'K%'D(3-(,(,'+" 4} 4 ~ 4e_ 4 sm 4  4  rrDc\rSrSr%SrSr\\\S'Sr \\ \S'Sr \\ \S'Sr \\ \S 'Sr\\ \S 'Sr\\\S 'Sr\\\ \S 'SS\\S\\ S S4Sjjr\S5rSrS\S\4SjrSrg)AmazonAnthropicClaude3ConfigtaN Reference: https://us-west-2.console.aws.amazon.com/bedrock/home?region=us-west-2#/providers?model=claude https://docs.anthropic.com/claude/docs/models-overview#model-comparison Supported Params for the Amazon / Anthropic Claude 3 models: - `max_tokens` Required (integer) max tokens. Default is 4096 - `anthropic_version` Required (string) version of anthropic for bedrock - e.g. "bedrock-2023-05-31" - `system` Optional (string) the system prompt, conversion from openai format to this is handled in factory.py - `temperature` Optional (float) The amount of randomness injected into the response - `top_p` Optional (float) Use nucleus sampling. - `top_k` Optional (int) Only sample from the top K options for each subsequent token - `stop_sequences` Optional (List[str]) Custom text sequences that cause the model to stop generating i max_tokenszbedrock-2023-05-31anthropic_versionNsystemrHtop_ptop_kstop_sequencesr+c[5nUR5H(upEUS:wdM UcM[URXE5 M* grKrL)rrnrorOrPr7s rr%AmazonAnthropicClaude3Config.__init__s7 (!--/JCf}!23*rc URR5VVs0sH_upURS5(aM[U[R [R [[45(aMXUcM]X_Ma snn$s snnfrSrUr^s rrb'AmazonAnthropicClaude3Config.get_configrdrec /SQ$)N) rnmax_completion_tokenstools tool_choicestreamstoprHrq extra_headersr(r)s rget_supported_openai_params8AmazonAnthropicClaude3Config.get_supported_openai_paramss   rr1r2cUR5HIup4US:XdUS:XaXBS'US:XaXBS'US:XaXBS'US:XaXBS'US:XaXBS'US:XdMEXBS'MK U$) Nrnryrzr|r}rsrHrqr4rr1r2r6r7s rmap_openai_params.AmazonAnthropicClaude3Config.map_openai_paramss.446LE $1H(H05 -+0( ,1)49 01 %16 .+0(7rr()NN)rrrr rfrnrrgrhrorBrprHrjrqrrrsrrr\rbrrArr!r(rrrlrlts !%J $';x}; FHSM #'K%'!E8E?!E8C=*.NHT#Y'.%)+/4SM4$C=4  4  "  D4rrlc(\rSrSr%Sr\R r\\ \ S'Sr \\ \ S'Sr \\\ S'Sr\\ \ S'Sr\\ \ S'Sr\\\ S 'SS\\ S\\ S\\S\\ S\\ S \\S S4S jjr\S 5rS rS\S\4SjrSrg)AmazonAnthropicConfiga Reference: https://us-west-2.console.aws.amazon.com/bedrock/home?region=us-west-2#/providers?model=claude Supported Params for the Amazon / Anthropic models: - `max_tokens_to_sample` (integer) max tokens, - `temperature` (float) model temperature, - `top_k` (integer) top k, - `top_p` (integer) top p, - `stop_sequences` (string[]) list of stop sequences - e.g. ["\n\nHuman:"], - `anthropic_version` (string) version of anthropic for bedrock - e.g. "bedrock-2023-05-31" max_tokens_to_sampleNrsrHrrrqror+c[5nUR5H(upUS:wdM U cM[URX5 M* grKrL) rrrsrHrrrqrorOrPr7s rrAmazonAnthropicConfig.__init__s7(!--/JCf}!23*rc URR5VVs0sH_upURS5(aM[U[R [R [[45(aMXUcM]X_Ma snn$s snnfrSrUr^s rrb AmazonAnthropicConfig.get_configrdrec /SQ$)N)rnryrHr}rqr|r(r)s rr1AmazonAnthropicConfig.get_supported_openai_paramss   rr1r2cUR5HFup4US:XdUS:XaXBS'US:XaXBS'US:XaXBS'US:XaXBS'US:XdM;US LdMBXBS'MH U$) NrnryrrHrqr}rsr|Trrs rr'AmazonAnthropicConfig.map_openai_paramss~.446LE $1H(H:? 67 %16 .+0(49 01 Ud],1)7rr()NNNNNN)rrrr rflitellmrnrrrgrhrsrirHrjrrrqrorBrr\rbrrArr!r(rrrrs +2*<*<(3-<%)NHTN)#'K%'E8C=E8C='+x}+/3)-'+##+/ 4&sm 4! 4e_ 4 } 4 } 4$C= 4  4  "   D 4 rrc \rSrSr%SrSr\\\S'Sr \\ \S'Sr \\ \S'S S\\S\\ S\\ SS4Sjjr \S 5rS rg) AmazonCohereConfigia Reference: https://us-west-2.console.aws.amazon.com/bedrock/home?region=us-west-2#/providers?model=command Supported Params for the Amazon / Cohere models: - `max_tokens` (integer) max tokens, - `temperature` (float) model temperature, - `return_likelihood` (string) n/a NrnrHreturn_likelihoodr+c[5nUR5H(upVUS:wdM UcM[URXV5 M* grKrL)rrnrHrrOrPr7s rrAmazonCohereConfig.__init__&7 (!--/JCf}!23*rc URR5VVs0sH_upURS5(aM[U[R [R [[45(aMXUcM]X_Ma snn$s snnfrSrUr^s rrbAmazonCohereConfig.get_config1rdrer(NNN)rrrr rfrnrrgrhrHrjrrBrr\rbr!r(rrrrs!%J $#'K%''+x}+%)'++/ 4SM 4e_ 4$C= 4  4  rrc\rSrSr%SrSr\\\S'Sr \\ \S'Sr \\ \S'Sr \\ \S'Sr\\\S'Sr\\\S 'Sr\\\S 'SS\\S\\ S\\ S\\ S\\S \\S \\S S4S jjr\S 5rSrg)AmazonAI21ConfigiDa# Reference: https://us-west-2.console.aws.amazon.com/bedrock/home?region=us-west-2#/providers?model=j2-ultra Supported Params for the Amazon / AI21 models: - `maxTokens` (int32): The maximum number of tokens to generate per result. Optional, default is 16. If no `stopSequences` are given, generation stops after producing `maxTokens`. - `temperature` (float): Modifies the distribution from which tokens are sampled. Optional, default is 0.7. A value of 0 essentially disables sampling and results in greedy decoding. - `topP` (float): Used for sampling tokens from the corresponding top percentile of probability mass. Optional, default is 1. For instance, a value of 0.9 considers only tokens comprising the top 90% probability mass. - `stopSequences` (array of strings): Stops decoding if any of the input strings is generated. Optional. - `frequencyPenalty` (object): Placeholder for frequency penalty object. - `presencePenalty` (object): Placeholder for presence penalty object. - `countPenalty` (object): Placeholder for count penalty object. N maxTokensrHrIrGfrequencePenaltypresencePenalty countPenaltyr+c[5nUR5H(upU S:wdM U cM[URX5 M* grKrL) rrrHrIrGrrrrOrPr7s rrAmazonAI21Config.__init__as7(!--/JCf}!23*rc URR5VVs0sH_upURS5(aM[U[R [R [[45(aMXUcM]X_Ma snn$s snnfrSrUr^s rrbAmazonAI21Config.get_configprdrer()NNNNNNN)rrrr rfrrrgrhrHrjrIrGrirrArrrr\rbr!r(rrrrDs( $Ix}##'K%' D(5/ $(M8D>('+htn+&*OXd^*#'L(4.'$('+ $(,+/*.'+ 4C= 4e_ 4uo 4  ~ 4 #4. 4"$ 4tn 4  4  rrc\rSrSrSrSrSrg)AnthropicConstantsiz Human: z Assistant: r(N)rrrr HUMAN_PROMPT AI_PROMPTr!r(rrrrs  L!Irrc \rSrSr%SrSr\\\S'Sr \\ \S'Sr \\ \S'S S\\S\\ S\\SS4S jjr \ S 5rS rg) AmazonLlamaConfigia* Reference: https://us-west-2.console.aws.amazon.com/bedrock/home?region=us-west-2#/providers?model=meta.llama2-13b-chat-v1 Supported Params for the Amazon / Meta Llama models: - `max_gen_len` (integer) max tokens, - `temperature` (float) temperature for model, - `top_p` (float) top p for model N max_gen_lenrHrIrFr+c[5nUR5H(upVUS:wdM UcM[URXV5 M* grKrL)rrFrHrIrOrPr7s rrAmazonLlamaConfig.__init__rrc URR5VVs0sH_upURS5(aM[U[R [R [[45(aMXUcM]X_Ma snn$s snnfrSrUr^s rrbAmazonLlamaConfig.get_configrdrer(r)rrrr rfrrrgrhrHrjrIrr\rbr!r(rrrrs"&K#%#'K%' D(5/ (,'+" 4} 4e_ 4sm 4  4  rrc \rSrSr%SrSr\\\S'Sr \\ \S'Sr \\ \S'Sr \\ \S'Sr \\\\S'S S\\S\\ S\\S\\ S\\\S S4 S jjr\S 5rS rg)AmazonMistralConfigia Reference: https://docs.aws.amazon.com/bedrock/latest/userguide/model-parameters-mistral.html Supported Params for the Amazon / Mistral models: - `max_tokens` (integer) max tokens, - `temperature` (float) temperature for model, - `top_p` (float) top p for model - `stop` [string] A list of stop sequences that if generated by the model, stops the model from generating further output. - `top_k` (float) top k for model NrnrHrqrrr}r+c[5nUR5H(upxUS:wdM UcM[URXx5 M* grKrL) rrnrHrqrrr}rOrPr7s rrAmazonMistralConfig.__init__s7(!--/JCf}!23*rc URR5VVs0sH_upURS5(aM[U[R [R [[45(aMXUcM]X_Ma snn$s snnfrSrUr^s rrbAmazonMistralConfig.get_configrdrer()NNNNN)rrrr rfrnrrgrhrHrjrqrrr}rrBrr\rbr!r(rrrrs !%J $#'K%'!E8E?!!E8E?! $D(49 $%)'+#!%$( 4SM 4e_ 4} 4  4 tCy! 4  4  rrc^U4SjnU$)z,Closure to capture the headers and add them.cn>TR5H up#URRX#5 M" g)z.Actual callback function that Boto3 will call.N)r4headers add_header)rkwargs header_name header_valuers rcallback#add_custom_header..callbacks()0 %K OO & &{ A*9rr()rrs` radd_custom_headerrsB Oraws_access_key_idaws_secret_access_keyr.aws_bedrock_runtime_endpointaws_session_nameaws_profile_name aws_role_nameaws_web_identity_tokenr~timeoutc [SS5n [SS5n UUUUUUUU/n [U 5H3upU(dMURS5(dM&[U5X'M5 U unnnnnnnn[R"S[ R 5nU(aO(U(aUnOU (aU nOU (aU nO [SSS9e[S5nU(aUnOU(aUnOS US 3nS SKn[U [5(aURRXS 9nOh[U [R5(a/URRU RU R S 9nOURR5nUbjUbgUbd[U5nUc [S SS9eUR#S5nUR%UUUSS9nUR#SUSSUSSUSSUUUUS9nOUbLUbIUR#SUUS9nUR'XuS9nUR#SUSSUSSUSSUUUUS9nOPUbUR#SUUUUUUS9nO7Ub!UR)US9R#SUUUUS9nOUR#SUUUUS9nU (a/UR*R,R/S[1U 55 U$)NAWS_REGION_NAME AWS_REGIONz os.environ/ SSL_VERIFYzRAWS region not set: set AWS_REGION_NAME or AWS_REGION env variable or in .env filei)rrAWS_BEDROCK_RUNTIME_ENDPOINTzhttps://bedrock-runtime.z.amazonaws.comr)connect_timeout read_timeoutz6OIDC token could not be retrieved from secret manager.stsi)RoleArnRoleSessionNameWebIdentityTokenDurationSecondszbedrock-runtime Credentials AccessKeyIdSecretAccessKey SessionToken) service_namerraws_session_tokenr- endpoint_urlconfigverify)rr)rr)rrrr-rrr) profile_name)rr-rrrzbefore-sign.bedrock-runtime.*)r enumeraterWosgetenvr ssl_verifyr boto3rXrjsessionConfigrTimeoutconnectreadclientassume_role_with_web_identity assume_roleSessionmetaeventsregisterr)r-rrr.rrrrrr~rlitellm_aws_region_namestandard_aws_region_nameparams_to_checkir6r env_aws_bedrock_runtime_endpointrrr oidc_token sts_client sts_responsers rinit_bedrock_clientrs)):DA),= $ Oo. 5U%%m44!+E!2O /  $<););O"P*=9.I#%    "'7'C\\ /"7" "--!. **=9-H".}"=>O"P*=9.I#%    &*/"7#%   %,<=DD*#% E *#%   ## +->}-M  Mrc2\rSrSrSrSrSrSrSrSr g) ModelResponseIteratoricXlSUlg)NF)model_responseis_done)rrs rrModelResponseIterator.__init__s, rcU$r'r(r)s r__iter__ModelResponseIterator.__iter__ rcVUR(a[eSUlUR$NT)r StopIterationrr)s r__next__ModelResponseIterator.__next__s" <<  """rcU$r'r(r)s r __aiter__ModelResponseIterator.__aiter__rrc^# UR(a[eSUlUR$7fr)rStopAsyncIterationrr)s r __anext__ModelResponseIterator.__anext__s% <<$ $ """s+-)rrN) rrrr rrrr r r!r(rrrrs ##rrresponse_tool_namer+c|U[RR;a[RRUnU$)z If litellm formatted the input tool name, we need to convert it back to the original name. Args: response_tool_name (str): The name of the tool as received from the response. Returns: str: The original name of the tool. )rbedrock_tool_name_mappings cache_dict)rs rget_bedrock_tool_namers:W??JJJ$??JJ   r) NNNNNNNNNNN) rfrrYenumrtypingrrrrrlitellm.secret_managers.mainr Exceptionr r$rDrlrrrrrrrrBrArjrrrrr(rrrs (( 3  9  $ $ N- - `PPfMM`* * Z< < ~"" * * Z/ / d'++/%)26&*&*#',0$(59s}s$C=sc] s #+3- s sm ssmsC=s%SMsD>seE5==01 2sl##4ccr