cCiǼ SrSSKJrJrJr SSKrSSKJr SSKJrJ r J r SSK J r SSK JrJr SS KJr SS KJr SS KJr SS KJr SS KJrJrJrJr SSKJrJr SSK J!r! SSK"J#r#J$r$J%r%J&r&J'r' SSK(J)r) SSK*J+r+ \&"5(a SSK,J-r- SSK.J/r/ \'R`"\15r2"SS\Rf5r4S3S\RjS\RlS\RlS\RlS\\RlS\7S\74Sjjr8"S S!\Rj5r9"S"S#\5r:\$"S$S%\55r;"S&S'\;5r<\$"S(S)\;55r="S*S+\;\5r>\$"S,S-9"S.S/\;55r?\$"S0S1\;55r@/S2QrAg)4zPyTorch OPT model.)CallableOptionalUnionN)nn)BCEWithLogitsLossCrossEntropyLossMSELoss)ACT2FN)Cache DynamicCache)GenerationMixin)AttentionMaskConverter)FlashAttentionKwargs)GradientCheckpointingLayer)BaseModelOutputWithPastCausalLMOutputWithPastQuestionAnsweringModelOutput SequenceClassifierOutputWithPast)ALL_ATTENTION_FUNCTIONSPreTrainedModel)Unpack)TransformersKwargsauto_docstringcan_return_tupleis_torch_flex_attn_availablelogging)deprecate_kwarg) OPTConfig) BlockMask)make_flex_block_causal_maskc^\rSrSrSrS\S\4U4SjjrS S\RS\S\ \R4U4S jjjr S r U=r $) OPTLearnedPositionalEmbedding3zF This module learns positional embeddings up to a fixed maximum size. num_embeddings embedding_dimcL>SUl[TU] XR-U5 gN)offsetsuper__init__)selfr&r' __class__s ^/home/james-whalen/.local/lib/python3.13/site-packages/transformers/models/opt/modeling_opt.pyr-&OPTLearnedPositionalEmbedding.__init__8s"  ++5}Eattention_maskpast_key_values_length position_idsc>Uc5[R"USS9nX1-S- R5nUSS2US24n[TU]X0R -5$)z3`input_ids_shape` is expected to be [bsz x seqlen].Nrdim)torchcumsumlongr,forwardr+)r.r3r4r5r/s r0r<%OPTLearnedPositionalEmbedding.forward>sZ   <<A>L(9A=CCEL'+A+B(BCLw|kk9::r2)r+)rN) __name__ __module__ __qualname____firstlineno____doc__intr-r9 LongTensorrr<__static_attributes__ __classcell__r/s@r0r$r$3s]FsF3F'(37 ;((;!$;u//0 ;;r2r$modulequerykeyvaluer3scalingdropoutc [R"XRSS55U-nUbX-n[RR US[R S9RUR5n[RRXURS9n[R"X5n U RSS5R5n X4$)N)r8dtypeptrainingrr*) r9matmul transposer functionalsoftmaxfloat32torQrMrT contiguous) rHrIrJrKr3rLrMkwargs attn_weights attn_outputs r0eager_attention_forwardr_Ps<<}}R'<=GL!#4 ==((2U]](SVVW\WbWbcL==((6??([L,,|3K''1-88:K  $$r2c6^\rSrSrSrSS\S\\4U4Sjjjr\ "SSSS 9SS \ RS\\ S \\ RS \\ RS \ S\\ RS\\ R\\ R\\ 44Sjj5rSrU=r$) OPTAttentiongz=Multi-headed attention from 'Attention Is All You Need' paperconfig layer_idxc >[TU]5 XlURUlUR UlURUlURUl X l Uc-[RSURRS35 URUR -UlSUlURUR -UR:wa&[#SURSUR S35eURS-Ul[&R("URURURS9Ul[&R("URURURS9Ul[&R("URURURS9Ul[&R("URURURS9Ulg) NzInstantiating z without passing a `layer_idx` is not recommended and will lead to errors during the forward call if caching is used. Please make sure to provide a `layer_idx` when creating this class.Tz;embed_dim must be divisible by num_heads (got `embed_dim`: z and `num_heads`: z).g࿩bias)r,r-rc hidden_size embed_dimnum_attention_heads num_headsattention_dropoutrM enable_biasrdlogger warning_oncer/r>head_dim is_causal ValueErrorrLrLineark_projv_projq_projout_proj)r.rcrdr\r/s r0r-OPTAttention.__init__js{  ++33// !--"      !8!8 9:,,  $..8  MMDNN *t~~ =MdnnM]$T^^$4B8 }}d* iiTEUEUV iiTEUEUV iiTEUEUV  $..$..tGWGWX r2past_key_valuepast_key_values4.58new_nameversion hidden_statesr3layer_head_maskoutput_attentionscache_positionreturnc UR5upn URU5UR-n U RUSURUR 5R SS5n URU5n URU5n U RUSURUR 5R SS5n U RUSURUR 5R SS5n Ub!URXURSU05up[nURRS:wa[URRnU"UU U U U4UR(dSO UR SS .UD6unnUR#XS5R%5nUR'U5nU(dSnUU4$) z#Input shape: Batch x Time x ChannelrOrr*Nreager?)rMrL)sizervrLviewrkrprVrtruupdaterdr_rc_attn_implementationrrTrMreshaper[rw)r.rrzr3rrrr\bsztgt_len_ query_states key_states value_statesattention_interfacer^r]s r0r<OPTAttention.forwards(,,.a{{=1DLL@ #((b$..$--PZZ[\^_` [[/ {{=1 __S"dnndmmLVVWXZ[\ #((b$..$--PZZ[\^_`  &'6'='=$..;K^:\( $J)@ ;; + +w 6"9$++:Z:Z"[ $7      % $}}C$,, %  % ! \"))#;FFH mmK0  LL((r2) rcrMrirmrprqrtrdrkrwrvrLruN)NNNFN)r>r?r@rArBr rrCr-rr9Tensorr booltupler<rErFrGs@r0raragsG $(!Y!YC=!Y!YF%0A6R,01526"'156)||6)"%6)!. 6) "%,,/ 6)  6)!.6) u||Xell3Xe_D E6)S6)r2rac^\rSrSrSS\S\\4U4Sjjjr\"SSSS9SS \ RS \\ RS \\ RS\\ S \\ S \\ S\\ RS\\ RS\\S\\ R"\\\ R"\ R"444Sjj5rSrU=r$)OPTDecoderLayerrcrdcp>[TU]5 URUl[ XS9UlUR UlURUl[URUl [R"URURS9Ul[R"URUR UR"S9Ul[R"UR URUR"S9Ul[R"URURS9Ulg)N)rcrdelementwise_affinerf)r,r-rhrira self_attndo_layer_norm_beforerMr activation_function activation_fnr LayerNormlayer_norm_elementwise_affineself_attn_layer_normrsffn_dimrmfc1fc2final_layer_norm)r.rcrdr/s r0r-OPTDecoderLayer.__init__s ++%VI$*$?$?!~~ #F$>$>?$&LL NNv/S/S% !99T^^V^^&BTBTU99V^^T^^&BTBTU " T^^PVPtPt ur2ryrzr{r|rr3rr use_cacher5rr\rc *Un UR(aURU5nUR"SUUUUUUUS.U D6up[RR XR UR S9nX-nUR(dURU5nURn URSURS55nUn UR(aURU5nURU5nURU5nURU5n[RR XR UR S9nX-RU 5nUR(dURU5nU4n U(aX4- n U $)a" Args: hidden_states (`torch.FloatTensor`): input to the layer of shape `(batch, seq_len, embed_dim)` attention_mask (`torch.FloatTensor`, *optional*): attention mask of size `(batch, 1, tgt_len, src_len)` where padding elements are indicated by very large negative values. layer_head_mask (`torch.FloatTensor`, *optional*): mask for attention heads in a given layer of size `(encoder_attention_heads,)`. output_attentions (`bool`, *optional*): Whether or not to return the attentions tensors of all attention layers. See `attentions` under returned tensors for more detail. use_cache (`bool`, *optional*): If set to `True`, `past_key_values` key value states are returned and can be used to speed up decoding (see `past_key_values`). past_key_values (`Cache`, *optional*): cached past key and value projection states cache_position (`torch.LongTensor` of shape `(sequence_length)`, *optional*): Indices depicting the position of the input sequence tokens in the sequence.. )rrzr5r3rrrrRrO)rrrrrWrMrTshaperrrrrrr)r.rr3rrzrrr5rr\residualself_attn_weightshidden_states_shapeoutputss r0r<OPTDecoderLayer.forwards>!  $ $ 55mDM,0>> , '+%)+/) ,  , (  --m||VZVcVc-d  0 (( 55mDM,11%--b-2D2DR2HI    $ $ 11-@M/ **=9 /  --m||VZVcVc-d !1778KL (( 11-@M "  + +Gr2) rrrMrirrrrrr)NNNFFNN)r>r?r@rAr rrCr-rr9rr rrDrrr FloatTensorr<rErFrGs@r0rrs/vyvXc]vv"%0A6R2626+/,1$)3715P||P!.P"%,,/ P "% P $D> PD>Pu//0P!.P-.P u  (51B1BEDUDU1U+V"WW XPSPr2rcH\rSrSr%\\S'SrSrS/rSr Sr Sr Sr Sr SrSrg) OPTPreTrainedModeli-rcmodelTrcURRn[U[R5(aWUR R RSUS9 URb%URR R5 gg[U[R5(adUR R RSUS9 URb2UR R URR5 gg[U[R5(aJUR R RS5 URR R5 gg)Nr)meanstdr)rcinit_std isinstancerrsweightdatanormal_rgzero_ Embedding padding_idxrfill_)r.rHrs r0 _init_weights OPTPreTrainedModel._init_weights:skk"" fbii ( ( MM   & &CS & 9{{&   &&('  - - MM   & &CS & 9!!- ""6#5#56<<>.  - - MM   $ $S ) KK   " " $.r2rN)r>r?r@rAr __annotations__base_model_prefixsupports_gradient_checkpointing_no_split_modules_supports_attention_backend_supports_flash_attn_supports_sdpa_supports_flex_attn_can_compile_fullgraphrrErr2r0rr-s? &*#*+"&N! %r2rc2^\rSrSrSrS\4U4SjjrSS\\RS4S\RS\RS \ S \ 4 S jjr \ S\RS \S \S\RS\RS\4 Sj5r\SS\\R&S\\RS\\RS \\ S\\R(S\\ S \\ S\\ S\\ S\\R&S\\RS\\S\\\44Sjj5rSrU=r$) OPTDecoderiIz Transformer decoder consisting of *config.num_hidden_layers* layers. Each layer is a [`OPTDecoderLayer`] Args: config: OPTConfig rcc F>[TU]U5 URUlURUlURUlUR UlURUl[R"URURUR 5Ul [UR UR5UlURUR:wa0[R "URURSS9UlOSUlURUR:wa0[R "URURSS9UlOSUlUR&(a@UR((d/[R*"URUR,S9UlOSUl[R0"[3UR45Vs/sH n[7XS9PM sn5UlSUlUR=5 gs snf)NFrfr)rd)r,r-rM layerdrop pad_token_idrmax_position_embeddingsmax_target_positions vocab_sizerrword_embed_proj_dim embed_tokensr$rhembed_positionsrs project_out project_inr_remove_final_layer_normrrr ModuleListrangenum_hidden_layersrlayersgradient_checkpointing post_init)r.rcir/s r0r-OPTDecoder.__init__Qs  ~~ ))!..$*$B$B! ++LL):):FT]n%>>*'7 MM  ""&,,Q/ !+??AMnell;;$$R(%7!; PP +')#))!, Q  KK , , 6*%%**.DD%  E*..I0CCK[Kr2rrrQrc UbUR5S:XaUnU$[R"U5Rn[R"X4XUR S9nUS:wa[R "USS9nU[R"X$R S9URSS5:-nUSSSS2SS24RUSSS5nUbUR5nURSn USS2SS2SS2SU 24USS2SSSS24RUR 5-n U S:Hn USS2SS2SS2SU 24RX5USS2SS2SS2SU 24'U$) a Creates a causal 4D mask of shape `(batch_size, 1, query_length, key_value_length)` from a 2D mask of shape `(batch_size, key_value_length)`, or if the input `attention_mask` is already 4D, do nothing. Args: attention_mask (`torch.Tensor`): A 2D attention mask of shape `(batch_size, key_value_length)` or a 4D attention mask of shape `(batch_size, 1, query_length, key_value_length)`. sequence_length (`int`): The sequence length being processed. target_length (`int`): The target length: when generating with static cache, the mask should be as long as the static cache, to account for the 0 padding, the part of the cache that is not filled yet. dtype (`torch.dtype`): The dtype to use for the 4D attention mask. cache_position (`torch.Tensor`): Indices depicting the position of the input sequence tokens in the sequence. batch_size (`torch.Tensor`): Batch size. N) fill_valuerQrr)diagonalrrOr)r8r9rrfullrtriuarangerexpandclonerrZ masked_fill) r3rrrQrrr\rr mask_length padding_masks r0r@OPTDecoder._prepare_4d_causal_attention_mask_with_cache_positions}>  %.*<*<*>!*C(K*' E*..I** 0Y\j\q\qK!##jjqA 5<< >S>STWeWmWmnprsWtt tK%dD!Q&67>>z1bRTUK))//1 ,2226 *1aL[L+@ANSTVZ\`bcScDdDgDg&&E   ,q0 5@Aq,;,AV5W5c5c 6 Aq!\k\12r2 input_ids head_maskrroutput_hidden_states return_dictr5r\rc nUbUOURRnUbUOURRnUbUOURRnU bU OURRn USLUSL- (a [ S5eUR (a/UR(aU(a[RS5 SnUbURSURS5nUcURU5nU(aUc[URS9nUbUR5OSn U c/[R "XURS-UR"S 9n Uc=XRS-n[R$"URSXR"S 9nUR'X%XU5nU c5[R("USS 9n X-S- R+5n U SS2U S24n UR-X-U S 9nUR.bUR/U5nUUR1UR"5-nU(aS OSnU(aS OSn[3U/S /5HnunnUcM UR55S[7UR85:wdM7[ SUS[7UR85SUR55SS35e [;UR85H|unnU(aUU4- nUR(a([R<"/5nUUR>:aMLU"U4UU UbUUOSUUUU S.U D6nUSnU(dMsUUS4- nM~ UR@bURAU5nURBbURCU5nU(aUU4- n[EUUUUS9$)a: Args: input_ids (`torch.LongTensor` of shape `(batch_size, sequence_length)`): Indices of input sequence tokens in the vocabulary. Padding will be ignored by default should you provide it. Indices can be obtained using [`AutoTokenizer`]. See [`PreTrainedTokenizer.encode`] and [`PreTrainedTokenizer.__call__`] for details. [What are input IDs?](../glossary#input-ids) attention_mask (`torch.Tensor` of shape `(batch_size, sequence_length)`, *optional*): Mask to avoid performing attention on padding token indices. Mask values selected in `[0, 1]`: - 1 for tokens that are **not masked**, - 0 for tokens that are **masked**. [What are attention masks?](../glossary#attention-mask) head_mask (`torch.Tensor` of shape `(num_hidden_layers, num_attention_heads)`, *optional*): Mask to nullify selected heads of the attention modules. Mask values selected in `[0, 1]`: - 1 indicates the head is **not masked**, - 0 indicates the head is **masked**. past_key_values (`Cache`, *optional*, returned when `use_cache=True` is passed or when `config.use_cache=True`): It is a [`~cache_utils.Cache`] instance. For more details, see our [kv cache guide](https://huggingface.co/docs/transformers/en/kv_cache). Contains pre-computed hidden-states (key and values in the self-attention blocks and in the cross-attention blocks) that can be used (see `past_key_values` input) to speed up sequential decoding. If `past_key_values` are used, the user can optionally input only the last `decoder_input_ids` (those that don't have their past key value states given to this model) of shape `(batch_size, 1)` instead of all `decoder_input_ids` of shape `(batch_size, sequence_length)`. inputs_embeds (`torch.FloatTensor` of shape `(batch_size, sequence_length, hidden_size)`, *optional*): Optionally, instead of passing `input_ids` you can choose to directly pass an embedded representation. This is useful if you want more control over how to convert `input_ids` indices into associated vectors than the model's internal embedding lookup matrix. output_attentions (`bool`, *optional*): Whether or not to return the attentions tensors of all attention layers. See `attentions` under returned tensors for more detail. output_hidden_states (`bool`, *optional*): Whether or not to return the hidden states of all layers. See `hidden_states` under returned tensors for more detail. return_dict (`bool`, *optional*): Whether or not to return a [`~utils.ModelOutput`] instead of a plain tuple. position_ids (`torch.LongTensor` of shape `(batch_size, sequence_length)`, *optional*): Indices of positions of each input sequence tokens in the position embeddings. Selected in the range `[0, config.n_positions - 1]`. for padding use -1. [What are position IDs?](../glossary#position-ids) cache_position (`torch.LongTensor` of shape `(sequence_length)`, *optional*): Indices depicting the position of the input sequence tokens in the sequence. Contrarily to `position_ids`, this tensor is not affected by padding. It is used to update the cache in the correct position and to infer the complete sequence length. Nz:You must specify exactly one of input_ids or inputs_embedszX`use_cache=True` is incompatible with gradient checkpointing. Setting `use_cache=False`.FrO)rcrrrr7)r5rrzThe `z` should be specified for z layers, but it is for .)r3r5rrzrrrlast_hidden_staterzr attentions)#rcrrruse_return_dictrrrrTrnrorrrr rr9r ronesrr:r;rrrZziprlenr enumeraterandrrrr)r.rr3rrzrrrrrr5rr\r seq_lengthr pos_embedsrall_hidden_statesall_self_attns attn_mask mask_nameidx decoder_layerdropout_probability layer_outputss r0r<OPTDecoder.forwardsN2C1N-TXT_T_TqTq$8$D $++JjJj "+!6IDKK?I   --i8M 0*$++>O?N?Z?99;`a  !"\\ ]5H5H5K"KTaThThN  !),?,?,BBJ"ZZ (;(;A(> SgSghN.. >L]   <<A>L(9A=CCEL'+;+<(<=L)).Ye)f ?? & OOM:M% m6J6J(KK #7BD0d%( k]$C Iy$>>#A&3t{{+;<$ {*DSEUDVW%NN,Q/03%D#,DKK"8 C#!m%55!}}&+jjn#&7) *)3<3H3d /"3#-  M*!,M  =#3"553#96  , 11-@M    ' ,,];M -!1 1 &+++%   r2) rMrrrrrrrrrrr)F NNNNNNNNNNN)r>r?r@rArBr r-rr9rr rr staticmethodrCrQrrrrDrrrrrr<rErFrGs@r0rrIs#y#X#( BellK78BllB B  B  BH4 444{{ 4  4  44l1515,0+/59$(,0/3&*3715t E,,-t !.t ELL) t "% t   1 12 t D>t $D>t 'tnt d^t u//0t !.t -.t  u-- .t t r2rcr^\rSrSrS\4U4SjjrSrSr\\ SS\ \ RS\ \ RS\ \ RS \ \S \ \ RS \ \S \ \S \ \S\ \S\ \ RS\ \ RS\\S\\\44Sjj55rSrU=r$)OPTModelirccd>[TU]U5 [U5UlUR 5 gr)r,r-rdecoderrr.rcr/s r0r-OPTModel.__init__s&  !&)  r2c.URR$rr0rr.s r0get_input_embeddingsOPTModel.get_input_embeddingss||(((r2c$XRlgrr4r.rKs r0set_input_embeddingsOPTModel.set_input_embeddingss $) !r2rr3rrzrrrrrr5rr\rc  ~UbUOURRnUbUOURRnUbUOURRnU bU OURRn UR "SUUU UUUUUUSU S. U D6n [ U RU RU RU RS9$)NT rr3r5rrzrrrrrrrr) rcrrrrr0rrrzrr)r.rr3rrzrrrrrr5rr\decoder_outputss r0r<OPTModel.forwards"2C1N-TXT_T_TqTq$8$D $++JjJj "+!6IDKKr?r@rAr r-r6r:rrrr9rDrr rrrrrrrr<rErFrGs@r0r.r.s9y )*1515,0+/59$(,0/3&*3715+ E,,-+ !.+ ELL) + "% +   1 12 + D>+ $D>+ 'tn+ d^+ u//0+ !.+ -.+  u-- .+ + r2r.c!^\rSrSrS/rU4SjrSrSrSrSr \ \ SS\ \ RS \ \ RS \ \ RS \ \S \ \ R"S \ \ RS\ \S\ \S\ \S\ \S\ \ RS\ \ RS\\S\\\44Sjj55rSrU=r$)OPTForCausalLMizlm_head.weightc>[TU]U5 [U5Ul[R "UR URSS9UlUR5 gNFrf) r,r-r.rrrsrrlm_headrr1s r0r-OPTForCausalLM.__init__sK  f% yy!;!;V=N=NUZ[  r2cBURRR$rrr0rr5s r0r6#OPTForCausalLM.get_input_embeddingszz!!...r2c8XRRlgrrGr9s r0r:#OPTForCausalLM.set_input_embeddings*/ 'r2c$XRlgrrr0)r.r0s r0 set_decoderOPTForCausalLM.set_decoders $ r2c.URR$rrNr5s r0 get_decoderOPTForCausalLM.get_decoderszz!!!r2rr3rrzrlabelsrrrrr5rr\rc  "UbUOURRnU bU OURRn U bU OURRn URR "SUUU UUUUUU SU S. U D6nUR US5R5nSnUbEURUR5nUR"UU4SURR0U D6n[UUURURURS9$)a labels (`torch.LongTensor` of shape `(batch_size, sequence_length)`, *optional*): Labels for computing the masked language modeling loss. Indices should either be in `[0, ..., config.vocab_size]` or -100 (see `input_ids` docstring). Tokens with indices set to `-100` are ignored (masked), the loss is only computed for the tokens with labels in `[0, ..., config.vocab_size]`. Example: ```python >>> from transformers import AutoTokenizer, OPTForCausalLM >>> model = OPTForCausalLM.from_pretrained("facebook/opt-350m") >>> tokenizer = AutoTokenizer.from_pretrained("facebook/opt-350m") >>> prompt = "Hey, are you conscious? Can you talk to me?" >>> inputs = tokenizer(prompt, return_tensors="pt") >>> # Generate >>> generate_ids = model.generate(inputs.input_ids, max_length=30) >>> tokenizer.batch_decode(generate_ids, skip_special_tokens=True, clean_up_tokenization_spaces=False)[0] "Hey, are you conscious? Can you talk to me?\nI'm not conscious. I'm just a little bit of a weirdo." ```NTr=rrlosslogitsrzrrr)rcrrrrr0rDr[rZr loss_functionrrrzrr)r.rr3rrzrrTrrrrr5rr\rrXrWs r0r<OPTForCausalLM.forwards7R2C1N-TXT_T_TqTq$8$D $++JjJj &1%r?r@rA_tied_weights_keysr-r6r:rOrRrrrr9rDrr rrrrrrrr<rErFrGs@r0rArAsm*+/0%"1515,0+/59-1$(,0/3&*3715P E,,-P !.P ELL) P "% P   1 12 P ))*P D>P $D>P 'tnP d^P u//0P !.P +,P  u,, -P P r2rAa The OPT Model transformer with a sequence classification head on top (linear layer). [`OPTForSequenceClassification`] uses the last token in order to do the classification, as other causal models (e.g. GPT-2) do. Since it does classification on the last token, it requires to know the position of the last token. If a `pad_token_id` is defined in the configuration, it finds the last token that is not a padding token in each row. If no `pad_token_id` is defined, it simply takes the last value in each row of the batch. Since it cannot guess the padding tokens when `inputs_embeds` are passed instead of `input_ids`, it does the same (take the last value in each row of the batch). ) custom_introc^^\rSrSrS\4U4Sjjr\SS\\RS\\RS\\RS\\ S\\RS \\RS \\ S \\ S \\ S \\ S\\RS\ \\44Sjj5rSrSrSrU=r$)OPTForSequenceClassificationiWrcc>[TU]U5 URUl[U5Ul[ R "URURSS9UlUR5 grC) r,r- num_labelsr.rrrsrscorerr1s r0r-%OPTForSequenceClassification.__init__fsT   ++f% YYv994??QVW  r2rr3rrzrrTrrrrr5rc U bU OURRn URUUUU UUUUU U S9 n U Sn URU 5nUbURSSunnOURSSunnURR cUS:wa [ S5eURR cSnOUbXRR :gRUR[R5n[R"URSUR[RS9nUU-RS5nO.Sn[RURR S 35 U[R"XRS 9U4nSnUGbURR"cUR$S:XaS URlOoUR$S:aNUR&[R(:XdUR&[R*:XaS URlOS URlURR"S :XaJ[-5nUR$S:Xa&U"UR/5UR/55nOU"UU5nOURR"S :Xa=[15nU"UR3SUR$5UR3S55nO-URR"S :Xa[55nU"UU5nU (dU4U SS-nUbU4U-$U$[7UUU R8U R:U R<S9$)ae labels (`torch.LongTensor` of shape `(batch_size,)`, *optional*): Labels for computing the sequence classification/regression loss. Indices should be in `[0, ..., config.num_labels - 1]`. If `config.num_labels == 1` a regression loss is computed (Mean-Square loss), If `config.num_labels > 1` a classification loss is computed (Cross-Entropy). N rzr3r5rrrrrrrr*rz=Cannot handle batch sizes > 1 if no padding token is defined.rO)rrQz will not detect padding tokens in `inputs_embeds`. Results may be unexpected if using padding tokens in conjunction with `inputs_embeds.`r regressionsingle_label_classificationmulti_label_classificationrV)rcrrrbrrrrrZrr9int32r argmaxrnror/r> problem_typerarQr;rCr squeezerrrrrzrr)r.rr3rrzrrTrrrrr5transformer_outputsrrXrrlast_non_pad_token non_pad_mask token_indices pooled_logitsrWloss_fctoutputs r0r<$OPTForSequenceClassification.forwardos*&1%!F!Fr!J !#    >>**+,ZZ  u||J}}MOaab   {{''/??a'/;DKK,__q(fllejj.HFLL\a\e\eLe/LDKK,/KDKK,{{''<7"9??a'#M$9$9$;V^^=MND#M6:D))-JJ+- 2 22t GUWY))-II,. v6#%(;AB(??F)-)9TGf$ Ev E/ /??-;;*55   r2cBURRR$rrGr5s r0r61OPTForSequenceClassification.get_input_embeddingsrIr2c8XRRlgrrGr9s r0r:1OPTForSequenceClassification.set_input_embeddingsrLr2)rrarbr+)r>r?r@rAr r-rrr9rDrr rrrrr<r6r:rErFrGs@r0r_r_Ws;y156:15+/59-1$(,0/3&*37\ E,,-\ !!2!23\ E--. \ "% \   1 12 \ ))*\ D>\ $D>\ 'tn\ d^\ u//0\  u66 7\ \ |/00r2r_c~^\rSrSrS\4U4Sjjr\SS\\RS\\RS\\RS\\ S\\RS \\RS \\RS \\ S \\ S \\ S\\ S\\RS\ \\44Sjj5rSrSrSrU=r$)OPTForQuestionAnsweringircc>[TU]U5 [U5Ul[R "UR S5UlUR5 gr)) r,r-r.rrrsr qa_outputsrr1s r0r- OPTForQuestionAnswering.__init__s@  f% ))F$>$>B r2rr3rrzrstart_positions end_positionsrrrrr5rc U bU OURRn URUUUU UUUU U U S9 n U SnURU5nUR SSS9unnUR S5R 5nUR S5R 5nSnUbUb[UR55S:aUR S5n[UR55S:aUR S5nURS5nURSU5RUR5nURSU5RUR5n[US9nU"UU5nU"UU5nUU-S- nU (dUU4U SS-nUbU4U-$U$[UUUU RU RS 9$) a Example: ```python >>> from transformers import AutoTokenizer, OPTForQuestionAnswering >>> import torch >>> torch.manual_seed(4) # doctest: +IGNORE_RESULT >>> tokenizer = AutoTokenizer.from_pretrained("facebook/opt-350m") >>> # note: we are loading a OPTForQuestionAnswering from the hub here, >>> # so the head will be randomly initialized, hence the predictions will be random >>> model = OPTForQuestionAnswering.from_pretrained("facebook/opt-350m") >>> question, text = "Who was Jim Henson?", "Jim Henson was a nice puppet" >>> inputs = tokenizer(question, text, return_tensors="pt") >>> with torch.no_grad(): ... outputs = model(**inputs) >>> answer_start_index = outputs.start_logits.argmax() >>> answer_end_index = outputs.end_logits.argmax() >>> answer_offset = len(tokenizer(question)[0]) >>> predict_answer_tokens = inputs.input_ids[ ... 0, answer_offset + answer_start_index : answer_offset + answer_end_index + 1 ... ] >>> predicted = tokenizer.decode(predict_answer_tokens) >>> predicted ' a nice puppet' ```NrerrrOr7) ignore_indexr*)rW start_logits end_logitsrr)rcrrr|splitrlr[rrclamprZrrrrr)r.rr3rrzrr~rrrrrr5rmrrXrr total_loss ignored_indexrr start_lossend_lossrss r0r<OPTForQuestionAnswering.forwards`&1%r?r@rAr r-rrr9rDrr rrrrr<r6r:rErFrGs@r0rzrzsTy156:15+/596:48$(,0/3&*37_ E,,-_ !!2!23_ E--. _ "% _   1 12 _ "%"2"23_   0 01_ D>_ $D>_ 'tn_ d^_ u//0_  u22 3_ _ B/00r2rz)rAr.rr_rz)r)BrBtypingrrrr9rtorch.nnrrr activationsr cache_utilsr r generationrmodeling_attn_mask_utilsrmodeling_flash_attention_utilsrmodeling_layersrmodeling_outputsrrrrmodeling_utilsrrprocessing_utilsrutilsrrrrrutils.deprecationrconfiguration_optr !torch.nn.attention.flex_attentionr!integrations.flex_attentionr" get_loggerr>rnrr$Modulerfloatr_rarrrr.rAr_rz__all__rr2r0rs,, AA!.)>B9 G&pp0( !!;J   H %;BLL;H% II% <<% % << % U\\* %  %%.])299])@c0cL%%%6_ #_ D : !: : zk 'k \  m0#5m0 m0`o00o0o0d r2