cCi@lSrSSKrSSKJr SSKJrJrJrJr SSK r SSK J r SSK J r SSK JrJrJr SS KJr SS KJr SS KJr SS KJrJrJrJr SS KJrJr SSKJ r SSK!J"r"J#r#J$r$J%r%J&r&J'r' SSK(J)r) SSK*J+r+J,r,J-r- \&R\"\/5r0SSS\ RbS\ RdS\\34Sjjr4STS\ RjS\ RdS\ RlS\34Sjjr7STSjr8\\$"SS9"SS\"555r9\\$"S S9"S!S"\"555r:"S#S$\ Rv5r<SUS%\ RvS&\ RbS'\ RbS(\ RbS)\\ RbS*\=S+\=4S,jjr>"S-S.\ Rv5r?"S/S0\ Rv5r@"S1S2\5rA"S3S4\ Rv5rB"S5S6\ Rv5rC"S7S8\ Rv5rD"S9S:\ Rv5rE"S;S<\ Rv5rF"S=S>\5rG"S?S@\ Rv5rH\$"SASB\55rI"SCSD\I5rJ"SESF\I5rK\$"SGS9"SHSI\I\55rL"SJSK\ Rv5rM\$"SLS9"SMSN\I55rN\$"SOS9"SPSQ\I\55rO/SRQrPg)VzPyTorch KOSMOS-2 model.N) dataclass)AnyCallableOptionalUnion)nn)ACT2FN)Cache DynamicCacheEncoderDecoderCache)GenerationMixin)FlashAttentionKwargs)GradientCheckpointingLayer)BaseModelOutput)BaseModelOutputWithPastAndCrossAttentionsBaseModelOutputWithPooling!CausalLMOutputWithCrossAttentions)ALL_ATTENTION_FUNCTIONSPreTrainedModel)Unpack) ModelOutputTransformersKwargsauto_docstringcan_return_tuplelogging torch_int)deprecate_kwarg) Kosmos2ConfigKosmos2TextConfigKosmos2VisionConfigmaskdtypetgt_lenc2UR5up4UbUOUnUSS2SSSS24RUSX$5RU5nSU- nURUR[R 5[R "U5R5$)zW Expands attention_mask from `[bsz, seq_len]` to `[bsz, 1, tgt_seq_len, src_seq_len]`. Nr?)sizeexpandto masked_filltorchboolfinfomin)r#r$r%bszsrc_len expanded_mask inverted_masks f/home/james-whalen/.local/lib/python3.13/site-packages/transformers/models/kosmos2/modeling_kosmos2.py _expand_maskr5-s99;LC ,g'GD$)*11#q'KNNuUM-'M  $ $]%5%5ejj%A5;;uCUCYCY ZZinput_ids_shapedevicepast_key_values_lengthc UupE[R"XU4[R"U5RUS9n[R"UR S5US9nUR XwS-RUR S5S5:S5 URU5nUS:a*[R"[R"XSXS9U/SS9nUSSSS2SS24RUSXUU-5$)z: Make causal mask used for bi-directional self-attention. )r8rrr$r8dimN) r,fullr.r/aranger( masked_fill_viewr*catzerosr))r7r$r8r9r0r%r# mask_conds r4_make_causal_maskrF;s #LC ::w(%++e*<*@*@ PD TYYr]6:Iiq="6"6tyy}a"HH!L 775>D!yy%++gUbdhioqr dAq ! ( (aDZ:Z [[r6cURU5R5n[R"USS9R U5U-U-nUR 5U-$)z Replace non-padding symbols with their position numbers. Position numbers begin at padding_idx+1. Padding symbols are ignored. This is modified from fairseq's `utils.make_positions`. Args: x: torch.Tensor x: Returns: torch.Tensor rr=)neintr,cumsumtype_aslong) input_ids padding_idxr9r#incremental_indicess r4"create_position_ids_from_input_idsrPMsW << $ ( ( *D <<!4<