cCi S SK JrJr S SKrS SKJr S SKJr SSKJ r J
r
SSK J r SSK
JrJrJr SS KJrJrJrJr " S
S \5 r\ " S S
\5 5 r " S S\\5 r\" SS9 " S S\\5 5 r/ SQrg) )OptionalUnionN) IJepaConfig )BaseModelOutputWithPoolingImageClassifierOutput)Unpack)TransformersKwargsauto_docstring torch_int )
ViTEmbeddingsViTForImageClassificationViTModelViTPreTrainedModelc ^ \ rS rSrSS\S\SS4U 4S jjjrS\R S\ S \ S\R 4S
jr
SS \R S \ \R S
\S\R 4S jjr
SrU =r$ )IJepaEmbeddings configuse_mask_tokenreturnNc > [ TU ] X5 U ?U R R n[
R " [ R " SX1R 5 5 U l
g )N ) super__init__ cls_tokenpatch_embeddings num_patchesnn Parametertorchrandn hidden_sizeposition_embeddings)selfr r r __class__s a/home/james-whalen/.local/lib/python3.13/site-packages/transformers/models/ijepa/modular_ijepa.pyr IJepaEmbeddings.__init__ sH
0 N++77 #%<< A{L^L^0_#`
embeddingsheightwidthc , UR S nU R R S n[ R R 5 ( d XE:X a X#:X a U R $ U R nUR S nX R
- nX0R
- n [
US- 5 n
UR SXU5 nUR SSSS5 n[ R R UX4SSS 9nUR SSSS5 R SSU5 nU$ )
a
This method allows to interpolate the pre-trained position encodings, to be able to use the model on higher resolution
images. This method is also adapted to support torch.jit tracing.
Adapted from:
- https://github.com/facebookresearch/dino/blob/de9ee3df6cf39fac952ab558447af1fa1365362a/vision_transformer.py#L174-L194, and
- https://github.com/facebookresearch/dinov2/blob/e1277af2ba9496fbadf7aec6eba56e8d882d1e35/dinov2/models/vision_transformer.py#L179-L211
r g ?r r r
bicubicF)sizemode
align_corners)
shaper$ r! jit
is_tracing
patch_sizer reshapepermuter
functional interpolateview) r% r* r+ r, r
num_positionspatch_pos_embeddim
new_height new_widthsqrt_num_positionss r' interpolate_pos_encoding(IJepaEmbeddings.interpolate_pos_encoding s !&&q) 0066q9
yy##%%+*F6?++ + 22 r".
__, &}c'9 :)11!5G]`a)11!Q1= --33 ( 4
*11!Q1=BB1b#N r) pixel_valuesbool_masked_posrB c n UR u pEpgU R XS9nUbX UR S n U R R XIS5 n
UR S5 R U
5 n USU -
- X- - nU( a XR
XU5 - nOXR - nU R U5 nU$ )N)rB r r. ?) r3 r
mask_tokenexpand unsqueezetype_asrB r$ dropout) r% rD rE rB
batch_size_r+ r, r*
seq_length mask_tokensmasks r' forwardIJepaEmbeddings.forward= s (4'9'9$
v**<*k
&#))!,J //00LK",,R088ED#sTz2[5GGJ $#&C&CJX]&^^J#&>&>>J\\*-
r) )r$ )F)NF)__name__
__module__ __qualname____firstlineno__r boolr r! TensorintrB r
BoolTensorrR __static_attributes__
__classcell__r& s @r' r r s a{ aD aT a a% 5<< % % UX % ]b]i]i % T 7;). ll "%"2"23 #'
r) r c n \ rS rSrS\\R \R \R 4 SS4S jr Sr
g)IJepaPreTrainedModelX moduler Nc [ U[ R [ R 45 ( a [ R R UR R R [ R 5 SU R R S9R UR R 5 UR l UR b% UR R R 5 gg[ U[ R 5 ( aJ UR R R 5 UR R R# S5 g[ U[$ 5 ( a [ R R UR& R R [ R 5 SU R R S9R UR& R 5 UR& l UR( b% UR( R R 5 ggg)zInitialize the weightsg )meanstdNrG )
isinstancer LinearConv2dinit
trunc_normal_weightdatator! float32r initializer_rangedtypebiaszero_ LayerNormfill_r r$ rH )r% rb s r'
_init_weights"IJepaPreTrainedModel._init_weightsZ s fryy")) 4 5 5 "$!6!6
""%%emm43DKKDaDa "7 "b $$%
MM {{& &&( '
-
- KK " " $ MM $ $S )
0
0.0gg.C.C**//225==AKK11 /D / b++112
& & +
,!!&&,,. -
1r) ) rT rU rV rW r r rg rh rs ru r\ rw r) r' r` r` X s. /E"))RYY *L$M /RV /r) r` c < ^ \ rS rSrSS\S\S\4U 4S jjjrSrU =r$ )
IJepaModelq r add_pooling_layerr c L > [ TU ] U5 Xl [ XS9U l g)z
add_pooling_layer (bool, *optional*, defaults to `True`):
Whether to add a pooling layer
use_mask_token (`bool`, *optional*, defaults to `False`):
Whether to use a mask token for masked image modeling.
)r N)r r r r r* )r% r r{ r r& s r' r IJepaModel.__init__r s# )&Pr) )r r* )FF) rT rU rV rW r rX r r\ r] r^ s @r' ry ry q s( Q{ Qt Q]a Q Qr) ry a
IJepa Model transformer with an image classification head on top (a linear layer on top of the final hidden states)
e.g. for ImageNet.
Note that it's possible to fine-tune IJepa on higher resolution images than the ones it has been trained on, by
setting `interpolate_pos_encoding` to `True` in the forward of the model. This will interpolate the pre-trained
position embeddings to the higher resolution.
) custom_introc
^ \ rS rSrS\4U 4S jjr S S\\R S\\R S\\R S\\ S\
\ S \ 4 S
jjr
S rU =r$ )
IJepaForImageClassification~ r c b > [ TU ] U5 [ USS9U l U R 5 g )NF)r{ )r r ry ijepa post_init)r% r r& s r' r $IJepaForImageClassification.__init__ s(
%@
r) rD head_masklabelsrB kwargsr c U R " U4UUS.UD6nUR nU R UR SS95 nSn Ub U R " X8U R
40 UD6n [
U UUR UR S9$ )ab
labels (`torch.LongTensor` of shape `(batch_size,)`, *optional*):
Labels for computing the image classification/regression loss. Indices should be in `[0, ...,
config.num_labels - 1]`. If `config.num_labels == 1` a regression loss is computed (Mean-Square loss), If
`config.num_labels > 1` a classification loss is computed (Cross-Entropy).
)r rB r )r> N)losslogits
hidden_states
attentions) r last_hidden_state
classifierrd
loss_functionr r r r )
r% rD r r rB r outputssequence_outputr r s
r' rR #IJepaForImageClassification.forward s /3jj /
%=/
/
"33 !5!5!!5!<= %%fdkkLVLD$!// ))
r) )r )NNNN)rT rU rV rW r r r r! rY rX r r
r rR r\ r] r^ s @r' r r ~ s { 04,0)-37 !
u||,!
ELL)!
& !
#+4. !
+,
!
!
!
r) r )r` ry r )typingr r r! torch.nnr -transformers.models.ijepa.configuration_ijepar modeling_outputsr r processing_utilsr utilsr
r r vit.modeling_vitr r r r r r` ry r __all__rw r) r' r s " E Q & B B e eGm GT /- / /0
Q%x
Q
'
"68Q '
'
T r)