+h_SSKrSSKJrJrJrJr SSKrSSKJr SSK J r J r SSK J r SSKJr \ R "\5r"SS \5r"S S \\ 5r"S S \R*5r"SS\R*5r"SS\R*5r"SS\R*5r"SS\R*5r"SS\R*5r"SS\R*5rg)N)CallableListOptionalUnion) ConfigMixinregister_to_config)logging) ModelMixinc ^\rSrSrSrS\S4U4SjjrSS\RS\ \\ S\\R4S jjr SS \ \ \R4S \S \S \S\ \ 4 Sjjr\S\ \ \ \R44Sj5rSrU=r$) MultiAdaptera} MultiAdapter is a wrapper model that contains multiple adapter models and merges their outputs according to user-assigned weighting. This model inherits from [`ModelMixin`]. Check the superclass documentation for common methods such as downloading or saving. Args: adapters (`List[T2IAdapter]`, *optional*, defaults to None): A list of `T2IAdapter` model instances. adapters T2IAdapterc&>[[U] 5 [U5Ul[ R "U5Ul[U5S:Xa [S5e[U5S:Xa [S5eUSRnUSRn[S[U55H\nXRU:wdXRU:wdM)[SUSUSUSXRSUS XR3 5e X l X0l g) NrzExpecting at least one adapterr zQFor a single adapter, please use the `T2IAdapter` class instead of `MultiAdapter`zjExpecting all adapters to have the same downscaling behavior, but got: adapters[0].total_downscale_factor=z adapters[0].downscale_factor=z adapter[`z`].total_downscale_factor=z`].downscale_factor=) superr__init__len num_adapternn ModuleListr ValueErrortotal_downscale_factordownscale_factorrange)selfr$first_adapter_total_downscale_factorfirst_adapter_downscale_factoridx __class__s R/home/james-whalen/.local/lib/python3.13/site-packages/diffusers/models/adapter.pyrMultiAdapter.__init__)s$ lD*,x= h/ x=A => > x=A pq q08{/Q/Q,)1!)E)E&CM*C 448\\=115SS ::^9_`44R3ST #u$>x}?c?c>de #u$89W9W8X Z +'K# >xsadapter_weightsreturncUc2[R"SUR- /UR-5nO[R"U5nSn[XUR5HeupEnU"U5nUc(Un[ [ U55H nXSU-X8'M M9[ [ U55HnX8==XWU-- ss'M Mg U$)a2 Args: xs (`torch.Tensor`): A tensor of shape (batch, channel, height, width) representing input images for multiple adapter models, concatenated along dimension 1(channel dimension). The `channel` dimension should be equal to `num_adapter` * number of channel per image. adapter_weights (`List[float]`, *optional*, defaults to None): A list of floats representing the weights which will be multiplied by each adapter's output before summing them together. If `None`, equal weights will be used for all adapters. Nr )torchtensorrziprrr) rr%r& accume_statexwadapterfeaturesis r"forwardMultiAdapter.forwardLs  "#llA0@0@,@+ADDTDT+TUO#ll?;O  dmmDMA'qzH#' s<01A&'q/&9LO2s8}-A OqA;6O.Er$save_directoryis_main_process save_functionsafe_serializationvariantc rSnUnURH#nURUUUUUS9 US- nUSU3-nM% g)a Save a model and its configuration file to a specified directory, allowing it to be re-loaded with the `[`~models.adapter.MultiAdapter.from_pretrained`]` class method. Args: save_directory (`str` or `os.PathLike`): The directory where the model will be saved. If the directory does not exist, it will be created. is_main_process (`bool`, optional, defaults=True): Indicates whether current process is the main process or not. Useful for distributed training (e.g., TPUs) and need to call this function on all processes. In this case, set `is_main_process=True` only for the main process to avoid race conditions. save_function (`Callable`): Function used to save the state dictionary. Useful for distributed training (e.g., TPUs) to replace `torch.save` with another method. Can also be configured using`DIFFUSERS_SAVE_MODE` environment variable. safe_serialization (`bool`, optional, defaults=True): If `True`, save the model using `safetensors`. If `False`, save the model with `pickle`. variant (`str`, *optional*): If specified, weights are saved in the format `pytorch_model..bin`. r)r5r6r7r8r _N)rsave_pretrained) rr4r5r6r7r8r model_path_to_saver/s r"r;MultiAdapter.save_pretrainedis[8+}}G  # #" /+#5 $  1HC!3#i!? %r$pretrained_model_pathc Sn/nUn[RRU5(a[[R"U40UD6nUR U5 US- nUSU3-n[RRU5(aM[[ R[U5SUS35 [U5S:Xa2[S[RRU5SUS-S35eU"U5$) al Instantiate a pretrained `MultiAdapter` model from multiple pre-trained adapter models. The model is set in evaluation mode by default using `model.eval()` (Dropout modules are deactivated). To train the model, set it back to training mode using `model.train()`. Warnings: *Weights from XXX not initialized from pretrained model* means that the weights of XXX are not pretrained with the rest of the model. It is up to you to train those weights with a downstream fine-tuning. *Weights from XXX not used in YYY* means that the layer XXX is not used by YYY, so those weights are discarded. Args: pretrained_model_path (`os.PathLike`): A path to a *directory* containing model weights saved using [`~diffusers.models.adapter.MultiAdapter.save_pretrained`], e.g., `./my_model_directory/adapter`. torch_dtype (`torch.dtype`, *optional*): Override the default `torch.dtype` and load the model under this dtype. output_loading_info(`bool`, *optional*, defaults to `False`): Whether or not to also return a dictionary containing missing keys, unexpected keys and error messages. device_map (`str` or `Dict[str, Union[int, str, torch.device]]`, *optional*): A map that specifies where each submodule should go. It doesn't need to be refined to each parameter/buffer name, once a given module name is inside, every submodule of it will be sent to the same device. To have Accelerate compute the most optimized `device_map` automatically, set `device_map="auto"`. For more information about each option see [designing a device map](https://hf.co/docs/accelerate/main/en/usage_guides/big_modeling#designing-a-device-map). max_memory (`Dict`, *optional*): A dictionary mapping device identifiers to their maximum memory. Default to the maximum memory available for each GPU and the available CPU RAM if unset. low_cpu_mem_usage (`bool`, *optional*, defaults to `True` if torch version >= 1.9.0 else `False`): Speed up model loading by not initializing the weights and only loading the pre-trained weights. This also tries to not use more than 1x model size in CPU memory (including peak memory) while loading the model. This is only supported when torch version >= 1.9.0. If you are using an older version of torch, setting this argument to `True` will raise an error. variant (`str`, *optional*): If specified, load weights from a `variant` file (*e.g.* pytorch_model..bin). `variant` will be ignored when using `from_flax`. use_safetensors (`bool`, *optional*, defaults to `None`): If `None`, the `safetensors` weights will be downloaded if available **and** if`safetensors` library is installed. If `True`, the model will be forcibly loaded from`safetensors` weights. If `False`, `safetensors` is not used. rr r:z adapters loaded from .zNo T2IAdapters found under z. Expected at least _0) ospathisdirrfrom_pretrainedappendloggerinforrdirname)clsr>kwargsr rmodel_path_to_loadr/s r"rEMultiAdapter.from_pretrainedsZ 3ggmm.// 001CNvNG OOG $ 1HC!61SE!B  ggmm.//  s8}o%;S.T-UUijCGkGjHHIJ 8}r$)rrrrN)TNTN)__name__ __module__ __qualname____firstlineno____doc__rrr)Tensorrfloatr2rstrrBPathLikeboolrr; classmethodrE__static_attributes__ __classcell__r!s@r"rrs !?l!3!?F%,,$u+9NZ^_d_k_kZl@!%"&#'!% (@c2;;./(@(@ (@ ! (@ # (@TAHU3 CS=T4UAAr$rc ^\rSrSrSr\S/SQSSS4S\S \\S \S \S \4 U4S jjj5r S\ RS\\ R4Sjr \ S5r\ S5rSrU=r$)ra A simple ResNet-like model that accepts images containing control signals such as keyposes and depth. The model generates multiple feature maps that are used as additional conditioning in [`UNet2DConditionModel`]. The model's architecture follows the original implementation of [Adapter](https://github.com/TencentARC/T2I-Adapter/blob/686de4681515662c0ac2ffa07bf5dda83af1038a/ldm/modules/encoders/adapter.py#L97) and [AdapterLight](https://github.com/TencentARC/T2I-Adapter/blob/686de4681515662c0ac2ffa07bf5dda83af1038a/ldm/modules/encoders/adapter.py#L235). This model inherits from [`ModelMixin`]. Check the superclass documentation for the common methods, such as downloading or saving. Args: in_channels (`int`, *optional*, defaults to `3`): The number of channels in the adapter's input (*control image*). Set it to 1 if you're using a gray scale image. channels (`List[int]`, *optional*, defaults to `(320, 640, 1280, 1280)`): The number of channels in each downsample block's output hidden state. The `len(block_out_channels)` determines the number of downsample blocks in the adapter. num_res_blocks (`int`, *optional*, defaults to `2`): Number of ResNet blocks in each downsample block. downscale_factor (`int`, *optional*, defaults to `8`): A factor that determines the total downscale factor of the Adapter. adapter_type (`str`, *optional*, defaults to `full_adapter`): Adapter type (`full_adapter` or `full_adapter_xl` or `light_adapter`) to use. @rcr full_adapter in_channelschannelsnum_res_blocksr adapter_typec>[TU]5 US:Xa[XX45UlgUS:Xa[ XX45UlgUS:Xa[ XX45Ulg[ SUS35e)Nrefull_adapter_xl light_adapterzUnsupported adapter_type: 'zH'. Choose either 'full_adapter' or 'full_adapter_xl' or 'light_adapter'.)rr FullAdapterr/ FullAdapterXL LightAdapterr)rrfrgrhrrir!s r"rT2IAdapter.__init__sq  > )&{n_DL . .(aDL _ ,' ~`DL-l^<88 r$r-r'c$URU5$)aj This function processes the input tensor `x` through the adapter model and returns a list of feature tensors, each representing information extracted at a different scale from the input. The length of the list is determined by the number of downsample blocks in the Adapter, as specified by the `channels` and `num_res_blocks` parameters during initialization. r/rr-s r"r2T2IAdapter.forward s||Ar$c.URR$rN)r/rrs r"r!T2IAdapter.total_downscale_factors||222r$cBURRR$)zThe downscale factor applied in the T2I-Adapter's initial pixel unshuffle operation. If an input image's dimensions are not evenly divisible by the downscale_factor then an exception will be raised. )r/ unshufflerrvs r"rT2IAdapter.downscale_factors ||%%666r$rr)rOrPrQrRrSr intrrVrr)rTr2propertyrrrZr[r\s@r"rrs44 !* s)     ,$u||*<3377r$rc ^\rSrSrSrS/SQSS4S\S\\S \S \4U4S jjjrS \RS \\R4Sjr Sr U=r $)rmi"* See [`T2IAdapter`] for more information. r_r`rrdrfrgrhrc >[TU]5 XS--n[R"U5Ul[R "XSSSS9Ul[R"[USUSU5/[S[U55Vs/sHn[X%S- X%USS9PM snQ5Ul US[U5S- --Ul gs snfNrrr_r  kernel_sizepaddingTdown) rrrPixelUnshuffleryConv2dconv_inr AdapterBlockrrbodyrrrfrgrhrr1r!s r"rFullAdapter.__init__'s !a$77 **+;<yyqkqRST MMXa[(1+~F #1c(m44!a%(+~TXY4   '7s8}q?P9Q&Q#s C r-r'cURU5nURU5n/nURHnU"U5nURU5 M U$)a This method processes the input tensor `x` through the FullAdapter model and performs operations including pixel unshuffling, convolution, and a stack of AdapterBlocks. It returns a list of feature tensors, each capturing information at a different stage of processing within the FullAdapter model. The number of feature tensors in the list is determined by the number of downsample blocks specified during initialization. ryrrrFrr-r0blocks r"r2FullAdapter.forwardAsN NN1  LLOYYEaA OOA r$rrrry rOrPrQrRrSr{rrr)rTr2rZr[r\s@r"rmrm"su 4 ! RRs)R R  RR4$u||*<r$rmc ^\rSrSrSrS/SQSS4S\S\\S \S \4U4S jjjrS \RS \\R4Sjr Sr U=r $)rniTr~r_r`rrfrgrhrc j>[TU]5 XS--n[R"U5Ul[R "XSSSS9Ul/Ul[[U55HnUS:Xa/URR[X%S- X%U55 M8US:Xa.URR[X%S- X%USS95 MlURR[X%X%U55 M [R"UR5UlUS-Ul gr)rrrrryrrrrrrFrrrrs r"rFullAdapterXL.__init__Ys !a$77 **+;<yyqkqRST  s8}%AAv   h1uox{N![\a   h1uox{Nae!fg   hk8;!WX &MM$)), &6&:#r$r-r'cURU5nURU5n/nURHnU"U5nURU5 M U$)z This method takes the tensor x as input and processes it through FullAdapterXL model. It consists of operations including unshuffling pixels, applying convolution layer and appending each block into list of feature tensors. rrs r"r2FullAdapterXL.forwardusN NN1  LLOYYEaA OOA r$rrr\s@r"rnrnTso 4 " ;;s); ;  ;;8$u||*<r$rnc ~^\rSrSrSrS S\S\S\S\4U4SjjjrS\RS \R4S jr S r U=r $) ria An AdapterBlock is a helper model that contains multiple ResNet-like blocks. It is used in the `FullAdapter` and `FullAdapterXL` models. Args: in_channels (`int`): Number of channels of AdapterBlock's input. out_channels (`int`): Number of channels of AdapterBlock's output. num_res_blocks (`int`): Number of ResNet blocks in the AdapterBlock. down (`bool`, *optional*, defaults to `False`): If `True`, perform downsampling on AdapterBlock's input. rf out_channelsrhrc@>[TU]5 SUlU(a[R"SSSS9UlSUlX:wa[R "XSS9Ul[R"[U5Vs/sHn[U5PM sn6Ul gs snf)NrTrstride ceil_moder r) rr downsampler AvgPool2din_convr SequentialrAdapterResnetBlockresnets)rrfrrhrr:r!s r"rAdapterBlock.__init__s}   llqdSDO  &99[ANDL}}8=n8M N8M1 .8M N  Ns<Br-r'cURbURU5nURbURU5nURU5nU$)z This method takes tensor x as input and performs operations downsampling and convolutional layers if the self.downsample and self.in_conv properties of AdapterBlock model are specified. Then it applies a series of residual blocks to the input tensor. rrrrss r"r2AdapterBlock.forwardsE ?? &"A << # QA LLOr$rF rOrPrQrRrSr{rXrr)rTr2rZr[r\s@r"rrsO   C  s  C  W[   %,,r$rcn^\rSrSrSrS\4U4SjjrS\RS\R4Sjr Sr U=r $) riz An `AdapterResnetBlock` is a helper model that implements a ResNet-like block. Args: channels (`int`): Number of channels of AdapterResnetBlock's input and output. rgc>[TU]5 [R"XSSS9Ul[R "5Ul[R"XSS9Ulg)Nr_r rrrrrrblock1ReLUactblock2rrgr!s r"rAdapterResnetBlock.__init__sA ii1M 779iiB r$r-r'clURURU55nURU5nX!-$)z This method takes input tensor x and applies a convolutional layer, ReLU activation, and another convolutional layer on the input tensor. It returns addition with the input tensor. rrrrr-hs r"r2AdapterResnetBlock.forward. HHT[[^ $ KKNu r$r rOrPrQrRrSr{rr)rTr2rZr[r\s@r"rrs6CC  %,,  r$rc ^\rSrSrSrS/SQSS4S\S\\S \S \4U4S jjjrS \RS \\R4Sjr Sr U=r $)roir~r_)rarbrcrdrfrgrhrc >[TU]5 XS--n[R"U5Ul[R "[ XSU5/[[U5S- 5Vs/sHn[ X%X%S-USS9PM snQ[ USUSUSS9P5Ul US[U5--Ul gs snf)Nrrr Tr) rrrrryrLightAdapterBlockrrrrrs r"rLightAdapter.__init__s !a$77 **+;<MM!+{NK #3x=1#455&hk8E?NY]^5  "(2, nSWX     '7!s8}:L&M#s*B= r-r'cURU5n/nURHnU"U5nURU5 M U$)z This method takes the input tensor x and performs downscaling and appends it in list of feature tensors. Each feature tensor corresponds to a different level of processing within the LightAdapter. )ryrrFrs r"r2LightAdapter.forwardsA NN1 YYEaA OOA r$)rrryrr\s@r"rorosu . ! NNs)N N  NN4  $u||*<  r$roc ~^\rSrSrSrS S\S\S\S\4U4SjjjrS\RS \R4S jr S r U=r $) ria  A `LightAdapterBlock` is a helper model that contains multiple `LightAdapterResnetBlocks`. It is used in the `LightAdapter` model. Args: in_channels (`int`): Number of channels of LightAdapterBlock's input. out_channels (`int`): Number of channels of LightAdapterBlock's output. num_res_blocks (`int`): Number of LightAdapterResnetBlocks in the LightAdapterBlock. down (`bool`, *optional*, defaults to `False`): If `True`, perform downsampling on LightAdapterBlock's input. rfrrhrcf>[TU]5 US-nSUlU(a[R"SSSS9Ul[R "XSS9Ul[R"[U5Vs/sHn[U5PM sn6Ul [R "XRSS9Ul gs snf)NrrTrr r) rrrrrrrrrLightAdapterResnetBlockrout_conv)rrfrrhr mid_channelsr:r!s r"rLightAdapterBlock.__init__s #q(   llqdSDOyyJ }}V[\jVk&lVkQR'>|'LVk&lm  ,!L 'ms5B.r-r'cURbURU5nURU5nURU5nURU5nU$)z This method takes tensor x as input and performs downsampling if required. Then it applies in convolution layer, a sequence of residual blocks, and out convolutional layer. )rrrrrss r"r2LightAdapterBlock.forwardsI ?? &"A LLO LLO MM! r$)rrrrrrr\s@r"rrsU  MC Ms MC MW[ M M  %,,  r$rcn^\rSrSrSrS\4U4SjjrS\RS\R4Sjr Sr U=r $) ri.z A `LightAdapterResnetBlock` is a helper model that implements a ResNet-like block with a slightly different architecture than `AdapterResnetBlock`. Args: channels (`int`): Number of channels of LightAdapterResnetBlock's input and output. rgc>[TU]5 [R"XSSS9Ul[R "5Ul[R"XSSS9Ulg)Nr_r rrrs r"r LightAdapterResnetBlock.__init__8sC ii1M 779ii1M r$r-r'clURURU55nURU5nX!-$)z This function takes input tensor x and processes it through one convolutional layer, ReLU activation, and another convolutional layer and adds it to input tensor. rrs r"r2LightAdapterResnetBlock.forward>rr$rrr\s@r"rr.s6NN  %,,  r$r)rBtypingrrrrr)torch.nnrconfiguration_utilsrr utilsr modeling_utilsr get_loggerrOrGrrModulermrnrrrorrr$r"rs 22 A&   H %y:yxD7[D7T/"))/d/BII/d-299-`<,299,^( (Vbiir$