+hp*VSSKJrJrJrJrJr SSKrSSKJr SSK J r "SS\ 5r g))DictListOptionalTupleUnionN)Image)VaeImageProcessorc~^\rSrSrSrSS.S\4U4SjjjrS\\\\ RS\S \ \\\ R\\\\\\44S jr S\\\ RS \\S \\\ R4S jrS\\\ RS \S\S \ \\\ R\\\\44SjrS\\\ RS \\\ R4SjrS\ \\4S \4SjrSS\\\\4S\\\\4S\\\\\\S \\S\\S\S\S \4SjjrSrU=r$)VisualClozeProcessora7 Image processor for the VisualCloze pipeline. This processor handles the preprocessing of images for visual cloze tasks, including resizing, normalization, and mask generation. Args: resolution (int, optional): Target resolution for processing images. Each image will be resized to this resolution before being concatenated to avoid the out-of-memory error. Defaults to 384. *args: Additional arguments passed to [~image_processor.VaeImageProcessor] **kwargs: Additional keyword arguments passed to [~image_processor.VaeImageProcessor] ) resolutionrc2>[TU]"U0UD6 Xlg)N)super__init__r)selfrargskwargs __class__s k/home/james-whalen/.local/lib/python3.13/site-packages/diffusers/pipelines/visualcloze/visualcloze_utils.pyrVisualClozeProcessor.__init__&s $)&)$ input_imagesvae_scale_factorreturnc[U5[US5pCSU-n[U5Vs/sHn/PM nn[U5Vs/sHnSPM nn/n [U5GHsn [U5Hn XU cM XU RXU R- n URUR-n [ X- S-5n[ X-5n[ X-S5U-n[ X-S5U-nX4X' O [U5Hn XU bRURXU XSXS5nXzRU5 XS- :XaU RS5 M[M][R"SX=(d URUR4S5nXzRU5 XS- :XdMU RS5 M GMv [U 5S:a[U 5S:aXS- S=(d Sn[[U55Hn [[Xz55Hzn XzU cM [ XzU RXU U R- -5n[ US - 5S -n[ US - 5S -nURXzU X5XzU 'M| M /n[[U55Hn URXzVs/sHnURUR/PM sn5 [Xz5H@un nURU5nURU5nURU5nUXzU 'MB M UUU 4$s snfs snfs snf) a Preprocesses input images for the VisualCloze pipeline. This function handles the preprocessing of input images by: 1. Resizing and cropping images to maintain consistent dimensions 2. Converting images to the Tensor format for the VAE 3. Normalizing pixel values 4. Tracking image sizes and positions of target images Args: input_images (List[List[Optional[Image.Image]]]): A nested list of PIL Images where: - Outer list represents different samples, including in-context examples and the query - Inner list contains images for the task - In the last row, condition images are provided and the target images are placed as None vae_scale_factor (int): The scale factor used by the VAE for resizing images Returns: Tuple containing: - List[List[torch.Tensor]]: Preprocessed images in tensor format - List[List[List[int]]]: Dimensions of each processed image [height, width] - List[int]: Target positions indicating which images are to be generated rNg?RGB)rrrr)lenrangewidthheightrintmax_resize_and_cropappendrnewsum enumerate pil_to_numpy numpy_to_pt normalize)rrr n_samples n_task_images divisible_processed_images resize_sizetarget_positionij aspect_ratio target_areanew_hnew_wtargetblank image_sizesimgimages rpreprocess_image%VisualClozeProcessor.preprocess_image*s6$'|#4c,q/6J=(( AFyAQ4RAQARAQ4RFKIFV7WFVFV 7W%'y!A=)?1%1#/?1#5#;#;loa>P>W>W#WL"&//DOO"CK!; CDE 45E 2A6BE 2A6BE&+^KN*=)?1%1!22 nnOIUVs/sH<n[R"SSUSRSUSRS4SS9PM> nnUR U5 M U$s snfs snf)a Generate masks for the VisualCloze pipeline. Args: input_images (List[List[Image.Image]]): Processed images from preprocess_image target_position (List[int]): Binary list marking the positions of target images (1 for target, 0 for condition) Returns: List[List[torch.Tensor]]: A nested list of mask tensors (1 for target positions, 0 for condition images) rrrr ) fill_value)r,r"torchfullshaper)) rrr6maskr7rowm row_masksr3s rpreprocess_mask$VisualClozeProcessor.preprocess_masks  -FAC %))`o`o[\EJJ1c!fll1os1v||AGTUV`o ap`o[\EJJ1c!fll1os1v||AGTUV`o KK ". s AC3ACr%r$cURUSSX#5nURU5nURU5nURU5nXASS'X#///nX4$)zProcess images for the upsampling stage in the VisualCloze pipeline. Args: input_images: Input image to process height: Target height width: Target width Returns: Tuple of processed image and its size r)resizer-r.r/)rrr%r$rAr?s rpreprocess_image_upsampling0VisualClozeProcessor.preprocess_image_upsamplingsm  LOA.>!!%(  'u%"Q() ((rc[R"SSUSSRSUSSRS45//$)Nrrrr )rFonesrH)rrs rpreprocess_mask_upsampling/VisualClozeProcessor.preprocess_mask_upsamplingsJQ<?1#5#;#;A#> QPQ@R@X@XYZ@[\]^__rsizec>SUSSUSSUSUS-S34nU$)NzA grid layout with rz rows and rz columns, displaying z images arranged side by side.)rrWlayout_instructions rget_layout_prompt&VisualClozeProcessor.get_layout_promptsO!$q'*T!WI=RSWXYSZ]abc]dSdRefD E "!r task_promptcontent_prompt upsamplingc[U[5(a U/nU/nU/n//U(dUO$[[U55Vs/sHnSPM snU///S.n [[U55Hn U(aSn O,UR [X:5[X:S545n U(a&UR X:XES9upUR U 5nO:URX:US9upnURX5nU SRU5 U SRU 5 U SRU 5 U S RU5 U S RU 5 M U $s snf) aProcess visual cloze inputs. Args: task_prompt: Task description(s) content_prompt: Content description(s) input_images: List of images or None for the target images height: Optional target height for upsampling stage width: Optional target width for upsampling stage upsampling: Whether this is in the upsampling processing stage Returns: Dictionary containing processed images, masks, prompts and metadata N) init_imagerIr]r^ layout_promptr6 image_sizer)r%r$)rr6rcrarIrb) isinstancestrr#r"r[rQrUrBrMr))rr]r^rr%r$r_rr3outputr7rbcur_processed_imagescur_image_sizecur_maskcur_target_positions r preprocessVisualClozeProcessor.preprocesss. k3 ' '&-K,-N(>L.8;USVWbScMd>eMdtMd>e,! s;'(A $ $ 6 6LO8LcR^RabcRdNe7f g 7;7W7W OF8X84$ ::;OPLPLaLa O6FMbMI$6I //0DZ()001DE < ' ' 7 < ' '(< = 6N ! !( + ? # * *= 9-)0 =?fs E)NNNFr!)__name__ __module__ __qualname____firstlineno____doc__r&rrrrrrFTensorrBrMrQrUrer[rboolrrk__static_attributes__ __classcell__)rs@rr r s 14%#%%S> hu{{&;!<=S>QTS> tD&'d49o)>S I JS>j ekk!23FJ3i d5<< !:)4 ,-)) ) tD%&T$s)_(== > )2`tD*=c49n-=tDhsm)<$=>? =  = } === ==rr ) typingrrrrrrFPILrimage_processorr r rYrrrys%65 0d,dr