+hM SSKrSSKJrJrJrJrJrJrJr SSK r SSK r SSK J r Jr SSKJrJr SSKJr SSKJr SSKJrJr SS KJr SS KJr SS KJrJrJ r SS K!J"r" S SK#J$r$ \"5(a SSK%J&s J'r( Sr)OSr)\RT"\+5r,Sr-SS\.S\/S\/S\/4Sjjr0SS\\.S\\\1\ Rd4S\\\.S\\\/4Sjjr3"SS\\5r4g) N)AnyCallableDictListOptionalTupleUnion) AutoTokenizerGlmModel)MultiPipelineCallbacksPipelineCallback)VaeImageProcessor)CogView4LoraLoaderMixin) AutoencoderKLCogView4Transformer2DModel)DiffusionPipeline)FlowMatchEulerDiscreteScheduler)is_torch_xla_availableloggingreplace_example_docstring) randn_tensor)CogView4PipelineOutputTFa Examples: ```python >>> import torch >>> from diffusers import CogView4Pipeline >>> pipe = CogView4Pipeline.from_pretrained("THUDM/CogView4-6B", torch_dtype=torch.bfloat16) >>> pipe.to("cuda") >>> prompt = "A photo of an astronaut riding a horse on mars" >>> image = pipe(prompt).images[0] >>> image.save("output.png") ``` base_seq_len base_shift max_shiftreturnc"X- S-nXC-U-nU$)Ng?) image_seq_lenrrrmmus h/home/james-whalen/.local/lib/python3.13/site-packages/diffusers/pipelines/cogview4/pipeline_cogview4.pycalculate_shiftr%;s"  %#-A  #B Inum_inference_stepsdevice timestepssigmasc bS[[R"UR5RR 55;nS[[R"UR5RR 55;nUbYUbVU(d U(d[ SURS35eUR"S X4US.UD6 URn[U5nX14$UbQUcNU(d[ SURS35eUR"S X2S.UD6 URn[U5nX14$UcQUbNU(d[ SURS35eUR"S XBS .UD6 URn[U5nX14$UR"U4S U0UD6 URnX14$) a Calls the scheduler's `set_timesteps` method and retrieves timesteps from the scheduler after the call. Handles custom timesteps. Any kwargs will be supplied to `scheduler.set_timesteps`. Args: scheduler (`SchedulerMixin`): The scheduler to get timesteps from. num_inference_steps (`int`): The number of diffusion steps used when generating samples with a pre-trained model. If used, `timesteps` must be `None`. device (`str` or `torch.device`, *optional*): The device to which the timesteps should be moved to. If `None`, the timesteps are not moved. timesteps (`List[int]`, *optional*): Custom timesteps used to override the timestep spacing strategy of the scheduler. If `timesteps` is passed, `num_inference_steps` and `sigmas` must be `None`. sigmas (`List[float]`, *optional*): Custom sigmas used to override the timestep spacing strategy of the scheduler. If `sigmas` is passed, `num_inference_steps` and `timesteps` must be `None`. Returns: `Tuple[torch.Tensor, int]`: A tuple where the first element is the timestep schedule from the scheduler and the second element is the number of inference steps. r)r*zThe current scheduler class z's `set_timesteps` does not support custom timestep or sigma schedules. Please check whether you are using the correct scheduler.)r)r*r(zx's `set_timesteps` does not support custom timestep schedules. Please check whether you are using the correct scheduler.)r)r(zv's `set_timesteps` does not support custom sigmas schedules. Please check whether you are using the correct scheduler.)r*r(r(r ) setinspect signature set_timesteps parameterskeys ValueError __class__r)len) schedulerr'r(r)r*kwargsaccepts_timestepsaccepts_sigmass r$retrieve_timestepsr9Fs>$s7+<+ .y/B/B.CDab  M)MfM'' !)n  ))  v1.y/B/B.CD_`  GvGG'' !)n  ))  3MFMfM''  ))r&c1^\rSrSrSr/rSr/SQrS\S\ S\ S\ S \ 4 U4S jjr S8S \\\\4S\S\\R(S\\R*4SjjrS9S \\\\4S\\\\\4S\S\S\\R0S\\R0S\\R(S\\R*S\4SjjrS:SjrS;Sjr\S5r\S5r\S5r\S5r \S 5r!\S!5r"\RF"5\$"\%5S S S S S"S S S#SS S S S S S$S%SS S S&/S 4S \\\\\4S\\\\\4S'\\S(\\S)\S*\\\S+\\\&S,\&S\S-\\\RN\\RN4S&\\RPS\\RPS\\RPS.\\)\\4S/\)\\4S0\S1\S2\\*\\+4S3\\\,\\\*/S 4\-\.4S4\\S\S5\\/\)44,S6jj55r0S7r1U=r2$)<CogView4Pipelinea Pipeline for text-to-image generation using CogView4. This model inherits from [`DiffusionPipeline`]. Check the superclass documentation for the generic methods the library implements for all the pipelines (such as downloading or saving, running on a particular device, etc.) Args: vae ([`AutoencoderKL`]): Variational Auto-Encoder (VAE) Model to encode and decode images to and from latent representations. text_encoder ([`GLMModel`]): Frozen text-encoder. CogView4 uses [glm-4-9b-hf](https://huggingface.co/THUDM/glm-4-9b-hf). tokenizer (`PreTrainedTokenizer`): Tokenizer of class [PreTrainedTokenizer](https://huggingface.co/docs/transformers/main/en/main_classes/tokenizer#transformers.PreTrainedTokenizer). transformer ([`CogView4Transformer2DModel`]): A text conditioned `CogView4Transformer2DModel` to denoise the encoded image latents. scheduler ([`SchedulerMixin`]): A scheduler to be used in combination with `transformer` to denoise the encoded image latents. ztext_encoder->transformer->vae)latents prompt_embedsnegative_prompt_embeds tokenizer text_encodervae transformerr5c>[TU]5 URXX4US9 [USS5(a/S[ UR R R5S- -OSUl[URS9Ul g)N)r@rArBrCr5rBr)vae_scale_factor) super__init__register_modulesgetattrr4rBconfigblock_out_channelsrGrimage_processor)selfr@rArBrCr5r3s r$rICogView4Pipeline.__init__s|  hq  W^^bdikoVpVpc$((//*L*L&MPQ&Q Rvw0$BWBWXr&Npromptmax_sequence_lengthr(dtypec U=(d URnU=(d URRn[U[5(aU/OUnUR USUSSSS9nUR nUR USSS9R nURSURS:a][R"Xg5(dBUR RUSS2US- S245n[RSUS U35 URSn S U S -- S -n U S :ad[R"URS U 4UR RURURS 9n [R "X/SS 9nURUR#U5SS9R$Sn U R#XCS9n U $)NlongestTpt)padding max_length truncationadd_special_tokensreturn_tensors)rXr\rzXThe following part of your input was truncated because `max_sequence_length` is set to z tokens: r) fill_valuerTr()dim)output_hidden_statesrTr()_execution_devicerArT isinstancestrr@ input_idsshapetorchequal batch_decodeloggerwarningfull pad_token_idr(catto hidden_states) rOrRrSr(rT text_inputstext_input_idsuntruncated_ids removed_textcurrent_length pad_lengthpad_idsr>s r$_get_glm_embeds CogView4Pipeline._get_glm_embedss14110**00'44&&nn *# % %....SW.Xbb   $(<(>66qJ]`aJadfJfGf7ghL NN'( ,A (--a0NR/0B6 >jj%%a(*5>>66$**%,, G #YY'@aHN)).*;*;F*CZ^)_mmnpq %((u(D r&Trnegative_promptdo_classifier_free_guidancenum_images_per_promptr>r?c U=(d URn[U[5(aU/OUnUb [U5n OURSn UcUR XXx5nUR S5n URSUS5nURX-U S5nU(aUcU=(d Sn[U[5(aX/-OUnUb;[U5[U5La$[S[U5S[U5S35eU [U5:wa![SUS [U5S US U S 3 5eUR X)Xx5nUR S5n URSUS5nURX-U S5nXV4$) a Encodes the prompt into text encoder hidden states. Args: prompt (`str` or `List[str]`, *optional*): prompt to be encoded negative_prompt (`str` or `List[str]`, *optional*): The prompt or prompts not to guide the image generation. If not defined, one has to pass `negative_prompt_embeds` instead. Ignored when not using guidance (i.e., ignored if `guidance_scale` is less than `1`). do_classifier_free_guidance (`bool`, *optional*, defaults to `True`): Whether to use classifier free guidance or not. num_images_per_prompt (`int`, *optional*, defaults to 1): Number of images that should be generated per prompt. torch device to place the resulting embeddings on prompt_embeds (`torch.Tensor`, *optional*): Pre-generated text embeddings. Can be used to easily tweak text inputs, *e.g.* prompt weighting. If not provided, text embeddings will be generated from `prompt` input argument. negative_prompt_embeds (`torch.Tensor`, *optional*): Pre-generated negative text embeddings. Can be used to easily tweak text inputs, *e.g.* prompt weighting. If not provided, negative_prompt_embeds will be generated from `negative_prompt` input argument. device: (`torch.device`, *optional*): torch device dtype: (`torch.dtype`, *optional*): torch dtype max_sequence_length (`int`, defaults to `1024`): Maximum sequence length in encoded prompt. Can be set to other values but may lead to poorer results. rrr]z?`negative_prompt` should be the same type to `prompt`, but got z != .z`negative_prompt`: z has batch size z, but `prompt`: zT. Please make sure that passed `negative_prompt` matches the batch size of `prompt`.) rdrerfr4rhrzsizerepeatviewtype TypeErrorr2) rOrRr|r}r~r>r?r(rTrS batch_sizeseq_lens r$ encode_promptCogView4Pipeline.encode_promptsP1411'44&&  VJ&,,Q/J   00f\M$$Q'%,,Q0EqI %**:+MwXZ[ &+A+I-3O@J?\_@`@`j+<Z+GA#i.AQR&<'gi u&Vr&c n^US-S:wd US-S:wa[SUSUS35eUbW[U4SjU55(d=[STRSUVs/sHoTR;dMUPM sn35eUbUb[S US US 35eUcUc [S 5eUbA[U[5(d,[U[ 5(d[S [ U535eUbUb[S USUS 35eUbUb[SUSUS 35eUbUbURSURS:wa&[SURSURS35eURSURS:wa&[SURSURS35egggs snf)Nr^rz8`height` and `width` have to be divisible by 16 but are z and rc3@># UHoTR;v M g7fN)_callback_tensor_inputs).0krOs r$ 0CogView4Pipeline.check_inputs..KsF 7Y!-- -7Ysz2`callback_on_step_end_tensor_inputs` has to be in z , but found zCannot forward both `prompt`: z and `prompt_embeds`: z2. Please make sure to only forward one of the two.zeProvide either `prompt` or `prompt_embeds`. Cannot leave both `prompt` and `prompt_embeds` undefined.z2`prompt` has to be of type `str` or `list` but is z and `negative_prompt_embeds`: z'Cannot forward both `negative_prompt`: zz`prompt_embeds` and `negative_prompt_embeds` must have the same batch size when passed directly, but got: `prompt_embeds` z and `negative_prompt_embeds` r]zy`prompt_embeds` and `negative_prompt_embeds` must have the same dimension when passed directly, but got: `prompt_embeds` )r2allrrerfrrrh) rOrRrrr|"callback_on_step_end_tensor_inputsr>r?rs ` r$ check_inputsCogView4Pipeline.check_inputs>s B;! urzQWX^W__dejdkklmn n - 9#F 7YF C C DTEaEaDbbn|^pH|^vwkokGkGbGpq|^pHoIJ   -";08N}o^00 ^ 5w  FC)@)@TZ\`IaIaQRVW]R^Q_`a a  "8"D09*++]_   &+A+M9/9JK*++]_   $)?)K""1%)?)E)Ea)HH --:-@-@,AB.445Q8 ""2&*@*F*Fr*JJ --:-@-@,AB.445Q8K*L $5pHs F21F2cUR$r_guidance_scalerOs r$guidance_scaleCogView4Pipeline.guidance_scalews###r&c URS:$)Nrrrs r$r},CogView4Pipeline.do_classifier_free_guidance~s##a''r&cUR$r)_num_timestepsrs r$ num_timestepsCogView4Pipeline.num_timestepss"""r&cUR$r)_attention_kwargsrs r$attention_kwargs!CogView4Pipeline.attention_kwargs%%%r&cUR$r)_current_timesteprs r$current_timestep!CogView4Pipeline.current_timesteprr&cUR$r) _interruptrs r$ interruptCogView4Pipeline.interrupts r&2g@)rrpilr=rrr'r)r*rr original_sizecrops_coords_top_left output_type return_dictrcallback_on_step_endrrc4[U[[45(a URnU=(d- URR R UR-nU=(d- URR R UR-nU=(d X44nX44nURUUUUUU U 5 Xl UUl SUl SUl Ub[U[5(aSnO3Ub![U[5(a [U5nOU R SnUR"nUR%UUUR&U U U UUS9upURR R(nUR+UU -UUU[,R.UU U 5n [,R0"U/U R2US9n[,R0"U/U R2US9n[,R0"U/U R2US9nUR5UU -S5nUR5UU -S5nUR5UU -S5nX0R-X@R--URR R6S--nUc6[8R:"UR<R R>SU5O[8R@"U5nURC[8RD5RC[8R.5nUc"X`R<R R>- OUn[GUUR<R RIS S 5UR<R RIS S 5UR<R RIS S55nUR<UUXgUS9upe[U5Ul&URR2n[O[U5XPR<RP-- S5nURSUS9n[UU5GHunn URV(aMU Ul U RYU5n!U R[U R S5n"URR]S5 UR U!U U"UUUUSS9Sn#SSS5 UR&(aSURR]S5 UR U!U U"UUUUSS9Sn$SSS5 W$UR^W#U$- --n%OW#n%UR<RaU%U U SS9Sn Ubs0n&UHn'[c5U'U&U''M U"UUUR<RdUU&5n(U(RgSU 5n U(RgSU 5n U(RgSU 5n U[U5S- :Xd)US-U:a0US-UR<RP-S:XaURi5 [j(dGM[lRn"5 GM SSS5 SUl US:XdeU RYURpR25URpR Rr- n URpRuU SU S9Sn)OU n)URvRyU)US9n)UR{5 U(dU)4$[}U)S9$!,(df  GN6=f!,(df  GN=f!,(df  N=f)a Function invoked when calling the pipeline for generation. Args: prompt (`str` or `List[str]`, *optional*): The prompt or prompts to guide the image generation. If not defined, one has to pass `prompt_embeds`. negative_prompt (`str` or `List[str]`, *optional*): The prompt or prompts not to guide the image generation. If not defined, one has to pass `negative_prompt_embeds` instead. Ignored when not using guidance (i.e., ignored if `guidance_scale` is less than `1`). height (`int`, *optional*, defaults to self.transformer.config.sample_size * self.vae_scale_factor): The height in pixels of the generated image. If not provided, it is set to 1024. width (`int`, *optional*, defaults to self.transformer.config.sample_size * self.vae_scale_factor): The width in pixels of the generated image. If not provided it is set to 1024. num_inference_steps (`int`, *optional*, defaults to `50`): The number of denoising steps. More denoising steps usually lead to a higher quality image at the expense of slower inference. timesteps (`List[int]`, *optional*): Custom timesteps to use for the denoising process with schedulers which support a `timesteps` argument in their `set_timesteps` method. If not defined, the default behavior when `num_inference_steps` is passed will be used. Must be in descending order. sigmas (`List[float]`, *optional*): Custom sigmas to use for the denoising process with schedulers which support a `sigmas` argument in their `set_timesteps` method. If not defined, the default behavior when `num_inference_steps` is passed will be used. guidance_scale (`float`, *optional*, defaults to `5.0`): Guidance scale as defined in [Classifier-Free Diffusion Guidance](https://huggingface.co/papers/2207.12598). `guidance_scale` is defined as `w` of equation 2. of [Imagen Paper](https://huggingface.co/papers/2205.11487). Guidance scale is enabled by setting `guidance_scale > 1`. Higher guidance scale encourages to generate images that are closely linked to the text `prompt`, usually at the expense of lower image quality. num_images_per_prompt (`int`, *optional*, defaults to `1`): The number of images to generate per prompt. generator (`torch.Generator` or `List[torch.Generator]`, *optional*): One or a list of [torch generator(s)](https://pytorch.org/docs/stable/generated/torch.Generator.html) to make generation deterministic. latents (`torch.FloatTensor`, *optional*): Pre-generated noisy latents, sampled from a Gaussian distribution, to be used as inputs for image generation. Can be used to tweak the same generation with different prompts. If not provided, a latents tensor will ge generated by sampling using the supplied random `generator`. prompt_embeds (`torch.FloatTensor`, *optional*): Pre-generated text embeddings. Can be used to easily tweak text inputs, *e.g.* prompt weighting. If not provided, text embeddings will be generated from `prompt` input argument. negative_prompt_embeds (`torch.FloatTensor`, *optional*): Pre-generated negative text embeddings. Can be used to easily tweak text inputs, *e.g.* prompt weighting. If not provided, negative_prompt_embeds will be generated from `negative_prompt` input argument. original_size (`Tuple[int]`, *optional*, defaults to (1024, 1024)): If `original_size` is not the same as `target_size` the image will appear to be down- or upsampled. `original_size` defaults to `(height, width)` if not specified. Part of SDXL's micro-conditioning as explained in section 2.2 of [https://huggingface.co/papers/2307.01952](https://huggingface.co/papers/2307.01952). crops_coords_top_left (`Tuple[int]`, *optional*, defaults to (0, 0)): `crops_coords_top_left` can be used to generate an image that appears to be "cropped" from the position `crops_coords_top_left` downwards. Favorable, well-centered images are usually achieved by setting `crops_coords_top_left` to (0, 0). Part of SDXL's micro-conditioning as explained in section 2.2 of [https://huggingface.co/papers/2307.01952](https://huggingface.co/papers/2307.01952). output_type (`str`, *optional*, defaults to `"pil"`): The output format of the generate image. Choose between [PIL](https://pillow.readthedocs.io/en/stable/): `PIL.Image.Image` or `np.array`. return_dict (`bool`, *optional*, defaults to `True`): Whether or not to return a [`~pipelines.stable_diffusion_xl.StableDiffusionXLPipelineOutput`] instead of a plain tuple. attention_kwargs (`dict`, *optional*): A kwargs dictionary that if specified is passed along to the `AttentionProcessor` as defined under `self.processor` in [diffusers.models.attention_processor](https://github.com/huggingface/diffusers/blob/main/src/diffusers/models/attention_processor.py). callback_on_step_end (`Callable`, *optional*): A function that calls at the end of each denoising steps during the inference. The function is called with the following arguments: `callback_on_step_end(self: DiffusionPipeline, step: int, timestep: int, callback_kwargs: Dict)`. `callback_kwargs` will include a list of all tensors as specified by `callback_on_step_end_tensor_inputs`. callback_on_step_end_tensor_inputs (`List`, *optional*): The list of tensor inputs for the `callback_on_step_end` function. The tensors specified in the list will be passed as `callback_kwargs` argument. You will only be able to include variables listed in the `._callback_tensor_inputs` attribute of your pipeline class. max_sequence_length (`int`, defaults to `224`): Maximum sequence length in encoded prompt. Can be set to other values but may lead to poorer results. Examples: Returns: [`~pipelines.cogview4.pipeline_CogView4.CogView4PipelineOutput`] or `tuple`: [`~pipelines.cogview4.pipeline_CogView4.CogView4PipelineOutput`] if `return_dict` is True, otherwise a `tuple`. When returning a tuple, the first element is a list with the generated images. NFrr)r~r>r?rSr(rcrEg?base_image_seq_lenr?r?)r#)totalcond)rrencoder_hidden_statestimestepr target_size crop_coordsrruncond)rr=r>r?latent)rr)r)images)?rerr tensor_inputsrCrL sample_sizerGrrrrrrfrr4rhrdrr} in_channelsrrifloat32tensorrTr patch_sizenplinspacer5num_train_timestepsarrayastypeint64r%getr9rmaxorder progress_bar enumeraterrqexpand cache_contextrsteplocalsr*popupdate XLA_AVAILABLExm mark_steprBscaling_factordecoderN postprocessmaybe_free_model_hooksr)*rOrRr|rrr'r)r*rr~rr=r>r?rrrrrrrrSrrr(latent_channelsr!r#transformer_dtypenum_warmup_stepsritlatent_model_inputrnoise_pred_condnoise_pred_uncond noise_predcallback_kwargsrcallback_outputsimages* r$__call__CogView4Pipeline.__call__sf *-=?U,V W W1E1S1S .V4++22>>AVAVVT))00<\]    # # . . 1   KK--AA3H[ \)$  $$RXX.55bjjA JP.^^22FFF^d   NN ! ! % %&:C @ NN ! ! % %lD 9 NN ! ! % %k4 8   *< NN/r* & ")n!,,22s9~0CnnFZFZ0ZZ\]^   %8  9\!),1>>)*&%,ZZ0A%B"88GMM!$45%%33F;&*&6&6&8.;!)&3$/$9)9$)'7 ' 'O<33))77A,0,<,<*<2H%-*7(3(=-=(--= - -)B"3T5H5HO^oLo5p!pJ!0J..--j!WRW-XYZ[(3&(O?-3Xa[*@';D!T^^EZEZ[\E]_n'o$.229gFG$4$8$8-$XM-=-A-ABZ\r-s*I**A9I/IqSTuX\XfXfXlXlNlpqNq '') =LLNo-:t"&h&jj0488??3Q3QQGHHOOG)OTUVWEE$$00K0P ##%8O%U33<;BA3: 9sD%A7\ [%64\ *[7D \ \ % [4 /\ 7 \ \  \)rrrrrrNrG)NrQNN)NTrNNNNrQr)NN)3__name__ __module__ __qualname____firstlineno____doc___optional_componentsmodel_cpu_offload_seqrr r rrrrIr rfrrrrir(rTrzboolTensorrrrpropertyrr}rrrrno_gradrEXAMPLE_DOC_STRINGfloat Generator FloatTensorrrrrrr rr__static_attributes__ __classcell__)r3s@r$r;r;s3(<TY YY Y 0 Y 3 Y$)-#')-'+ )c49n%)!)& )  $ )\<@,0%&049=)-'+#'M5c49n%M5"%T#Y"78M5&* M5 # M5   - M5!) 6M5&M5 $M5!M5^2#7r$$ ((##&&&& ]]_1237;? $##%)-(, #%&MQ/359>B3717  59 9B #'1Y4sDI~./Y4"%T#Y"78Y4 Y4 } Y4 ! Y4DI&Y4e%Y4Y4 #Y4E%//43H"HIJY4%++,Y4  1 12Y4!)):): ;Y4 c3h0Y4 %S#X!Y4"#Y4$%Y4&#4S>2'Y4(' (Cd+T124DF\\ ] )Y4.-1I/Y40!1Y42 %u, -3Y43Y4r&r;)rrr)NNNN)5r-typingrrrrrrr numpyrri transformersr r callbacksr rrNrloadersrmodelsrrpipelines.pipeline_utilsr schedulersrutilsrrrutils.torch_utilsrpipeline_outputrtorch_xla.core.xla_modelcore xla_modelrr get_loggerrrlrrrr%rfr(r9r;r r&r$rs DDD 0A0.?99OO-3))MM   H % $    *.15%)$( @*!#@* U3 ,- .@*S " @* T%[ ! @*Fd4(*Ad4r&