wiIPddlmZmZmZmZmZddlZddlZddl m Z m Z ddl m Z ddlmZmZmZddlmZdd lmZmZd d lmZd d lmZejeZd ZGddeZdS))CallableDictListOptionalUnionN) CLIPTextModel CLIPTokenizer)DDPMWuerstchenScheduler) deprecateloggingreplace_example_docstring) randn_tensor)DiffusionPipelineImagePipelineOutput) PaellaVQModel)WuerstchenDiffNeXta Examples: ```py >>> import torch >>> from diffusers import WuerstchenPriorPipeline, WuerstchenDecoderPipeline >>> prior_pipe = WuerstchenPriorPipeline.from_pretrained( ... "warp-ai/wuerstchen-prior", torch_dtype=torch.float16 ... ).to("cuda") >>> gen_pipe = WuerstchenDecoderPipeline.from_pretrain("warp-ai/wuerstchen", torch_dtype=torch.float16).to( ... "cuda" ... ) >>> prompt = "an image of a shiba inu, donning a spacesuit and helmet" >>> prior_output = pipe(prompt) >>> images = gen_pipe(prior_output.image_embeddings, prompt=prompt) ``` c^eZdZdZdZgdZ d&dededede d e d e d d ffd Z dZ d'dZedZedZedZejeed dd dd dd d ddd dgf deejeejfdeeeefdedeee de deeeeefded eeejeejfdeejd!eed"ed#eeeee gd fd$eefd%Z!xZ"S)(WuerstchenDecoderPipelineaR Pipeline for generating images from the Wuerstchen model. This model inherits from [`DiffusionPipeline`]. Check the superclass documentation for the generic methods the library implements for all the pipelines (such as downloading or saving, running on a particular device, etc.) Args: tokenizer (`CLIPTokenizer`): The CLIP tokenizer. text_encoder (`CLIPTextModel`): The CLIP text encoder. decoder ([`WuerstchenDiffNeXt`]): The WuerstchenDiffNeXt unet decoder. vqgan ([`PaellaVQModel`]): The VQGAN model. scheduler ([`DDPMWuerstchenScheduler`]): A scheduler to be used in combination with `prior` to generate image embedding. latent_dim_scale (float, `optional`, defaults to 10.67): Multiplier to determine the VQ latent space size from the image embeddings. If the image embeddings are height=24 and width=24, the VQ latent shape needs to be height=int(24*10.67)=256 and width=int(24*10.67)=256 in order to match the training conditions. ztext_encoder->decoder->vqgan)latentstext_encoder_hidden_statesnegative_prompt_embedsimage_embeddingsףp= W%@ tokenizer text_encoderdecoder schedulervqganlatent_dim_scalereturnNct||||||||dS)N)rrrr r!)r")super__init__register_modulesregister_to_config)selfrrrr r!r" __class__s /root/.openclaw/workspace/chatterbox_venv_py311/lib/python3.11/site-packages/diffusers/pipelines/wuerstchen/pipeline_wuerstchen.pyr&z"WuerstchenDecoderPipeline.__init__Ssg  %     1ABBBBBc|t||||}n:|j|krtd|jd|||}||jz}|S)N) generatordevicedtypezUnexpected latents shape, got z , expected )rshape ValueErrortoinit_noise_sigma)r)r1r0r/r.rr s r+prepare_latentsz)WuerstchenDecoderPipeline.prepare_latentsgsr ?"5IfTYZZZGG}%% !c'-!c!c\a!c!cdddjj((GI66r,c t|trt|nd}||d|jjdd}|j}|j} ||ddj} | jd|jdkrtj || s|j | dd|jjdz df} t d |jjd | |ddd|jjf}| ddd|jjf} | ||| | } | j} | |d } d}|r^|dg|z}nt#|t#|ur0t%dt#|dt#|dt|t&r|g}n>|t|kr)t)d|dt|d|d|d |}||d|jjdd}| |j||j| }|j}|jd}|d|d}|||z|d}| |fS)Nr max_lengthTpt)paddingr7 truncationreturn_tensorslongest)r9r;z\The following part of your input was truncated because CLIP can only handle sequences up to z tokens: )attention_maskrdimz?`negative_prompt` should be the same type to `prompt`, but got z != .z`negative_prompt`: z has batch size z, but `prompt`: zT. Please make sure that passed `negative_prompt` matches the batch size of `prompt`.) isinstancelistlenrmodel_max_length input_idsr>r1torchequal batch_decodeloggerwarningrr3last_hidden_staterepeat_interleavetype TypeErrorstrr2repeatview)r)promptr/num_images_per_promptdo_classifier_free_guidancenegative_prompt batch_size text_inputstext_input_idsr>untruncated_ids removed_texttext_encoder_outputr!uncond_text_encoder_hidden_states uncond_tokens uncond_input*negative_prompt_embeds_text_encoder_outputseq_lens r+ encode_promptz'WuerstchenDecoderPipeline.encode_promptrs%/vt$<$<CS[[[! nn  ~6 %  %.$3..SW.XXb   $(66qqq$.JilmJmprJrGr7sttL NNMN3MM>JMM   ,AAA/P1P/P,PQN+AAA/P1P/P,PQN"//0A0A&0I0IZhZkZklrZsZs/tt%8%J"%?%Q%QRgmn%Q%o%o",0) && &!#z 1 fT/%:%:::(VZ[jVkVk((V (((OS11 0!0 1 s?3333 3/333K_K_33330:333 !0 >>$>:# *L:>9J9J&))&11,B]B`B`agBhBh:K:: 61[0l -8=a@G0Q0X0XYZ\qst0u0u -0Q0V0V22GR11 -*+LLLr,c|jSN_guidance_scaler)s r+guidance_scalez(WuerstchenDecoderPipeline.guidance_scales ##r,c|jdkS)Nrrfrhs r+rVz5WuerstchenDecoderPipeline.do_classifier_free_guidances#a''r,c|jSre)_num_timestepsrhs r+ num_timestepsz'WuerstchenDecoderPipeline.num_timestepss ""r, grpilTrrrTnum_inference_steps timestepsrirWrUr. output_type return_dictcallback_on_step_end"callback_on_step_end_tensor_inputsc |dd}|dd}|tddd|tddd| Atfd| Ds&tdjd fd | Dj}jj}|_t|ts9t|tr|g}n td t|d jrP|Nt|ts9t|tr|g}n td t|d t|trtj|d}t|t"jr*tj|||}t|tjs tdt|d t|t*s tdt|d|||d|zj|\}}|tj||gn|}jr(tj|tj|gn|}t+|djjz}t+|djjz}|d|zd||f}|8j||jj}t=|}n(j||jj}||||| j} t=|dd_ tC"|ddD]\}}|#| d|}jrtj| gdzn| jrtj|gdzn|||}jr3|$d\}}tj%||j&}j'||| |j(} | ni} | D]}!tS|!| |!<| ||| }"|"d| } |"d|}|"d|}|/||zdkr&|tUjdd z}#||#|| | d!vrtd"| | d#ksj+jj,| z} j+-| j./dd }$| d$krO|$0dddd 123}$nl| d%krc|$0dddd 123}$4|$}$n| }$5| s|$Stm|$S)&ae Function invoked when calling the pipeline for generation. Args: image_embedding (`torch.Tensor` or `List[torch.Tensor]`): Image Embeddings either extracted from an image or generated by a Prior Model. prompt (`str` or `List[str]`): The prompt or prompts to guide the image generation. num_inference_steps (`int`, *optional*, defaults to 12): The number of denoising steps. More denoising steps usually lead to a higher quality image at the expense of slower inference. timesteps (`List[int]`, *optional*): Custom timesteps to use for the denoising process. If not defined, equal spaced `num_inference_steps` timesteps are used. Must be in descending order. guidance_scale (`float`, *optional*, defaults to 0.0): Guidance scale as defined in [Classifier-Free Diffusion Guidance](https://arxiv.org/abs/2207.12598). `decoder_guidance_scale` is defined as `w` of equation 2. of [Imagen Paper](https://arxiv.org/pdf/2205.11487.pdf). Guidance scale is enabled by setting `decoder_guidance_scale > 1`. Higher guidance scale encourages to generate images that are closely linked to the text `prompt`, usually at the expense of lower image quality. negative_prompt (`str` or `List[str]`, *optional*): The prompt or prompts not to guide the image generation. Ignored when not using guidance (i.e., ignored if `decoder_guidance_scale` is less than `1`). num_images_per_prompt (`int`, *optional*, defaults to 1): The number of images to generate per prompt. generator (`torch.Generator` or `List[torch.Generator]`, *optional*): One or a list of [torch generator(s)](https://pytorch.org/docs/stable/generated/torch.Generator.html) to make generation deterministic. latents (`torch.Tensor`, *optional*): Pre-generated noisy latents, sampled from a Gaussian distribution, to be used as inputs for image generation. Can be used to tweak the same generation with different prompts. If not provided, a latents tensor will ge generated by sampling using the supplied random `generator`. output_type (`str`, *optional*, defaults to `"pil"`): The output format of the generate image. Choose between: `"pil"` (`PIL.Image.Image`), `"np"` (`np.array`) or `"pt"` (`torch.Tensor`). return_dict (`bool`, *optional*, defaults to `True`): Whether or not to return a [`~pipelines.ImagePipelineOutput`] instead of a plain tuple. callback_on_step_end (`Callable`, *optional*): A function that calls at the end of each denoising steps during the inference. The function is called with the following arguments: `callback_on_step_end(self: DiffusionPipeline, step: int, timestep: int, callback_kwargs: Dict)`. `callback_kwargs` will include a list of all tensors as specified by `callback_on_step_end_tensor_inputs`. callback_on_step_end_tensor_inputs (`List`, *optional*): The list of tensor inputs for the `callback_on_step_end` function. The tensors specified in the list will be passed as `callback_kwargs` argument. You will only be able to include variables listed in the `._callback_tensor_inputs` attribute of your pipeline class. Examples: Returns: [`~pipelines.ImagePipelineOutput`] or `tuple` [`~pipelines.ImagePipelineOutput`] if `return_dict` is True, otherwise a `tuple`. When returning a tuple, the first element is a list with the generated image embeddings. callbackNcallback_stepsz1.0.0zhPassing `callback` as an input argument to `__call__` is deprecated, consider use `callback_on_step_end`znPassing `callback_steps` as an input argument to `__call__` is deprecated, consider use `callback_on_step_end`c3*K|] }|jvVdSre_callback_tensor_inputs.0kr)s r+ z5WuerstchenDecoderPipeline.__call__..*sDF F 23A- -F F F F F F r,z2`callback_on_step_end_tensor_inputs` has to be in z , but found c&g|] }|jv |Srzr|s r+ z6WuerstchenDecoderPipeline.__call__...sVpHpHpHvwbckokGbGbGpqbGbGbGr,z2'prompt' must be of type 'list' or 'str', but got rBz;'negative_prompt' must be of type 'list' or 'str', but got rr?)r/)r0zI'image_embeddings' must be of type 'torch.Tensor' or 'np.array', but got z5'num_inference_steps' must be of type 'int', but got zo In Case you want to provide explicit timesteps, please use the 'timesteps' argument.rr )rqr/r=)reffnetclip) model_outputtimestepsampler.rrrorderr)r8nprolatentzSOnly the output types `pt`, `np`, `pil` and `latent` are supported not output_type=rrro)7popr allr2r{_execution_devicerr0rgrCrDrQrPrOrVrHcatrndarrayTensorr3intrcsize zeros_likeconfigr"r set_timestepsrqrEr5rl enumerate progress_barexpandchunklerpristep prev_samplelocalsgetattrr! scale_factordecoderclamppermutecpufloatnumpy numpy_to_pilmaybe_free_model_hooksr)%r)rrTrprqrirWrUr.rrrrsrtrukwargsrwrxr/r0 prompt_embedsrrr latent_height latent_widthlatent_features_shapeitratiopredicted_latentspredicted_latents_textpredicted_latents_uncondcallback_kwargsr~callback_outputsstep_idximagess%` r+__call__z"WuerstchenDecoderPipeline.__call__sT::j$//$4d;;   z     %  A    . 9#F F F F 7YF F F C C 9JTEaJJpHpHpHpH|^pHpHpHJJ  ' "-&$'' f&#&& f  dUYZ`UaUa d d deee  + *:ot3T3T*os33'6&7OO#nVZ[jVkVknnn & - - B$y)9qAAA  & 3 3 ]$|, > >jw # / "EI')9:J)K)KL M M M! ,11!44t{7SSTT +0033dk6RRSS !1!6!6q!9!9GZ&&w//6<rs798888888888888 55555555111111BBBBBBBBBB------CCCCCCCC333333<<<<<<  H % %(C+C+C+C+C+ 1C+C+C+C+C+r,