wiEddlmZmZmZmZddlZddlmZddlm Z m Z ddl m Z m Z ddlmZmZddlmZd d lmZmZd d lmZejeZd ZddZGddeZdS))CallableListOptionalUnionN)XLMRobertaTokenizer)UNet2DConditionModelVQModel) DDIMScheduler DDPMScheduler)loggingreplace_example_docstring) randn_tensor)DiffusionPipelineImagePipelineOutput)MultilingualCLIPav Examples: ```py >>> from diffusers import KandinskyPipeline, KandinskyPriorPipeline >>> import torch >>> pipe_prior = KandinskyPriorPipeline.from_pretrained("kandinsky-community/Kandinsky-2-1-prior") >>> pipe_prior.to("cuda") >>> prompt = "red cat, 4k photo" >>> out = pipe_prior(prompt) >>> image_emb = out.image_embeds >>> negative_image_emb = out.negative_image_embeds >>> pipe = KandinskyPipeline.from_pretrained("kandinsky-community/kandinsky-2-1") >>> pipe.to("cuda") >>> image = pipe( ... prompt, ... image_embeds=image_emb, ... negative_image_embeds=negative_image_emb, ... height=768, ... width=768, ... num_inference_steps=100, ... ).images >>> image[0].save("cat.png") ``` cz||dzz}||dzzdkr|dz }||dzz}||dzzdkr|dz }||z||zfS)Nrrr)hw scale_factornew_hnew_ws /root/.openclaw/workspace/chatterbox_venv_py311/lib/python3.11/site-packages/diffusers/pipelines/kandinsky/pipeline_kandinsky.py get_new_h_wrBsm q E<?a   q E<?a   < !5 55c!4eZdZdZdZdedededee e fde f fd Z d Z d"d Zejee d#deeeefdeejeejfdeejeejfdeeeeefdedededededeeejeejfdeejdeedeeeeejgd fded efd!ZxZS)$KandinskyPipelinea1 Pipeline for text-to-image generation using Kandinsky This model inherits from [`DiffusionPipeline`]. Check the superclass documentation for the generic methods the library implements for all the pipelines (such as downloading or saving, running on a particular device, etc.) Args: text_encoder ([`MultilingualCLIP`]): Frozen text-encoder. tokenizer ([`XLMRobertaTokenizer`]): Tokenizer of class scheduler (Union[`DDIMScheduler`,`DDPMScheduler`]): A scheduler to be used in combination with `unet` to generate image latents. unet ([`UNet2DConditionModel`]): Conditional U-Net architecture to denoise the image embedding. movq ([`VQModel`]): MoVQ Decoder to generate the image from the latents. ztext_encoder->unet->movq text_encoder tokenizerunet schedulermovqct||||||dt|jjjdz z|_dS)N)r"r#r$r%r&rr)super__init__register_moduleslenr&configblock_out_channelsmovq_scale_factor)selfr"r#r$r%r& __class__s rr)zKandinskyPipeline.__init__bsr  %    "#s49+;+N'O'ORS'S!Trc|t||||}n:|j|krtd|jd|||}||jz}|S)N) generatordevicedtypezUnexpected latents shape, got z , expected )rshape ValueErrortoinit_noise_sigma)r/r5r4r3r2latentsr%s rprepare_latentsz!KandinskyPipeline.prepare_latentsvsr ?"5IfTYZZZGG}%% !c'-!c!c\a!c!cdddjj((GI66rNc t|trt|nd}||dddddd}|j}||ddj} | jd |jd krrt j|| s]|j| dd|jj dz d f} t d |jj d | | |}|j |} ||| \} } | |d } | |d } | |d } |r|dg|z}nt!|t!|ur0t#dt!|dt!|dt|t$r|g}n>|t|kr)t'd|dt|d|d|d |}||dddddd}|j |}|j |}||| \}}|jd}|d|}|||z|}|jd}|d|d}|||z|d }||d }t j|| g} t j|| g} t j|| g} | | | fS)Nr max_lengthTMpt)padding truncationr<return_attention_maskadd_special_tokensreturn_tensorslongest)r?rCz\The following part of your input was truncated because CLIP can only handle sequences up to z tokens: ) input_idsattention_maskrdimz?`negative_prompt` should be the same type to `prompt`, but got z != .z`negative_prompt`: z has batch size z, but `prompt`: zT. Please make sure that passed `negative_prompt` matches the batch size of `prompt`.)r?r<r@rArBrC) isinstancelistr+r#rFr5torchequal batch_decodemodel_max_lengthloggerwarningr7rGr"repeat_interleavetype TypeErrorstrr6repeatviewcat)r/promptr3num_images_per_promptdo_classifier_free_guidancenegative_prompt batch_size text_inputstext_input_idsuntruncated_ids removed_text text_mask prompt_embedstext_encoder_hidden_states uncond_tokens uncond_inputuncond_text_input_idsuncond_text_masknegative_prompt_embeds!uncond_text_encoder_hidden_statesseq_lens r_encode_promptz KandinskyPipeline._encode_promptsC%/vt$<$<CS[[[! nn  "&#%  %...SW.XXb   $(66qqq$.JilmJmprJrGr7sttL NNMN3MM>JMM    (**622.11&99 484E4E$Y5F5 5 1 1&778MST7UU %?%Q%QRgmn%Q%o%o"//0E1/MM &9 A&!#z 1 fT/%:%:::(VZ[jVkVk((V (((OS11 0!0 1 s?3333 3/333K_K_33330:333 !0 >>$&*#'#*L%1$:$=$=f$E$E !+:==fEE HLHYHY/@PIZII E "$E -215G%;%B%B1F[%\%\ "%;%@%@NcAcel%m%m "7=a@G0Q0X0XYZ\qst0u0u -0Q0V0V22GR11 - 0AABW]^A__ "I'=}&MNNM).4UWq3r)s)s & #3Y"?@@I8)CCrd@rpilTr[ image_embedsnegative_image_embedsr^heightwidthnum_inference_stepsguidance_scaler\r2r9 output_typecallbackcallback_steps return_dictcjt|trd}nDt|trt|}nt dt ||j}|| z}|dk}|||| ||\}}}t|trtj |d}t|trtj |d}|r`| | d}| | d}tj ||gd |j |}|j |||j j}|jjj}t'|||j\}}|||||f|j || | |j } t-||D]g\}}|rtj | gdzn| }||d }|||||d d}|rx|| jdd\}}|d\}}|d\}} ||||z zz}tj || gd}t7|j jd r|j jjd vs%|| jdd\}}|j ||| | j} | /||zdkr&|t?|j ddz}!| |!|| i|j !| dd}"|"| dvrt d| | dvrl|"dzdz}"|"#dd}"|"$%dddd&'}"| dkr|(|"}"|s|"fStS|"S)a Function invoked when calling the pipeline for generation. Args: prompt (`str` or `List[str]`): The prompt or prompts to guide the image generation. image_embeds (`torch.Tensor` or `List[torch.Tensor]`): The clip image embeddings for text prompt, that will be used to condition the image generation. negative_image_embeds (`torch.Tensor` or `List[torch.Tensor]`): The clip image embeddings for negative text prompt, will be used to condition the image generation. negative_prompt (`str` or `List[str]`, *optional*): The prompt or prompts not to guide the image generation. Ignored when not using guidance (i.e., ignored if `guidance_scale` is less than `1`). height (`int`, *optional*, defaults to 512): The height in pixels of the generated image. width (`int`, *optional*, defaults to 512): The width in pixels of the generated image. num_inference_steps (`int`, *optional*, defaults to 100): The number of denoising steps. More denoising steps usually lead to a higher quality image at the expense of slower inference. guidance_scale (`float`, *optional*, defaults to 4.0): Guidance scale as defined in [Classifier-Free Diffusion Guidance](https://arxiv.org/abs/2207.12598). `guidance_scale` is defined as `w` of equation 2. of [Imagen Paper](https://arxiv.org/pdf/2205.11487.pdf). Guidance scale is enabled by setting `guidance_scale > 1`. Higher guidance scale encourages to generate images that are closely linked to the text `prompt`, usually at the expense of lower image quality. num_images_per_prompt (`int`, *optional*, defaults to 1): The number of images to generate per prompt. generator (`torch.Generator` or `List[torch.Generator]`, *optional*): One or a list of [torch generator(s)](https://pytorch.org/docs/stable/generated/torch.Generator.html) to make generation deterministic. latents (`torch.Tensor`, *optional*): Pre-generated noisy latents, sampled from a Gaussian distribution, to be used as inputs for image generation. Can be used to tweak the same generation with different prompts. If not provided, a latents tensor will ge generated by sampling using the supplied random `generator`. output_type (`str`, *optional*, defaults to `"pil"`): The output format of the generate image. Choose between: `"pil"` (`PIL.Image.Image`), `"np"` (`np.array`) or `"pt"` (`torch.Tensor`). callback (`Callable`, *optional*): A function that calls every `callback_steps` steps during inference. The function is called with the following arguments: `callback(step: int, timestep: int, latents: torch.Tensor)`. callback_steps (`int`, *optional*, defaults to 1): The frequency at which the `callback` function is called. If not specified, the callback is called at every step. return_dict (`bool`, *optional*, defaults to `True`): Whether or not to return a [`~pipelines.ImagePipelineOutput`] instead of a plain tuple. Examples: Returns: [`~pipelines.ImagePipelineOutput`] or `tuple` rz2`prompt` has to be of type `str` or `list` but is g?rrH)r4r3)r3r) text_embedsrsF)sampletimestepencoder_hidden_statesadded_cond_kwargsr| variance_type)learned learned_range)r2NorderT)force_not_quantizer)r>nprrzIOnly the output types `pt`, `pil` and `np` are supported not output_type=)rrrg?rrr)images)*rLrWrMr+r6rU_execution_devicernrNrZrTr7r4r% set_timesteps timestepsr$r, in_channelsrr.r: enumerate progress_barsplitr5chunkhasattrrstep prev_samplegetattrr&decodemaybe_free_model_hooksclampcpupermutefloatnumpy numpy_to_pilr)#r/r[rsrtr^rurvrwrxr\r2r9ryrzr{r|r_r3r]rerf_timesteps_tensornum_channels_latentsitlatent_model_inputr noise_pred variance_prednoise_pred_uncondnoise_pred_textvariance_pred_textstep_idximages# r__call__zKandinskyPipeline.__call__sR fc " " bJJ  % % bVJJ`RVW]R^R^``aa a'"77 &4s&:#7;7J7J F13NP_8 8 4 11 lD ) ) : 9\q999L +T 2 2 L$)I.C$K$K$K ! & '99:OUV9WWL$9$K$KLagh$K$i$i ! 9&;\%JPQRRRUU#)&VL $$%8$HHH>3#y/;#FE43IJJ && -vu = & ,    N   d//0@AABB$ /$ /DAq=X!eG9q=!9!9!9^e 0=| \ \ )&@"3! #  J+ P,6,<,(>%%.?UfCf1gg "Y 4F'GQOOO -?? JN)7;WWW * 0 0q1Aq 0 I I An))# *  #N(:a(?(? C CC1g...   T BB8L ##%%% 1 1 1vitvvww w - ' 'CK#%EKK1%%EIIKK''1a3399;;AACCE %  %%e,,E 8O"%0000r)N) NrororprqrNNrrNrT)__name__ __module__ __qualname____doc__model_cpu_offload_seqrrr rr r r r)r:rnrNno_gradrEXAMPLE_DOC_STRINGrWrTensorrintr Generatorrboolr __classcell__)r0s@rr!r!Ls\&7U&U'U# U  56 U  UUUUUU(   " dDdDdDdDLU]__122 <@#& #%&MQ*.%*GK !n1n1c49n%n1EL$u|*<<=n1 %U\4 3E%EF n1 "%T#Y"78 n1  n1n1!n1n1 #n1E%/43H"HIJn1%,'n1c]n18S#u|$rsY322222222222 4333333366666666.-----CCCCCCCC******  H % %>6666K1K1K1K1K1)K1K1K1K1K1r