wivkddlmZmZmZmZmZddlZddlmZm Z ddl m Z ddl m Z mZddlmZddlmZmZmZdd lmZd d lmZmZejeZd ZddZGddee ZdS))CallableDictListOptionalUnionN)T5EncoderModel T5Tokenizer)LoraLoaderMixin)Kandinsky3UNetVQModel) DDPMScheduler) deprecateloggingreplace_example_docstring) randn_tensor)DiffusionPipelineImagePipelineOutputa Examples: ```py >>> from diffusers import AutoPipelineForText2Image >>> import torch >>> pipe = AutoPipelineForText2Image.from_pretrained( ... "kandinsky-community/kandinsky-3", variant="fp16", torch_dtype=torch.float16 ... ) >>> pipe.enable_model_cpu_offload() >>> prompt = "A photograph of the inside of a subway train. There are raccoons sitting on the seats. One of them is reading a newspaper. The window shows the city in the background." >>> generator = torch.Generator(device="cpu").manual_seed(0) >>> image = pipe(prompt, num_inference_steps=25, generator=generator).images[0] ``` cz||dzz}||dzzdkr|dz }||dzz}||dzzdkr|dz }||z||zfS)Nrr)heightwidth scale_factor new_height new_widths /root/.openclaw/workspace/chatterbox_venv_py311/lib/python3.11/site-packages/diffusers/pipelines/kandinsky3/pipeline_kandinsky3.pydownscale_height_and_widthr (sm<?*J a1$$a q(I |Q!##Q  $i,&> >>c%@eZdZdZgdZdedededede f fd Z d Z e j d*dee jdee jdee jdee jfdZdZ d+dZedZedZedZe j eed ddd d ddd d d d d dd d d dgfdeeeefdeded eeeeefd!eed"eed#eed$eee jee jfdee jdee jdee jdee jd%eed&ed'ee eee!gd fd(eef d)Z"xZ#S),Kandinsky3Pipelineztext_encoder->unet->movq)latents prompt_embedsnegative_prompt_embedsnegative_attention_maskattention_mask tokenizer text_encoderunet schedulermovqc|t||||||dS)N)r)r*r+r,r-)super__init__register_modules)selfr)r*r+r,r- __class__s rr0zKandinsky3Pipeline.__init__<sO  lQZae      r!c|rktj||dk||dk<|ddz}|ddd|f}|ddd|f}||fS)Nrr)torch zeros_likesummax)r2 embeddingsr( cut_contextmax_seq_lengths rprocess_embedsz!Kandinsky3Pipeline.process_embedsJs  @.3.>z.\]J]?^._._J~* ++//337799A=N#AAA$67J+AAA,>?N>))r!TrNFr%r&r(r'c :|P|Nt|t|ur0tdt|dt|d||j}|t|trd} n4|%t|t rt |} n |jd} d} |||d| d d } | j |}| j |} | || }|d}| || |\}} || d z}|j |j j}nd}| ||}|j\}}}|d|d}|||z|d}| |d} |rF|C|dg| z}nWt|tr|g}n>| t |kr)t%d|dt |d|d| d |}|||ddd d d }|j |}|j |} | || }|d}|ddd|jdf}| ddd|jdf} || d z}n(t'j|}t'j| } |r||jd}| ||}|j|jkrG|d|d}|| |z|d}| |d} nd}d} ||| | fS)aX Encodes the prompt into text encoder hidden states. Args: prompt (`str` or `List[str]`, *optional*): prompt to be encoded device: (`torch.device`, *optional*): torch device to place the resulting embeddings on num_images_per_prompt (`int`, *optional*, defaults to 1): number of images that should be generated per prompt do_classifier_free_guidance (`bool`, *optional*, defaults to `True`): whether to use classifier free guidance or not negative_prompt (`str` or `List[str]`, *optional*): The prompt or prompts not to guide the image generation. If not defined, one has to pass `negative_prompt_embeds`. instead. If not defined, one has to pass `negative_prompt_embeds`. instead. Ignored when not using guidance (i.e., ignored if `guidance_scale` is less than `1`). prompt_embeds (`torch.Tensor`, *optional*): Pre-generated text embeddings. Can be used to easily tweak text inputs, *e.g.* prompt weighting. If not provided, text embeddings will be generated from `prompt` input argument. negative_prompt_embeds (`torch.Tensor`, *optional*): Pre-generated negative text embeddings. Can be used to easily tweak text inputs, *e.g.* prompt weighting. If not provided, negative_prompt_embeds will be generated from `negative_prompt` input argument. attention_mask (`torch.Tensor`, *optional*): Pre-generated attention mask. Must provide if passing `prompt_embeds` directly. negative_attention_mask (`torch.Tensor`, *optional*): Pre-generated negative attention mask. Must provide if passing `negative_prompt_embeds` directly. Nz?`negative_prompt` should be the same type to `prompt`, but got z != .rr max_lengthTpt)paddingrA truncationreturn_tensors)r(r)dtypedevicer5z`negative_prompt`: z has batch size z, but `prompt`: zT. Please make sure that passed `negative_prompt` matches the batch size of `prompt`.)rCrArDreturn_attention_maskrE)type TypeError_execution_device isinstancestrlistlenshaper) input_idstor(r*r= unsqueezerFrepeatview ValueErrorr6r7)r2promptdo_classifier_free_guidancenum_images_per_promptrGnegative_promptr%r& _cut_contextr(r' batch_sizerA text_inputstext_input_idsrFbs_embedseq_len_ uncond_tokens uncond_inputs r encode_promptz Kandinsky3Pipeline.encode_promptRsT  /"=F||4#8#888(VZ[jVkVk((V ((( >+F  *VS"9"9 JJ  Jvt$<$< VJJ&,Q/J  ..$%# )K)255f==N(7::6BBN ---.M*!,M,0,?,? ~_k,l,l )M>)N,D,DQ,G,GGM   (%+EEE%((uV(DD ,2'1%,,Q0EqII %**86K+KWVXYY '../DaHH && K+A+I&!#z 1 OS11 0!0 1 s?3333 3/333K_K_33330:333 !0 *#~~!("#*.#' .   ".!7!:!:6!B!B*6*E*H*H*P*P')-):):"#:*;**&*@)B&)?C[]EXYZE[C[@[)\&*A!!!E]}GZ[\G]E]B]*^')?BYBcBcdeBfBf)f&&*/)9-)H)H&*/*:>*J*J' & +,215G%;%>%>USY%>%Z%Z "%+}/BBB)?)F)FqJ_ab)c)c&)?)D)DZRgEgiprt)u)u&*A*H*HI^`a*b*b' &* "&* #4nF]]]r!c|t||||}n:|j|krtd|jd|||}||jz}|S)N) generatorrGrFzUnexpected latents shape, got z , expected )rrQrWrSinit_noise_sigma)r2rQrFrGrgr$r,s rprepare_latentsz"Kandinsky3Pipeline.prepare_latentssr ?"5IfTYZZZGG}%% !c'-!c!c\a!c!cdddjj((GI66r!c |>t|tr|dkr#td|dt|d|At fd|Ds&tdjdfd|D||td |d |d ||td |It|t s4t|tstd t|||td|d|d |2|0|j|jkr td|jd|jd||td|B|@|jdd|jkr(td|jddd|jd||td|B|B|jdd|jkr,td|jddd|jddSdSdS)Nrz5`callback_steps` has to be a positive integer but is z of type r?c3*K|] }|jvVdSN_callback_tensor_inputs.0kr2s r z2Kandinsky3Pipeline.check_inputs..DF F 23A- -F F F F F F r!2`callback_on_step_end_tensor_inputs` has to be in , but found c&g|] }|jv |Srrmros r z3Kandinsky3Pipeline.check_inputs..VpHpHpHvwbckokGbGbGpqbGbGbGr!zCannot forward both `prompt`: z and `prompt_embeds`: z2. Please make sure to only forward one of the two.zeProvide either `prompt` or `prompt_embeds`. Cannot leave both `prompt` and `prompt_embeds` undefined.z2`prompt` has to be of type `str` or `list` but is z'Cannot forward both `negative_prompt`: z and `negative_prompt_embeds`: zu`prompt_embeds` and `negative_prompt_embeds` must have the same shape when passed directly, but got: `prompt_embeds` z != `negative_prompt_embeds` zLPlease provide `negative_attention_mask` along with `negative_prompt_embeds`rz`negative_prompt_embeds` and `negative_attention_mask` must have the same batch_size and token length when passed directly, but got: `negative_prompt_embeds` z != `negative_attention_mask` z:Please provide `attention_mask` along with `prompt_embeds`z`prompt_embeds` and `attention_mask` must have the same batch_size and token length when passed directly, but got: `prompt_embeds` z != `attention_mask` ) rMintrWrJallrnrNrOrQ) r2rXcallback_stepsr[r%r&"callback_on_step_end_tensor_inputsr(r's ` r check_inputszKandinsky3Pipeline.check_inputss  %z.#/N/N %R`deReRe,,,((,,,  . 9#F F F F 7YF F F C C 9JTEaJJpHpHpHpH|^pHpHpHJJ   -";000}000 ^ 5w  FC)@)@ TZ\`IaIa `RVW]R^R^``aa a  &+A+M_/__*___   $)?)K"&<&BBB 8-:-@88.4888 " -2I2Qkll l ! -2I2U%+BQB/3J3PPP 96L6RSUTUSU6V99/5999  $)?YZZ Z  $)C"2A2&.*>>> 0-:-@!-D00&,000 % $)C)C>>r!c|jSrl_guidance_scaler2s rguidance_scalez!Kandinsky3Pipeline.guidance_scale9s ##r!c|jdkS)Nrrrs rrYz.Kandinsky3Pipeline.do_classifier_free_guidance=s#a''r!c|jSrl)_num_timestepsrs r num_timestepsz Kandinsky3Pipeline.num_timestepsAs ""r!g@ipilr$rXnum_inference_stepsrr[rZrrrg output_type return_dictcallback_on_step_endr|c p |dd}|dd}|tddd|tddd|Atfd|Ds&tdjd fd |Dd }j}|||| | || | |_|t|trd }n4|%t|trt|}n | j d } |j|||| | || |  \} } } } jr>tj| | g} tj| | g} j||jj}t+||d\}}||zd||f| j|||j}t1dr jjt||jjzz }t|_|5}t=|D]\}}jrtj|gdzn|}||| | dd }jr&| d\}} |dz| z||zz }j!||||j"}|i}!|D]}"tG|"|!|"<||||!}#|#d|}|#d| } |#d| } |#d| } |#d| } |t|d z ks|d z|kr[|d zjjzd krE|$|/||zd kr&|tKjdd z}$||$||| dvrtd | | d!ksj&'|d "d#}%| d$vrl|%d%zd%z}%|%(d d }%|%)*d dd&d +,}%| d'kr-|%}%n|}%.|s|%fcdddSt_|%(cdddS#1swxYwYdS))u" Function invoked when calling the pipeline for generation. Args: prompt (`str` or `List[str]`, *optional*): The prompt or prompts to guide the image generation. If not defined, one has to pass `prompt_embeds`. instead. num_inference_steps (`int`, *optional*, defaults to 25): The number of denoising steps. More denoising steps usually lead to a higher quality image at the expense of slower inference. timesteps (`List[int]`, *optional*): Custom timesteps to use for the denoising process. If not defined, equal spaced `num_inference_steps` timesteps are used. Must be in descending order. guidance_scale (`float`, *optional*, defaults to 3.0): Guidance scale as defined in [Classifier-Free Diffusion Guidance](https://arxiv.org/abs/2207.12598). `guidance_scale` is defined as `w` of equation 2. of [Imagen Paper](https://arxiv.org/pdf/2205.11487.pdf). Guidance scale is enabled by setting `guidance_scale > 1`. Higher guidance scale encourages to generate images that are closely linked to the text `prompt`, usually at the expense of lower image quality. negative_prompt (`str` or `List[str]`, *optional*): The prompt or prompts not to guide the image generation. If not defined, one has to pass `negative_prompt_embeds` instead. Ignored when not using guidance (i.e., ignored if `guidance_scale` is less than `1`). num_images_per_prompt (`int`, *optional*, defaults to 1): The number of images to generate per prompt. height (`int`, *optional*, defaults to self.unet.config.sample_size): The height in pixels of the generated image. width (`int`, *optional*, defaults to self.unet.config.sample_size): The width in pixels of the generated image. eta (`float`, *optional*, defaults to 0.0): Corresponds to parameter eta (η) in the DDIM paper: https://arxiv.org/abs/2010.02502. Only applies to [`schedulers.DDIMScheduler`], will be ignored for others. generator (`torch.Generator` or `List[torch.Generator]`, *optional*): One or a list of [torch generator(s)](https://pytorch.org/docs/stable/generated/torch.Generator.html) to make generation deterministic. prompt_embeds (`torch.Tensor`, *optional*): Pre-generated text embeddings. Can be used to easily tweak text inputs, *e.g.* prompt weighting. If not provided, text embeddings will be generated from `prompt` input argument. negative_prompt_embeds (`torch.Tensor`, *optional*): Pre-generated negative text embeddings. Can be used to easily tweak text inputs, *e.g.* prompt weighting. If not provided, negative_prompt_embeds will be generated from `negative_prompt` input argument. attention_mask (`torch.Tensor`, *optional*): Pre-generated attention mask. Must provide if passing `prompt_embeds` directly. negative_attention_mask (`torch.Tensor`, *optional*): Pre-generated negative attention mask. Must provide if passing `negative_prompt_embeds` directly. output_type (`str`, *optional*, defaults to `"pil"`): The output format of the generate image. Choose between [PIL](https://pillow.readthedocs.io/en/stable/): `PIL.Image.Image` or `np.array`. return_dict (`bool`, *optional*, defaults to `True`): Whether or not to return a [`~pipelines.stable_diffusion.IFPipelineOutput`] instead of a plain tuple. callback (`Callable`, *optional*): A function that will be called every `callback_steps` steps during inference. The function will be called with the following arguments: `callback(step: int, timestep: int, latents: torch.Tensor)`. callback_steps (`int`, *optional*, defaults to 1): The frequency at which the `callback` function will be called. If not specified, the callback will be called at every step. clean_caption (`bool`, *optional*, defaults to `True`): Whether or not to clean the caption before creating embeddings. Requires `beautifulsoup4` and `ftfy` to be installed. If the dependencies are not installed, the embeddings will be created from the raw prompt. cross_attention_kwargs (`dict`, *optional*): A kwargs dictionary that if specified is passed along to the `AttentionProcessor` as defined under `self.processor` in [diffusers.models.attention_processor](https://github.com/huggingface/diffusers/blob/main/src/diffusers/models/attention_processor.py). Examples: Returns: [`~pipelines.ImagePipelineOutput`] or `tuple` callbackNr{z1.0.0zhPassing `callback` as an input argument to `__call__` is deprecated, consider use `callback_on_step_end`znPassing `callback_steps` as an input argument to `__call__` is deprecated, consider use `callback_on_step_end`c3*K|] }|jvVdSrlrmros rrrz.Kandinsky3Pipeline.__call__..rsr!rtruc&g|] }|jv |Srrmros rrwz/Kandinsky3Pipeline.__call__..rxr!Trr)rZrGr[r%r&r\r(r')rGrtext_encoder_offload_hook)totalrF)encoder_hidden_statesencoder_attention_maskrg?)rgr$r%r&r(r'order)rBnprlatentzSOnly the output types `pt`, `pil`, `np` and `latent` are supported not output_type=r)force_not_quantizesample)rrg?r r)images)0poprrzrWrnrLr}rrMrNrOrPrQrerYr6catboolr, set_timesteps timestepsr rirFhasattrroffloadrr progress_bar enumerater+chunkstep prev_samplelocalsupdategetattrr-decodeclampcpupermutefloatnumpy numpy_to_pilmaybe_free_model_hooksr)&r2rXrrr[rZrrrgr%r&r(r'rrr$rr|kwargsrr{r;rGr]rnum_warmup_stepsritlatent_model_input noise_prednoise_pred_uncondnoise_pred_textcallback_kwargsrqcallback_outputsstep_idximages&` r__call__zKandinsky3Pipeline.__call__Es(@::j$//$4d;;   z     %  A    . 9#F F F F 7YF F F C C 9JTEaJJpHpHpHpH|^pHpHpHJJ  '      " .  #  .  *VS"9"9 JJ  Jvt$<$< VJJ&,Q/JZ^YkYk   ,"7+'#9$)$;Zl Z Z V -~?V  + Y!I'=}&MNNM"Y(?'PQQVVXXN $$%8$HHHN, 365!DD && / /FE B      N    44 5 5 5$:X:d  * 2 2 4 4 4y>>,?$.BV,VV!)nn   %8  9 9E 5\!),,* 7* 71AEAa%nUYy1}%=%=%=gn""YY&*7+9 % '   3o9C9I9I!9L9L6%"03"6/!IN]nLn!nJ.--' .  (3&(O?99-3XXa[**';';D!Q'X'X$.229gFFG$4$8$8-$X$XM-=-A-ABZ\r-s-s*%5%9%9:JN%[%[N.>.B.BC\^u.v.v+I***A9I/I/IqSTuX\XfXlNlpqNqNq '')))+N0Ba0G0G#$(K(K#K 1g666"??? wjuww(** ((T(JJ8T-//!CK#-E!KK1--E!IIKK//1a;;AACCIIKKE%'' --e44E  ' ' ) ) ) xGE 5E 5E 5E 5E 5E 5E 5E 5J'e444KE 5E 5E 5E 5E 5E 5E 5E 5E 5E 5E 5E 5E 5E 5E 5E 5E 5E 5sJ#T+T++T/2T/) TrNNNNFNN)NNNNNN)$__name__ __module__ __qualname__model_cpu_offload_seqrnr rr rr r0r=r6no_gradrTensorrerir}propertyrrYrrEXAMPLE_DOC_STRINGrrNrryr Generatorrrrr __classcell__)r3s@rr#r#2sq6    %    !         ***U]__%)049=15:>S^S^  -S^!) 6S^!.S^"*%,!7S^S^S^_S^j   #+/ $DDDDL$$X$((X(##X#U]__122)-#% #;?/0 $#MQ049=15:>%* KO9B %y5y5c49n%y5!y5 y5 "%T#Y"78 y5 (} y5 y5}y5E%/43H"HIJy5  -y5!) 6y5!.y5"*%,!7y5c]y5y5"'xc40@$0F'GH#y5$-1I%y5y5y532_y5y5y5y5y5r!r#)r) typingrrrrrr6 transformersrr loadersr modelsr r schedulersrutilsrrrutils.torch_utilsrpipeline_utilsrr get_loggerrloggerrr r#rr!rrs[88888888888888 44444444&&&&&&--------'''''' .-----CCCCCCCC  H % %(????N5N5N5N5N5*ON5N5N5N5N5r!