wi͇ddlZddlmZmZmZmZddlZddlZddl m Z m Z ddl m Z ddlmZmZmZ d d ZGd d ee ZdS)N)ListOptionalTupleUnion) ConfigMixinregister_to_config) deprecate)KarrasDiffusionSchedulersSchedulerMixinSchedulerOutput+?cosinec F|dkrd}n|dkrd}ntd|g}t|D]J}||z }|dz|z }|td||||z z |Kt j|tjS)a Create a beta schedule that discretizes the given alpha_t_bar function, which defines the cumulative product of (1-beta) over time from t = [0,1]. Contains a function alpha_bar that takes an argument t and transforms it to the cumulative product of (1-beta) up to that part of the diffusion process. Args: num_diffusion_timesteps (`int`): the number of betas to produce. max_beta (`float`): the maximum beta to use; use values lower than 1 to prevent singularities. alpha_transform_type (`str`, *optional*, default to `cosine`): the type of noise schedule for alpha_bar. Choose from `cosine` or `exp` Returns: betas (`np.ndarray`): the betas used by the scheduler to step the model outputs rc\tj|dzdz tjzdz dzS)NgMb?gT㥛 ?r)mathcospits ~/root/.openclaw/workspace/chatterbox_venv_py311/lib/python3.11/site-packages/diffusers/schedulers/scheduling_deis_multistep.py alpha_bar_fnz)betas_for_alpha_bar..alpha_bar_fn7s,8QY%/$'9A=>>!C Cexpc0tj|dzS)Ng()rrrs rrz)betas_for_alpha_bar..alpha_bar_fn<s8AI&& &rz"Unsupported alpha_transform_type: r dtype) ValueErrorrangeappendmintorchtensorfloat32)num_diffusion_timestepsmax_betaalpha_transform_typerbetasit1t2s rbetas_for_alpha_barr-s.x'' D D D D  & & ' ' ' 'T>RTTUUU E * + +MM ( (!e. . S\\"-- R0@0@@@(KKLLLL <U] 3 3 33rc"8eZdZdZdeDZdZe dDdede de de de e j dede dede de de de ded e ed!e d"ef d#Zed$Zed%ZdEd&efd'ZdFd(ed)ee ejffd*Zd+ejd,ejfd-Zd.Zd/Zd0ejd,ejfd1Zdd2d3ejd+ejd,ejfd4Zdd2d3ejd+ejd,ejfd5Zdd2d6eejd+ejd,ejfd7Z dd2d6eejd+ejd,ejfd8Z!dFd9Z"d:Z# dGd3ejd;ed+ejdZ'd?ejd@ejdAej(d,ejfdBZ)dCZ*dS)HDEISMultistepScheduleru `DEISMultistepScheduler` is a fast high order solver for diffusion ordinary differential equations (ODEs). This model inherits from [`SchedulerMixin`] and [`ConfigMixin`]. Check the superclass documentation for the generic methods the library implements for all schedulers such as loading and saving. Args: num_train_timesteps (`int`, defaults to 1000): The number of diffusion steps to train the model. beta_start (`float`, defaults to 0.0001): The starting `beta` value of inference. beta_end (`float`, defaults to 0.02): The final `beta` value. beta_schedule (`str`, defaults to `"linear"`): The beta schedule, a mapping from a beta range to a sequence of betas for stepping the model. Choose from `linear`, `scaled_linear`, or `squaredcos_cap_v2`. trained_betas (`np.ndarray`, *optional*): Pass an array of betas directly to the constructor to bypass `beta_start` and `beta_end`. solver_order (`int`, defaults to 2): The DEIS order which can be `1` or `2` or `3`. It is recommended to use `solver_order=2` for guided sampling, and `solver_order=3` for unconditional sampling. prediction_type (`str`, defaults to `epsilon`): Prediction type of the scheduler function; can be `epsilon` (predicts the noise of the diffusion process), `sample` (directly predicts the noisy sample`) or `v_prediction` (see section 2.4 of [Imagen Video](https://imagen.research.google/video/paper.pdf) paper). thresholding (`bool`, defaults to `False`): Whether to use the "dynamic thresholding" method. This is unsuitable for latent-space diffusion models such as Stable Diffusion. dynamic_thresholding_ratio (`float`, defaults to 0.995): The ratio for the dynamic thresholding method. Valid only when `thresholding=True`. sample_max_value (`float`, defaults to 1.0): The threshold value for dynamic thresholding. Valid only when `thresholding=True`. algorithm_type (`str`, defaults to `deis`): The algorithm type for the solver. lower_order_final (`bool`, defaults to `True`): Whether to use lower-order solvers in the final steps. Only valid for < 15 inference steps. use_karras_sigmas (`bool`, *optional*, defaults to `False`): Whether to use Karras sigmas for step sizes in the noise schedule during the sampling process. If `True`, the sigmas are determined according to a sequence of noise levels {σi}. timestep_spacing (`str`, defaults to `"linspace"`): The way the timesteps should be scaled. Refer to Table 2 of the [Common Diffusion Noise Schedules and Sample Steps are Flawed](https://huggingface.co/papers/2305.08891) for more information. steps_offset (`int`, defaults to 0): An offset added to the inference steps, as required by some model families. cg|] }|j S)name).0es r z!DEISMultistepScheduler.ys>>>qAF>>>rr -C6?{Gz?linearNrepsilonFףp= ??deislogrhoTlinspacernum_train_timesteps beta_startbeta_end beta_schedule trained_betas solver_orderprediction_type thresholdingdynamic_thresholding_ratiosample_max_valuealgorithm_type solver_typelower_order_finaluse_karras_sigmastimestep_spacing steps_offsetc6|&tj|tj|_n|dkr(tj|||tj|_nk|dkr1tj|dz|dz|tjdz|_n4|dkrt ||_nt |d|jd|jz |_tj |jd |_ tj |j |_ tj d |j z |_ tj|j tj|j z |_d |j z |j z dz|_d|_| d vr4| d vr|dnt | d|j| dvr5| dvr|dnt d| d|jd|_t)jd |d z |t(jddd}tj||_dg|z|_d |_d|_d|_|jd|_dS)Nrr9 scaled_linear?rsquaredcos_cap_v2z is not implemented for r<rdimr )r=) dpmsolverz dpmsolver++r=)rJ)r>)midpointheunbh1bh2r>)rKz solver type cpu)r#r$r%r)r?r-NotImplementedError __class__alphascumprodalphas_cumprodsqrtalpha_tsigma_tloglambda_tsigmasinit_noise_sigmar num_inference_stepsnpcopy from_numpy timesteps model_outputslower_order_nums _step_index _begin_indexto)selfr@rArBrCrDrErFrGrHrIrJrKrLrMrNrOrms r__init__zDEISMultistepScheduler.__init__|s(  $m5=IIIDJJ h & & H>QY^YfgggDJJ o - - C3H[chcpqqquvvDJJ 1 1 1,-@AADJJ%&`&`PTP^&`&`aa aDJ& #mDKQ???z$"566 z!d&9"9::  $,//%)DL2I2II D//43FF3N !$  ) )!===''v'>>>>)^*e*eUYUc*e*efff j ( (@@@''H'====)*n*n*n^b^l*n*nooo$( K#6#:.s)!Z!Z!Z%$"2"25*"E"E!Z!Z!Zrr}rr\)!configrNrjr?r@roundrkastypeint64arangerOrarrayrarMreflip_convert_to_karras concatenater%interplenr#rlrgrrrmrirErnrorprq)rsrir}rm step_ratiorg sigma_lastrs` @r set_timestepsz$DEISMultistepScheduler.set_timestepss ; ': 5 5 At{>BDWZ[D[\\2"s$!! I [ )Y 6 68=PST=TUJ1&9A&=>>KRRTTUYUYWYUYZ[^\^[^_ddffmmnpnvwwI 1 1II [ )Z 7 78;NNJ $+"A1zkRRXXZZ__aahhikiqrrI NII;/KKK A 33t7JJsRSS ; ( OJWV__))++F,,vSf,ggF!Z!Z!Z!Z!ZSY!Z!Z!Z[[aaccI^VVBCC[$9::AA"*MMFFYy")As6{{*C*CVLLFt21559LQ9OOTWWJ^Vj\$:;;BB2:NNF&v.. ))4477vU[7YY#&y>>   K $%!"  knnU++ rsamplereturnc,|j}|j^}}}|tjtjfvr|}|||tj|z}| }tj ||j j d}tj |d|j j}|d}tj || ||z }|j||g|R}||}|S)as "Dynamic thresholding: At each sampling step we set s to a certain percentile absolute pixel value in xt0 (the prediction of x_0 at timestep t), and if s > 1, then we threshold xt0 to the range [-s, s] and then divide by s. Dynamic thresholding pushes saturated pixels (those near -1 and 1) inwards, thereby actively preventing pixels from saturation at each step. We find that dynamic thresholding results in significantly better photorealism as well as better image-text alignment, especially when using very large guidance weights." https://arxiv.org/abs/2205.11487 r rT)r"max)rshaper#r%float64floatreshaperjprodabsquantilerrHclamprI unsqueezerr)rsrr batch_sizechannelsremaining_dims abs_sampless r_threshold_samplez(DEISMultistepScheduler._threshold_samples 06 - H~  6 6 6\\^^F Hrw~7N7N,NOOZZ\\ N:t{'MST U U U K 1$+6    KKNNVaR++a/ HF~FFF5!! rctjtj|d}||ddtjfz }tj|dkdd|jddz }|dz}||}||}||z ||z z } tj| dd} d| z |z| |zz} | |j} | S)Ng|=r)axisr)rr ) rjremaximumnewaxiscumsumargmaxcliprr) rsrr log_sigmadistslow_idxhigh_idxlowhighwrs rrz"DEISMultistepScheduler._sigma_to_t9sF2:eU3344 Jqqq"*}55)UaZq11188a8@@EE*JZ[\J]`aJaEbbQ;!(#9_t , GAq!  Ug H , IIek " "rc0d|dzdzdzz }||z}||fS)Nr rrRr1)rsrrcrds r_sigma_to_alpha_sigma_tz.DEISMultistepScheduler._sigma_to_alpha_sigma_tQs-q1 ,-'/rrczt|jdr |jj}nd}t|jdr |jj}nd}||n|d}||n|d}d}t jdd|}|d|z z}|d|z z}||||z zz|z} | S)z6Constructs the noise schedule of Karras et al. (2022). sigma_minN sigma_maxr[rg@r )hasattrrrritemrjr?) rsrrirrrhoramp min_inv_rho max_inv_rhorgs rrz)DEISMultistepScheduler._convert_to_karrasXs 4; , ,  -III 4; , ,  -III!*!6IIIbM4yy1}}a !RSSS   Z     DO,77>> ; &) 3 3, 66'AGG [ (H 4 4"GG [ (N : :&<)??GGBDK,GBBB  ; # 6,,W55G ; % / /Ww..'9 9%&OPP PrcXt|dkr|dn|dd}t|dkr|dn|dd}|+t|dkr |d}ntd|tdd d |tdd d |j|jdz|j|j}}||\} }||\} }tj| tj|z } tj| tj|z } | | z } |j j d kr'| | z |z|tj | d z z|zz }ntd|S)au One step for the first-order DEIS (equivalent to DDIM). Args: model_output (`torch.Tensor`): The direct output from the learned diffusion model. timestep (`int`): The current discrete timestep in the diffusion chain. prev_timestep (`int`): The previous discrete timestep in the diffusion chain. sample (`torch.Tensor`): A current instance of a sample created by the diffusion process. Returns: `torch.Tensor`: The sample tensor at the previous timestep. rrNr prev_timestepr0 missing `sample` as a required keyward argumentrmrrPassing `prev_timestep` is deprecated and has no effect as model output conversion is now handled via an internal counter `self.step_index`r=r<r) rrrr rgrwrr#rerrJrr])rsrrrrrrrdsigma_srcalpha_srflambda_shx_ts rdeis_first_order_updatez.DEISMultistepScheduler.deis_first_order_updates0"$ii!mm477J1M1M#&t99q==QfjjRV6W6W >4yy1}}a !STTT   Z     $ ^     ;t':;T[=Y77@@77@@9W%% '(:(::9W%% '(:(:: x  ; % / /W$.'UYq\\C=O2PT`1``CC%&OPP P rmodel_output_listct|dkr|dn|dd}t|dkr|dn|dd}|+t|dkr |d}ntd|tddd |tddd |j|jdz|j|j|j|jdz } }}||\} }||\} }|| \} } |d |d }} || z || z | | z }}}|jjd krOd}||||||||z }||||||||z }| || z || zz||zzz}|Std)a One step for the second-order multistep DEIS. Args: model_output_list (`List[torch.Tensor]`): The direct outputs from learned diffusion model at current and latter timesteps. sample (`torch.Tensor`): A current instance of a sample created by the diffusion process. Returns: `torch.Tensor`: The sample tensor at the previous timestep. r timestep_listNr rrrrPassing `timestep_list` is deprecated and has no effect as model output conversion is now handled via an internal counter `self.step_index`rr[r=c|tj| tj|zdz ztj|tj|z z S)Nr rjre)rbcs rind_fnzIDEISMultistepScheduler.multistep_deis_second_order_update..ind_fnsCRVAYYJ2Q6726!99rvayy;PQQrr rrrr rgrwrrrJr])rsrrrrrrrdsigma_s0sigma_s1rcalpha_s0alpha_s1m0m1rho_trho_s0rho_s1rcoef1coef2rs r"multistep_deis_second_order_updatez9DEISMultistepScheduler.multistep_deis_second_order_updates)($'t99q==QfjjRV6W6W #&t99q==QfjjRV6W6W >4yy1}}a !STTT  $ ^     $ ^    K!+ , K ( K!+ ,$  77@@!99(CC(!99(CC("2&(9"(=B '' 18h3FS[H[vv ; % / / R R RF5&&11FF6664R4RREF5&&11FF6664R4RREVh.;ebjHICJ%&OPP Prc\t|dkr|dn|dd}t|dkr|dn|dd}|+t|dkr |d}ntd|tddd |tddd |j|jdz|j|j|j|jdz |j|jdz f\}}} } ||\} }||\} }|| \} } || \}} |d |d |d }}}|| z || z | | z | |z f\}}}}|jjdkrvd}||||||||||z }||||||||||z }||||||||||z }| || z ||zz||zz||zzz}|Std)a One step for the third-order multistep DEIS. Args: model_output_list (`List[torch.Tensor]`): The direct outputs from learned diffusion model at current and latter timesteps. sample (`torch.Tensor`): A current instance of a sample created by diffusion process. Returns: `torch.Tensor`: The sample tensor at the previous timestep. rrNr rrz/ missing`sample` as a required keyward argumentrrrr[rr=c|tj|tj|tj|z dzztj|tj|zz tj|ztj|dzzdtj|zz dzz}tj|tj|z tj|tj|z z}||z S)Nr rr)rrrd numerator denominators rrzHDEISMultistepScheduler.multistep_deis_third_order_update..ind_fnjsF1IIRVAYY!6!:;fQii"&))+,fQii fQii1n%"&))m $    "vayy26!994RVAYY9NO  ;..rrr)rsrrrrrrrdrrsigma_s2rcrralpha_s2rrm2rrrrho_s2rrrcoef3rs r!multistep_deis_third_order_updatez8DEISMultistepScheduler.multistep_deis_third_order_update)s*$'t99q==QfjjRV6W6W #&t99q==QfjjRV6W6W >4yy1}}a !RSSS  $ ^     $ ^    K!+ , K ( K!+ , K!+ , 1 -8X 77@@!99(CC(!99(CC(!99(CC(&r*,=b,ACTUWCXB g  x  x  x  ) %vvv ; % / / / / /F5&&&99FF66SY[a#;;t~'<==#66x@@D   #0D   rr return_dictc|jtd|j|||jt |jdz ko#|jjot |jdk}|jt |jdz ko#|jjot |jdk}|||}t|jj dz D]}|j |dz|j |<||j d<|jj dks |j dks|r| ||}nV|jj dks |j dks|r||j |}n||j |}|j |jj kr|xj dz c_ |xjdz c_|s|fSt#|S) a Predict the sample from the previous timestep by reversing the SDE. This function propagates the sample with the multistep DEIS. Args: model_output (`torch.Tensor`): The direct output from learned diffusion model. timestep (`float`): The current discrete timestep in the diffusion chain. sample (`torch.Tensor`): A current instance of a sample created by the diffusion process. return_dict (`bool`): Whether or not to return a [`~schedulers.scheduling_utils.SchedulerOutput`] or `tuple`. Returns: [`~schedulers.scheduling_utils.SchedulerOutput`] or `tuple`: If return_dict is `True`, [`~schedulers.scheduling_utils.SchedulerOutput`] is returned, otherwise a tuple is returned where the first element is the sample tensor. NzaNumber of inference steps is 'None', you need to run 'set_timesteps' after creating the schedulerr rrr[) prev_sample)rirrwrrrmrrLrr rErnrorrrrpr) rsrrrrrLlower_order_secondr*rs rstepzDEISMultistepScheduler.steps 6  # +s  ? "  ! !( + + +_DN 3 3a 7 7 wT[=Z w_bcgcq_r_ruw_w _DN 3 3a 7 7 wT[=Z w_bcgcq_r_ruw_w 00f0MM t{/!344 > >A$($6q1u$=D q ! !!-2 ; #q ( (D,AA,E,EIZ,E66|F6SSKK [ % * *d.Ca.G.GK].GAA$BT]cAddKK@@AS\b@ccK  4;#; ; ;  ! !Q & ! ! A "> !;7777rc|S)a? Ensures interchangeability with schedulers that need to scale the denoising model input depending on the current timestep. Args: sample (`torch.Tensor`): The input sample. Returns: `torch.Tensor`: A scaled input sample. r1)rsrrrs rscale_model_inputz(DEISMultistepScheduler.scale_model_inputs  roriginal_samplesnoisermc j|j|j}|jjdkrft j|rRj|jt j ||jt j}n9j|j ||j}j  fd|D}n4j j g|j dz}nj g|j dz}|| }t|j t|j kr?|d}t|j t|j k?|\}}||z||zz} | S)Nrmpsrc<g|]}|Sr1)r)r3rrrss rr5z4DEISMultistepScheduler.add_noise..s*^^^qD33A7IJJ^^^rrr[)rgrrr}rtyper#is_floating_pointrmr%rzrwrflattenrrr) rsr r rmrg step_indicesrrcrd noisy_samplesrs ` @r add_noisez DEISMultistepScheduler.add_noises'7'>FVF\]]  " '5 0 0U5LY5W5W 0!%!2!23C3JRWR_!2!`!` ! %5%>"22Wu_D rc|jjSN)rr@rvs r__len__zDEISMultistepScheduler.__len__s {..r)r6r7r8r9Nrr:Fr;r<r=r>TFr?r)rr)T)+__name__ __module__ __qualname____doc__r _compatiblesorderr intrstrrrjndarrayboolrtpropertyrwrzr|rr#r}rrrrrrrrrrrrrrrr r  IntTensorrrr1rrr/r/Js,,\?>$=>>>L E$("%.2(",1"%$#"&,1 *#@,@, @,@, @,  @,   + @,@,@,@,%*@, @,@,@, @,$D>@, !@,"#@,@,@,@,D  X !!X!((3((((<,<,<,eCDU>V<,<,<,<,~ D0   ELRWR^: $ 7Q7Q7Ql7Q 7Q  7Q7Q7Q7Qz $ 888l8 8  8888| $ CQCQCQ -CQ CQ  CQCQCQCQR $ VQVQVQ -VQ VQ  VQVQVQVQr( 1 1 1"! ?8?8l?8?8 ?8  ?8 % & ?8?8?8?8B  %,     , | ?      D/////rr/)rr)rtypingrrrrnumpyrjr#configuration_utilsrr utilsr scheduling_utilsr r rr-r/r1rrr+s$ //////////// AAAAAAAAXXXXXXXXXX !)4)4)4)4XL /L /L /L /L /^[L /L /L /L /L /r