wiCddlmZmZddlZddlmcmZddlmZddlm Z ddl m Z m Z ddl mZdd lmZmZdd lmZmZmZmZmZdd lmZmZdd lmZdd lmZmZddl m!Z!m"Z"Gddee eZ#Gddej$Z%Gddej$Z&Gddej$Z'Gddej$Z(dS))DictUnionN)nn checkpoint) ConfigMixinregister_to_config)PeftAdapterMixin)BasicTransformerBlockSkipFFTransformerBlock)ADDED_KV_ATTENTION_PROCESSORSCROSS_ATTENTION_PROCESSORSAttentionProcessorAttnAddedKVProcessor AttnProcessor)TimestepEmbeddingget_timestep_embedding) ModelMixin)GlobalResponseNormRMSNorm) Downsample2D Upsample2Dc.2eZdZdZe d1dededededededededededededededed ed!ed"ed#ed$ed%ed&ef,fd' Zd2d(ed)d*fd+Z d3d,Z e d)e e effd-Zd.eee e efffd/Zd0ZxZS)4 UVit2DModelTF@ r  ư>@ hidden_sizeuse_biashidden_dropoutcond_embed_dimmicro_cond_encode_dimmicro_cond_embed_dimencoder_hidden_size vocab_size codebook_size in_channelsblock_out_channelsnum_res_blocks downsampleupsampleblock_num_headsnum_hidden_layersnum_attention_headsattention_dropoutintermediate_sizelayer_norm_epsln_elementwise_affine sample_sizec ttj||_t |_t| | ||_t||z|_ t| | ||| d |_ t | |_ tj| |_tjfdt!|D|_t |_tj| |_t| | ||d| |_t+| | | |_d|_dS)Nbias)sample_proj_biasFcPg|]"}tzd#S)ada_norm_continuous)dimr:attention_head_dimdropoutcross_attention_dimattention_bias norm_type-ada_norm_continous_conditioning_embedding_dimnorm_elementwise_affinenorm_eps ada_norm_bias ff_inner_dimff_biasattention_out_bias)r ) .0_r,r*r<r=r>r:r+s n/root/.openclaw/workspace/chatterbox_venv_py311/lib/python3.11/site-packages/diffusers/models/unets/uvit_2d.py z(UVit2DModel.__init__..ksj   "!&#(;'26I'I*(3#+3BM,A+"*!2$'/   )r6r7)super__init__rLinear encoder_projrencoder_proj_layer_normUVit2DConvEmbedembedr cond_embed UVitBlock down_blockproject_to_hidden_normproject_to_hidden ModuleListrangetransformer_layersproject_from_hidden_normproject_from_hiddenup_block ConvMlmLayer mlm_layergradient_checkpointing)selfr*r+r,r-r.r/r0r1r2r3r4r5r6r7r8r9r:r;r<r=r>r? __class__s ``` ` ``` rUrYzUVit2DModel.__init__*sD I&9;XVVV'.{NLa'b'b$$ +Z9NP^`h   , > 1;QY   $     !         '..@.Rg&h&h#!#+={QY!Z!Z!Z"$-          "011#   # # ,)0 ^Mb(c(c%#%9[:LS[#\#\#\ !     !         &  X7Ln^k  ',###rWvaluereturnNcdSN)rmmoduleros rU_set_gradient_checkpointingz'UVit2DModel._set_gradient_checkpointings rWcT||}||}t||jjdd}||jddf}tj ||gd}| |j }| | |j }| |}|||||}|j\}} } } |dd d d|| | z| }||}||}|jD])|jr |jrfd } n} | |||d |i }*||}||}||| | | dd dd }|||||}||} | S)NTr)flip_sin_to_cosdownscale_freq_shiftrF)dtype)pooled_text_embencoder_hidden_statescross_attention_kwargsr rctg|RSrrr)argslayers rUlayer_z#UVit2DModel.forward..layer_s%e3d3333rWr})r~radded_cond_kwargs)r[r\rflattenconfigr.reshapeshapetorchcattor|r_r^rapermuterbrcrftrainingrlrgrhrirk)rm input_idsr~r} micro_condsrmicro_cond_embeds hidden_states batch_sizechannelsheightwidthrlogitsrs @rUforwardzUVit2DModel.forwards} $ 1 12G H H $ < <=R S S2    ! !4;#DVZqr   .55yq7I26NOO)_6G$HaPPP),,4:,>>///::==>S>YZZ 9--  +"7#9 (  /<.A+ Hfe%--aAq99AA*fW\n^fgg 33MBB ..}== ,  E} !< 444444"F&;'=#4o"F MM55mDD 00?? %--j&%RRZZ[\^_abdeff  +"7#9 &   .. rWci}dtdtjjdttt fffd |D]\}}||||S)z Returns: `dict` of attention processors: A dictionary containing all attention processors used in the model with indexed by its weight name. namert processorsct|dr|d||d<|D]\}}|d||||S)N get_processorT)return_deprecated_lora .processor.)hasattrrnamed_children)rrtrsub_namechildfn_recursive_add_processorss rUrz@UVit2DModel.attn_processors..fn_recursive_add_processorssv// d282F2F^b2F2c2c d.../#)#8#8#:#: U U%++t,@,@h,@,@%TTTT rW)strrrModulerrr)rmrrrtrs @rUattn_processorszUVit2DModel.attn_processorss  c 58? X\]`bt]tXu      !//11 B BLD& ' 'fj A A A ArW processorc zt|j}t|tr9t||kr&t dt|d|d|ddt dtjj ffd | D]\}}|||dS) a4 Sets the attention processor to use to compute attention. Parameters: processor (`dict` of `AttentionProcessor` or only `AttentionProcessor`): The instantiated processor class or a dictionary of processor classes that will be set as the processor for **all** `Attention` layers. If `processor` is a dict, the key needs to define the path to the corresponding cross attention processor. This is strongly recommended when setting trainable attention processors. z>A dict of processors was passed, but the number of processors z0 does not match the number of attention layers: z. Please make sure to pass z processor classes.rrtc,t|drVt|ts||n+|||d|D]\}}|d|||dS)N set_processorrr)r isinstancedictrpopr)rrtrrrfn_recursive_attn_processors rUrzCUVit2DModel.set_attn_processor..fn_recursive_attn_processorsv// M!)T22M((3333(($7J7J7J)K)KLLL#)#8#8#:#: T T%++t,@,@h,@,@%SSSS T TrWN) lenrkeysrr ValueErrorrrrrr)rmrcountrrtrs @rUset_attn_processorzUVit2DModel.set_attn_processorsD(--//00 i & & 3y>>U+B+BmQTU^Q_Q_mm05mmRWmmm   Tc T58? T T T T T T!//11 A ALD& ' 'fi @ @ @ @ A ArWc td|jDrt}ntd|jDrt }nCt dt t|j||dS)ze Disables custom attention processors and sets the default attention implementation. c32K|]}|jtvVdSrr)rnrrSprocs rU z9UVit2DModel.set_default_attn_processor..s*ii4t~!>>iiiiiirWc32K|]}|jtvVdSrr)rnrrs rUrz9UVit2DModel.set_default_attn_processor..s*hh$#==hhhhhhrWzOCannot call `set_default_attn_processor` when attention processors are of type N) allrvaluesrrrnextiterr)rmrs rUset_default_attn_processorz&UVit2DModel.set_default_attn_processors ii4K_KfKfKhKhiii i i ,..II hh$J^JeJeJgJghhh h h %IINbfgklpmAmHmHmJmJhKhKcLcLNN   *****rW)rFrrr r!rr"r#rrrFFr$r%r&rr'r(Tr))Frr)__name__ __module__ __qualname__ _supports_gradient_checkpointingr intboolfloatrYrurpropertyrrrrrrr __classcell__rns@rUrr'sE'+$  #!%($(#&!"% !!##%#&!% $&*?j,j,j, j,  j,j, #j,"j,!j,j,j,j,  !j,"#j,$%j,&'j,()j,,-j,.!/j,2!3j,67j,:;j,< $=j,>?j,j,j,j,j,j,X   $    ;;;;zc+=&=!>X0 AE2Dd3PbKbFc2c,d A A A AF + + + + + + +rWrc$eZdZfdZdZxZS)r]cttj|||_t ||||_tj||d||_dSNrz) kernel_sizerB) rXrYr Embedding embeddingsr layer_normConv2dconv)rmr3r4r1elementwise_affineepsrBrns rUrYzUVit2DConvEmbed.__init__%s` ,z;??!+s4FGGIk+=1SWXXX rWc||}||}|dddd}||}|S)Nrrrzr )rrrr)rmrrs rUrzUVit2DConvEmbed.forward+sT__Y// __Z00 ''1a33 YYz** rWrrrrYrrrs@rUr]r]$sLYYYYY rWr]c2eZdZdededeffd ZdZxZS)r`r5r6r7c  t| rtddddd |_nd|_t jfdt |D|_t j fdt |D|_| r tdddd dd |_ dSd|_ dS) NTrConv2d_0r rms_norm)use_convpaddingrrrKrrrBc 8g|]}tSrs) ConvNextBlock)rSirr,r*r=r>r+s rUrVz&UVitBlock.__init__..TsG   ")"    rWc Fg|]}tz S))rJrR)r)rSrTr;r8rr*r+s rUrVz&UVitBlock.__init__..bsZ   '#/%#+'/      rWrF) use_conv_transposerrrrKrrrB interpolate) rXrYrr6rrdre res_blocksattention_blocksrr7) rmrr5r*r,r>r=r+r8r;r6r7rns ` ``````` rUrYzUVitBlock.__init__4sl   #*$"#8   DOO#DO-         ~..      !#         ~..   ! ! "  !&#'$"#8!   DMMM!DMMMrWc|j||}t|j|jD]\}}|||}|j\}}} } |||| | zddd}||||}|ddd||| | }|j||}|S)Nrr rz)r~r)r6ziprrrviewrr7) rmxr}r~r res_blockattention_blockrrrrs rUrzUVitBlock.forwards ? &""A*-dot?T*U*U M M &I !_--A23' /J&%z8Ve^<<DDQ1MMA)>WmA !Q""'' HfeLLAA = $ a  ArW)rrrrrrYrrrs@rUr`r`3stL!L!L!L!L!L!L!L!L!\rWr`c(eZdZ dfd ZdZxZS)rcHttj||dd|||_t ||||_tj|t||z||_ tj |_ tt||z|_ tjt||z|||_tj||_tj||dz||_dS)Nrrz)rrgroupsrBrAr )rXrYrr depthwisernormrZrchannelwise_linear_1GELUchannelwise_actrchannelwise_normchannelwise_linear_2Dropoutchannelwise_dropoutcond_embeds_mapper) rmrr=r>r+r,r*res_ffn_factorrns rUrYzConvNextBlock.__init__s       Hn6KLL $&IhH~eZdZdedededededef fd ZdZxZS) rjr4r3r+r>r=r2cttj||d||_t ||||_tj||d||_dSr)rXrYrrconv1rrconv2)rmr4r3r+r>r=r2rns rUrYzConvMlmLayer.__init__si Y1;AT\]]] !+~?TUUY{MqxXXX rWc||}||dddddddd}||}|S)Nrr rrz)rrrr)rmrrs rUrzConvMlmLayer.forwardsf =11  (=(=aAq(I(IJJRRSTVWYZ\]^^ M** rW) rrrrrrrYrrrs@rUrjrjs Y Y Y Y $ Y  Y Y Y Y Y Y YrWrj))typingrrrtorch.nn.functionalr functionalrtorch.utils.checkpointrconfiguration_utilsr r loadersr attentionr rattention_processorrrrrrrrrmodeling_utilsr normalizationrrresnetrrrrr]r`rrjrsrWrUrsH  ------BBBBBBBB''''''EEEEEEEECBBBBBBB''''''77777777--------z+z+z+z+z+*k+;z+z+z+z     bi   ````` ```F*****BI***Z29rW