wiAddlmZddlmZmZmZmZddlZddlm cm Z ddl m Z ddlmZddlmZmZddlmZmZmZmZmZmZd d lmZerdd lmZmZdd lm Z m!Z!m"Z"m#Z#ej$e%Z&Gd dZ'dS))Path)DictListOptionalUnionN)validate_hf_hub_args) safe_open)_LOW_CPU_MEM_USAGE_DEFAULTload_state_dict)USE_PEFT_BACKEND_get_model_fileis_accelerate_availableis_torch_versionis_transformers_availablelogging)_maybe_expand_lora_scales)CLIPImageProcessorCLIPVisionModelWithProjection) AttnProcessorAttnProcessor2_0IPAdapterAttnProcessorIPAdapterAttnProcessor2_0c eZdZdZe d deeeeeee j ffdeeeefdeeeefde efdZ dZ d Zd S) IPAdapterMixinzMixin for handling IP Adapters. image_encoder%pretrained_model_name_or_path_or_dict subfolder weight_nameimage_encoder_folderc t|ts|g}t|ts|g}t|dkr|t|z}t|ts|g}t|dkr|t|z}t|t|krtdt|t|krtd|dd}|dd}|dd}|d d} |d d} |d d} |d d} |d t } | r*t sd} td| durtddstdddd}g}t|||D]\}}}t|tst|||||| | | | || }|driid}t|dd5}|D]}|dr3|||d|dd<J|dr2|||d |dd< dddn #1swxYwYnt)|}n|}t|}|dd gkrtd!||t-|d"rt/|d"d|t|tstd#||d$d%kr#t5||}n!t5|}t9j||| &|j|j '}|!|(n)td)td*t-|d+r5t/|d+d$tE}|!|,t-|d-st/||j#n|j$}|%|| .|&|}|ikrtNstd/dSt/|d0i}|(D]D\}}d1||vr6|)|d1|2|*d1|gd3g4CdSdS)5ac Parameters: pretrained_model_name_or_path_or_dict (`str` or `List[str]` or `os.PathLike` or `List[os.PathLike]` or `dict` or `List[dict]`): Can be either: - A string, the *model id* (for example `google/ddpm-celebahq-256`) of a pretrained model hosted on the Hub. - A path to a *directory* (for example `./my_model_directory`) containing the model weights saved with [`ModelMixin.save_pretrained`]. - A [torch state dict](https://pytorch.org/tutorials/beginner/saving_loading_models.html#what-is-a-state-dict). subfolder (`str` or `List[str]`): The subfolder location of a model file within a larger model repository on the Hub or locally. If a list is passed, it should have the same length as `weight_name`. weight_name (`str` or `List[str]`): The name of the weight file to load. If a list is passed, it should have the same length as `weight_name`. image_encoder_folder (`str`, *optional*, defaults to `image_encoder`): The subfolder location of the image encoder within a larger model repository on the Hub or locally. Pass `None` to not load the image encoder. If the image encoder is located in a folder inside `subfolder`, you only need to pass the name of the folder that contains image encoder weights, e.g. `image_encoder_folder="image_encoder"`. If the image encoder is located in a folder other than `subfolder`, you should pass the path to the folder that contains image encoder weights, for example, `image_encoder_folder="different_subfolder/image_encoder"`. cache_dir (`Union[str, os.PathLike]`, *optional*): Path to a directory where a downloaded pretrained model configuration is cached if the standard cache is not used. force_download (`bool`, *optional*, defaults to `False`): Whether or not to force the (re-)download of the model weights and configuration files, overriding the cached versions if they exist. resume_download: Deprecated and ignored. All downloads are now resumed by default when possible. Will be removed in v1 of Diffusers. proxies (`Dict[str, str]`, *optional*): A dictionary of proxy servers to use by protocol or endpoint, for example, `{'http': 'foo.bar:3128', 'http://hostname': 'foo.bar:4012'}`. The proxies are used on each request. local_files_only (`bool`, *optional*, defaults to `False`): Whether to only load local model weights and configuration files or not. If set to `True`, the model won't be downloaded from the Hub. token (`str` or *bool*, *optional*): The token to use as HTTP bearer authorization for remote files. If `True`, the token generated from `diffusers-cli login` (stored in `~/.huggingface`) is used. revision (`str`, *optional*, defaults to `"main"`): The specific model version to use. It can be a branch name, a tag name, a commit id, or any identifier allowed by Git. low_cpu_mem_usage (`bool`, *optional*, defaults to `True` if torch version >= 1.9.0 else `False`): Speed up model loading only loading the pretrained weights and not initializing the weights. This also tries to not use more than 1x model size in CPU memory (including peak memory) while loading the model. Only supported for PyTorch >= 1.9.0. If you are using an older version of PyTorch, setting this argument to `True` will raise an error. rzT`weight_name` and `pretrained_model_name_or_path_or_dict` must have the same length.z8`weight_name` and `subfolder` must have the same length. cache_dirNforce_downloadFresume_downloadproxieslocal_files_onlytokenrevisionlow_cpu_mem_usagea,Cannot initialize model with low cpu memory usage because `accelerate` was not found in the environment. Defaulting to `low_cpu_mem_usage=False`. It is strongly recommended to install `accelerate` for faster and less memory-intense model loading. You can do so with: ``` pip install accelerate ``` .Tz>=z1.9.0z~Low memory initialization requires torch >= 1.9.0. Please either update your PyTorch version or set `low_cpu_mem_usage=False`.attn_procs_weightspytorch) file_type framework) weights_namer#r$r%r&r'r(r)r user_agentz .safetensors) image_proj ip_adapterptcpu)r.devicez image_proj.r1z ip_adapter.r2zNRequired keys are (`image_proj` and `ip_adapter`) missing from the state dict.rzloading image_encoder from /r)rr*)dtyperza`image_encoder` cannot be loaded because `pretrained_model_name_or_path_or_dict` is a state dict.zimage_encoder is not loaded since `image_encoder_folder=None` passed. You will not be able to use `ip_adapter_image` when calling the pipeline with IP-Adapter.Use `ip_adapter_image_embeds` to pass pre-generated image embedding instead.feature_extractorr:unet)r*z/PEFT backend is required to load these weights. peft_configfaceid_) adapter_nameg?)adapter_weights)+ isinstancelistlen ValueErrorpopr rloggerwarningrNotImplementedErrorzipdictrendswithr keys startswith get_tensorreplacer appendhasattrgetattrinfocountras_posixrfrom_pretrainedtor5r8register_modulesr unet_namer<_load_ip_adapter_weights_load_ip_adapter_lorasr itemsload_lora_weights set_adapters)selfrrr r!kwargsr#r$r%r&r'r(r)r*r0 state_dicts model_file state_dictfkeyrLimage_encoder_subfolderrr:r< extra_lorasr=kloras l/root/.openclaw/workspace/chatterbox_venv_py311/lib/python3.11/site-packages/diffusers/loaders/ip_adapter.pyload_ip_adapterzIPAdapterMixin.load_ip_adapter6s|+t,, (&-K?FF \5Z4[ 1 4 5 5 : :4Y\_`k\l\l4l 1)T** $" I y>>Q  !C $4$44I {  s#HII I Istt t {  s9~~ - -WXX XJJ{D11 $4e<< **%6==**Y--!::&8$?? 7D))::j$//"JJ':%>  %  NN.     $ $-=dG-L-L $%.  ."   MP 1; N N A KA K I 1; CTJJ C,9!,'#1$3#%5%')    ''77 =02"!E!EJ":eLLLmPQ#$6688mmC"~~m<<m[\[g[ghk[l[l < 8]TV9W9W X X!$ !>!>m[\[g[ghk[l[l < 8]TV9W9W X mmmmmmmmmmmmmmmm"1!> "  # RPQQQQQ&dM2>> *0022RRGAt$}}K77..t-A--.PPP))=Q==/C5)QQQ   RRs0B'L$$L( +L( c t|dst||jn|j}t |t s|g}t ||d}|jD]\}}t |ttfrt|t|j kr5tdt|dt|j dt|dkr|t|j z}t|D]^\}}t |tr:|D]$\}} ||r | |j |<%T||j |<_dS) a Set IP-Adapter scales per-transformer block. Input `scale` could be a single config or a list of configs for granular control over each IP-Adapter behavior. A config can be a float or a dictionary. Example: ```py # To use original IP-Adapter scale = 1.0 pipeline.set_ip_adapter_scale(scale) # To use style block only scale = { "up": {"block_0": [0.0, 1.0, 0.0]}, } pipeline.set_ip_adapter_scale(scale) # To use style+layout blocks scale = { "down": {"block_2": [0.0, 1.0]}, "up": {"block_0": [0.0, 1.0, 0.0]}, } pipeline.set_ip_adapter_scale(scale) # To use style and layout from 2 reference images scales = [{"down": {"block_2": [0.0, 1.0]}}, {"up": {"block_0": [0.0, 1.0, 0.0]}}] pipeline.set_ip_adapter_scale(scales) ``` r<g) default_scalezCannot assign z scale_configs to z IP-Adapter.rN)rQrRrYr<rArBrattn_processorsr\rrrCscalerD enumeraterJrM) r_ror< scale_configs attn_nameattn_processori scale_configrhss rjset_ip_adapter_scalez#IPAdapterMixin.set_ip_adapter_scales<5>> # Assuming `pipeline` is already loaded with the IP Adapter weights. >>> pipeline.unload_ip_adapter() >>> ... ``` rNr9safety_checkerr:r;scaled_dot_product_attention)rQrRrregister_to_configr:r<encoder_hid_projconfigencoder_hid_dim_typernr\FrrrArr __class__set_attn_processor)r_ attn_procsnamevalueattn_processor_classs rjunload_ip_adapterz IPAdapterMixin.unload_ip_adapter,sp 4 ) ) @gdOT.R.R.^!%D   # #4, # ? ? ?t-.. Ht011 HgdDWY]6^6^6j)-&''4,'GGG&* "+/ ( 94::<<  KD%&-a1O&P&Pe """VcVeVe ! e&<>W%XYY'$$__&& t   $$Z00000rxNr9)__name__ __module__ __qualname____doc__rrstrrrtorchTensorrrkrwrrxrjrr3s)) /> RR/4S$s)T#u|J[E\5\/]Rd3i(R3S >* R 'sm RRRRB2?2?2?h'1'1'1'1'1rxr)(pathlibrtypingrrrrrtorch.nn.functionalnn functionalrhuggingface_hub.utilsr safetensorsr models.modeling_utilsr r utilsr rrrrrunet_loader_utilsr transformersrrmodels.attention_processorrrrr get_loggerrrFrrrxrjrs............ 666666!!!!!!OOOOOOOO988888    H % %`1`1`1`1`1`1`1`1`1`1rx