wij&ddlZddlmZddlmZmZmZddlZddlm Z ddl Z ddl mZmZddlmZmZddlmZmZddlmZmZddlmZmZmZmZdd lmZd d l m!Z!m"Z"d d l#m$Z$m%Z%m&Z&m'Z'm(Z(m)Z)d dl*m+Z+e)j,e-Z.Gdde(Z/dS)N)UnpicklingError)AnyDictUnion) FrozenDictunfreeze) from_bytesto_bytes) flatten_dictunflatten_dict) create_repohf_hub_download)EntryNotFoundErrorRepositoryNotFoundErrorRevisionNotFoundErrorvalidate_hf_hub_args) HTTPError) __version__is_torch_available) CONFIG_NAMEFLAX_WEIGHTS_NAMEHUGGINGFACE_CO_RESOLVE_ENDPOINT WEIGHTS_NAMEPushToHubMixinlogging)"convert_pytorch_state_dict_to_flaxc eZdZdZeZgdZgdZedZ dde e e fde jded efd Zdde e e fdefd Zdde e e fdefd Zdde e e fdefd Zdejd e fdZeee jfde eejfde jfdZ dde eejfde e e fdedefdZdS)FlaxModelMixina* Base class for all Flax models. [`FlaxModelMixin`] takes care of storing the model configuration and provides methods for loading, downloading and saving models. - **config_name** ([`str`]) -- Filename to save a model to when calling [`~FlaxModelMixin.save_pretrained`]. )_diffusers_version _class_name _name_or_path)nameparentdtypec ||fi|S)zZ All context managers that the model should be initialized under go here. )clsconfigkwargss t/root/.openclaw/workspace/chatterbox_venv_py311/lib/python3.11/site-packages/diffusers/models/modeling_flax_utils.py _from_configzFlaxModelMixin._from_config@s s6$$V$$$Nparamsr&maskreturnc&fd}|tj||St|}tj|\}}t ||D]\}} |r|| } || || <t |S)zk Helper method to cast floating-point values of given parameter `PyTree` to given `dtype`. ct|tjr9tj|jtjr|}|SN) isinstancejnpndarray issubdtyper&floatingastype)paramr&s r,conditional_castz:FlaxModelMixin._cast_floating_to..conditional_castMsD%-- ,#.cl2[2[ , U++Lr.)jaxtree_mapr tree_flattenzipkeysr ) selfr/r&r0r< flat_params flat_mask_maskedkeyr;s ` r,_cast_floating_toz FlaxModelMixin._cast_floating_toGs       << 0&99 9"6** '-- 1y+*:*:*<*<== ; ;KFC ;#C(#3#3E#:#: C k***r.cD||tj|S)a Cast the floating-point `params` to `jax.numpy.bfloat16`. This returns a new `params` tree and does not cast the `params` in place. This method can be used on a TPU to explicitly convert the model parameters to bfloat16 precision to do full half-precision training or to save weights in bfloat16 for inference in order to save memory and improve speed. Arguments: params (`Union[Dict, FrozenDict]`): A `PyTree` of model parameters. mask (`Union[Dict, FrozenDict]`): A `PyTree` with same structure as the `params` tree. The leaves should be booleans. It should be `True` for params you want to cast, and `False` for those you want to skip. Examples: ```python >>> from diffusers import FlaxUNet2DConditionModel >>> # load model >>> model, params = FlaxUNet2DConditionModel.from_pretrained("runwayml/stable-diffusion-v1-5") >>> # By default, the model parameters will be in fp32 precision, to cast these to bfloat16 precision >>> params = model.to_bf16(params) >>> # If you don't want to cast certain parameters (for example layer norm bias and scale) >>> # then pass the mask as follows >>> from flax import traverse_util >>> model, params = FlaxUNet2DConditionModel.from_pretrained("runwayml/stable-diffusion-v1-5") >>> flat_params = traverse_util.flatten_dict(params) >>> mask = { ... path: (path[-2] != ("LayerNorm", "bias") and path[-2:] != ("LayerNorm", "scale")) ... for path in flat_params ... } >>> mask = traverse_util.unflatten_dict(mask) >>> params = model.to_bf16(params, mask) ```)rHr6bfloat16rBr/r0s r,to_bf16zFlaxModelMixin.to_bf16_sJ%%fclDAAAr.cD||tj|S)a Cast the floating-point `params` to `jax.numpy.float32`. This method can be used to explicitly convert the model parameters to fp32 precision. This returns a new `params` tree and does not cast the `params` in place. Arguments: params (`Union[Dict, FrozenDict]`): A `PyTree` of model parameters. mask (`Union[Dict, FrozenDict]`): A `PyTree` with same structure as the `params` tree. The leaves should be booleans. It should be `True` for params you want to cast, and `False` for those you want to skip. Examples: ```python >>> from diffusers import FlaxUNet2DConditionModel >>> # Download model and configuration from huggingface.co >>> model, params = FlaxUNet2DConditionModel.from_pretrained("runwayml/stable-diffusion-v1-5") >>> # By default, the model params will be in fp32, to illustrate the use of this method, >>> # we'll first cast to fp16 and back to fp32 >>> params = model.to_f16(params) >>> # now cast back to fp32 >>> params = model.to_fp32(params) ```)rHr6float32rKs r,to_fp32zFlaxModelMixin.to_fp32s2%%fck4@@@r.cD||tj|S)a Cast the floating-point `params` to `jax.numpy.float16`. This returns a new `params` tree and does not cast the `params` in place. This method can be used on a GPU to explicitly convert the model parameters to float16 precision to do full half-precision training or to save weights in float16 for inference in order to save memory and improve speed. Arguments: params (`Union[Dict, FrozenDict]`): A `PyTree` of model parameters. mask (`Union[Dict, FrozenDict]`): A `PyTree` with same structure as the `params` tree. The leaves should be booleans. It should be `True` for params you want to cast, and `False` for those you want to skip. Examples: ```python >>> from diffusers import FlaxUNet2DConditionModel >>> # load model >>> model, params = FlaxUNet2DConditionModel.from_pretrained("runwayml/stable-diffusion-v1-5") >>> # By default, the model params will be in fp32, to cast these to float16 >>> params = model.to_fp16(params) >>> # If you want don't want to cast certain parameters (for example layer norm bias and scale) >>> # then pass the mask as follows >>> from flax import traverse_util >>> model, params = FlaxUNet2DConditionModel.from_pretrained("runwayml/stable-diffusion-v1-5") >>> flat_params = traverse_util.flatten_dict(params) >>> mask = { ... path: (path[-2] != ("LayerNorm", "bias") and path[-2:] != ("LayerNorm", "scale")) ... for path in flat_params ... } >>> mask = traverse_util.unflatten_dict(mask) >>> params = model.to_fp16(params, mask) ```)rHr6float16rKs r,to_fp16zFlaxModelMixin.to_fp16sJ%%fck4@@@r.rngc&td|)Nz.init_weights method has to be implemented for )NotImplementedError)rBrSs r, init_weightszFlaxModelMixin.init_weightss!"YSW"Y"YZZZr.pretrained_model_name_or_pathc j|dd}|dd}|dd}|dd}|dd} |dd} |d d} |d d} |d d} |d d}td dd}||j|f|d|| | | | | |d |\}}|j|f|dd|\}}||ntj||}tj|rn|rtjtj|tstdtd|dtj|t}ntjtj|tr'tj|t}ntjtj|trttd|dtdtdtd|d t||stnt||| | | | |||  }n#t$rt|dt$rt| d|dt$rt|dtdt $r}td|d |d}~wt"$r,td!t$d"|d#tdtd$ t$r'td%|d&|d'tdtd wxYw|r@t'rd(d)lm}ntd*||}t-||}n t/|d+5}t1||}dddn #1swxYwYn#t4t6jjf$r} t/|5}|d,rt?d-t"|#1swxYwYn(#t@t"f$rtd.|d/wxYwYd}~nd}~wwxYwtBj"#d0|}tI|}tCj%|j&tBj'(d12}tStItU|+}tItU|}|tS|+z }tS|+|z } |r(tX-d3|d4|d5||_.|+D]Q}!|!|vrK||!j/||!j/kr/t#d6|!d7||!j/d8||!j/d9R| D]}"||"=ta| d1kr n(tX3d?|j1j2d@ta|d1kr/tX-dA|j1j2dB|dC|dDn8tX3dE|j1j2dF|dG|j1j2dH|ti|fS)IaY Instantiate a pretrained Flax model from a pretrained model configuration. Parameters: pretrained_model_name_or_path (`str` or `os.PathLike`): Can be either: - A string, the *model id* (for example `runwayml/stable-diffusion-v1-5`) of a pretrained model hosted on the Hub. - A path to a *directory* (for example `./my_model_directory`) containing the model weights saved using [`~FlaxModelMixin.save_pretrained`]. dtype (`jax.numpy.dtype`, *optional*, defaults to `jax.numpy.float32`): The data type of the computation. Can be one of `jax.numpy.float32`, `jax.numpy.float16` (on GPUs) and `jax.numpy.bfloat16` (on TPUs). This can be used to enable mixed-precision training or half-precision inference on GPUs or TPUs. If specified, all the computation will be performed with the given `dtype`. This only specifies the dtype of the *computation* and does not influence the dtype of model parameters. If you wish to change the dtype of the model parameters, see [`~FlaxModelMixin.to_fp16`] and [`~FlaxModelMixin.to_bf16`]. model_args (sequence of positional arguments, *optional*): All remaining positional arguments are passed to the underlying model's `__init__` method. cache_dir (`Union[str, os.PathLike]`, *optional*): Path to a directory where a downloaded pretrained model configuration is cached if the standard cache is not used. force_download (`bool`, *optional*, defaults to `False`): Whether or not to force the (re-)download of the model weights and configuration files, overriding the cached versions if they exist. resume_download: Deprecated and ignored. All downloads are now resumed by default when possible. Will be removed in v1 of Diffusers. proxies (`Dict[str, str]`, *optional*): A dictionary of proxy servers to use by protocol or endpoint, for example, `{'http': 'foo.bar:3128', 'http://hostname': 'foo.bar:4012'}`. The proxies are used on each request. local_files_only(`bool`, *optional*, defaults to `False`): Whether to only load local model weights and configuration files or not. If set to `True`, the model won't be downloaded from the Hub. revision (`str`, *optional*, defaults to `"main"`): The specific model version to use. It can be a branch name, a tag name, a commit id, or any identifier allowed by Git. from_pt (`bool`, *optional*, defaults to `False`): Load the model weights from a PyTorch checkpoint save file. kwargs (remaining dictionary of keyword arguments, *optional*): Can be used to update the configuration object (after it is loaded) and initiate the model (for example, `output_attentions=True`). Behaves differently depending on whether a `config` is provided or automatically loaded: - If a configuration is provided with `config`, `kwargs` are directly passed to the underlying model's `__init__` method (we assume all relevant updates to the configuration have already been done). - If a configuration is not provided, `kwargs` are first passed to the configuration class initialization function [`~ConfigMixin.from_config`]. Each key of the `kwargs` that corresponds to a configuration attribute is used to override said attribute with the supplied `kwargs` value. Remaining keys that do not correspond to any configuration attribute are passed to the underlying model's `__init__` function. Examples: ```python >>> from diffusers import FlaxUNet2DConditionModel >>> # Download model and configuration from huggingface.co and cache. >>> model, params = FlaxUNet2DConditionModel.from_pretrained("runwayml/stable-diffusion-v1-5") >>> # Model was saved using *save_pretrained('./test/saved_model/')* (for example purposes, not runnable). >>> model, params = FlaxUNet2DConditionModel.from_pretrained("./test/saved_model/") ``` If you get the error message below, you need to finetune the weights for your downstream task: ```bash Some weights of UNet2DConditionModel were not initialized from the model checkpoint at runwayml/stable-diffusion-v1-5 and are newly initialized because the shapes did not match: - conv_in.weight: found shape torch.Size([320, 4, 3, 3]) in the checkpoint and torch.Size([320, 9, 3, 3]) in the model instantiated You should probably TRAIN this model on a down-stream task to be able to use it for predictions and inference. ``` r*N cache_dirforce_downloadFfrom_ptresume_downloadproxieslocal_files_onlytokenrevision subfoldermodelflax) diffusers file_type frameworkT) rYreturn_unused_kwargsrZr\r]r^r_r`ra)r&rgzError no file named z found in directory  z file found in directory z-. Please load the model using `from_pt=True`.z or .) filenamerYrZr]r\r^r_ user_agentrar`z is not a local folder and is not a valid model identifier listed on 'https://huggingface.co/models' If this is a private repository, make sure to pass a token having permission to this repo with `token` or log in with `huggingface-cli login`.z is not a valid git identifier (branch name, tag name or commit id) that exists for this model name. Check the model page at 'https://huggingface.co/z' for available revisions.z& does not appear to have a file named z:There was a specific connection error when trying to load z: zWe couldn't connect to 'zM' to load this model, couldn't find it in the cached files and it looks like z8 is not the path to a directory containing a file named z. Checkout your internet connection or see how to run the library in offline mode at 'https://huggingface.co/docs/transformers/installation#offline-mode'.zCan't load the model for 'z'. If you were trying to load it from 'https://huggingface.co/models', make sure you don't have a local directory with the same name. Otherwise, make sure 'z=' is the correct path to a directory containing a file named r)load_state_dictz|Can't load the model in PyTorch format because PyTorch is not installed. Please, install PyTorch or use native Flax weights.rbversionzYou seem to have cloned a repository without having git-lfs installed. Please install git-lfs and run `git lfs install` followed by `git lfs pull` in the folder you cloned.zUnable to convert z to Flax deserializable object. c^tj|tjddS)Ncpu)backendr)r= device_put local_devices)xs r,z0FlaxModelMixin.from_pretrained..s'3CT]bCcCcCcdeCf1g1gr.r)rSzThe checkpoint z is missing required keys: zI. Make sure to call model.init_weights to initialize the missing weights.z)Trying to load the pretrained weight for z failed: checkpoint has shape z, which is incompatible with the model shape z. z(Some weights of the model checkpoint at z! were not used when initializing z: z, - This IS expected if you are initializing zU from the checkpoint of a model trained on another task or with another architecture.z9All model checkpoint weights were used when initializing z. zSome weights of z3 were not initialized from the model checkpoint at z and are newly initialized: zo You should probably TRAIN this model on a down-stream task to be able to use it for predictions and inference.zAll the weights of z/ were initialized from the model checkpoint at zf. If your task is similar to the task the model of the checkpoint was trained on, you can already use z* for predictions without further training.)5popr load_config from_configospathjoinisdirisfilerEnvironmentErrorrrrrrr ValueErrorrrmodeling_utilsrlropenr readrmsgpack exceptions ExtraData startswithOSErrorUnicodeDecodeErrorr= tree_utilr>r eval_shaperVrandomPRNGKeysetrrAloggerwarning _missing_keysshapelen __class____name__infor )#r)rWr& model_argsr+r*rYrZr[r\r]r^r_r`rark unused_kwargsrb model_kwargspretrained_path_with_subfolder model_fileerrrlpytorch_model_filestatestate_fefparams_shape_treerequired_params shape_state missing_keysunexpected_keysrGunexpected_keys# r,from_pretrainedzFlaxModelMixin.from_pretraineds[ xHd++JJ{D11 $4e<<**Y.. **%6==**Y--!::&8%@@ 7D))::j$//JJ{D11 %    >$3CO- %#%)- /!1!# % % % % !FM.cofnEX\nn`mnn|   * );YGG ' 7==7 8 8I  w~~bgll3QS_&`&`aa*r|rrQorrr W\\*H,WW  -KM^ _ _`` W\\*HJ[\\  -K\ Z Z[[ &#-->\--- '9+<99,995999 3 ,16=O..<'#1#$3%5)'%    +   &4 )   &ii/Liii &   &4pp\mppp   &Qn   &]/N]]>[]]:K]]Q]]]]$   &V1NVV-JVV0AVVGSVVV   n!## ;;;;;;;&J "1!#>?? &UZZ\\):):: ejjll++o=  - NNZ"?ZZ\hZZZ   !-C ::<<  Ck!!eCj&6+c:J:P&P&P pppSz'ppU`adUeUkppp . & &Nn%%   ! # # NN.;X..!&!9..=L..!&!9...     KKqTYTcTlqqq r r r |  q NNn5?#;nn1nnO[nnn     KKeo&>18=8P   nU++++s (K55A&OM00A(OQ).#Q Q)Q!!Q)$Q!%Q))TS?SS SS ST %TT  TTFsave_directoryis_main_process push_to_hubc tj|r td|ddStj|d|r|dd}|dd}|d d}|d d} |d |tjjd } t| d|| j } |} |r| |tj |t} t| d5} t|}| |dddn #1swxYwYtd| |r||| | ||dSdS)a Save a model and its configuration file to a directory so that it can be reloaded using the [`~FlaxModelMixin.from_pretrained`] class method. Arguments: save_directory (`str` or `os.PathLike`): Directory to save a model and its configuration file to. Will be created if it doesn't exist. params (`Union[Dict, FrozenDict]`): A `PyTree` of model parameters. is_main_process (`bool`, *optional*, defaults to `True`): Whether the process calling this is the main process or not. Useful during distributed training and you need to call this function on all processes. In this case, set `is_main_process=True` only on the main process to avoid race conditions. push_to_hub (`bool`, *optional*, defaults to `False`): Whether or not to push your model to the Hugging Face model hub after saving it. You can specify the repository you want to push to with `repo_id` (will default to the name of `save_directory` in your namespace). kwargs (`Dict[str, Any]`, *optional*): Additional key word arguments passed along to the [`~utils.PushToHubMixin.push_to_hub`] method. zProvided path (z#) should be a directory, not a fileNT)exist_okcommit_messageprivateF create_prr_repo_id)rrr_wbzModel weights saved in )r_rr)ryrzr}rerrormakedirsrvsplitsepr r save_configr{rrr writer_upload_folder)rBrr/rrr+rrrr_r model_to_saveoutput_model_filer model_bytess r,save_pretrainedzFlaxModelMixin.save_pretraineds8 7>>. ) )  LL^>^^^ _ _ _ F NT2222  `#ZZ(8$??NjjE22G ;66IJJw--EjjN,@,@,M,Mb,QRRG!'D'QVWWW_G   6  % %n 5 5 5GLL9JKK #T * * !a"6**K GGK  ! ! ! ! ! ! ! ! ! ! ! ! ! ! !  A.?AABBB     -#        s%FF  F r4)TF)r __module__ __qualname____doc__r config_name_automatically_saved_args_flax_internal_args classmethodr-rrrr6r&rrHrLrOrRr=ArrayrVrrNstrryPathLikerboolrr(r.r,r r 2s4K V V V555%%[% ++dJ.>(?+ +Y\+hk++++0%B%BeD*$45%BS%B%B%B%BNAAeD*$45ASAAAA6%A%AeD*$45%AS%A%A%A%AN[ [d[[[[;g,g,',S"+-='>g,yg,g,g,[g,Z !%! @@c2;./@dJ&'@ @  @@@@@@r.r )0rypicklertypingrrrr= jax.numpynumpyr6msgpack.exceptionsrflax.core.frozen_dictrrflax.serializationr r flax.traverse_utilr r huggingface_hubr rhuggingface_hub.utilsrrrrrequestsrrrutilsrrrrrrmodeling_flax_pytorch_utilsr get_loggerrrr r(r.r,rs  """"""########## 6666666633333333;;;;;;;;88888888 ........LKKKKK  H % %DDDDD^DDDDDr.