#
IP-Adapter

IP-Adapter is a lightweight adapter that enables prompting a diffusion model with an image. This method decouples the cross-attention layers of the image and text features. The image features are generated from an image encoder. Files generated from IP-Adapter are only ~100MBs.

Learn how to load an IP-Adapter checkpoint and image in the IP-Adapter loading guide.

##
IPAdapterMixin

###
class diffusers.loaders.IPAdapterMixin

< source >

( )

Mixin for handling IP Adapters.

####
load_ip_adapter

< source >

( pretrained_model_name_or_path_or_dict: typing.Union[str, typing.Dict[str, torch.Tensor]] subfolder: str weight_name: str **kwargs )

Parameters

- pretrained_model_name_or_path_or_dict (`str` or `os.PathLike` or `dict`) — Can be either:
  - A string, the model id (for example `google/ddpm-celebahq-256`) of a pretrained model hosted on the Hub.
  - A path to a directory (for example `./my_model_directory`) containing the model weights saved with ModelMixin.save_pretrained().
  - A torch state dict.
- cache_dir (`Union[str, os.PathLike]`, optional) — Path to a directory where a downloaded pretrained model configuration is cached if the standard cache is not used.
- force_download (`bool`, optional, defaults to `False`) — Whether or not to force the (re-)download of the model weights and configuration files, overriding the cached versions if they exist.
- resume_download (`bool`, optional, defaults to `False`) — Whether or not to resume downloading the model weights and configuration files. If set to `False`, any incompletely downloaded files are deleted.
- proxies (`Dict[str, str]`, optional) — A dictionary of proxy servers to use by protocol or endpoint, for example, `{'http': 'foo.bar:3128', 'http://hostname': 'foo.bar:4012'}`. The proxies are used on each request.
- local_files_only (`bool`, optional, defaults to `False`) — Whether to only load local model weights and configuration files or not. If set to `True`, the model won’t be downloaded from the Hub.
- token (`str` or bool, optional) — The token to use as HTTP bearer authorization for remote files. If `True`, the token generated from `diffusers-cli login` (stored in `~/.huggingface`) is used.
- revision (`str`, optional, defaults to `"main"`) — The specific model version to use. It can be a branch name, a tag name, a commit id, or any identifier allowed by Git.
- subfolder (`str`, optional, defaults to `""`) — The subfolder location of a model file within a larger model repository on the Hub or locally.

####
unload_ip_adapter

< source >

( )

Unloads the IP Adapter weights

Examples:

    >>> # Assuming `pipeline` is already loaded with the IP Adapter weights.
    >>> pipeline.unload_ip_adapter()
    >>> ...