Skip to content

Spandrel gives your project support for various PyTorch architectures meant for AI Super-Resolution, restoration, and inpainting. Based on the model support implemented in chaiNNer.

License

Notifications You must be signed in to change notification settings

cozy-creator/spandrel

 
 

Repository files navigation

Spandrel

PyPI package version number PyPi Downloads Python Version Documentation

Actions Status License Contributors

Spandrel is a library for loading and running pre-trained PyTorch models. It automatically detects the model architecture and hyperparameters from model files, and provides a unified interface for running models.

After seeing many projects extract out chaiNNer's model support into their own projects, I decided to create this PyPi package for the architecture support and model loading functionality. I'm also hoping that by having a central package anyone can use, the community will be encouraged to help add support for more models.

This package does not yet have easy inference code, but porting that code is planned as well.

Installation

Spandrel is available through pip:

pip install spandrel

Basic Usage

While Spandrel supports different kinds of models, this is how you would run a super resolution model (e.g. ESRGAN, SwinIR, HAT, etc.):

from spandrel import ImageModelDescriptor, ModelLoader
import torch

# load a model from disk
model = ModelLoader().load_from_file(r"path/to/model.pth")

# make sure it's an image to image model
assert isinstance(model, ImageModelDescriptor)

# send it to the GPU and put it in inference mode
model.cuda().eval()

# use the model
def process(image: torch.Tensor) -> torch.Tensor:
    with torch.no_grad():
        return model(image)

Note that model is a ModelDescriptor object, which is a wrapper around the actual PyTorch model. This wrapper provides a unified interface for running models, and also contains metadata about the model. See ImageModelDescriptor for more details about the metadata contained and how to call the model.

NOTE: ImageModelDescriptor will NOT convert an image to a tensor for you. It is purely making the forward passes of these models more convenient to use, since the actual forward passes are not always as simple as image in/image out.

More architectures

If you are working on a non-commercial open-source project or a private project, you should use spandrel and spandrel_extra_arches to get everything spandrel has to offer. The spandrel package only contains architectures with permissive and public domain licenses (MIT, Apache 2.0, public domain), so it is fit for every use case. Architectures with restrictive licenses (e.g. non-commercial) are implemented in the spandrel_extra_arches package.

from spandrel import ImageModelDescriptor, MAIN_REGISTRY, ModelLoader
from spandrel_extra_arches import EXTRA_REGISTRY
import torch

# add extra architectures before `ModelLoader` is used
MAIN_REGISTRY.add(*EXTRA_REGISTRY)

# load a model from disk
model = ModelLoader().load_from_file(r"path/to/model.pth")

... # use model

Supported File Types

Spandrel mainly supports loading .pth files for all supported architectures. This is what you will typically find from official repos and community trained models. However, Spandrel also supports loading TorchScript traced models (.pt), certain types of .ckpt files, and .safetensors files for any supported architecture saved in one of these formats.

Model Architecture Support

NOTE: By its very nature, Spandrel will never be able to support every model architecture. The goal is just to support as many as is realistically possible.

Spandrel currently supports a limited amount of network architectures. If the architecture you need is not supported, feel free to request it or try adding it.

Single Image Super Resolution

Face Restoration

Inpainting

Denoising

DeJPEG

Colorization

Dehazing

Low-light Enhancement

(All architectures marked with a + are only part of spandrel_extra_arches.)

Security

Use .safetensors files for guaranteed security.

As you may know, loading .pth files poses a security risk due to python's pickle module being inherently unsafe and vulnerable to arbitrary code execution (ACE). To mitigate this, Spandrel only allows deserializing certain types of data. This helps to improve security, but it still doesn't fully solve the issue of ACE.

Used By

Here are some cool projects that use Spandrel:

License

This repo is bounded by the MIT license. However, the code of implemented architectures (everything inside an arch/ directory) is bound by their original respective licenses (which are included in their respective arch/ directories).

About

Spandrel gives your project support for various PyTorch architectures meant for AI Super-Resolution, restoration, and inpainting. Based on the model support implemented in chaiNNer.

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages

  • Python 100.0%