Diffusers Dynamic Offloader

DDO is a model-agnostic offload manager for low-VRAM Diffusers-style inference. It provides dense-weight routes and official Diffusers compatibility routes while preserving quantized backend math.

Getting started

Follow the installation guide, then choose full-pipeline, staged or Modular Diffusers integration.

Normal linear use requires no native allocator or compiler. Native VMM is optional: it uses a matching packaged DLL/.so and bootstrap before CUDA initialization. Experimental version 0.1.0rc3 is available on PyPI. The installation guide uses published packages, without requiring a source clone or local wheel files.

from diffusers_dynamic_offloader import enable_offload

result = enable_offload(transformer, preset="auto", component="transformer")
transformer = result.module

This snippet selects the default linear route. See native VMM integration for its separate loading contract.

Documentation

Read the documentation website or browse the Markdown guides below.

Examples and benchmarks

Model runners, custom-block integration and measured comparisons live in DDO examples. See the test hardware and support boundaries when interpreting results. Selected workloads do not establish universal speedups or model support.

License

Apache-2.0; preserve required copyright, license and attribution notices.