Vapoursynth SwinIR (mlrt)
SwinIR (mlrt) runs the color denoising and JPEG artifact removal models of SwinIR, a Swin Transformer restoration network, as ONNX models through vs-mlrt at the input size. It works on NVIDIA and Intel hardware and on the CPU, but the network is large and practical only on a GPU; the super-resolution models are in Vapoursynth SwinIR Resize (mlrt).
Filtering → Vapoursynth → Other

What it does
Every frame is restored on its own by the chosen model. The denoise models are trained for one noise level (15, 25 or 50), the JPEG models for one JPEG quality (10 to 40); pick the one closest to the source.
Requirements
- Hybrid's vs-mlrt add-on with the SwinIR (mlrt) models and at least one backend.
- GPU backends: TensorRT and ONNX (CUDA) for NVIDIA, TensorRT RTX for RTX cards, OpenVINO (GPU) for Intel; on macOS ONNX (CoreML).
Settings
| Control | Parameter | Range / values | Hybrid default | Meaning |
|---|---|---|---|---|
| Model | model | denoise 15, denoise 25, denoise 50, compression artifact removal jpeg10, jpeg20, jpeg30, jpeg40 | denoise 15 | Network: color denoising for a noise level, or JPEG artifact removal for a quality. |
| Backend | backend | ONNX (CPU/CUDA), OpenVINO (CPU/GPU), TensorRT, TensorRT RTX (macOS: ONNX CoreML/CPU) | ONNX (CPU) | Runtime that executes the network; only installed backends are listed. |
| Tiles | tiles | 1 – 64 | 1 | Number of tiles the frame is split into to save memory; 1 = whole frame. |
| Tile pad | overlap | 0 – 64 | 16 | Overlap of the tiles; only with more than one tile. |
| FP16 | fp16 | on/off | off | Half precision; greyed out for the CPU backends, which run at full precision. The GPU backends, OpenVINO (GPU) included, allow it. |
| BF16 | bf16 | on/off | off | bfloat16; TensorRT and OpenVINO (CPU) only. Switches FP16 off. |
| Device | device_id | 0 – 4 | 0 | GPU to use; not for CPU and CoreML backends. |
| Streams | num_streams | 1 – 64 | 1 | Parallel streams; TensorRT, TensorRT RTX, OpenVINO (GPU) and CoreML only. |
| CUDA Graph | use_cuda_graph | on/off | off | TensorRT, TensorRT RTX and ONNX (CUDA) only. |
| TRT backend optimization | builder_optimization_level | 0 – 5 | 3 | TensorRT engine build effort. |
| Dynamic Shapes (opt/min/max) | opt_shapes, min_shapes, max_shapes | 0 – 8192 | unset | TensorRT engine sizes; with a minimum size the engine becomes dynamic. |
Example
Default settings on a 10-bit YUV 4:2:0 source (640x356):
clip = core.std.AddBorders(clip=clip, left=0, right=0, top=2, bottom=2) # add borders to achieve mod 8 (vsSwinIRmlrt) - 640x360 # adjusting color space from YUV420P10 to RGBS for vsSwinIRmlrt clip = core.resize.Bicubic(clip=clip, format=vs.RGBS, matrix_in_s="470bg", range_in_s="limited", range_s="full") # denoising using SwinIR (mlrt) from vsmlrt import Backend clip = vsmlrt.SwinIR(clip, model=17, backend=Backend.ORT_CPU(fp16=False,output_format=0)) clip = core.std.Crop(clip=clip, left=0, right=0, top=2, bottom=2) # removing added borders from mod requirement (vsSwinIRmlrt) - 640x356
How Hybrid calls it
- The clip is converted to 32-bit float RGB (16-bit float with TensorRT and FP16), full range, and padded to mod 8.
- Model and backend options are always written; tiles and tile pad only when they differ from the defaults.
- TensorRT builds an engine on the first run for each size and stores it in the engine path from Hybrid's path settings (or the temp folder). TensorRT cannot be used in vs-mlrt filters and torch filters in the same script.
- SwinIR (mlrt) also offers the common options (Masked, ApplyOnlyTo, Merge, OnRegion, Gimmick).
Background
SwinIR
SwinIR (Liang et al., ICCV workshops 2021) is an image restoration network built from Swin Transformer blocks. Besides super-resolution it has models for gray and color denoising and for gray and color JPEG artifact reduction; this filter uses the color ones.
vs-mlrt
vs-mlrt runs ONNX models in VapourSynth with several backends. Its SwinIR wrapper expects 16- or 32-bit float input and uses an overlap of 16 between tiles unless told otherwise.
Sources
- vs-mlrt README
- SwinIR
- Hybrid's tooltips