Vapoursynth SCUNet (mlrt): Difference between revisions
New filter page (via create-page on MediaWiki MCP Server) |
torch filters are Windows only (via update-page on MediaWiki MCP Server) |
||
| Line 12: | Line 12: | ||
* Hybrid's vs-mlrt add-on with the SCUNet models and at least one backend. | * Hybrid's vs-mlrt add-on with the SCUNet models and at least one backend. | ||
* GPU backends: TensorRT and ONNX (CUDA) for NVIDIA, OpenVINO (GPU) for Intel; on macOS ONNX (CoreML). | * GPU backends: TensorRT and ONNX (CUDA) for NVIDIA, OpenVINO (GPU) for Intel; on macOS ONNX (CoreML). | ||
* Fix Color needs the torch add-on. | * Fix Color needs the torch add-on (Windows only). | ||
== Settings == | == Settings == | ||
Revision as of 08:03, 15 November 2026
SCUNet (mlrt) runs the SCUNet denoising network as an ONNX model through vs-mlrt. It works with NVIDIA and Intel hardware and on the CPU (slow), and can restore the original colors afterwards (ColorFix).
Filtering → Vapoursynth → DeNoise

What it does
Every frame is passed through SCUNet ("Swin-Conv-UNet"). The real models handle real-world noise of unknown strength; the color and gray models are trained for Gaussian noise of a fixed level. The gray models only filter the luma plane; the chroma planes are kept.
Requirements
- Hybrid's vs-mlrt add-on with the SCUNet models and at least one backend.
- GPU backends: TensorRT and ONNX (CUDA) for NVIDIA, OpenVINO (GPU) for Intel; on macOS ONNX (CoreML).
- Fix Color needs the torch add-on (Windows only).
Settings
| Control | Parameter | Range / values | Hybrid default | Meaning |
|---|---|---|---|---|
| Model | model | color_15/25/50, real_psnr, real_gan, gray_15/25/50 | real_psnr | Network to use; real_psnr keeps the colors better than real_gan. |
| Tiles | tiles | 1 – 64 | 1 | Splits the frame to save memory. |
| Tile pad | overlap | 0 – 64 | 64 | Overlap of the tiles. |
| Backend | backend | ONNX (CUDA/CPU), OpenVINO (GPU/CPU), TensorRT, TensorRT RTX (macOS: ONNX CoreML/CPU) | ONNX (CPU) | Runtime that executes the network. |
| Device | device_id | 0 – 4 | 0 | GPU to use; not for CPU and CoreML backends. |
| Streams | num_streams | 1 – 64 | 1 | Parallel streams; TensorRT, TensorRT RTX, OpenVINO (GPU) and CoreML only. |
| FP16 / BF16 | fp16, bf16 | on/off | off / off | Half precision; BF16 only with TensorRT and OpenVINO (CPU). The two exclude each other. |
| CUDA Graph | use_cuda_graph | on/off | off | TensorRT, TensorRT RTX and ONNX (CUDA) only. |
| TRT backend optimization | builder_optimization_level | 0 – 5 | 3 | TensorRT engine build effort. |
| Dynamic Shapes (min/opt/max) | min_shapes, opt_shapes, max_shapes | 0 – 8192 | unset | TensorRT engine sizes. |
Fix Color: the same options as in SCUNet (torch) (average or wavelet mode, radius, fast, wavelets, plane).
Example
Default settings on a 10-bit YUV 4:2:0 source (640×356, padded to mod 8):
clip = core.std.AddBorders(clip=clip, left=0, right=0, top=2, bottom=2) # add borders to achieve mod 8 (vsSCUNetmlrt) - 640x360
# changing range from limited to full range for vsSCUNetmlrt
clip = core.resize.Bicubic(clip, format=vs.YUV420P10, range_in_s="limited", range_s="full")
# setting color range to PC (full) range.
prop_name = '_Range' if core.core_version.release_major >= 74 else '_ColorRange'
clip = core.std.SetFrameProps(clip=clip, **{prop_name: vs.RANGE_FULL})
# adjusting color space from YUV420P10 to RGBS for vsSCUNetmlrt
clip = core.resize.Bicubic(clip=clip, format=vs.RGBS, matrix_in_s="470bg", range_in_s="full", range_s="full")
# denoising using SCUNet (mlrt)
from vsmlrt import Backend
clip = vsmlrt.SCUNet(clip, model=3, backend=Backend.ORT_CPU(fp16=False,output_format=0))
clip = core.std.Crop(clip=clip, left=0, right=0, top=2, bottom=2) # removing added borders from mod requirement (vsSCUNetmlrt) - 640x356How Hybrid calls it
- The clip is padded to mod 8 and converted to 32-bit float RGB (16-bit float with FP16); for the gray models a float YUV clip is used and only its luma plane is filtered.
- The model and the backend options are always written; tiles and tile pad only when they differ from the defaults.
- TensorRT builds an engine on the first run for each size and cannot be mixed with torch TensorRT filters in the same script.
- SCUNet (mlrt) also offers the common options (Masked, ApplyOnlyTo, Merge, OnRegion, Gimmick).
Background
SCUNet
SCUNet (Zhang et al., Machine Intelligence Research 2023) combines Swin Transformer and residual convolution branches in a UNet and was trained only on synthetic degradations. vs-mlrt chooses real_psnr as its default model because the GAN model can shift colors.