Vapoursynth SCUNet: Difference between revisions

From wiki.selur.net
Jump to navigation Jump to search
torch filters are Windows only (via update-page on MediaWiki MCP Server)
Update to current Hybrid behaviour; code as <pre> (via update-page on MediaWiki MCP Server)
 
Line 47: Line 47:
== Example ==
== Example ==
Default settings on a 10-bit YUV 4:2:0 source:
Default settings on a 10-bit YUV 4:2:0 source:
<syntaxhighlight lang="python">
<pre>
# adjusting color space from YUV420P10 to RGBS for vsSCUNet
# adjusting color space from YUV420P10 to RGBS for vsSCUNet
clip = core.resize.Bicubic(clip=clip, format=vs.RGBS, matrix_in_s="470bg", range_in_s="limited", range_s="full")
clip = core.resize.Bicubic(clip=clip, format=vs.RGBS, matrix_in_s="470bg", range_in_s="limited", range_s="full")
# denoising using SCUNet
# denoising using SCUNet
clip = SCUNet(clip, model=4)
clip = SCUNet(clip, model=4)
</syntaxhighlight>
</pre>


== How Hybrid calls it ==
== How Hybrid calls it ==

Latest revision as of 09:51, 16 November 2026

SCUNet (torch) is a neural network for blind denoising: it removes noise without being told its strength. It runs on an NVIDIA GPU through PyTorch, optionally with TensorRT, and can restore the original colors afterwards (ColorFix).

Filtering → Vapoursynth → DeNoise

SCUNet (torch) controls in Hybrid
SCUNet (torch) controls in Hybrid

What it does

Every frame is passed through SCUNet ("Swin-Conv-UNet"), a UNet whose blocks combine a Swin Transformer branch with a residual convolution branch. The real models handle real-world noise of unknown strength; the color models are trained for Gaussian noise of a fixed level.

Requirements

  • Windows, Hybrid's torch add-on with the SCUNet component and an NVIDIA GPU with CUDA.
  • TRT needs the Torch-TensorRT component; it cannot be combined with TensorRT in vs-mlrt filters in the same script.

Settings

Control Parameter Range / values Hybrid default Meaning
Model model color_15, color_25, color_50, real_psnr, real_gan real_gan Network to use: Gaussian noise of level 15/25/50, or real-world noise (PSNR-oriented or GAN-sharpened).
GPU ID device_index 0 – 4 0 CUDA device.
Tile (x/y) tile 0 – 1024 0 / 0 Tile size to save GPU memory; 0 = no tiling.
Tile pad tile_pad 0 – 64 32 Overlap of the tiles; only with tiling.
Streams num_streams 1 – 64 1 Number of CUDA streams.
FP16 – on/off off Half precision (16-bit float clip); faster and needs less memory.
TRT / optimization trt, trt_optimization_level on/off, 0 – 5 off, 3 TensorRT; the engine is built on the first run and cached.

Fix Color (optional)

Control Range / values Hybrid default Meaning
Mode average, wavelet average How the colors of the input are brought back: by comparing blurred averages, or by exchanging the low-frequency wavelet bands (CUDA).
Radius / Fast 1 – 100 / on/off 10 / off Average mode: blur radius and a faster approximation.
Wavelets 1 – 15 5 Wavelet mode: number of levels.
Plane all, chroma only, luma only all Planes that are corrected (the clip is RGB here: chroma only = G and B, luma only = R).

Example

Default settings on a 10-bit YUV 4:2:0 source:

# adjusting color space from YUV420P10 to RGBS for vsSCUNet
clip = core.resize.Bicubic(clip=clip, format=vs.RGBS, matrix_in_s="470bg", range_in_s="limited", range_s="full")
# denoising using SCUNet
clip = SCUNet(clip, model=4)

How Hybrid calls it

  • The clip is converted to 32-bit float RGB (16-bit float with FP16) and back afterwards.
  • The model is always written; device and streams only when they differ from the defaults; the tile pad whenever tiling is on.
  • With Fix Color a copy of the input is kept as reference and the colors are corrected after SCUNet.
  • SCUNet also offers the common options (Masked, ApplyOnlyTo, Merge, OnRegion, Gimmick).
  • The same network is available without PyTorch as SCUNet (mlrt).

Background

SCUNet

SCUNet (Zhang et al., Machine Intelligence Research 2023) was trained only on synthetic data from a randomly shuffled degradation pipeline, without real noisy/clean pairs, and generalizes well to real-world noise.

vs-scunet

HolyWu's vs-scunet runs the models in VapourSynth with PyTorch; frames are padded internally to multiples of 64, and TensorRT engines are cached per model, size, precision and GPU.

ColorFix

The Fix Color step uses pifroggi's vs_colorfix: the wavelet method comes from sd-webui-stablesr, the average method from an idea in chaiNNer.

Sources