Vapoursynth SRMD

From wiki.selur.net
Jump to navigation Jump to search

SRMD (Super-Resolution for Multiple Degradations) is a CNN that upscales by 2, 3 or 4 and denoises at the same time, with an adjustable denoise strength. It is one of the resizers of Vapoursynth Resizer and runs through ncnn and Vulkan on NVIDIA, AMD and Intel GPUs.

Filtering → Vapoursynth → Frame → Resize

SRMD controls in Hybrid
SRMD controls in Hybrid

What it does

Every frame is enlarged by a whole factor and cleaned by the network in one pass; Denoise sets how much noise it removes. Hybrid picks the factor itself and resizes the result to the exact output size afterwards.

Requirements

  • A Vulkan-capable GPU.
  • The SRMD plugin (part of Hybrid's base installation on Windows; on Linux/macOS it must be in the VapourSynth plugin folder). Without it the entry is not offered.

Settings

Control Parameter Range / values Hybrid default Meaning
Denoise noise -1 – 10 3 Denoise strength; higher removes more noise.
GPU (id) gpu_id 0 – 4 0 GPU to use.
Threads gpu_thread 0 – 1024 0 GPU threads; 0 = the plugin's choice.
Force tile size – on/off off Enables the tile sizes; otherwise the plugin picks them from the free VRAM.
tile x / tile y tilesize_x, tilesize_y 32 – 1024 100 / 100 Tile size; sizes that divide the frame evenly are best, very small tiles are slow.
TTA tta on/off off Test-time augmentation: slightly better, much slower.
Downsizer – point, rect, linear, cubic, lanczos, blackman, blackmanminlobe, spline36, spline64, spline144, spline, gauss, sinc spline64 Kernel that brings the result to the exact output size.

Example

Default settings on a 10-bit YUV 4:2:0 source (640x356), resized to 1280x712:

# adjusting color space from YUV420P10 to RGBS for vsSRMD
clip = core.resize.Bicubic(clip=clip, format=vs.RGBS, matrix_in_s="470bg", range_in_s="limited", range_s="full")
# resizing using srmd
# Zoomfactor: 2 
clip = core.srmdnv.SRMD(clip, scale=2) # 1280x712

How Hybrid calls it

  • Selected with Resize enabled and Resizer set to SRMD (see Vapoursynth Resizer).
  • The clip is converted to 32-bit float RGB and brought to mod 4 before the call.
  • There is no scale control: Hybrid uses the smallest whole factor (2 – 4) that reaches the output size in both directions; when even 4 is not enough a script comment says so. For downscaling factor 2 is used and then shrunk.
  • If the upscaled size differs from the target, fmtconv resizes it with the Downsizer kernel to the exact size. With separately filtered fields the target height is halved.
  • The scale is always written; denoise, GPU and threads only when they differ from the defaults; tile sizes only with Force tile size; TTA only when on.

Background

SRMD

SRMD (Zhang, Zuo and Zhang, CVPR 2018) takes the blur and noise level as an extra input, so one network handles many degradations instead of only clean bicubic downscaling. The ncnn port ships the SRMD models for 2×, 3× and 4× and the noise-free SRMDNF variants.

The plugin

Hybrid uses an API-4 fork of the VapourSynth SRMD ncnn Vulkan plugin, based on nihui's srmd-ncnn-vulkan. It accepts 32-bit float RGB only. The fork builds against a recent ncnn and fixes wrong colors of the older Windows build.

Sources