Vapoursynth RealSR

From wiki.selur.net
Revision as of 09:01, 15 November 2026 by Selur (talk | contribs) (New filter page (via create-page on MediaWiki MCP Server))
(diff) ← Older revision | Latest revision (diff) | Newer revision → (diff)
Jump to navigation Jump to search

RealSR is a CNN super-resolution network trained on real-world camera and photo degradations. It always upscales by 4 and is one of the resizers of Vapoursynth Resizer; it runs through ncnn and Vulkan on NVIDIA, AMD and Intel GPUs.

Filtering → Vapoursynth → Frame → Resize

RealSR controls in Hybrid
RealSR controls in Hybrid

What it does

Every frame is enlarged 4× by the network, which also removes blur and noise typical of real recordings; the DF2K_JPEG model also removes JPEG/compression artifacts. Hybrid resizes the 4× result to the exact output size afterwards. Because the factor is fixed at 4, it is slow.

Requirements

  • A Vulkan-capable GPU.
  • The RealSR plugin (part of Hybrid's base installation on Windows; on Linux/macOS it must be in the VapourSynth plugin folder). Without it the entry is not offered.

Settings

Control Parameter Range / values Hybrid default Meaning
Model model DF2K, DF2K_JPEG DF2K_JPEG DF2K_JPEG is also trained to remove compression artifacts.
GPU (id) gpu_id 0 – 4 0 GPU to use.
Threads gpu_thread 0 – 1024 0 GPU threads; 0 = the plugin's choice.
Force tile size – on/off off Enables the tile sizes.
tile x / tile y tilesize_x, tilesize_y 32 – 1024 100 / 100 Tile size; the plugin recommends 100 and advises against larger tiles.
TTA tta on/off off Test-time augmentation: slightly better, much slower.
Downsizer – point, rect, linear, cubic, lanczos, blackman, blackmanminlobe, spline36, spline64, spline144, spline, gauss, sinc spline64 Kernel that brings the 4× result to the exact output size.

Example

Default settings on a 10-bit YUV 4:2:0 source (640x356), resized to 1280x712:

# adjusting color space from YUV420P10 to RGBS for vsRealSR
clip = core.resize.Bicubic(clip=clip, format=vs.RGBS, matrix_in_s="470bg", range_in_s="limited", range_s="full")
# resizing using RealSR
clip = core.rsnv.RealSR(clip, scale=4) # 2560x1424
# resizing 2560x1424 to 1280x712
clip = core.fmtc.resample(clip, w=1280, h=712, kernel="spline64", interlaced=False, interlacedd=False)

How Hybrid calls it

  • Selected with Resize enabled and Resizer set to RealSR (see Vapoursynth Resizer).
  • The clip is converted to 32-bit float RGB and brought to mod 4 before the call.
  • The scale is always 4. If 4× the input differs from the target, fmtconv resizes it with the Downsizer kernel to the exact size (also downwards). With separately filtered fields the target height is halved.
  • The model, GPU and threads are only written when they differ from the defaults (which are also the plugin defaults); tile sizes only with Force tile size; TTA only when on.

Background

RealSR

RealSR (Ji et al., CVPR workshops 2020) estimates realistic blur kernels and noise from real photos and trains on low-resolution images degraded that way, instead of only bicubic downscaling. It won the NTIRE 2020 real-world super-resolution challenge.

The plugin

Hybrid uses an API-4 fork of the VapourSynth RealSR ncnn Vulkan plugin, based on nihui's realsr-ncnn-vulkan with the DF2K and DF2K_JPEG 4× models. It accepts 32-bit float RGB only.

Sources