Vapoursynth RealCUGAN

From wiki.selur.net
Revision as of 09:02, 15 November 2026 by Selur (talk | contribs) (New filter page (via create-page on MediaWiki MCP Server))
(diff) ← Older revision | Latest revision (diff) | Newer revision → (diff)
Jump to navigation Jump to search

Real-CUGAN (Real Cascade U-Nets) is bilibili's super-resolution network for anime that upscales by 2, 3 or 4 and can denoise in several steps. It is one of the resizers of Vapoursynth Resizer and runs through ncnn and Vulkan on NVIDIA, AMD and Intel GPUs.

Filtering → Vapoursynth → Frame → Resize

RealCUGAN controls in Hybrid
RealCUGAN controls in Hybrid

What it does

Every frame is enlarged by Scale and, depending on the denoise model, cleaned at the same time. Hybrid resizes the result to the exact output size afterwards.

Requirements

  • A Vulkan-capable GPU.
  • The RealCUGAN plugin (part of Hybrid's base installation on Windows; on Linux/macOS it must be in the VapourSynth plugin folder). Without it the entry is not offered.

Settings

Control Parameter Range / values Hybrid default Meaning
Model model no-se, se, pro se Model family; it limits the scales and denoise levels.
Scale scale 2 – 4 (no-se: 2, pro: 2 – 3) 2 Upscale factor.
(De-)Noise noise none (-1), light (0), medium (1), strong (2), heavy (3) light Denoise model: -1 = conservative model, 0 = no denoising, 1 – 3 = denoise steps. Only installed levels are listed (see below).
Sync gap syncgap without (0), lossless (1), some background, less texture (2), more background, less texture (3) 2 Sync gap mode of the network.
GPU (id) gpu_id 0 – 4 0 GPU to use.
Threads gpu_thread 0 – 1024 0 GPU threads; 0 = the plugin's choice.
Force tile size – on/off off Enables the tile sizes; otherwise the plugin chooses them.
tile x / tile y tilesize_x, tilesize_y 32 – 1024 100 / 100 Tile size.
TTA tta on/off off Test-time augmentation: slightly better, much slower.
Downsizer – point, rect, linear, cubic, lanczos, blackman, blackmanminlobe, spline36, spline64, spline144, spline, gauss, sinc spline64 Kernel that brings the result to the exact output size.

Denoise levels per model: no-se offers only light (no denoising); se at scale 2 offers all five; se at scale 3 or 4 and pro offer none, light and heavy.

Example

Default settings on a 10-bit YUV 4:2:0 source (640x356), resized to 1280x712:

# adjusting color space from YUV420P10 to RGBS for vsRealCUGAN
clip = core.resize.Bicubic(clip=clip, format=vs.RGBS, matrix_in_s="470bg", range_in_s="limited", range_s="full")
# resizing using RealCUGAN
clip = core.rcnv.RealCUGAN(clip, noise=0, syncgap=2) # 1280x712

How Hybrid calls it

  • Selected with Resize enabled and Resizer set to RealCUGAN (see Vapoursynth Resizer).
  • The clip is converted to 32-bit float RGB and brought to mod 4 before the call.
  • The factor is the Scale setting. If input × scale differs from the target, fmtconv resizes the result with the Downsizer kernel to the exact size (up or down). With separately filtered fields the target height is halved.
  • Denoise and sync gap are always written; model, scale, GPU and threads only when they differ from the defaults (which are also the plugin defaults); tile sizes only with Force tile size; TTA only when on.

Background

Real-CUGAN

Real-CUGAN comes from bilibili's AI lab and was trained on anime. The ncnn port by nihui ships the models of 2022-07-28: no-se only for 2× without denoising, se for 2× with all denoise levels and for 3×/4× with conservative, no-denoise and the strongest denoise model, and pro for 2× and 3× with the same three.

The plugin

Hybrid uses an API-4 fork of the VapourSynth RealCUGAN ncnn Vulkan plugin. It accepts 32-bit float RGB only; a tile size of 0 lets it choose automatically.

Sources