Vapoursynth RealESRGAN Resize

From wiki.selur.net
Revision as of 09:51, 16 November 2026 by Selur (talk | contribs) (Update to current Hybrid behaviour; code as <pre> (via update-page on MediaWiki MCP Server))
(diff) ← Older revision | Latest revision (diff) | Newer revision → (diff)
Jump to navigation Jump to search

Real-ESRGAN is a blind super-resolution network that upscales by a fixed factor while removing blur, noise and compression artifacts. It is one of the resizers of Vapoursynth Resizer and runs on an NVIDIA GPU through PyTorch; besides the original models it offers community anime models.

Filtering → Vapoursynth → Frame → Resize

RealESRGAN controls in Hybrid
RealESRGAN controls in Hybrid

What it does

Every frame is enlarged by the factor of the chosen model (2× or 4×) and restored at the same time. Hybrid resizes the result to the exact output size afterwards.

Requirements

  • Windows, Hybrid's torch add-on and an NVIDIA GPU with CUDA.
  • Tensor RT additionally needs Torch-TensorRT from the add-on; it cannot be combined with TensorRT in vs-mlrt filters in the same script.

Settings

Control Parameter Range / values Hybrid default Meaning
Model model x4 ESRGAN, x2plus, x4plus, x4plus_anime, realesr-anime, realesr-general, AnimeJaNai (c, uc, suc, sharp c/uc/suc, SD c, v2 c/uc/suc), AniScale2, OpenProteus, Ani4Kv2 (c, uc) AnimeJaNai v2 c Network; x2plus and all community models upscale 2×, the others 4×.
Denoise strength denoise_strength 0 – 1 0.5 Only for realesr-general: 0 keeps the noise, 1 removes it strongly.
Tile (x/y) tile 32 – 8192 off, 256 / 256 Splits the frame into tiles to save GPU memory.
Tile pad tile_pad 0 – 64 10 Overlap of the tiles; only with tiling.
GPU ID device_index 0 – 4 0 CUDA device.
Streams num_streams 1 – 64 1 Number of CUDA streams.
Batches batch_size 1 – 64 1 Frames processed together.
Tensor RT trt on/off off Uses TensorRT; the engine is built on the first run and cached.
FP16 – on/off off Half precision (16-bit float clip), with and without Tensor RT.
max WSS trt_workspace_size 0 – 1073741824 0 TensorRT workspace size; 0 = not set.
Resizer – point, rect, linear, cubic, lanczos, blackman, blackmanminlobe, spline36, spline64, spline144, spline, gauss, sinc spline64 Kernel that brings the result to the exact output size.

Example

Default settings on a 10-bit YUV 4:2:0 source (640x356), resized to 1280x712:

from vsrealesrgan import realesrgan as RealESRGAN
# adjusting color space from YUV420P10 to RGBS for vsRealESRGAN
clip = core.resize.Bicubic(clip=clip, format=vs.RGBS, matrix_in_s="470bg", range_in_s="limited", range_s="full")
# resizing using RealESRGAN
clip = RealESRGAN(clip, model=107, device_index=0) # 1280x712

How Hybrid calls it

  • Selected with Resize enabled and Resizer set to RealESRGAN (see Vapoursynth Resizer).
  • The clip is converted to 32-bit float RGB (16-bit float with FP16) and padded to mod 4; the padding is cropped afterwards.
  • The output is input × the model's factor. If that differs from the target, fmtconv resizes it with the Resizer kernel to the exact size. With separately filtered fields the target height is halved.
  • Model and GPU ID are always written, and so is the tile pad while tiling is on; denoise strength, streams, batches and max WSS only when they differ from the defaults.
  • With Tensor RT the engines are cached in the engine path from Hybrid's path settings (or the temp folder); one engine per resolution.
  • The 1× models (refiner, denoise, fix) are in the separate RealESRGAN filter.

Background

Real-ESRGAN

Real-ESRGAN (Wang et al., ICCV workshops 2021) extends ESRGAN with a high-order degradation model, so it is trained on synthetic data that resembles real-world blur, noise and compression. realesr-anime (animevideov3) and realesr-general (general x4v3) are its smaller video models.

vs-realesrgan

HolyWu's vs-realesrgan runs Real-ESRGAN and the community models (AnimeJaNai, AniScale2, OpenProteus, Ani4Kv2 – c, uc and suc are Compact, UltraCompact and SuperUltraCompact) in VapourSynth with PyTorch, optionally with TensorRT. It accepts 16- or 32-bit float RGB.

Sources