Vapoursynth RVRT Resize

From wiki.selur.net
Revision as of 09:52, 16 November 2026 by Selur (talk | contribs) (Update to current Hybrid behaviour; code as <pre> (via update-page on MediaWiki MCP Server))
(diff) ← Older revision | Latest revision (diff) | Newer revision → (diff)
Jump to navigation Jump to search

RVRT (Recurrent Video Restoration Transformer) restores video from several frames at once; this resizer uses its 4× video super-resolution models. It is one of the resizers of Vapoursynth Resizer, runs on an NVIDIA GPU through PyTorch and needs a lot of GPU memory.

Filtering → Vapoursynth → Frame → Resize

RVRT controls in Hybrid
RVRT controls in Hybrid

What it does

Frames are processed in groups; each group is aligned to its neighbors and refined recurrently, so the result of a frame depends on the frames around it. Long clips are split into chunks that overlap. Hybrid resizes the 4× result to the exact output size afterwards.

Requirements

  • Windows, Hybrid's torch add-on (with its vsrvrt build) and an NVIDIA GPU with CUDA.
  • The RVRT models are licensed for non-commercial use only.

Settings

Control Parameter Range / values Hybrid default Meaning
Model model reds, vimeo_bi, vimeo_bd reds Training set: REDS, or Vimeo-90K with bicubic (bi) or blur and downscaling (bd). All 4×.
Force Tile size – on/off off Enables the tile size; otherwise the plugin picks tiles from the free GPU memory.
Tile T / H / W tile_size T 0 – 1024, H/W 0 – 1024 in steps of 8 64 / 256 / 256 Tile size in frames, height and width; 0/0/0 = no tiling. Spatial sizes should be multiples of 8.
Force Tile Overlap size – on/off off With Force Tile size: enables the tile overlap.
Overlap T / H / W tile_overlap 0 – 512 2 / 20 / 20 Overlap of the tiles in frames, height and width.
FP16 use_fp16 on/off on Half precision: about twice as fast with half the memory.
Chunking use_chunking on/off on Processes the clip in chunks of frames.
Chunk size / overlap chunk_size, chunk_overlap 0 – 127 / 0 – 63 off, 64 / 16 Forces the chunk size and overlap in frames; otherwise the plugin's values are used.
Lazy chunking preview_mode on/off off Processes chunks on demand and independently; for quick previews.
GPU ID device 0 – 8 0 CUDA device.
Resizer – point, rect, linear, cubic, lanczos, blackman, blackmanminlobe, spline36, spline64, spline144, spline, gauss, sinc spline64 Kernel that brings the 4× result to the exact output size.

Example

Default settings on a 10-bit YUV 4:2:0 source (640x356), resized to 1280x712:

# adjusting color space from YUV420P10 to RGBS for VsRVRT
clip = core.resize.Bicubic(clip=clip, format=vs.RGBS, matrix_in_s="470bg", range_in_s="limited", range_s="full")
# resizing using RVRT
import vsrvrt
clip = vsrvrt.SuperRes(clip, device="cuda:0", use_chunking=True) # 2560x1424
# resizing 2560x1424 to 1280x712
clip = core.fmtc.resample(clip, w=1280, h=712, kernel="spline64", interlaced=False, interlacedd=False)

How Hybrid calls it

  • Selected with Resize enabled and Resizer set to RVRT (see Vapoursynth Resizer).
  • The clip is converted to RGB; there is no size multiple.
  • The output is always 4× the input. If that differs from the target, fmtconv resizes it with the Resizer kernel to the exact size. With separately filtered fields the target height is halved.
  • The device (from GPU ID) and Chunking are always written; model and FP16 only when they differ from the defaults; tile size and overlap only when forced; chunk size and overlap only when forced with Chunking on.
  • Denoising and deblurring with RVRT are in the separate RVRT filter.

Background

RVRT

RVRT (Liang et al., NeurIPS 2022) splits a video into short clips, processes the frames of a clip in parallel and passes features recurrently from clip to clip, aligned with guided deformable attention. This sits between fully parallel and fully recurrent video restoration models.

vs-rvrt

Lyra-Vhess's vs-rvrt offers RVRT denoising, deblurring and 4× super-resolution for VapourSynth. Its README suggests spatial tiles of 256 and multiples of 8, chunks below 128 frames, and about (8, 256, 256) tiles with chunk size 16 on a 12 GB card. Without an explicit setting it chunks on its own when a clip is long or would need too much memory.

Sources