Vapoursynth RIFE

From wiki.selur.net
Revision as of 09:51, 16 November 2026 by Selur (talk | contribs) (Update to current Hybrid behaviour; code as <pre> (via update-page on MediaWiki MCP Server))
(diff) ← Older revision | Latest revision (diff) | Newer revision → (diff)
Jump to navigation Jump to search

RIFE (Real-Time Intermediate Flow Estimation) is a neural network that estimates the optical flow between two frames and synthesizes the frame in between. This variant runs through ncnn on any GPU with Vulkan; no CUDA is needed.

Filtering → Vapoursynth → Frame → Interpolation

RIFE controls in Hybrid
RIFE controls in Hybrid

What it does

For every pair of neighbor frames the network predicts the motion and blends the warped frames into new in-between frames. The v4 models can produce any target rate; older models only double the frame rate. Frames at scene changes can be duplicated instead of interpolated, and nearly identical frame pairs can be skipped.

Requirements

  • A GPU with Vulkan support (Windows, Linux; macOS via MoltenVK).
  • Skip threshold needs the VMAF plugin.

Settings

Tick Frame Interpolation and pick RIFE in the method list.

Control Parameter Range / values Hybrid default Meaning
Model model, model_path 74 models from rife to rife-v4.26-large, plus custom rife-v4.18 (ensemble=True) Network; custom loads a model folder chosen with the Model button.
Factor – on/off off Off: absolute target rate; on: multiple of the input rate. Only for v4 and custom models.
Target FPS (num/den) fps_num, fps_den 1 – 200000 60/1 Target frame rate with Factor off.
FPS Factor (num/den) factor_num, factor_den 1 – 200000 2/1 Multiple of the input rate with Factor on.
Force gpu / GPU id gpu_id 0 – 4 off, 0 Picks the GPU; otherwise the plugin chooses.
GPU-threads gpu_thread 1 – 8 2 Threads per GPU.
TTA tta on/off off Test-time augmentation (slower). Only for models before v4.
UHD uhd on/off off Mode for high-resolution content.
Scene change / threshold sc on/off, 0.001 – 1 off, 0.1 Detects cuts before RIFE; frames at a cut are not interpolated.
Manual scene changes sc on/off, frame list off List of the last frames before each cut; excludes the automatic detection.
Skip threshold skip, skip_threshold on/off, 0.01 – 60 off, 60 PSNR value: frame pairs at or above it count as static and are not interpolated. 0 does not switch it off; the checkbox does.
Tools: SCD – scd, misc (Windows), std, mv – Scene change detection, shown with Scene change on; mv uses its own motion thresholds instead of the threshold.

Example

Default settings on a 10-bit YUV 4:2:0 source (640x356, 25 fps), interpolated to 50 fps:

# adjusting color space from YUV420P10 to RGBS for vsRIFE
clip = core.resize.Bicubic(clip=clip, format=vs.RGBS, matrix_in_s="470bg", range_in_s="limited", range_s="full")
# adjusting frame count&rate with RIFE, target fps: 50fps
clip = core.rife.RIFE(clip, model=58, fps_num=50, fps_den=1, sc=False) # new fps: 50

How Hybrid calls it

  • The clip is converted to 32-bit float RGB.
  • With a v4 or custom model the target is the absolute Target FPS or, with Factor ticked, the factor (written only when it is not 2/1). Older models always double the rate; the rate controls are disabled. A target below the input rate is passed to the plugin as is.
  • The model is written as its index in the plugin's model table.
  • With Scene change Hybrid runs a scene detection first (RGB clips are converted to 16-bit YUV 4:4:4 for it) and writes sc=True. With manual scene changes it marks the listed frames as the last frame before a cut and also writes sc=True.
  • GPU-threads, TTA and UHD are only written when they differ from the defaults.

Background

RIFE ncnn Vulkan

Hybrid uses styler00dollar's version of the VapourSynth RIFE ncnn Vulkan plugin. It only takes 32-bit float RGB; a target rate overrides the factor; custom rates are only possible with v4 models, and v4 models do not support TTA. The plugin's scene change mode reads cut markers set by a scene detection, and the skip mode measures frame similarity with VMAF's PSNR.

Other RIFE versions

RIFE (mlrt) runs the same networks through vs-mlrt (also on the CPU), TorchRIFE in PyTorch on NVIDIA GPUs.

Sources