Vapoursynth TorchRIFE: Difference between revisions

From wiki.selur.net
Jump to navigation Jump to search
New filter page (via create-page on MediaWiki MCP Server)
 
Update to current Hybrid behaviour; code as <pre> (via update-page on MediaWiki MCP Server)
 
Line 11: Line 11:
== Requirements ==
== Requirements ==
* Windows with an NVIDIA graphics card (CUDA) and Hybrid's torch add-on (package vsrife); without it the method is not offered.
* Windows with an NVIDIA graphics card (CUDA) and Hybrid's torch add-on (package vsrife); without it the method is not offered.
* TensorRT acceleration needs Torch-TensorRT.
* TensorRT acceleration needs Torch-TensorRT; the TensorRT checkbox is only enabled when it is installed.


== Settings ==
== Settings ==
Line 47: Line 47:
== Example ==
== Example ==
Default settings on a 10-bit YUV 4:2:0 source (640x356, 25 fps), interpolated to 50 fps:
Default settings on a 10-bit YUV 4:2:0 source (640x356, 25 fps), interpolated to 50 fps:
<syntaxhighlight lang="python">
<pre>
from vsrife import rife
from vsrife import rife
# adjusting color space from YUV420P10 to RGBS for vsTorchRIFE
# adjusting color space from YUV420P10 to RGBS for vsTorchRIFE
Line 53: Line 53:
# adjusting frame count&rate with RIFE (torch)
# adjusting frame count&rate with RIFE (torch)
clip = rife(clip) # new fps: 50
clip = rife(clip) # new fps: 50
</syntaxhighlight>
</pre>


== How Hybrid calls it ==
== How Hybrid calls it ==

Latest revision as of 09:53, 16 November 2026

TorchRIFE runs HolyWu's PyTorch version of the RIFE frame interpolation network on an NVIDIA GPU, optionally accelerated with TensorRT. It offers the RIFE 4.x models up to 4.26.

Filtering → Vapoursynth → Frame → Interpolation

TorchRIFE controls in Hybrid
TorchRIFE controls in Hybrid

What it does

RIFE estimates the optical flow between two neighbor frames and synthesizes the frames in between, either as a multiple of the input rate or for an absolute target rate. Frames marked as the last frame before a cut are not interpolated.

Requirements

  • Windows with an NVIDIA graphics card (CUDA) and Hybrid's torch add-on (package vsrife); without it the method is not offered.
  • TensorRT acceleration needs Torch-TensorRT; the TensorRT checkbox is only enabled when it is installed.

Settings

Tick Frame Interpolation and pick TorchRIFE in the method list.

Control Parameter Range / values Hybrid default Meaning
Factor/FPS – on/off on On: multiple of the input rate; off: absolute target rate.
Factor (num/den) factor_num, factor_den 1 – 120000 2/1 Multiple of the input rate.
FPS (num/den) fps_num, fps_den 1 – 120000 2/1 Target frame rate; the default of 2 fps must be changed before this mode does anything.
Model model 4.0 – 4.26.heavy (36 entries incl. .lite/.heavy) 4.25 RIFE model version.
GPU id device_index 0 – 4 0 GPU to use.
FP16 – on/off off Half precision: the clip is fed as 16-bit float RGB.
TensorRT trt on/off off Builds a TensorRT engine; not for models 4.0 and 4.1.
max WSS trt_workspace_size 0 – 1073741824 0 TensorRT workspace limit in bytes; 0 = no limit.
Scale scale 0.25, 0.5, 1.0, 2.0, 4.0 1.0 Resolution of the flow estimation; not with Ensemble.
Ensemble ensemble on/off off Ensemble variant of the model (slower).
Scene detection / threshold sc on/off, 0.001 – 1 off, 0.1 Detects cuts before RIFE so they are not interpolated.
Manual scene changes sc on/off, frame list off List of the last frames before each cut; excludes the automatic detection.
Tools: SCD – scd, misc (Windows), std, mv – Scene change detection, shown with Scene detection on; mv uses its own motion thresholds.

Example

Default settings on a 10-bit YUV 4:2:0 source (640x356, 25 fps), interpolated to 50 fps:

from vsrife import rife
# adjusting color space from YUV420P10 to RGBS for vsTorchRIFE
clip = core.resize.Bicubic(clip=clip, format=vs.RGBS, matrix_in_s="470bg", range_in_s="limited", range_s="full")
# adjusting frame count&rate with RIFE (torch)
clip = rife(clip) # new fps: 50

How Hybrid calls it

  • The clip is converted to 32-bit float RGB (16-bit with FP16).
  • In factor mode the factor is written only when it is not 2/1; in FPS mode the target rate is always written. If the new rate would not be higher than the input rate, the filter is skipped with a note in the script and the frame rate stays.
  • The model is written when it is not 4.25 (also the package default). TensorRT engines are cached in Hybrid's temporary folder.
  • With scene detection Hybrid runs the detection itself before RIFE (RGB clips are converted to 16-bit YUV 4:4:4 for it) and writes sc=True; manual scene changes mark the listed frames and also write sc=True.
  • Models 4.0 and 4.1 get mod-16 padding.

Background

vs-rife

vs-rife needs at least two frames of 16- or 32-bit float RGB and a CUDA GPU. The package itself does not contain the models; they are downloaded separately. The package pads the frames to the model's modulo itself. With its own scene detection it uses the misc plugin; otherwise it reads cut markers already set on the frames, which is how Hybrid uses it.

Other RIFE versions

RIFE runs on Vulkan GPUs through ncnn, RIFE (mlrt) through vs-mlrt (also on the CPU). TorchDRBA is a RIFE-based interpolator aimed at animation.

Sources