Vapoursynth vsMLRT
vsMLRT runs any ONNX model you supply, or one of the bundled ArtCNN models, through vs-mlrt on the CPU or on GPU backends. Upscaling models are scaled back to the input size by default, so they act as restoration filters; an optional color fix matches the colors back to the source. To keep the larger size use Vapoursynth VSMLRT Resize.
Filtering → Vapoursynth → Other

What it does
Every frame is passed through the selected model. Hybrid takes the model's factor from the file name and, with Undo resize, resizes the result back to the input size afterwards. Without a model the filter is skipped.
Requirements
- Hybrid's vs-mlrt add-on with at least one backend.
- ONNX model files whose names start with the scale factor, e.g. "1x_name.onnx" or "4x_name.onnx". Models without such a prefix are skipped with a message in the log. ArtCNN needs the ArtCNN models of the add-on.
- GPU backends: TensorRT and ONNX (CUDA) for NVIDIA, TensorRT RTX for RTX cards, OpenVINO (GPU) for Intel, NCNN for Vulkan GPUs, DML for DirectX 12 GPUs (Windows); on macOS ONNX (CoreML).
Settings
| Control | Parameter | Range / values | Hybrid default | Meaning |
|---|---|---|---|---|
| Model / Select | network_path | ONNX file | empty | Model to run; the factor is the number before "x_" in the file name. |
| ArtCNN | model | on/off; ArtCNN_C4F32, C4F32_DS, C16F64, C16F64_DS, C4F32_Chroma, C16F64_Chroma, R16F96, R8F64, R8F64_DS, R8F64_Chroma, C4F16, C4F16_DS, R16F96_Chroma, C4F16_DN, C4F32_DN, R8F64_JPEG420, R8F64_JPEG444, R8F64_Chroma_DN | off, ArtCNN_C16F64 | Uses a bundled ArtCNN model instead of a file. Luma models work on Y (2×), chroma models on YUV 4:4:4, JPEG models on RGB. |
| Serialize | path_is_serialization | on/off | off | Loads the ONNX file in the script and hands the model to vs-mlrt in memory instead of as a path; not for TensorRT. |
| MultiInput / count | – | on/off, 2 – 50 | off, 2 | For models that take several consecutive frames: feeds that many neighbor frames. |
| Backend | backend | OpenVINO (CPU/GPU/NPU), ONNX (CPU/CUDA), DML (GPU), TensorRT, TensorRT RTX, NCNN (macOS: ONNX CoreML/CPU) | OpenVINO (CPU) | Runtime that executes the network; only installed backends are listed. |
| 32bit | fp16 | on/off | on | Full precision; off runs in half precision. Forced on for OpenVINO and TensorRT RTX. |
| BF16 | bf16 | on/off | off | bfloat16; TensorRT and OpenVINO (CPU) only. Switches 32bit on. |
| Device | device_id | 0 – 4 | 0 | GPU to use; not for CPU and CoreML backends. |
| Streams | num_streams | 1 – 16 | 1 | Parallel streams. |
| Cuda Graph | use_cuda_graph | on/off | off | TensorRT, TensorRT RTX and ONNX (CUDA) only. |
| TRT backend optimization | builder_optimization_level | 0 – 5 | 3 | TensorRT engine build effort. |
| TRT max workspace | workspace | 0 – 1048576 MiB | 0 | TensorRT workspace limit; 0 = TensorRT decides. |
| Tile (wxh) | tilesize | 1 – 1024 | off, 256×256 | Splits the frame to save memory; off = whole frame. |
| Overlap | overlap | 2, 4, 8, 16 – 624 | 8 | Overlap of the tiles; only with tiling. |
| Dynamic Shapes (opt/min/max) | opt_shapes, min_shapes, max_shapes | 0 – 8192 | unset | TensorRT engine sizes; with a minimum size the engine becomes dynamic. |
| Undo resize | – | on/off | on | Resizes the result back to the input size. |
| Resizer | – | Bilinear, Bicubic, Point, Lanczos, Spline16, Spline36, Spline64, DPID | Spline64 | Kernel used to resize back to the input size. |
| Requires: Luma scale / Matrix / Modulo | – | no, pc, tv / matrix list / 1 – 128 | pc / no / 1 | Overrides the range, matrix and size multiple Hybrid prepares for the model; "no" keeps Hybrid's choice. |
| Fix Color: Mode, Radius, Wavelets, Planes, Fast | – | average/wavelet, radius 1 – 100, wavelets 1 – 15 | off (average, radius 10, wavelets 5, all planes) | Matches the colors of the output to the clip before the model; Radius and Fast with average, Wavelets with wavelet. |
Example
Model F:/Hybrid/64bit/onnx_models/2x_AnimeJaNai_Standard_V1_Compact_net_g_120000.onnx selected (no model is set by default), otherwise default settings, on a 10-bit YUV 4:2:0 source (640x356):
clip = core.std.AddBorders(clip=clip, left=0, right=0, top=2, bottom=2) # add borders to achieve mod 8 (vsVSMLRTFilter) - 640x360
# changing range from limited to full range for vsVSMLRTFilter
clip = core.resize.Bicubic(clip, format=vs.YUV420P10, range_in_s="limited", range_s="full")
# setting color range to PC (full) range.
prop_name = '_Range' if core.core_version.release_major >= 74 else '_ColorRange'
clip = core.std.SetFrameProps(clip=clip, **{prop_name: vs.RANGE_FULL})
# adjusting color space from YUV420P10 to RGBS for vsVSMLRTFilter
clip = core.resize.Bicubic(clip=clip, format=vs.RGBS, matrix_in_s="470bg", range_in_s="full", range_s="full")
# filtering using VSMLRT
from vsmlrt import Backend
clip = vsmlrt.inference([clip], network_path="F:/Hybrid/64bit/onnx_models/2x_AnimeJaNai_Standard_V1_Compact_net_g_120000.onnx", backend=Backend.OV_CPU(fp16=False,bf16=False))
# undo resizing 1280x720 to 640x360
clip = core.resize.Spline64(clip=clip, width=640, height=360)
clip = core.std.Crop(clip=clip, left=0, right=0, top=2, bottom=2) # removing added borders from mod requirement (vsVSMLRTFilter) - 640x356
# making sure 0-1 limits are respected
clip = core.vszip.Limiter(clip=clip, min=[0,0,0], max=[1,1,1])
# adjusting output color from RGBS to YUV420P10 for NVEncModel
clip = core.resize.Bicubic(clip=clip, format=vs.YUV420P10, matrix_s="470bg", range_in_s="full", range_s="full", dither_type="error_diffusion")
# changing range from full to limited range for matching filter target luma scale
clip = core.resize.Bicubic(clip, format=vs.YUV420P10,range_in_s="full", range_s="limited")
# setting color range to TV (limited) range.
prop_name = '_Range' if core.core_version.release_major >= 74 else '_ColorRange'
clip = core.std.SetFrameProps(clip=clip, **{prop_name: vs.RANGE_LIMITED})
How Hybrid calls it
- The clip is converted to 32-bit float RGB (16-bit float with 32bit off), full range, and brought to mod 8 (mod 1 for TensorRT) unless the Requires settings say otherwise.
- With tiling the overlap is always written (vs-mlrt's own default would be 0).
- With MultiInput the model gets the requested number of neighbor frames; at the start and end of the clip the first or last frame is repeated.
- TensorRT builds an engine on the first run for each size. TensorRT cannot be used in vs-mlrt filters and torch filters in the same script.
- The color fix runs last.
- vsMLRT also offers the common options (Masked, ApplyOnlyTo, Merge, OnRegion, Gimmick).
- The PyTorch counterpart for user-supplied models is VSGAN.
Background
vs-mlrt
vs-mlrt runs ONNX models in VapourSynth with several backends (OpenVINO, ONNX Runtime, TensorRT, NCNN and others) behind one Python wrapper. Its generic inference function takes any ONNX model and one or more input clips; by default it processes the whole frame and uses no tile overlap. The wrapper also has ready-made functions for model families such as ArtCNN.
Sources
- vs-mlrt README
- Hybrid's tooltips