Vapoursynth VSMLRT Resize: Difference between revisions
New filter page (via create-page on MediaWiki MCP Server) |
Update to current Hybrid behaviour; code as <pre> (via update-page on MediaWiki MCP Server) |
||
| Line 58: | Line 58: | ||
== Example == | == Example == | ||
A 2x ONNX model on a 10-bit YUV 4:2:0 source (640x356), resized to 1280x712 (default backend, no tiling): | A 2x ONNX model on a 10-bit YUV 4:2:0 source (640x356), resized to 1280x712 (default backend, no tiling): | ||
< | <pre> | ||
clip = core.std.AddBorders(clip=clip, left=0, right=0, top=2, bottom=2) # add borders to achieve mod 8 (vsVSMLRT) - 640x360 | clip = core.std.AddBorders(clip=clip, left=0, right=0, top=2, bottom=2) # add borders to achieve mod 8 (vsVSMLRT) - 640x360 | ||
# changing range from limited to full range for vsVSMLRT | # changing range from limited to full range for vsVSMLRT | ||
| Line 73: | Line 73: | ||
# making sure 0-1 limits are respected | # making sure 0-1 limits are respected | ||
clip = core.vszip.Limiter(clip=clip, min=[0,0,0], max=[1,1,1]) | clip = core.vszip.Limiter(clip=clip, min=[0,0,0], max=[1,1,1]) | ||
</ | </pre> | ||
== How Hybrid calls it == | == How Hybrid calls it == | ||
Latest revision as of 09:53, 16 November 2026
VSMLRT runs any ONNX super-resolution model you supply (for example ESRGAN-type "2x_…" or "4x_…" models) through vs-mlrt, on the CPU or on GPU backends. It is one of the resizers of Vapoursynth Resizer; several models can be chained, and an optional color fix matches the colors back to the source.
Filtering → Vapoursynth → Frame → Resize

What it does
Every frame is passed through the selected model(s). Hybrid takes the upscale factor from the file name and resizes the result to the exact output size afterwards. Without a model the network step is skipped and the clip is only resized.
Requirements
- Hybrid's vs-mlrt add-on with at least one backend, and one or more ONNX model files.
- GPU backends: TensorRT and ONNX (CUDA) for NVIDIA, TensorRT RTX for RTX cards, OpenVINO (GPU) for Intel, NCNN for Vulkan GPUs, DML for DirectX 12 GPUs (Windows); on macOS ONNX (CoreML).
Settings
| Control | Parameter | Range / values | Hybrid default | Meaning |
|---|---|---|---|---|
| Model / Select | network_path | ONNX file(s) | empty | Model to run; the factor is the number before "x_" in the file name. |
| Multi | – | on/off | off | Select several models, applied in order. |
| No adjust | – | on/off | off | With Multi: do not resize back to the input size between the models. |
| Serialize | path_is_serialization | on/off | off | Loads the ONNX file in the script and hands the model to vs-mlrt in memory instead of as a path; not for TensorRT and TensorRT RTX. |
| Backend | backend | OpenVINO (CPU/GPU), ONNX (CPU/CUDA), DML (GPU), TensorRT, TensorRT RTX, NCNN (macOS: ONNX CoreML/CPU) | OpenVINO (CPU) (macOS: CoreML) | Runtime that executes the network. |
| 32bit | fp16 | on/off | on | Full precision; off runs in half precision. Forced on for OpenVINO and TensorRT RTX. |
| BF16 | bf16 | on/off | off | bfloat16; TensorRT and OpenVINO (CPU) only. Switches 32bit on. |
| Device | device_id | 0 – 4 | 0 | GPU to use; not for CPU and CoreML backends. |
| Streams | num_streams | 1 – 16 | 1 | Parallel streams; TensorRT, TensorRT RTX, OpenVINO (GPU), NCNN and CoreML only. |
| Cuda Graph | use_cuda_graph | on/off | off | TensorRT and ONNX (CUDA) only. |
| TRT backend optimization | builder_optimization_level | 0 – 5 | 3 | TensorRT engine build effort. |
| TRT max workspace | workspace | 0 – 1048576 MiB | 0 | TensorRT workspace limit; 0 = not set. |
| Tile (wxh) | tilesize | 1 – 1024 | off, 256×256 | Splits the frame to save memory; off = whole frame. |
| Overlap | overlap | 2 – 624 | 8 | Overlap of the tiles; only with tiling. |
| MultiInput / count | – | on/off, 2 – 50 | off, 2 | For models that take several consecutive frames: feeds that many neighbor frames. |
| Requires: Luma scale / Matrix / Modulo | – | no, pc, tv / matrix list / 1 – 128 | pc / no / 1 | Overrides the range, matrix and size multiple Hybrid prepares for the model. |
| Dynamic Shapes (opt/min/max) | opt_shapes, min_shapes, max_shapes | 0 – 8192 | unset | TensorRT engine sizes; with a minimum size the engine becomes dynamic. |
| Fix Color: Radius, Plane, Mode, Wavelets, Fast | – | radius 1 – 100, average/wavelet, wavelets 1 – 15 | off (radius 10, wavelets 5) | Matches the colors of the output to the clip before the model. |
| Resizer | – | point, rect, linear, cubic, lanczos, blackman, blackmanminlobe, spline36, spline64, spline144, spline, gauss, sinc | spline64 | Kernel that brings the result to the exact output size. |
Example
A 2x ONNX model on a 10-bit YUV 4:2:0 source (640x356), resized to 1280x712 (default backend, no tiling):
clip = core.std.AddBorders(clip=clip, left=0, right=0, top=2, bottom=2) # add borders to achieve mod 8 (vsVSMLRT) - 640x360
# changing range from limited to full range for vsVSMLRT
clip = core.resize.Bicubic(clip, format=vs.YUV420P10, range_in_s="limited", range_s="full")
# setting color range to PC (full) range.
prop_name = '_Range' if core.core_version.release_major >= 74 else '_ColorRange'
clip = core.std.SetFrameProps(clip=clip, **{prop_name: vs.RANGE_FULL})
# adjusting color space from YUV420P10 to RGBS for vsVSMLRT
clip = core.resize.Bicubic(clip=clip, format=vs.RGBS, matrix_in_s="470bg", range_in_s="full", range_s="full")
# resizing using VSMLRT, target: 1280x712
from vsmlrt import Backend
clip = vsmlrt.inference([clip], network_path="F:/Hybrid/64bit/onnx_models/2x_AnimeJaNai_Standard_V1_Compact_net_g_120000.onnx", backend=Backend.OV_CPU(fp16=False,bf16=False))
clip = core.std.Crop(clip=clip, left=0, right=0, top=4, bottom=4) # removing added borders from mod requirement (vsVSMLRT) - 1280x712
# making sure 0-1 limits are respected
clip = core.vszip.Limiter(clip=clip, min=[0,0,0], max=[1,1,1])
How Hybrid calls it
- Selected with Resize enabled and Resizer set to VSMLRT (see Vapoursynth Resizer).
- The clip is converted to 32-bit float RGB (16-bit float with 32bit off), full range, and brought to mod 8 (mod 1 for TensorRT) unless the Requires settings say otherwise.
- The factor comes from the file name: "4x_name.onnx" counts as 4×; a prefix that is not a number is assumed to be 4×; a file without "x_" is skipped with a comment in the script.
- After each model the size is multiplied by its factor. With Multi, Hybrid resizes back to the input size between the models unless No adjust is set. At the end fmtconv resizes the result with the Resizer kernel to the exact size if needed; with separately filtered fields the target height is halved. The color fix runs last.
- With tiling the overlap is always written (vs-mlrt's own default would be 0).
- With MultiInput the model gets the requested number of neighbor frames; at the start and end of the clip the first or last frame is repeated.
- TensorRT builds an engine on the first run for each size. TensorRT cannot be used in vs-mlrt filters and torch filters in the same script.
- The PyTorch counterpart for user-supplied models is VSGAN.
Background
vs-mlrt
vs-mlrt runs ONNX models in VapourSynth with several backends (OpenVINO, ONNX Runtime, TensorRT, NCNN and others) behind one Python wrapper. Its generic inference function takes any ONNX model and one or more input clips; by default it processes the whole frame and uses no tile overlap.
Sources
- vs-mlrt README
- Hybrid's tooltips