Vapoursynth VSGAN

From wiki.selur.net
Jump to navigation Jump to search

VSGAN runs a PyTorch model you supply – an ESRGAN-family network or an EGVSR video model – on an NVIDIA GPU and scales the result back to the input size, so an upscaling model is used only for its cleaning effect. Several models can be chained, and an optional color fix matches the colors back to the source. To keep the larger size use Vapoursynth VSGAN Resize.

Filtering → Vapoursynth → Other

VSGAN controls in Hybrid
VSGAN controls in Hybrid

What it does

Every frame is passed through the selected .pth model(s); there is no built-in model list. Each model enlarges the frame by its factor, and at the end Hybrid brings the result back to the input size with the chosen resizer. Without a model the filter is skipped.

Requirements

  • Windows, Hybrid's torch add-on and an NVIDIA GPU with CUDA.
  • One or more model files whose names start with the scale factor, e.g. "2x_name.pth" or "4x_name.pth". Models without such a prefix are skipped with a message in the log.
  • The color fix needs the color fix package.

Settings

Control Parameter Range / values Hybrid default Meaning
Model / Select load() .pth file(s) empty Model to run; the factor is the number before "x_" in the file name.
Multi – on/off off Select several models, applied in order.
Overlap overlap none, 16 – 624 none Overlap of the tiles vsgan uses when the GPU memory runs out; none = vsgan's default (16). Not used for EGVSR; not available with Duplicate aware.
32bit – on/off on Full precision; off runs in half precision.
GPU ID device 0 – 8 0 CUDA device.
Undo resize – on/off on With several models: resize back to the input size between the models. The final resize back to the input size always happens.
Resizer – Bilinear, Bicubic, Point, Lanczos, Spline16, Spline36, Spline64, DPID Spline64 Kernel used to resize back to the input size.
Duplicate aware – on/off off Processes only the first frame of a run of near-identical frames and repeats the result.
Threshold – 0.001 – 1 0.001 Largest difference that still counts as a duplicate.
Requires: Luma scale / Matrix / Modulo – no, pc, tv / matrix list / 1 – 128 pc / no / 1 Overrides the range, matrix and size multiple Hybrid prepares for the model; "no" keeps Hybrid's choice.
Fix Color: Mode, Radius, Wavelets, Planes, Fast – average/wavelet, radius 1 – 100, wavelets 1 – 15 off (average, radius 10, wavelets 5, all planes) Matches the colors of the output to the clip before the model; Radius and Fast with average, Wavelets with wavelet.

Example

Model F:/Hybrid/64bit/vsgan_models/2x_AniScale_30000.pth selected (no model is set by default), otherwise default settings, on a 10-bit YUV 4:2:0 source (640x356):

# changing range from limited to full range for vsVSGANFilter
clip = core.resize.Bicubic(clip, format=vs.YUV420P10, range_in_s="limited", range_s="full")
# setting color range to PC (full) range.
prop_name = '_Range' if core.core_version.release_major >= 74 else '_ColorRange'
clip = core.std.SetFrameProps(clip=clip, **{prop_name: vs.RANGE_FULL})
# adjusting color space from YUV420P10 to RGBS for vsVSGANFilter
clip = core.resize.Bicubic(clip=clip, format=vs.RGBS, matrix_in_s="470bg", range_in_s="full", range_s="full")
# filtering using VSGANFilter
from vsgan import ESRGAN
vsgan = ESRGAN(clip=clip, device="cuda:0")
model = "F:/Hybrid/64bit/vsgan_models/2x_AniScale_30000.pth"
vsgan.load(model)
vsgan.apply() # 1280x712
clip = vsgan.clip
# undo resizing 1280x712 to 640x356
clip = core.resize.Spline64(clip=clip, width=640, height=356) # resolution 640x356
# making sure 0-1 limits are respected
clip = core.vszip.Limiter(clip=clip, min=[0,0,0], max=[1,1,1])
# adjusting output color from RGBS to YUV420P10 for NVEncModel
clip = core.resize.Bicubic(clip=clip, format=vs.YUV420P10, matrix_s="470bg", range_in_s="full", range_s="full", dither_type="error_diffusion")
# changing range from full to limited range for matching filter target luma scale
clip = core.resize.Bicubic(clip, format=vs.YUV420P10,range_in_s="full", range_s="limited")
# setting color range to TV (limited) range.
prop_name = '_Range' if core.core_version.release_major >= 74 else '_ColorRange'
clip = core.std.SetFrameProps(clip=clip, **{prop_name: vs.RANGE_LIMITED})

How Hybrid calls it

  • The clip is converted to 32-bit float RGB (16-bit float with 32bit off), full range; there is no size multiple unless Requires sets one.
  • File names containing "EGVSR" use the EGVSR network, all others ESRGAN. The device is written as "cuda:" plus the GPU ID.
  • Each model works on the output of the previous one, with a new network object per model. With Multi and Undo resize the clip is brought back to the input size between the models.
  • With Duplicate aware Hybrid's duplicate-aware script runs the model chain only on frames that are not duplicates.
  • The color fix runs last.
  • VSGAN also offers the common options (Masked, ApplyOnlyTo, Merge, OnRegion, Gimmick).
  • The ONNX counterpart for user-supplied models is vsMLRT.

Background

vsgan

vsgan (rlaphoenix) is a PyTorch module for super-resolution and restoration in VapourSynth. The installed version offers the ESRGAN and EGVSR networks; the ESRGAN loader detects from the model file whether it is a classic ESRGAN or a compact Real-ESRGAN network. vsgan first tries the whole frame and only on a CUDA out-of-memory error splits it into parts; the overlap hides the seams between them. EGVSR works on intervals of frames instead.

Sources