Vapoursynth DDColor

From wiki.selur.net
Jump to navigation Jump to search

DDColor colorizes black-and-white video with the DDColor neural network ("Towards Photo-Realistic Image Colorization via Dual Decoders").

Filtering → Vapoursynth → Color → Coloring

DDColor controls in Hybrid
DDColor controls in Hybrid

What it does

Each frame is scaled down to a square working size and turned into a gray image. The network predicts the color (the a/b channels of the Lab color space); that color is scaled back to the full frame size and combined with the original lightness, so the detail and sharpness of the source are kept. Every frame is colorized on its own, without temporal smoothing, so colors can flicker; HAVC adds temporal stabilization on top of DDColor and other models.

Requirements

  • Windows with an NVIDIA graphics card (CUDA).
  • Windows, Hybrid's torch add-on (package vsddcolor with its two model files).

Settings

Control Parameter Range / values Hybrid default Meaning
Model model ModelScope, Artistic Artistic Which of the two trained DDColor models is used; Artistic usually gives more saturated colors.
Input size input_size 1 – 8192 512 Edge length of the square picture the network sees; the models were made for 512. It does not limit the output resolution.
FP16 – on/off off On: the clip is converted to half-precision RGB and the network runs in bfloat16 (faster, less memory). Off: 32-bit float.
Streams num_streams 1 – 64 1 Number of CUDA streams working in parallel.
GPU ID device_index 0 – 4 0 CUDA device to use.

Example

Default settings on a black-and-white 10-bit YUV 4:2:0 source:

# adjusting color space from YUV420P10 to RGBS for vsDDColor
clip = core.resize.Bicubic(clip=clip, format=vs.RGBS, matrix_in_s="470bg", range_in_s="limited", range_s="full")
# adding colors using DDColor
from vsddcolor import ddcolor
clip = ddcolor(clip, model=1)
# adjusting output color from RGBS to YUV420P10 for NVEncModel
clip = core.resize.Bicubic(clip=clip, format=vs.YUV420P10, matrix_s="470bg", range_in_s="full", range_s="limited", dither_type="error_diffusion")

How Hybrid calls it

  • The clip is converted to 32-bit float RGB (half precision with FP16) before the filter and back afterwards.
  • The model is always written; Input size, Streams and GPU ID only when they differ from the defaults.
  • DDColor also offers the common options (Masked, ApplyOnlyTo, Merge, OnRegion, Gimmick).

Background

vs-ddcolor

vs-ddcolor by HolyWu brings DDColor to VapourSynth via PyTorch (PyTorch 2.1.1+, VapourSynth R62+; models are downloaded with python -m vsddcolor). It accepts RGBH or RGBS clips and needs CUDA. Hybrid ships a version modified by Dan64 for HAVC, which adds a scene-change mode Hybrid does not use.

Sources