Vapoursynth DPIR Denoise: Difference between revisions

From wiki.selur.net
Jump to navigation Jump to search
New filter page (via create-page on MediaWiki MCP Server)
 
Update to current Hybrid behaviour; code as <pre> (via update-page on MediaWiki MCP Server)
 
(One intermediate revision by the same user not shown)
Line 10: Line 10:


== Requirements ==
== Requirements ==
* Hybrid's torch add-on and an NVIDIA GPU with CUDA.
* Windows, Hybrid's torch add-on and an NVIDIA GPU with CUDA.
* TRT additionally needs Torch-TensorRT from the add-on; it cannot be combined with TensorRT in vs-mlrt filters in the same script.
* TRT additionally needs Torch-TensorRT from the add-on; it cannot be combined with TensorRT in vs-mlrt filters in the same script.


Line 34: Line 34:
== Example ==
== Example ==
Default settings on a 10-bit YUV 4:2:0 source:
Default settings on a 10-bit YUV 4:2:0 source:
<syntaxhighlight lang="python">
<pre>
# adjusting color space from YUV420P10 to RGBS for vsDPIRDenoise
# adjusting color space from YUV420P10 to RGBS for vsDPIRDenoise
clip = core.resize.Bicubic(clip=clip, format=vs.RGBS, matrix_in_s="470bg", range_in_s="limited", range_s="full")
clip = core.resize.Bicubic(clip=clip, format=vs.RGBS, matrix_in_s="470bg", range_in_s="limited", range_s="full")
# denoising using DPIRDenoise
# denoising using DPIRDenoise
clip = DPIR(clip, strength=5.000, task="denoise", device_index=0)
clip = DPIR(clip, strength=5.000, task="denoise", device_index=0)
</syntaxhighlight>
</pre>


== How Hybrid calls it ==
== How Hybrid calls it ==

Latest revision as of 09:48, 16 November 2026

DPIR (torch) removes noise with DRUNet, the denoiser network of DPIR (Deep Plug-and-Play Image Restoration). It runs on an NVIDIA GPU through PyTorch and keeps detail well even at high noise levels.

Filtering → Vapoursynth → DeNoise

DPIR (torch) controls in Hybrid
DPIR (torch) controls in Hybrid

What it does

Every frame is passed through DRUNet (the color model for RGB, the gray model for gray clips). Strength tells the network how much noise to expect; higher values smooth more and lose more detail.

Requirements

  • Windows, Hybrid's torch add-on and an NVIDIA GPU with CUDA.
  • TRT additionally needs Torch-TensorRT from the add-on; it cannot be combined with TensorRT in vs-mlrt filters in the same script.

Settings

Control Parameter Range / values Hybrid default Meaning
Strength strength 0.001 – 1000 5 Noise level the network assumes, on a 0–255 scale; useful values are about 0–50.
Tile (w/h) tile 0 – 8192 0 / 0 Splits the frame into tiles to save GPU memory; 0 = no tiling.
Tile pad tile_pad 0 – 64 8 Overlap of the tiles; only with tiling.
Streams num_streams 1 – 64 1 Number of CUDA streams.
Device device_index 0 – 4 0 CUDA device.
TRT trt on/off off Uses TensorRT; the engine is built on the first run and cached.
FP16 – on/off off Runs the network in half precision (16-bit float clip); faster and needs less memory.

Example

Default settings on a 10-bit YUV 4:2:0 source:

# adjusting color space from YUV420P10 to RGBS for vsDPIRDenoise
clip = core.resize.Bicubic(clip=clip, format=vs.RGBS, matrix_in_s="470bg", range_in_s="limited", range_s="full")
# denoising using DPIRDenoise
clip = DPIR(clip, strength=5.000, task="denoise", device_index=0)

How Hybrid calls it

  • The clip is converted to 32-bit float RGB (gray clips to 32-bit float gray), or to 16-bit float with FP16, and back afterwards.
  • Strength, task and device are always written; tiles, tile pad, streams and TRT only when they differ from the defaults.
  • With TRT the engines are cached in the engine path from Hybrid's path settings (or the temp folder).
  • DPIR also offers the common options (Masked, ApplyOnlyTo, Merge, OnRegion, Gimmick).
  • The same network is available without PyTorch as DPIRDenoise (mlrt); for compression artifacts see DPIR Deblock.

Background

DPIR

DPIR (Zhang et al., TPAMI 2021) uses the DRUNet denoiser as a plug-in prior for image restoration tasks such as deblurring and super-resolution. DRUNet was trained on noise levels of about 0–50 and is one of the strongest Gaussian denoisers.

vs-dpir

HolyWu's vs-dpir runs DPIR in VapourSynth with PyTorch, optionally with TensorRT. TensorRT engines are built per model, size, precision and GPU and cannot be moved to another machine.

Sources