Vapoursynth BM3D

From wiki.selur.net
Jump to navigation Jump to search

BM3D (Block-Matching and 3D filtering) is a strong, slow denoiser that keeps texture well. It groups similar blocks of the picture and filters each group together; with a radius above 0 it also searches the neighbor frames (V-BM3D).

Filtering → Vapoursynth → DeNoise

BM3D controls in Hybrid
BM3D controls in Hybrid

What it does

  1. Basic estimate: for every block, similar blocks are searched and stacked; the stack is transformed, small coefficients are removed, and the blocks are put back.
  2. Final estimate (Refine passes): the same search on the basic estimate, now with Wiener filtering, which gives a cleaner result.

RGB can be filtered in the opponent color space (OPP), which separates brightness from color and denoises better.

Settings

General

Control Parameter Range / values Hybrid default Meaning
Color space – YUV, OPP YUV Working format: OPP filters RGB in the opponent color space.
CBM3D chroma on/off off Reuses the luma block matching for U and V; only with vszip, bm3dvk, vsfeel and bm3dmetal.

Basic (0 = automatic: the profile value with bm3d and bm3dvk2, the plugin default otherwise)

Control Parameter Range / values Hybrid default Meaning
Sigma Y/U/V sigma 0 – 100 5 / 5 / 5 Strength per plane on the 8-bit scale; 0 leaves the plane unchanged.
Radius radius1 0 – 16 0 Frames on each side (V-BM3D); needs much more time and memory.
Profile profile1 fast, lc, np, high, vn fast Parameter preset; bm3d and bm3dvk2 only.
Block size / Group size block_size1, group_size1 0 – 64 / 0 – 256 0 Block size and blocks per group; bm3d and bm3dvk2 only.
Block step block_step1 0 – 64 0 Distance between processed blocks; smaller is better and slower.
BM range / BM step bm_range1, bm_step1 0 – 64 0 Search range and step of the block matching (BM step: bm3d and bm3dvk2 only).
PS num / PS range / PS step ps_num1, ps_range1, ps_step1 0 – 64 0 Predictive search in the neighbor frames; only with a radius above 0 (PS step: bm3d and bm3dvk2 only).
th_mse th_mse1 0 – 65535 0 Similarity threshold of the block matching; bm3d and bm3dvk2 only.
hard_thr hard_thr 0 – 100 0 Hard threshold of the basic estimate; bm3d and bm3dvk2 only.

Final

Control Parameter Range / values Hybrid default Meaning
Refine refine 0 – 5 1 Number of final passes; 0 = basic estimate only (the other final controls are disabled).
Own sigma, Sigma Y/U/V sigma2 0 – 100 off, 5 / 5 / 5 Separate strength for the final passes.
Radius radius2 -1 – 16 -1 (= Basic) Temporal radius of the final passes.
Profile, block and search values profile2, …2 as in Basic = Basic 0 (or "same as Basic") takes the Basic value; th_mse keeps its own automatic value.

Tools: the BM3D implementation – vszip (CPU, default on Windows and Linux), bm3d (CPU), bm3dvk2 (Vulkan, VapourSynth R81, default on macOS), bm3dvk (Vulkan), vsfeel (Vulkan), bm3dmetal (macOS) – and a Device field for the GPU implementations. The GPU implementations are offered when a Vulkan 1.4 GPU is found.

Example

Default settings on a 10-bit YUV 4:2:0 source:

# adjusting color space from YUV420P10 to YUV420PS for vsBM3D
clip = core.resize.Bicubic(clip=clip, format=vs.YUV420PS)
# denoising using BM3D
clip = denoise.BM3D(clip, tools={'bm3d': 'vszip'})

How Hybrid calls it

  • The clip is converted to 32-bit float: RGB for OPP; YUV 4:4:4 when U/V are filtered with bm3d or bm3dvk2; otherwise the float version of the current subsampling.
  • Only values that differ from the defaults are written; the sigmas stay on the 8-bit scale. Final values left at 0 take the Basic value, final options are only written with Refine above 0, and the PS values only with a radius above 0.
  • With bm3d and bm3dvk2 the color matrix of a YUV clip is passed.
  • An implementation that cannot start (for example a GPU port without a usable GPU) is skipped with a warning, and the next one is used.
  • BM3D also offers the common options (Masked, ApplyOnlyTo, Merge, OnRegion, Gimmick).

Background

VapourSynth-BM3D

The original plugin implements the basic and final estimates as separate filters, the V-BM3D variants for several frames, and the conversion to and from the opponent color space. It recommends OPP for the best quality, needs 4:4:4 to filter chroma, and uses a lot of memory for V-BM3D. Its own default strength is sigma 10.

Ports

vszip offers a CPU version with the parameter set of the BM3DCUDA family (32-bit float, CBM3D with chroma); its output is close to the GPU versions but not identical. bm3dvk2 is a Vulkan port that runs basic and final estimate in one call, takes Gray or RGB, and knows the full parameter set of the original including the profiles.

Sources