Vapoursynth CDEF: Difference between revisions

From wiki.selur.net
Jump to navigation Jump to search
New filter page (via create-page on MediaWiki MCP Server)
 
Update to current Hybrid behaviour; code as <pre> (via update-page on MediaWiki MCP Server)
 
Line 30: Line 30:
== Example ==
== Example ==
Default settings on a 10-bit YUV 4:2:0 source (640x356):
Default settings on a 10-bit YUV 4:2:0 source (640x356):
<syntaxhighlight lang="python">
<pre>
clip = core.std.AddBorders(clip=clip, left=0, right=0, top=2, bottom=2) # add borders to achieve mod 8 (vsCDEF) - 640x360
clip = core.std.AddBorders(clip=clip, left=0, right=0, top=2, bottom=2) # add borders to achieve mod 8 (vsCDEF) - 640x360
clip = core.fb.FillBorders(clip=clip, left=0, right=0, top=2, bottom=2) # fill borders
clip = core.fb.FillBorders(clip=clip, left=0, right=0, top=2, bottom=2) # fill borders
Line 36: Line 36:
clip = core.cdef.CDEF(clip, pri_strength=[3,1], sec_strength=[1,0], pri_damping=[2], sec_damping=[1])
clip = core.cdef.CDEF(clip, pri_strength=[3,1], sec_strength=[1,0], pri_damping=[2], sec_damping=[1])
clip = core.std.Crop(clip=clip, left=0, right=0, top=2, bottom=2) # removing added borders from mod requirement (vsCDEF) -  640x356
clip = core.std.Crop(clip=clip, left=0, right=0, top=2, bottom=2) # removing added borders from mod requirement (vsCDEF) -  640x356
</syntaxhighlight>
</pre>


== How Hybrid calls it ==
== How Hybrid calls it ==

Latest revision as of 09:48, 16 November 2026

CDEF (Constrained Directional Enhancement Filter) is the in-loop deringing filter of the AV1 codec, here as a stand-alone VapourSynth filter built on SVT-AV1 code. It removes ringing and mosquito noise next to sharp edges while the edges stay sharp.

Filtering → Vapoursynth → DeRing

CDEF controls in Hybrid
CDEF controls in Hybrid

What it does

For every 8×8 block CDEF finds the main edge direction. The primary filter smooths along that direction, the secondary filter along the two directions 45° off it. Because it smooths along edges and not across them, the edges survive. Damping limits how far a single sample can pull a pixel, so strong detail is kept.

Settings

Control Parameter Range / values Hybrid default Meaning
Pri. strength Y pri_strength (luma) 0 – 64 3 Strength of the along-edge filter on luma; raise it first when the result is too weak. 0 = off. AV1 itself uses 0 – 15.
Pri. strength C pri_strength (chroma) 0 – 64 1 Same for U and V.
Pri. damping pri_damping 0 – 16 2 Damping of the primary filter for all planes; higher lets more correction through, lower protects detail.
Sec. strength Y sec_strength (luma) 0 – 64 1 Strength of the diagonal filter on luma; set it to 0 first when the result is too soft. AV1 uses 0, 1, 2 or 4.
Sec. strength C sec_strength (chroma) 0 – 64 0 Same for U and V; 0 = off.
Sec. damping sec_damping 0 – 16 1 Damping of the secondary filter for all planes.
Planes planes all, chroma only, luma only, only U, only V, only Y,U, only Y,V all Planes that are filtered. The directions always come from luma; planes that are not selected are copied from the source, so leaving planes out saves no time.

Example

Default settings on a 10-bit YUV 4:2:0 source (640x356):

clip = core.std.AddBorders(clip=clip, left=0, right=0, top=2, bottom=2) # add borders to achieve mod 8 (vsCDEF) - 640x360
clip = core.fb.FillBorders(clip=clip, left=0, right=0, top=2, bottom=2) # fill borders
# applying deringing using CDEF
clip = core.cdef.CDEF(clip, pri_strength=[3,1], sec_strength=[1,0], pri_damping=[2], sec_damping=[1])
clip = core.std.Crop(clip=clip, left=0, right=0, top=2, bottom=2) # removing added borders from mod requirement (vsCDEF) -  640x356

How Hybrid calls it

  • The plugin needs 10- or 12-bit integer YUV. Hybrid raises 8-bit input to 10 bit and lowers float and 14-bit input to 12 bit; the chroma subsampling stays. Gray sources are converted to YUV.
  • 16-bit input stays 16 bit: CDEF runs on a 12-bit copy and only the change is added back, so untouched pixels stay bit-identical.
  • Sizes that are not a multiple of 8 are padded with edge pixels and cropped back afterwards, so the padding creates no false edge.
  • Strengths are on the 8-bit scale at every bit depth; the plugin scales them itself. The four strength and damping values are always written, Planes only when it is not "all".
  • CDEF also offers the common options (Masked, ApplyOnlyTo, Merge, OnRegion, Gimmick).

Background

Ranges

The plugin accepts wider values than AV1 (primary strength 0 – 15, secondary 0/1/2/4, damping 3 – 6). The ranges in Hybrid are wider still; the defaults are those of the plugin author. The author shows CDEF combined with an MVTools denoiser for heavy mosquito noise.

Sources