Vapoursynth HINet: Difference between revisions

From wiki.selur.net
Jump to navigation Jump to search
torch filters are Windows only (via update-page on MediaWiki MCP Server)
Update to current Hybrid behaviour; code as <pre> (via update-page on MediaWiki MCP Server)
 
Line 34: Line 34:
{| class="wikitable"
{| class="wikitable"
! Control !! Range !! Hybrid default !! Meaning
! Control !! Range !! Hybrid default !! Meaning
|-
| Method || difference, replace || difference || ''difference'': only the change HINet makes in the scaled-down picture is scaled up and added to the original frame, so everything else keeps its full detail. ''replace'': the processed picture is scaled back up and replaces the frame (softer).
|-
|-
| Resolution mode || divider, manual || divider || How the frame is resized before the network.
| Resolution mode || divider, manual || divider || How the frame is resized before the network.
Line 52: Line 50:
== Example ==
== Example ==
Default settings on a 10-bit YUV 4:2:0 source (640×356, padded to mod 8):
Default settings on a 10-bit YUV 4:2:0 source (640×356, padded to mod 8):
<syntaxhighlight lang="python">
<pre>
clip = core.std.AddBorders(clip=clip, left=0, right=0, top=2, bottom=2) # add borders to achieve mod 8 (vsHINet) - 640x360
clip = core.std.AddBorders(clip=clip, left=0, right=0, top=2, bottom=2) # add borders to achieve mod 8 (vsHINet) - 640x360
# adjusting color space from YUV420P10 to RGBS for vsHINet
# adjusting color space from YUV420P10 to RGBS for vsHINet
Line 59: Line 57:
clip = HINet(clip, device_index=0) #  call 0
clip = HINet(clip, device_index=0) #  call 0
clip = core.std.Crop(clip=clip, left=0, right=0, top=2, bottom=2) # removing added borders from mod requirement (vsHINet) -  640x356
clip = core.std.Crop(clip=clip, left=0, right=0, top=2, bottom=2) # removing added borders from mod requirement (vsHINet) -  640x356
</syntaxhighlight>
</pre>


== How Hybrid calls it ==
== How Hybrid calls it ==
* The clip is padded to mod 8 and converted to 32-bit float RGB, the only format HINet takes, and cropped and converted back afterwards.
* The clip is padded to mod 8 and converted to 32-bit float RGB, the only format HINet takes, and cropped and converted back afterwards.
* The device is always written; mode, FP16 and the tile values only when they differ from the defaults (tiles only when both sizes are above 0).
* The device is always written; mode, FP16 and the tile values only when they differ from the defaults (tiles only when both sizes are above 0).
* With Scratch the network is wrapped: resize, levels adjustment, invert, HINet, invert, the exact reverse levels adjustment, and then – depending on ''Method'' – either the difference to the prepared picture is scaled up and added to the source, or the picture is scaled back up.
* With Scratch the network is wrapped: resize, levels adjustment, invert, HINet, invert, reverse levels adjustment, resize back.
* The whole chain (Scratch preparation, the HINet calls, the Scratch undo) runs as one step, so the mod-8 padding is removed only after all of it and the common options apply to the whole chain.
* HINet also offers the [[Vapoursynth common filter options|common options]] (Masked, ApplyOnlyTo, Merge, OnRegion, Gimmick).
* HINet also offers the [[Vapoursynth common filter options|common options]] (Masked, ApplyOnlyTo, Merge, OnRegion, Gimmick).



Latest revision as of 09:50, 16 November 2026

HINet (torch) runs Megvii's Half Instance Normalization Network, a neural network for image restoration. Hybrid offers its four trained models: deblurring (GoPro or REDS), denoising (SIDD) and deraining (Rain13k). The derain model can also be used to remove film scratches.

Filtering → Vapoursynth → DeNoise

HINet controls in Hybrid
HINet controls in Hybrid

What it does

Every frame is passed through the network of the chosen mode; there is no temporal part. With Scratch (derain mode only), Hybrid prepares the frame so that vertical film scratches look like rain streaks, lets the derain model remove them, and undoes the preparation afterwards.

Requirements

  • Windows, Hybrid's torch add-on and an NVIDIA GPU with CUDA; the models are about 355 MB each.

Settings

Control Parameter Range / values Hybrid default Meaning
Mode mode deblur - GoPro, deblur - REDS, denoise, derain deblur - GoPro The model that is used.
FP16 fp16 on/off off Half precision; not available for the deblur modes.
CPU device_type on/off off Run on the CPU (very slow).
Device device_index 0 – 4 0 CUDA device.
Tile (x/y) tile_x, tile_y 0 – 8192 0 / 0 Tile size to save GPU memory; 0 = no tiling.
Tile pad tile_pad 0 – 64 16 Overlap of the tiles; only with tiling.
Count – 1 – 10 1 Applies HINet several times in a row.

Scratch (derain only)

Control Range Hybrid default Meaning
Resolution mode divider, manual divider How the frame is resized before the network.
Divider (width / height) 1 – 100 3.5 / 1.5 The frame size is divided by these factors.
Resolution (manual) 0 – 8192 0 / 0 Fixed size; 0 keeps the source width or height.
In (min / max) 0 – 1 0.11 / 0.82 Input range of the levels adjustment.
Out (min / max) 0 – 1 0 / 1 Output range of the levels adjustment.
Gamma 0 – 1 0.8 Gamma of the levels adjustment.

Example

Default settings on a 10-bit YUV 4:2:0 source (640×356, padded to mod 8):

clip = core.std.AddBorders(clip=clip, left=0, right=0, top=2, bottom=2) # add borders to achieve mod 8 (vsHINet) - 640x360
# adjusting color space from YUV420P10 to RGBS for vsHINet
clip = core.resize.Bicubic(clip=clip, format=vs.RGBS, matrix_in_s="470bg", range_in_s="limited", range_s="full")
# denoising/debluring/deraining using HINet
clip = HINet(clip, device_index=0) #  call 0
clip = core.std.Crop(clip=clip, left=0, right=0, top=2, bottom=2) # removing added borders from mod requirement (vsHINet) -  640x356

How Hybrid calls it

  • The clip is padded to mod 8 and converted to 32-bit float RGB, the only format HINet takes, and cropped and converted back afterwards.
  • The device is always written; mode, FP16 and the tile values only when they differ from the defaults (tiles only when both sizes are above 0).
  • With Scratch the network is wrapped: resize, levels adjustment, invert, HINet, invert, reverse levels adjustment, resize back.
  • HINet also offers the common options (Masked, ApplyOnlyTo, Merge, OnRegion, Gimmick).

Background

HINet

HINet (Megvii, 2021) builds on Half Instance Normalization blocks in a two-stage network and won the NTIRE 2021 deblurring challenge (track 2). The models were trained on SIDD (denoising), GoPro and REDS (deblurring) and Rain13k (deraining).

vs-hinet

HolyWu's vs-hinet (archived) wraps HINet for VapourSynth with PyTorch and only accepts 32-bit float RGB. Frames that are not divisible by 16 are padded internally.

Sources