<stdin> |

My Thoughts, Trials and Adventures

Vapoursynth - Merge clips using MaskedMerge

Posted at — Mar 3, 2020 | Last Modified on — Mar 7, 2021

std.MaskedMerge(clip clipa, clip clipb, clip mask[, int[] planes, bint first_plane=0, bint premultiplied=0])

MaskedMerge merges clipa with clipb using the per pixel weights in the mask, where 0 means that clipa is returned unchanged. If mask is a grayscale clip or if first_plane is true, the mask’s first plane will be used as the mask for merging all planes. The mask will be bilinearly resized if necessary.

Objective

To demonstrate the use of std.MaskedMerge function. I will show how you can merge two clips using a mask.

Two small clips (00:01:00 each) are extracted from Gemini Man (film) for example usage. No copyright infringement intended.

The code and resources used in this article are availabe in Github repo.
Video Files:
GM_clip_1.mkv
GM_clip_2.mkv

Requirements:

  1. We use lsmas.LWLibavSource to load the clip and save a frame as .png
clip_a = core.lsmas.LWLibavSource(script_path + "/GM_clip_1.mkv",threads=6)

you can use any other application/video player such as VLC player to save a screenshot.

Frame 214 from GM_clip_1.mkv

  1. Open the png in photoshop and select the area you want to mask. Right-click on the selected area and choose Layer via cut

  2. After removing the original layer, we are left with our required mask.

Frame 214 Mask in Photoshop

  1. You can view the mask in vapoursynth editor by using the snippet below.
mask = core.imwri.Read(script_path + "/mask.png",alpha=True)[1]

mask.set_output()

while using alpha=True with core.imwri.Read returns list with 2 clips, a normal RGB clip and Gray8 clip.the alpha clip is of index 1 (or -1).

  1. Putting it all together..
import vapoursynth as vs
import os

script_path = os.path.dirname(os.path.abspath(__file__))

core = vs.get_core()

clip_a = core.lsmas.LWLibavSource(script_path + "/GM_clip_1.mkv",threads=6)
clip_b = core.lsmas.LWLibavSource(script_path + "/GM_clip_2.mkv",threads=6)

mask = core.imwri.Read(script_path + "/mask.png",alpha=True)[1]

merged_clip = core.std.MaskedMerge(clip_a,clip_b,mask)
merged_clip.set_output()

Merged