MaskUpscalePlugin

Segmentation mask upscale plugin

Performs smart upscaling of a segmentation mask on a pixel level using input image as a context provider. Increases the resolution of a segmentation mask preserving the consistency of foreground and background regions. BodyMaskHDPlugin is a processing (pre-render) stage plugin that can be attached to a @geenee/armature!ShaderRenderer. It depends on MaskUploadPlugin that must be attached to a renderer. High resolution mask will replace current maskTex within tracking results as described in MaskUploadPlugin. It can be utilized by plugins next in the rendering pipeline. Higher resolution segmentation increases quality and accuracy of partial patching and occluder components of the rendering.

Extends

• Plugin<SegmentationResult>

Constructors

new MaskUpscalePlugin()

new MaskUpscalePlugin(thresh, steps, size): MaskUpscalePlugin

Constructor

Parameters

• thresh: number = 0.55

Foreground threshold

• steps: number = 2

Number of x2 upscale steps

• size: Size = ...

Size of input segmentation mask

Returns

MaskUpscalePlugin

Overrides

Plugin.constructor

Properties

loaded

loaded: boolean

Loaded state

Inherited from

Plugin.loaded

---

ordinal

ordinal: number

Ordinal number

Inherited from

Plugin.ordinal

---

renderer?

protected optional renderer: Renderer<SegmentationResult>

Renderer loaded the plugin

Inherited from

Plugin.renderer

---

size

protected size: Size

Size of input segmentation mask

---

steps

protected steps: number = 2

Number of x2 upscale steps

---

thresh

protected thresh: number = 0.55

Foreground threshold

Methods

dispose()

dispose(): void

Dispose video plugin

Returns

void

Inherited from

Plugin.dispose

---

load()

load(renderer): Promise<void>

Initialize plugin

Initializes resources required for shader effect.

Parameters

• renderer: Renderer<SegmentationResult>

Renderer this plugin is attached to

Returns

Promise<void>

Promise resolving when initialization is finished

Overrides

Plugin.load

---

setupCamera()

setupCamera(ratio, angle): void

Set camera parameters

Could be overridden to adjust plugin’s pipeline.

Parameters

• ratio: number

Aspect ration of input video

• angle: number

Vertical field of view in radians

Returns

void

Inherited from

Plugin.setupCamera

---

setupVideo()

setupVideo(size): void

Set video size

Could be overridden to adjust plugin’s pipeline.

Parameters

• size: Size

Resolution of input video

Returns

void

Inherited from

Plugin.setupVideo

---

unload()

unload(): void

Reset plugin

Releases all resources and instances created in load().

Returns

void

Overrides

Plugin.unload

---

update()

update(result, stream): Promise<void>

Upscale segmentation masks

Performs smart pixel-level upscaling of a segmentation mask to a higher resolution. Replaces maskTex of tracks with HD mask.

Parameters

• result: SegmentationResult

Results of video processing

• stream: HTMLCanvasElement

Captured video frame

Returns

Promise<void>

Overrides

Plugin.update