MaskSmoothPlugin
Smoothing plugin for segmentation mask
Performs a smoothing convolution on a segmentation mask. Plugin depends on MaskUploadPlugin that must be attached to the renderer to upload mask buffer in texture. One may utilize MaskUpscalePlugin providing higher resolution segmentation mask that significantly increases mask’s accuracy. Plugin may be combined with other mask post-processing plugins. Order in which operation on mask are applied is defined by order of plugins’ attachment.
Extends
Section titled “Extends”MaskProcessPlugin
Constructors
Section titled “Constructors”Constructor
Section titled “Constructor”new MaskSmoothPlugin(
smooth,size,radius):MaskSmoothPlugin
Constructor
Parameters
Section titled “Parameters”smooth
Section titled “smooth”number = 2.0
Smoothing strength
Size = ...
Size of segmentation mask
radius
Section titled “radius”number = ...
Convolution radius
Returns
Section titled “Returns”MaskSmoothPlugin
Overrides
Section titled “Overrides”MaskProcessPlugin.constructor
Properties
Section titled “Properties”fragSrc?
Section titled “fragSrc?”
protectedoptionalfragSrc:string
Code of fragment shader (copy shader by default)
Inherited from
Section titled “Inherited from”MaskProcessPlugin.fragSrc
inputs
Section titled “inputs”
protectedinputs:string[]
Shader texiwure inputs (names of sampler uniforms)
Inherited from
Section titled “Inherited from”MaskProcessPlugin.inputs
loaded
Section titled “loaded”loaded:
boolean
Loaded state
Inherited from
Section titled “Inherited from”MaskProcessPlugin.loaded
ordinal
Section titled “ordinal”ordinal:
number
Ordinal number
Inherited from
Section titled “Inherited from”MaskProcessPlugin.ordinal
radius
Section titled “radius”
protectedradius:number
Convolution radius
renderer?
Section titled “renderer?”
protectedoptionalrenderer:Renderer<SegmentationResult>
Renderer loaded the plugin
Inherited from
Section titled “Inherited from”MaskProcessPlugin.renderer
shader?
Section titled “shader?”
protectedoptionalshader:ShaderProgram
Processing shader
Inherited from
Section titled “Inherited from”MaskProcessPlugin.shader
protectedsize:Size
Size of segmentation mask texture
Inherited from
Section titled “Inherited from”MaskProcessPlugin.size
uniforms
Section titled “uniforms”
protecteduniforms:object={}
Shader uniforms as name-type map object
Index Signature
Section titled “Index Signature”[key: string]: UniformType
Inherited from
Section titled “Inherited from”MaskProcessPlugin.uniforms
vertSrc?
Section titled “vertSrc?”
protectedoptionalvertSrc:string
Vertex shader source (copy shader by default)
Inherited from
Section titled “Inherited from”MaskProcessPlugin.vertSrc
Methods
Section titled “Methods”dispose()
Section titled “dispose()”dispose():
void
Dispose video plugin
Returns
Section titled “Returns”void
Inherited from
Section titled “Inherited from”MaskProcessPlugin.dispose
load()
Section titled “load()”load(
renderer):Promise<void>
Initialize plugin
Initializes smoothing shader programs.
Parameters
Section titled “Parameters”renderer
Section titled “renderer”Renderer this plugin is attached to
Returns
Section titled “Returns”Promise<void>
Promise resolving when initialization is finished
Overrides
Section titled “Overrides”MaskProcessPlugin.load
process()
Section titled “process()”process(
result):void
Update segmentation masks
Updates the mask applying smoothing convolution.
Replaces maskTex of tracks with the updated mask.
Parameters
Section titled “Parameters”result
Section titled “result”Results of video processing
Returns
Section titled “Returns”void
Overrides
Section titled “Overrides”MaskProcessPlugin.process
setupCamera()
Section titled “setupCamera()”setupCamera(
ratio,angle):void
Set camera parameters
Could be overridden to adjust plugin’s pipeline.
Parameters
Section titled “Parameters”number
Aspect ration of input video
number
Vertical field of view in radians
Returns
Section titled “Returns”void
Inherited from
Section titled “Inherited from”MaskProcessPlugin.setupCamera
setupVideo()
Section titled “setupVideo()”setupVideo(
size):void
Set video size
Could be overridden to adjust plugin’s pipeline.
Parameters
Section titled “Parameters”Resolution of input video
Returns
Section titled “Returns”void
Inherited from
Section titled “Inherited from”MaskProcessPlugin.setupVideo
unload()
Section titled “unload()”unload():
void
Reset plugin
Releases all resources initialized in load().
Returns
Section titled “Returns”void
Overrides
Section titled “Overrides”MaskProcessPlugin.unload
update()
Section titled “update()”update(
result,stream):Promise<void>
Update segmentation masks
Updates segmentation mask applying processing shader.
The mask is updated in place within tracking result
object, therefore next plugins in the pipeline will
access updated mask and mask processings can be chained.
Parameters
Section titled “Parameters”result
Section titled “result”Results of video processing
stream
Section titled “stream”HTMLCanvasElement
Captured video frame
Returns
Section titled “Returns”Promise<void>
Inherited from
Section titled “Inherited from”MaskProcessPlugin.update