Plugin
Generic renderer plugin
Plugin can be attached to a PluginRenderer instance. Usually they perform simple tasks that can be separated from bigger app context into atomic building blocks, for example control object on a scene to follow (be attached to) user’s head, apply image effect (smoothing, beautification), recognize gestures or poses, notify about state changes or perform other kinds of transformations, pre/post-processing, or analyzes with a 3D scene, video stream or raw data from a Processor. Plugin is a abstraction level to single out ready-made helpers that can be reused as atomic building blocks of an application. Plugins are very similar to Renderer and also should implement two basic methods load() and update(). Renderer initializes a plugin calling load() and providing itself as argument, plugin in turn remembers renderer it’s attached to and gets access to required resources of the renderer, for example 2d or webgl context of a canvas or reference to a 3d scene. In update() method plugins implement actual logic given processing results. Renderer applies attached plugins in order defined by their ordinal number. In some cases priority of plugins execution is important, e.g. there can be a plugin that filters results of pose processing by some constraint, let’s say it passes only poses where upper body is in the field of view and asks a user to step back for better virtual try-on experience, this plugin should update poses before plugin that renders virtual apparel.
Extended by
Section titled “Extended by”VideoPluginShaderPluginScenePluginBrightnessPluginCallbackPluginFaceFilterPluginMaskFilterPluginMaskSharpPluginMaskUploadPluginMaskUpscalePluginPerfDevPluginPoseFilterPlugin
Type Parameters
Section titled “Type Parameters”ResultT
Section titled “ResultT”ResultT extends object = { }
Type of processing results
Constructors
Section titled “Constructors”Constructor
Section titled “Constructor”new Plugin<
ResultT>():Plugin<ResultT>
Returns
Section titled “Returns”Plugin<ResultT>
Properties
Section titled “Properties”loaded
Section titled “loaded”loaded:
boolean=false
Loaded state
ordinal
Section titled “ordinal”ordinal:
number=PluginOrdinal.Render
Ordinal number
renderer?
Section titled “renderer?”
protectedoptionalrenderer:Renderer<ResultT>
Renderer loaded the plugin
Methods
Section titled “Methods”dispose()
Section titled “dispose()”dispose():
void
Dispose video plugin
Returns
Section titled “Returns”void
load()
Section titled “load()”load(
renderer):Promise<void>
Initialize plugin
Initializes resources/instances needed by plugin. Overridden by derived classes for particular task.
Parameters
Section titled “Parameters”renderer
Section titled “renderer”Renderer<ResultT>
Renderer this plugin is attached to
Returns
Section titled “Returns”Promise<void>
Promise resolving when initialization is finished
setupCamera()
Section titled “setupCamera()”setupCamera(
ratio,angle):void
Set camera parameters
Could be overridden to adjust plugin’s pipeline.
Parameters
Section titled “Parameters”number
Aspect ration of input video
number
Vertical field of view in radians
Returns
Section titled “Returns”void
setupVideo()
Section titled “setupVideo()”setupVideo(
size):void
Set video size
Could be overridden to adjust plugin’s pipeline.
Parameters
Section titled “Parameters”Resolution of input video
Returns
Section titled “Returns”void
unload()
Section titled “unload()”unload():
void
Reset plugin
Releases all resources/instances created in load(). Overridden by derived classes for particular task.
Returns
Section titled “Returns”void
update()
Section titled “update()”update(
result,stream):Promise<void>
Update
Main method implementing the logic of the plugin. Overridden by derived classes for particular task.
Parameters
Section titled “Parameters”result
Section titled “result”ResultT
Results of video processing
stream
Section titled “stream”HTMLCanvasElement
Captured video frame
Returns
Section titled “Returns”Promise<void>
Promise resolving when update is finished