Skip to main content

Module: @geenee/bodyrenderers-common

Renderer​

@geenee/armature!Renderer is the core visualization and logical part of any application. It's attached to the @geenee/armature!Engine. Basically, renders define two methods load() and update(). The first one is used to initialize assets and prepare the scene (lightning, environment map). The second is used to update the scene according to results of video processing. This's where all the logic happens. Renderers can be extended with plugins. Plugins do simple rendering task, for example add object to that follows the head or render avatar overlay.

This package defines common utilities, types, and helpers used by renderers. Additionally, it contains universal plugins that provide features not related to final rendering. Ready-made renderers and plugins are provided by the following packages:

Common Plugins​

BodyPatchPlugin​

Plugin patches (inpaints/erases) foreground region of image defined by body segmentation mask from @geenee/bodyprocessors!PoseProcessor. It can be used in avatar virtual try to remove parts of a user's body that stick out (not covered). Evaluation of body segmentation must be enabled in init() method enabling (@geenee/bodyprocessors!PoseParams#mask) flag.

BodyPatchHDPlugin​

Advanced version of BodyPatchPlugin that uses high resolution segmentation mask evaluated by BodyMaskHDPlugin. High resolution segmentation significantly increases accuracy of patching and reduces patch region. BodyMaskHDPlugin must be added to ShaderRenderer pipeline.

BodyMaskHDPlugin​

Performs smart upscaling of a segmentation mask on a pixel level using input image as a context provider. Increases the resolution of a segmentation mask preserving the consistency of foreground and background regions. BodyMaskHDPlugin is a processing (pre-render) stage plugin that can be attached to a @geenee/armature!ShaderRenderer. High resolution mask will be added to @geenee/bodyprocessors!Pose of tracking results and can be used by plugins next in the rendering pipeline. Higher resolution segmentation increases accuracy of patching and occluder components of the pipeline. Evaluation of body segmentation must be enabled when calling init() by setting @geenee/bodyprocessors!PoseParams#mask flag.

BrightnessPlugin​

Plugin estimates parameters of lighting (brightness) observed in the current image and calls the provided callback with these parameters. This plugin can be used for automatic adjustment of intensities of lights on the scene, for example in callback user can scale intensity properties of lights and environment map according to estimated brightness. One can set output range, estimated value will not go below the minimum and [min..max] range will be scaled to [min..1.0]. Adaptive low pass smoothing is applied to the result.

BilateralPlugin​

Plugin applying bilateral filter: non-linear, edge- preserving, and noise-reducing smoothing image filter.

Utility Plugins​

CallbackPlugin​

Simple plugin executing provided callback on processing results. Useful when, for example, application needs to show a message when no object is detected in camera's field of view or pose is not optimal for virtual try on.

PoseFilterPlugin​

Simple plugin filtering a list of detected poses by the provided predicate. Useful when application may need to ignore certain poses to provide the best quality of virtual try-on. For example to ensure that at least upper body of user is in the field of view.

FaceFilterPlugin​

Simple plugin filtering a list of detected faces by the provided predicate. Useful when application may need to ignore certain head poses, for example when rotation angle is too big for virtual try-on.

PerfDevPlugin​

Simple plugin measuring basic performance metrics of a renderer (fps). Executes provided callback when metrics are updated (every 30 rendered frames).

Classes​

Interfaces​

Functions​

maskRectUniform​

▸ maskRectUniform(b): [number, number, number, number]

Convert segmentation mask box to rect shader uniform

Parameters​

NameTypeDescription
bBoxMask box

Returns​

[number, number, number, number]

Mask rect shader uniform