Release v0.7.1
Alignment for Hands
New HandAlignPlugin is a universal plugin aligning node’s armature and the hand pose estimated by HandProcessor. Plugin supports rigs compatible with Clo3D and Marvelous Designer avatars. This is the most common standard of rigs in cloth/apparel modeling software. At the moment the plugin is in “Work in Progress” state, we are optimizing alignment algorithms to provide more natural fitting of 3D models.
Background Replacement
We introduce the new shader plugin replacing background region of an image BgReplacePlugin. Evaluated segmentation mask defines image foreground that stays untouched. Foreground-background classification is based on two thresholds defining uncertainty interval. Probability above foreground threshold classifies pixel as foreground, below background threshold as background. FG pixels are kept untouched, BG pixels are replaced with corresponding pixels from the background texture. For pixels within uncertainty region weighted interpolation between image and background textures takes place. Weight is evaluated by scaling uncertainty interval and probability to [0..1]. BgReplacePlugin has universal API, it allocates and provides access to a texture which content will be used as background substitution, this texture can be loaded from any source like canvas, image, or video.
Deprecated Plugins
OccluderPlugin was in deprecated state for a while and has been removed in this release. Please, use OccluderMaterial or OccluderMaskPlugin directly to transform scene meshes into occluders, this is a more flexible and universal approach to control rendering of 3D objects in a scene.
Related PoseOutfitPlugin has also been removed. It was making child meshes of the attached scene node an occluders based on their names. The same logic can be implemented externally. One can use PoseAlignPlugin to align node’s rig with the detected pose and manually make required children meshes an occluders using OccluderMaterial or OccluderMaskMaterial provided by OccluderMaskPlugin.
Other Changes
Common utility methods used within armature alignment algorithms are moved to PoseUtils namespace within @geenee/bodyrenderers-babylon package to be available for all plugins fitting rigs into detected poses. This allows to share implementation and reuse the same API across plugins providing more robust code base.
Other notable minor changes are:
- Performance optimization of partial body patching.
- Changes in body pose alignment are adopted in PoseTwingPlugin.
- @geenee/bodyrenderers-babylon uses the latest version of Babylon.js.