Skip to content
Engeenee
Blog

Release v0.8.0

Evgeny Peshkov
Evgeny Peshkov
CTO @ Geenee

Major Release

Section titled “Major Release”

This is a major release with lots of API breaking changes. Engeenee SDK has been significantly refactored but math and algorithms are mostly untouched.

API Restructure

Section titled “API Restructure”

We’ve introduced MaskProcessPlugin - base plugin for post-processing of a segmentation mask. It applies an image processing shader on the mask texture and updates the corresponding field of tracking results. Mask processing plugins can be combined and chained in a pipeline. Order in which operation on segmentation mask are applied corresponds to the order in which mask plugins are attached to the renderer. This plugin helps to simplify API and internal implementations. The next plugins now reuse base implementation inheriting MaskProcessPlugin: MaskSmoothPlugin, MaskMorphPlugin, MaskDilationPlugin, MaskErosionPlugin, MaskBinaryPlugin, MaskStepPlugin.

We removed attaching of a scene node that will be controlled by a plugin in its constructor. Usually, plugin pipeline is assembled within Renderer’s constructor, or completely outside of the Renderer. In both cases there’s no access to the scene or no assets are loaded yet; scene initialization happens in load() method and assets are loaded asynchronously. So almost always undefined was passed as node parameter. Now attaching of a node can be done only via setNode() method of a plugin. For example, refer new HeadFitPlugin(), node can be attached later, after constructor, with setNode().

Renaming of Plugins

Section titled “Renaming of Plugins”

There are three big changes in naming convention for plugins.

We now use word Fit instead of Align. For example, we rename HeadTrackPlugin to HeadFitPlugin. Plugins that fit scene nodes according to tracking data were renamed:

  • HeadTrackPlugin -> HeadFitPlugin
  • FaceTrackPlugin -> FacePointPlugin
  • HandAlignPlugin -> HandFitPlugin
  • WristTrackPlugin -> WristFitPlugin
  • HandFitPlugin -> FingersFitPlugin
  • HandGeometry -> FingersGeometry

Another change regards supported armature types. We support two armatures: compatible with Ready Player Me and Mixamo, and compatible with Clo3D and Marvelous Designer. At the moment Clo3D armature is considered the main one, and all updates and improvements are implemented for this armature first. Historically, plugins for Mixamo armature were called PoseSomethingPlugin and for Clo3D armature ClothSomethingPlugin. We resolve this ambiguity and from now one will call the main Clo3D armature Pose, and the secondary legacy Mixamo armature PoseB. Later it will be possible to introduce PoseC, PoseD, etc without breaking naming convention. Armature fitting plugins were renamed the next way:

  • ClothAlignPlugin -> PoseFitPlugin
  • ClothTwinPlugin -> PoseTwinPlugin
  • ClothSkirtPlugin ->SkirtFitPlugin
  • PoseAlignPlugin -> PoseBFitPlugin
  • PoseTwinPlugin -> PoseBTwinPlugin
  • PoseSkirtPlugin -> SkirtBFitPlugin

As patching can be applied to any segmentation mask, word Body doesn’t fit anymore, so we renamed patching plugins as well:

  • BodyPatchPlugin -> PatchPlugin
  • BodypartPatchPlugin -> PatchPartPlugin

The complete list of renamed plugins:

  • ClothAlignPlugin -> PoseFitPlugin
  • ClothTwinPlugin -> PoseTwinPlugin
  • ClothSkirtPlugin ->SkirtFitPlugin
  • PoseAlignPlugin -> PoseBFitPlugin
  • PoseTwinPlugin -> PoseBTwinPlugin
  • PoseSkirtPlugin -> SkirtBFitPlugin
  • HeadTrackPlugin -> HeadFitPlugin
  • FaceTrackPlugin -> FacePointPlugin
  • HandAlignPlugin -> HandFitPlugin
  • WristTrackPlugin -> WristFitPlugin
  • HandFitPlugin -> FingersFitPlugin
  • HandGeometry -> FingersGeometry
  • BodyPatchPlugin -> PatchPlugin
  • BodypartPatchPlugin -> PatchPartPlugin

Infrastructure

Section titled “Infrastructure”

Engeenee SDK moves to the new cloud infrastructure. On the front side the only changes are URLs of our npm registry and where to get SDK tokens. Registry is now hosted on npm.geen.ee, to direct npm to our registry for @geenee/ packages add this line to your .npmrc file @geenee:registry=https://npm.geen.ee.

Access tokens are managed from new SDK Manager. Here one can create, delete, or update SDK access tokens for urls where apps are hosted. And get NPM token as well to download SDK packages.

Example of .npmrc file:

registry=https://registry.npmjs.org
@geenee:registry=https://npm.geen.ee
//registry.npmjs.org/:_authToken="registry.npmjs.org_TOKEN"
//npm.geen.ee/:_authToken="npm.geen.ee_TOKEN"

Other Changes

Section titled “Other Changes”

We’ve removed deprecated OccluderPlugin and PoseOutfitPlugin from three.js library. Please, use OccluderMaterial directly to transform scene meshes into occluders, this is a more flexible and universal approach to control rendering of 3D objects in a scene.

Three.js plugins are refactored to use named imports that reduces final bundle size. Additionally, we’ve revisited and optimized their code.

spineCurve parameter of PoseFitPlugin is treated more robustly and not overriden with 0.0 when other pose tuning parameter are specified and it’s left undefined.

Tags: