Release v0.8.7
Repacker
Section titled “Repacker”We have added another extra utility module to Engeenee SDK - @geenee/repacker Repacker remuxes input video files to improve compatibility with various video players and media sharing apps. Additionally, it fixes common issues and artifacts of MediaRecorder outputs. The most common trait of video containers recorded in a browser is a file fragmentation. Browser records a video in chunks and flushes them periodically, so for example we get fragmented mp4 file that may be not supported by some apps for sharing, or they will truncate the video to the first chunk/fragment of mp4 file. Repacker remuxes a file to obtain a whole monolithic container. Being compiled into wasm it’s blazingly fast and memory efficient. Utilizing efficient memory map and zero-copy strategies it remuxes buffers in place avoiding redundant deep copies and conversions. Repacker supports mp4 container and h264/avc1 video codec. The set of supported formats is limited to make size of the wasm binary as small as possible (mp4+h264 combination is the most widely used). Support for more containers and codecs may be added in the future.
Repacker is very straightforward to use and essentially has one method implementing remuxing of media file stored in memory. First we need to initialize an instance of Repacker:
import { Repacker } from "@geenee/repacker";const repacker = new Repacker();await repacker.init();Asynchronous init() method is required to finish until further
usage of Repacker, it downloads and initialized corresponding
.wasm module distributed with the package. This module contains
codecs, muxers, and native implementation of streams remuxing.
One needs to put .wasm module in the root of a web app, e.g.
into public/ folder like we usually do with bodyutils.wasm.
After repacker.init() is successfully completed you can call
repacker.remuxBlob(blob) or repacker.remuxBuf(buf) to fix
fragmentation of any mp4 file. For example it’s recommended to
remux files captured with Recorder from @geenee/armature:
let blob = await recorder?.stop();blob = blob && await repacker.remuxBlob(blob);The whole capture + remux pipeline may look like this:
// Recorderconst repacker = new Repacker();await repacker.init();const recorder = new UniRecorder("video/mp4;codecs=avc1,opus", 8 * 1024 * 1024);const recordBtn = Button({ size: "icon-lg", variant: "outline", class: "absolute top-4 left-4 z-1 bg-opacity-60", onclick: async () => { recorder?.start(renderer, 0, renderer.audioTrack); setTimeout(async () => { let blob = await recorder?.stop(); blob = blob && await repacker.remuxBlob(blob); if (!blob) return; const url = URL.createObjectURL(blob); const link = document.createElement("a"); link.hidden = true; link.href = url; link.download = "capture.mp4"; link.click(); link.remove(); URL.revokeObjectURL(url); }, 5000); }Reliability
Section titled “Reliability”To increase the stability of web applications built using the Engeenee SDK, we have implemented several fail-safe features. The SDK now monitors the health of the GPU contexts and the camera being utilized. While the SDK itself is free from memory leaks and undefined behaviors, external code or third-party packages within an application might occasionally fail to release GPU resources (such as textures). Furthermore, the camera connection or its driver may experience temporary failures and subsequent automatic recovery. These new features are designed to mitigate the impact of such external issues. Given the specific nature of the browser environment, where a memory leak in one context can lead to failure in another, the SDK monitors the state of all controlled resources critical for its lifecycle. A global exception is triggered if any of these resources fail.
We track health of the next contexts:
- GPGPU context which is used by the tracking engine for neural networks inference and image post- and pre-processing.
- GPU context of the renderer.
- Camera capturing context.
Adding the following line of code at the very beginning of index.ts enables the page
to catch these exceptions and automatically reload, recovering from the failure:
window.onerror = () => { window.location.reload(); };