01 — Motion synthesis
Language- and audio-conditioned human motion
Extending masked motion models to richer pose representations. Moving from a
263-dimensional to a 272-dimensional representation lets the model carry direct SMPL joint
rotations in 6D form, which removes the inverse-kinematics artefacts that normally appear
during BVH conversion and preserves subtle detail like joint twist.
The current system reaches an FID of 0.093 with roughly two-to-three second inference, and
exports animation-ready output for Unity, Unreal Engine, and Blender.
Masked motion modelsSMPL 6D
HumanML3DBVH
02 — Representation learning
Discrete motion embeddings
A VQ-VAE encoder/decoder that compresses motion into a discrete codebook so a transformer
can model movement the way a language model handles tokens. Training used a 512-entry
codebook with 2× temporal downsampling.
Codebook resets and EMA stabilisation brought reconstruction error down to an L2 of 0.0159
and an L1 of 0.0640 — low enough that the decoded motion is visually indistinguishable
from ground truth.
VQ-VAECodebook resetsEMA
03 — Clinical AI
Decision support that clinicians will use
With the Duke Heart Center and the UNC School of Medicine I have worked on algorithms for
organ donation after circulatory death, timing tools for ECMO in SVC syndrome, and a
meta-analysis of whether deployed AI actually improves patient outcomes.
The donation algorithm projects a 94% improvement in success rate while reducing ICU
workload — the kind of result that only matters if the workflow around it is realistic.
DCDECMOBiomedCLIPSmolVLM
04 — Systems
Making research runnable
Research code that only runs on one machine is research nobody can build on. I spend a
lot of time on the deployment path: CUDA optimisation that took text-to-motion inference
from 90 seconds to 7, audio preprocessing from four minutes to 30 seconds, and FastAPI
services with pre-loaded models behind them.
Generated motion is retargeted onto TADA avatars and rendered in-browser with Three.js, so
a result can be inspected the moment it exists.
CUDAFastAPINext.jsThree.js
