publications
2025
-
Portable Biomechanics Laboratory: Clinically Accessible Movement Analysis from a Handheld SmartphoneJ.D. Peiffer, Kunal Shah, Irina Djuraskovic, and 6 more authors2025Background The way a person moves is a direct reflection of their neurological and musculoskeletal health, yet it remains one of the most underutilized vital signs in clinical practice. Although clinicians visually observe movement impairments, they lack accessible and validated methods to objectively measure movement in routine care. This gap prevents wider use of biomechanical measurements in practice, which could enable more sensitive outcome measures or earlier identification of impairment. Methods In this work, we present our Portable Biomechanics Laboratory (PBL), which includes a secure, cloud-enabled smartphone app for data collection and a novel algorithm for fitting biomechanical models to this data. We extensively validated PBL’s biomechanical measures using a large, clinically representative and heterogeneous dataset with synchronous ground truth. Next, we tested the usability and utility of our system in both a neurosurgery and sports medicine clinic. Results We found joint angle errors within 3 degrees and pelvis translation errors within several centimeters across participants with neurological injury, lower-limb prosthesis users, pediatric inpatients, and controls. In addition to being easy and quick to use, gait metrics computed from the PBL showed high reliability (ICCs \textgreater 0.9) and were sensitive to clinical differences. For example, in individuals undergoing decompression surgery for cervical myelopathy, the modified Japanese Orthopedic Association (mJOA) score is a common patient-reported outcome measure; we found that PBL gait metrics not only correlated with mJOA scores but also demonstrated greater responsiveness to surgical intervention than the patient-reported outcomes. Conclusions These findings support the use of handheld smartphone video as a scalable, low-burden tool for capturing clinically meaningful biomechanical data, offering a promising path toward remote, accessible monitoring of mobility impairments in clinical populations. To promote further research and clinical translation, we release the first method for measuring whole-body kinematics from handheld smartphone video validated in clinical populations: https://IntelligentSensingAndRehabilitation.
2024
-
Fusing Uncalibrated IMUs and Handheld Smartphone Video to Reconstruct Knee KinematicsJ.D. Peiffer, Kunal Shah, Shawana Anarwala, and 2 more authorsIn 2024 10th IEEE RAS/EMBS International Conference for Biomedical Robotics and Biomechatronics (BioRob), 2024ISSN: 2155-1782Video and wearable sensor data provide complementary information about human movement. Video provides a holistic understanding of the entire body in the world while wearable sensors, namely inertial measurement units (IMUs), provide high-resolution measurements of specific body segments. A robust method to fuse these modalities and obtain biomechanically accurate kinematics would have substantial utility for clinical assessment and monitoring. Although multiple video-sensor fusion methods exist, most assume that a time-intensive and often brittle sensor-body calibration process has already been completed. In this work, we employ an implicit function to combine handheld smartphone video and uncalibrated IMU data at their full temporal resolution. Our monocular, video-only, biomechanical reconstruction already performs well, with only 3.91 (1.55) degrees of mean adjusted angular error in knee kinematics across 60 recordings. Re-constructing from a fusion of video and IMU data reduces this error to 2.9 (1.27) degrees. We validate this method in a diverse group including individuals with no gait impairments, lower limb prosthesis users, and those with a history of stroke. We also show that IMU data allows accurate tracking through periods of visual occlusion, equivalent to video-only.
-
Hyperpolarized 129Xe MRI, 99mTc scintigraphy, and SPECT in lung ventilation imaging: a quantitative comparisonJ.D. Peiffer, Talissa Altes, Iulian C. Ruset, and 6 more authorsAcademic Radiology, 2024 -
Differentiable Biomechanics for Markerless Motion Capture in Upper Limb Stroke Rehabilitation: A Comparison with Optical Motion CaptureTim Unger, Arash Sal Moslehian, J.D. Peiffer, and 5 more authors2024Marker-based Optical Motion Capture (OMC) paired with biomechanical modeling is currently considered the most precise and accurate method for measuring human movement kinematics. However, combining differentiable biomechanical modeling with Markerless Motion Capture (MMC) offers a promising approach to motion capture in clinical settings, requiring only minimal equipment, such as synchronized webcams, and minimal effort for data collection. This study compares key kinematic outcomes from biomechanically modeled MMC and OMC data in 15 stroke patients performing the drinking task, a functional task recommended for assessing upper limb movement quality. We observed a high level of agreement in kinematic trajectories between MMC and OMC, as indicated by high correlations (median r above 0.95 for the majority of kinematic trajectories) and median RMSE values ranging from 2-5 degrees for joint angles, 0.04 m/s for end-effector velocity, and 6 mm for trunk displacement. Trial-to-trial biases between OMC and MMC were consistent within participant sessions, with interquartile ranges of bias around 1-3 degrees for joint angles, 0.01 m/s in end-effector velocity, and approximately 3mm for trunk displacement. Our findings indicate that our MMC for arm tracking is approaching the accuracy of marker-based methods, supporting its potential for use in clinical settings. MMC could provide valuable insights into movement rehabilitation in stroke patients, potentially enhancing the effectiveness of rehabilitation strategies.
-
Biomechanical Arm and Hand Tracking with Multiview Markerless Motion CapturePouyan Firouzabadi, Wendy Murray, Anton R Sobinov, and 4 more authorsIn 2024 10th IEEE RAS/EMBS International Conference for Biomedical Robotics and Biomechatronics (BioRob), 2024ISSN: 2155-1782Human arm and hand function is extremely complex with many degrees of freedom. It is also a common target for clinical interventions. However, precisely measuring upper extremity movement in both clinical and research settings is logistically challenging. We overcame this challenge through a novel approach to reconstructing arm biomechanics from markerless motion capture from multiple synchronized videos. Our approach directly opti-mizes the kinematics of an accurate biomechanical arm and hand that allows end-to-end minimization of the errors between the reconstructed movements and keypoints detected by computer vision. Key to this is an implicit function that maps from time to joint kinematics, which provides a learnable trajectory representation that can be differentiated through the biomechanical model, and supports GPU acceleration using MuJoCo-MJX. This optimization solves for the inverse kinematic solution consistent with the measured keypoints, consistent with biomechanical constraints, in addition to scaling the model while solving for the kinematics. We compare different hand keypoint detectors and find the best produces a fit with only several millimeters of reconstruction error. We also find that end-to-end optimization outperforms a two-stage fitting procedure, equivalent to more traditional biomechanical pipelines, where we first compute 3D marker trajectories and then perform inverse kinematics fitting in OpenSim. We anticipate this framework will reduce the barriers to biomechanical analysis of the arm and hand in both clinical and research settings.
2023
-
Enhanced selectivity of transcutaneous spinal cord stimulation by multielectrode configurationNoah Bryson, Lorenzo Lombardi, Rachel Hawthorn, and 4 more authorsJournal of Neural Engineering, 2023 -
Self-Supervised Learning of Gait-Based BiomarkersR. James Cotton, J.D. Peiffer, Kunal Shah, and 5 more authorsIn Predictive Intelligence in Medicine, 2023Markerless motion capture (MMC) is revolutionizing gait analysis in clinical settings by making it more accessible, raising the question of how to extract the most clinically meaningful information from gait data. In multiple fields ranging from image processing to natural language processing, self-supervised learning (SSL) from large amounts of unannotated data produces very effective representations for downstream tasks. However, there has only been limited use of SSL to learn effective representations of gait and movement, and it has not been applied to gait analysis with MMC. One SSL objective that has not been applied to gait is contrastive learning, which finds representations that place similar samples closer together in the learned space. If the learned similarity metric captures clinically meaningful differences, this could produce a useful representation for many downstream clinical tasks. Contrastive learning can also be combined with causal masking to predict future timesteps, which is an appealing SSL objective given the dynamical nature of gait. We applied these techniques to gait analyses performed with MMC in a rehabilitation hospital from a diverse clinical population. We find that contrastive learning on unannotated gait data learns a representation that captures clinically meaningful information. We probe this learned representation using the framework of biomarkers and show it holds promise as both a diagnostic and response biomarker, by showing it can accurately classify diagnosis from gait and is responsive to inpatient therapy, respectively. We ultimately hope these learned representations will enable predictive and prognostic gait-based biomarkers that can facilitate precision rehabilitation through greater use of MMC to quantify movement in rehabilitation.
-
Markerless Motion Capture and Biomechanical Analysis PipelineR. James Cotton, Allison DeLillo, Anthony Cimorelli, and 5 more authors2023Markerless motion capture using computer vision and human pose estimation (HPE) has the potential to expand access to precise movement analysis. This could greatly benefit rehabilitation by enabling more accurate tracking of outcomes and providing more sensitive tools for research. There are numerous steps between obtaining videos to extracting accurate biomechanical results and limited research to guide many critical design decisions in these pipelines. In this work, we analyze several of these steps including the algorithm used to detect keypoints and the keypoint set, the approach to reconstructing trajectories for biomechanical inverse kinematics and optimizing the IK process. Several features we find important are: 1) using a recent algorithm trained on many datasets that produces a dense set of biomechanically-motivated keypoints, 2) using an implicit representation to reconstruct smooth, anatomically constrained marker trajectories for IK, 3) iteratively optimizing the biomechanical model to match the dense markers, 4) appropriate regularization of the IK process. Our pipeline makes it easy to obtain accurate biomechanical estimates of movement in a rehabilitation hospital.
- Optimizing Trajectories and Inverse Kinematics for Biomechanical Analysis of Markerless Motion Capture DataR. James Cotton, Allison DeLillo, Anthony Cimorelli, and 5 more authorsIn 2023 International Conference on Rehabilitation Robotics (ICORR), Singapore, Singapore, 2023
Markerless motion capture using computer vision and human pose estimation (HPE) has the potential to expand access to precise movement analysis. This could greatly benefit rehabilitation by enabling more accurate tracking of outcomes and providing more sensitive tools for research. There are numerous steps between obtaining videos to extracting accurate biomechanical results and limited research to guide many critical design decisions in these pipelines. In this work, we analyze several of these steps including the algorithm used to detect keypoints and the keypoint set, the approach to reconstructing trajectories for biomechanical inverse kinematics and optimizing the IK process. Several features we find important are: 1) using a recent algorithm trained on many datasets that produces a dense set of biomechanically-motivated keypoints, 2) using an implicit representation to reconstruct smooth, anatomically constrained marker trajectories for IK, 3) iteratively optimizing the biomechanical model to match the dense markers, 4) appropriate regularization of the IK process. Our pipeline makes it easy to obtain accurate biomechanical estimates of movement in a rehabilitation hospital.