Human Activity Recognition with Qualitative Spatial Relations

MIT 16.S948 Algorithmic Human-Robot Interaction

January 2025 - May 2025

Cambridge, Massachusetts

Human Activity Recognition, Human Pose Estimation, Qualitative Spatial Relations, Bayesian Inference, Dynamic Bayesian Networks, Particle Filtering, Expectation Maximization, Computer Vision

Qualitative Spatial Relationships (QSR) are an abstract approach to human activity recognition that let robots “think” in a way that is intuitive to humans. The goal of this project was to extend prior work from the Model Based Embedded and Robotic Systems (MERS) Group so that models that predict QSRs from RGB video data leverage model learning in their transition and observation functions.

This was an individual research project in which I was responsible for reviewing prior implementations of QSR models, identifying areas for further improvement, updating the models, and collecting data to demonstrate better overall results.

Leveraging the key insight that different QSR types are probabilistically dependent on each other, I modeled QSRs in a dynamic Bayesian Network (DBN) and learned the transition probabilities between the QSRs in real time with the Expectation Maximization (EM) algorithm. These improvements in the model resulted in a 12.1% increase in QSR prediction accuracy compared to the baseline, demonstrating a new level of generalizability for QSR recognition. I presented my work to 20+ other students at the end of the semester and recieved positive feedback from my professor.

This project taught me about model-based methods for inference and how they can be applied to the field of human activity recognition. This gave me a better appreciation for the importance of building on human intuition and our own prior knowledge about a problem space, as this work makes it significantly easier and more efficient for robots to learn and execute autonomously.