First Person Vision for Activity Prediction Using Probabilistic Modeling
Journal: Mehran University Research Journal of Engineering and Technology (Vol.37, No. 4)Publication Date: 2018-10-01
Authors : Shaheena Noor Vali Uddin;
Page : 545-558
Keywords : First Person Vision; Activity Recognition; Activity Prediction; Context-Aware System; Hidden Markov Models; Artificial Neural Networks;
Abstract
Identifying activities of daily living is an important area of research with applications in smart-homes and healthcare for elderly people. It is challenging due to reasons like human self-occlusion, complex natural environment and the human behavior when performing a complicated task. From psychological studies, we know that human gaze is closely linked with the thought process and we tend to “look” at the objects before acting on them. Hence, we have used the object information present in gaze images as the context and formed the basis for activity prediction. Our system is based on HMM (Hidden Markov Models) and trained using ANN (Artificial Neural Network). We begin with extracting motion information from TPV (Third Person Vision) streams and object information from FPV (First Person Vision) cameras. The advantage of having FPV is that the object information forms the context of the scene. When context is included as input to the HMM for activity recognition, the precision increases. For testing, we used two standard datasets from TUM (Technische Universitaet Muenchen) and GTEA Gaze+ (Georgia Tech Egocentric Activities). In the first round, we trained our ANNs only with activity information and in the second round added the object information as well. We saw a significant increase in the precision (and accuracy) of predicted activities from 55.21% (respectively 85.25%) to 77.61% (respectively 93.5%). This confirmed our initial hypothesis that including the focus of attention of the actor in the form of object seen in FPV can help in predicting activities better.
Other Latest Articles
- Cloud Based Remote FPGA Lab Platform: An Application of Internet of Things
- Ontology-Based Verification of UML Class/OCL Model
- Comparison of Effects of Entropy Coding Schemes Cascaded with Set Partitioning in Hierarchical Trees
- Implementation of Tuned Schema Merging Approach
- Effect of Temperature and Time on Nickel Aluminide Coating Deposition
Last modified: 2018-10-12 16:45:25