Human Identification and Action Recognition Using Small Data and Deep Domain Adaptation

Human identification and human action recognition problems are two important research areas for real-world security and surveillance applications. In both human identification and action recognition, it is necessary to operate by collecting small datasets in the field, possibly in a short time window of observation. This dissertation studies and develops computational modeling and high-performance machine learning (ML) and deep learning (DL) models for human identification and human action recognition using small amounts of data. These methods and computational models may be useful for different security and surveillance applications. This dissertation on human recognition develops a ML computational model to estimate bone length from 3D Lidar sensor data. We hypothesize that the 3D Lidar sensor may be amenable to bone length estimation, and bone length, in turn, may be used as an invariant feature to identify a human. We obtain 3D joint location and subsequent bone length estimations using a computational model based on several image processing steps and novel human silhouette and 3D skeleton estimation algorithms. This computational model estimates the 3D skeleton without needing to train a deep model that would require large data for training. In comparison to existing methods, we achieve competitive results with far field Lidar data, which is better suited to surveillance where human identification at a longer distance offers a better security product. Then, this dissertation explores human activity recognition by using a DL method for deep domain adaptation to estimate human action. Training data for human action recognition is expensive to collect, so we use domain adaptation to learn from multiple publicly available datasets for human action recognition to solve recognition on an unlabeled target dataset. This may be useful in a real-world application where we may train the models with a large amount of publicly available yet unrelated human subject data while the target test dataset is specific and small and may be collected in the field. We develop a novel Bayesian DL framework that can learn from different multiple source domain adaptation during the training process. We show that this framework outperforms the existing state-of-the-art multidomain adaptation methods by 17% on an example target Daily-DA dataset in the literature.