Deep Convolutional Bidirectional LSTM for Complex Activity Recognition with Missing Data

Swapnil Sayan Saha, Sandeep Singh Sandha, Mani Srivastava

Research output: Chapter in Book/Report/Conference proceedingChapter

4 Scopus citations


Complex activity recognition using multiple on-body sensors is challenging due to missing samples, misaligned data times tamps across sensors, and variations in sampling rates. In this paper, we introduce a robust training pipeline that handles sampling rate variability, missing data, and misaligned data time stamps using intelligent data augmentation techniques. Specifically, we use controlled jitter in window length and add artificial misalignments in data timestamps between sensors, along with masking representations of missing data. We evaluate our pipeline on the Cooking Activity Dataset with Macro and Micro Activities, benchmarking the performance of deep convolutional bidirectional long short-term memory (DCBL) classifier. In our evaluations, DCBL achieves test accuracies of 88% and 72%, respectively, for macro- and micro-activity classifications, exceeding performance over state-of-the-art vanilla activity classifiers.
Original languageEnglish (US)
Title of host publicationHuman Activity Recognition Challenge
PublisherSpringer Singapore
Number of pages15
ISBN (Print)9789811582684
StatePublished - Nov 21 2020
Externally publishedYes

Bibliographical note

KAUST Repository Item: Exported on 2021-07-01
Acknowledged KAUST grant number(s): Sensor Innovation research program
Acknowledgements: The research reported in this paper was sponsored in part by the CONIX Research Center, one of six centers in JUMP, a Semiconductor Research Corporation (SRC) program sponsored by DARPA, the Army Research Laboratory (ARL) under Cooperative Agreement W911NF-17-2-0196 and the King Abdullah University of Science and Technology (KAUST) through its Sensor Innovation research program. Any findings in this material are those of the author(s) and do not reflect the views of any of the above funding agencies. The U.S. Government is authorized to reproduce and distribute reprints for Government purposes notwithstanding any copyright notation hereon.
This publication acknowledges KAUST support, but has no KAUST affiliated authors.


Dive into the research topics of 'Deep Convolutional Bidirectional LSTM for Complex Activity Recognition with Missing Data'. Together they form a unique fingerprint.

Cite this