Fusion from Multimodal Gait Spatiotemporal Data for Human Gait Speed Classifications

Abdullah S. Alharthi; Krikor B. Ozanyan

doi:10.1109/SENSORS47087.2021.9639816

Fusion from Multimodal Gait Spatiotemporal Data for Human Gait Speed Classifications

Abdullah S. Alharthi, Krikor B. Ozanyan

Computer, Electrical and Mathematical Sciences and Engineering

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

2 Scopus citations

Abstract

Human gait pattens remain largely undefined when relying on a single sensing modality. We report a pilot implementation of sensor fusion to classify gait spatiotemporal signals, from a publicly available dataset of 50 participants, harvested from four different type of sensors. For fusion we propose a hybrid Convolutional Neural Network and Long Short-Term Memory (hybrid CNN+LSTM) and Multi-stream CNN. The classification results are compared to single modality data using Single-stream CNN, a state-of-the-art Vision Transformer, and statistical classifiers algorithms. The fusion models outperformed the single modality methods and classified gait speed of previously unseen 10 random subjects with 97% F1-score prediction accuracy of the four gait speed classes.

Original language	English (US)
Title of host publication	2021 IEEE Sensors, SENSORS 2021 - Conference Proceedings
Publisher	Institute of Electrical and Electronics Engineers Inc.
ISBN (Electronic)	9781728195018
DOIs	https://doi.org/10.1109/SENSORS47087.2021.9639816
State	Published - 2021
Event	20th IEEE Sensors, SENSORS 2021 - Virtual, Online, Australia Duration: Oct 31 2021 → Nov 4 2021

Publication series

Name	Proceedings of IEEE Sensors
Volume	2021-October
ISSN (Print)	1930-0395
ISSN (Electronic)	2168-9229

Conference

Conference	20th IEEE Sensors, SENSORS 2021
Country/Territory	Australia
City	Virtual, Online
Period	10/31/21 → 11/4/21

Bibliographical note

Publisher Copyright:
© 2021 IEEE.

Keywords

Deep Convolutional Neural Networks (CNN)
Gait Speed
Long Short-Term Memory (LSTM)
Multimodal Data
Transformers

ASJC Scopus subject areas

Electrical and Electronic Engineering

Access to Document

10.1109/SENSORS47087.2021.9639816

Cite this

@inproceedings{9d488511a9424eb6a821241264777f55,

title = "Fusion from Multimodal Gait Spatiotemporal Data for Human Gait Speed Classifications",

abstract = "Human gait pattens remain largely undefined when relying on a single sensing modality. We report a pilot implementation of sensor fusion to classify gait spatiotemporal signals, from a publicly available dataset of 50 participants, harvested from four different type of sensors. For fusion we propose a hybrid Convolutional Neural Network and Long Short-Term Memory (hybrid CNN+LSTM) and Multi-stream CNN. The classification results are compared to single modality data using Single-stream CNN, a state-of-the-art Vision Transformer, and statistical classifiers algorithms. The fusion models outperformed the single modality methods and classified gait speed of previously unseen 10 random subjects with 97% F1-score prediction accuracy of the four gait speed classes.",

keywords = "Deep Convolutional Neural Networks (CNN), Gait Speed, Long Short-Term Memory (LSTM), Multimodal Data, Transformers",

author = "Alharthi, {Abdullah S.} and Ozanyan, {Krikor B.}",

note = "Publisher Copyright: {\textcopyright} 2021 IEEE.; 20th IEEE Sensors, SENSORS 2021 ; Conference date: 31-10-2021 Through 04-11-2021",

year = "2021",

doi = "10.1109/SENSORS47087.2021.9639816",

language = "English (US)",

series = "Proceedings of IEEE Sensors",

publisher = "Institute of Electrical and Electronics Engineers Inc.",

booktitle = "2021 IEEE Sensors, SENSORS 2021 - Conference Proceedings",

address = "United States",

}

Alharthi, AS & Ozanyan, KB 2021, Fusion from Multimodal Gait Spatiotemporal Data for Human Gait Speed Classifications. in 2021 IEEE Sensors, SENSORS 2021 - Conference Proceedings. Proceedings of IEEE Sensors, vol. 2021-October, Institute of Electrical and Electronics Engineers Inc., 20th IEEE Sensors, SENSORS 2021, Virtual, Online, Australia, 10/31/21. https://doi.org/10.1109/SENSORS47087.2021.9639816

Fusion from Multimodal Gait Spatiotemporal Data for Human Gait Speed Classifications. / Alharthi, Abdullah S.; Ozanyan, Krikor B.
2021 IEEE Sensors, SENSORS 2021 - Conference Proceedings. Institute of Electrical and Electronics Engineers Inc., 2021. (Proceedings of IEEE Sensors; Vol. 2021-October).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - Fusion from Multimodal Gait Spatiotemporal Data for Human Gait Speed Classifications

AU - Alharthi, Abdullah S.

AU - Ozanyan, Krikor B.

PY - 2021

Y1 - 2021

N2 - Human gait pattens remain largely undefined when relying on a single sensing modality. We report a pilot implementation of sensor fusion to classify gait spatiotemporal signals, from a publicly available dataset of 50 participants, harvested from four different type of sensors. For fusion we propose a hybrid Convolutional Neural Network and Long Short-Term Memory (hybrid CNN+LSTM) and Multi-stream CNN. The classification results are compared to single modality data using Single-stream CNN, a state-of-the-art Vision Transformer, and statistical classifiers algorithms. The fusion models outperformed the single modality methods and classified gait speed of previously unseen 10 random subjects with 97% F1-score prediction accuracy of the four gait speed classes.

AB - Human gait pattens remain largely undefined when relying on a single sensing modality. We report a pilot implementation of sensor fusion to classify gait spatiotemporal signals, from a publicly available dataset of 50 participants, harvested from four different type of sensors. For fusion we propose a hybrid Convolutional Neural Network and Long Short-Term Memory (hybrid CNN+LSTM) and Multi-stream CNN. The classification results are compared to single modality data using Single-stream CNN, a state-of-the-art Vision Transformer, and statistical classifiers algorithms. The fusion models outperformed the single modality methods and classified gait speed of previously unseen 10 random subjects with 97% F1-score prediction accuracy of the four gait speed classes.

KW - Deep Convolutional Neural Networks (CNN)

KW - Gait Speed

KW - Long Short-Term Memory (LSTM)

KW - Multimodal Data

KW - Transformers

UR - http://www.scopus.com/inward/record.url?scp=85123643285&partnerID=8YFLogxK

U2 - 10.1109/SENSORS47087.2021.9639816

DO - 10.1109/SENSORS47087.2021.9639816

M3 - Conference contribution

AN - SCOPUS:85123643285

T3 - Proceedings of IEEE Sensors

BT - 2021 IEEE Sensors, SENSORS 2021 - Conference Proceedings

PB - Institute of Electrical and Electronics Engineers Inc.

T2 - 20th IEEE Sensors, SENSORS 2021

Y2 - 31 October 2021 through 4 November 2021

ER -

Fusion from Multimodal Gait Spatiotemporal Data for Human Gait Speed Classifications

Abstract

Publication series

Conference

Bibliographical note

Keywords

ASJC Scopus subject areas

Access to Document

Other files and links

Fingerprint

Cite this