Event-Based Near-Eye Gaze Tracking Beyond 10,000 Hz

Anastasios N. Angelopoulos; Julien N.P. Martel; Amit P. Kohli; Jorg Conradt; Gordon Wetzstein

doi:10.1109/tvcg.2021.3067784

Event-Based Near-Eye Gaze Tracking Beyond 10,000 Hz

Anastasios N. Angelopoulos, Julien N.P. Martel, Amit P. Kohli, Jorg Conradt, Gordon Wetzstein

Research output: Contribution to journal › Article › peer-review

46 Scopus citations

Abstract

Fast and accurate eye tracking is crucial for many applications. Current camera-based eye tracking systems, however, are fundamentally limited by their bandwidth, forcing a tradeoff between image resolution and framerate, i.e. between latency and update rate. Here, we propose a hybrid frame-event-based near-eye gaze tracking system offering update rates beyond 10,000 Hz with an accuracy that matches that of high-end desktop-mounted commercial eye trackers when evaluated in the same conditions. Our system builds on emerging event cameras that simultaneously acquire regularly sampled frames and adaptively sampled events. We develop an online 2D pupil fitting method that updates a parametric model every one or few events. Moreover, we propose a polynomial regressor for estimating the gaze vector from the parametric pupil model in real time. Using the first hybrid frame-event gaze dataset, which will be made public, we demonstrate that our system achieves accuracies of 0.45 degrees -- 1.75 degrees for fields of view ranging from 45 degrees to 98 degrees.

Original language	English (US)
Pages (from-to)	2577-2586
Number of pages	10
Journal	IEEE Transactions on Visualization and Computer Graphics
Volume	27
Issue number	5
DOIs	https://doi.org/10.1109/tvcg.2021.3067784
State	Published - May 2021
Externally published	Yes

Bibliographical note

KAUST Repository Item: Exported on 2021-06-24
Acknowledgements: A.N.A. was supported by a National Science Foundation (NSF) Fellowship and a Berkeley Fellowship. J.N.P.M. was supported by a Swiss
National Foundation (SNF) Fellowship (P2EZP2 181817), G.W. was supported by an NSF CAREER Award (IIS 1553333), a Sloan Fellowship, by the KAUST Office of Sponsored Research through the Visual Computing Center CCF grant, and a PECASE by the ARL. Thanks to Stephen Boyd and Mert Pilanci for helpful conversations.
This publication acknowledges KAUST support, but has no KAUST affiliated authors.

ASJC Scopus subject areas

Signal Processing
Computer Graphics and Computer-Aided Design
Software
Computer Vision and Pattern Recognition

Access to Document

10.1109/tvcg.2021.3067784

https://repository.kaust.edu.sa/bitstream/10754/662526/1/Preprintfile1.pdf

Cite this

@article{2b5dbb631b1640dab5372b0b64f23a50,

title = "Event-Based Near-Eye Gaze Tracking Beyond 10,000 Hz",

abstract = "Fast and accurate eye tracking is crucial for many applications. Current camera-based eye tracking systems, however, are fundamentally limited by their bandwidth, forcing a tradeoff between image resolution and framerate, i.e. between latency and update rate. Here, we propose a hybrid frame-event-based near-eye gaze tracking system offering update rates beyond 10,000 Hz with an accuracy that matches that of high-end desktop-mounted commercial eye trackers when evaluated in the same conditions. Our system builds on emerging event cameras that simultaneously acquire regularly sampled frames and adaptively sampled events. We develop an online 2D pupil fitting method that updates a parametric model every one or few events. Moreover, we propose a polynomial regressor for estimating the gaze vector from the parametric pupil model in real time. Using the first hybrid frame-event gaze dataset, which will be made public, we demonstrate that our system achieves accuracies of 0.45 degrees -- 1.75 degrees for fields of view ranging from 45 degrees to 98 degrees.",

author = "Angelopoulos, {Anastasios N.} and Martel, {Julien N.P.} and Kohli, {Amit P.} and Jorg Conradt and Gordon Wetzstein",

note = "KAUST Repository Item: Exported on 2021-06-24 Acknowledgements: A.N.A. was supported by a National Science Foundation (NSF) Fellowship and a Berkeley Fellowship. J.N.P.M. was supported by a Swiss National Foundation (SNF) Fellowship (P2EZP2 181817), G.W. was supported by an NSF CAREER Award (IIS 1553333), a Sloan Fellowship, by the KAUST Office of Sponsored Research through the Visual Computing Center CCF grant, and a PECASE by the ARL. Thanks to Stephen Boyd and Mert Pilanci for helpful conversations. This publication acknowledges KAUST support, but has no KAUST affiliated authors.",

year = "2021",

month = may,

doi = "10.1109/tvcg.2021.3067784",

language = "English (US)",

volume = "27",

pages = "2577--2586",

journal = "IEEE Transactions on Visualization and Computer Graphics",

issn = "1077-2626",

publisher = "IEEE Computer Society",

number = "5",

}

TY - JOUR

T1 - Event-Based Near-Eye Gaze Tracking Beyond 10,000 Hz

AU - Angelopoulos, Anastasios N.

AU - Martel, Julien N.P.

AU - Kohli, Amit P.

AU - Conradt, Jorg

AU - Wetzstein, Gordon

N1 - KAUST Repository Item: Exported on 2021-06-24 Acknowledgements: A.N.A. was supported by a National Science Foundation (NSF) Fellowship and a Berkeley Fellowship. J.N.P.M. was supported by a Swiss National Foundation (SNF) Fellowship (P2EZP2 181817), G.W. was supported by an NSF CAREER Award (IIS 1553333), a Sloan Fellowship, by the KAUST Office of Sponsored Research through the Visual Computing Center CCF grant, and a PECASE by the ARL. Thanks to Stephen Boyd and Mert Pilanci for helpful conversations. This publication acknowledges KAUST support, but has no KAUST affiliated authors.

PY - 2021/5

Y1 - 2021/5

N2 - Fast and accurate eye tracking is crucial for many applications. Current camera-based eye tracking systems, however, are fundamentally limited by their bandwidth, forcing a tradeoff between image resolution and framerate, i.e. between latency and update rate. Here, we propose a hybrid frame-event-based near-eye gaze tracking system offering update rates beyond 10,000 Hz with an accuracy that matches that of high-end desktop-mounted commercial eye trackers when evaluated in the same conditions. Our system builds on emerging event cameras that simultaneously acquire regularly sampled frames and adaptively sampled events. We develop an online 2D pupil fitting method that updates a parametric model every one or few events. Moreover, we propose a polynomial regressor for estimating the gaze vector from the parametric pupil model in real time. Using the first hybrid frame-event gaze dataset, which will be made public, we demonstrate that our system achieves accuracies of 0.45 degrees -- 1.75 degrees for fields of view ranging from 45 degrees to 98 degrees.

AB - Fast and accurate eye tracking is crucial for many applications. Current camera-based eye tracking systems, however, are fundamentally limited by their bandwidth, forcing a tradeoff between image resolution and framerate, i.e. between latency and update rate. Here, we propose a hybrid frame-event-based near-eye gaze tracking system offering update rates beyond 10,000 Hz with an accuracy that matches that of high-end desktop-mounted commercial eye trackers when evaluated in the same conditions. Our system builds on emerging event cameras that simultaneously acquire regularly sampled frames and adaptively sampled events. We develop an online 2D pupil fitting method that updates a parametric model every one or few events. Moreover, we propose a polynomial regressor for estimating the gaze vector from the parametric pupil model in real time. Using the first hybrid frame-event gaze dataset, which will be made public, we demonstrate that our system achieves accuracies of 0.45 degrees -- 1.75 degrees for fields of view ranging from 45 degrees to 98 degrees.

UR - http://hdl.handle.net/10754/662526

UR - https://ieeexplore.ieee.org/document/9389490/

U2 - 10.1109/tvcg.2021.3067784

DO - 10.1109/tvcg.2021.3067784

M3 - Article

SN - 1077-2626

VL - 27

SP - 2577

EP - 2586

JO - IEEE Transactions on Visualization and Computer Graphics

JF - IEEE Transactions on Visualization and Computer Graphics

IS - 5

ER -

Event-Based Near-Eye Gaze Tracking Beyond 10,000 Hz

Abstract

Bibliographical note

ASJC Scopus subject areas

Access to Document

Other files and links

Fingerprint

Cite this