Asymptotic performance of regularized quadratic discriminant analysis based classifiers

Khalil Elkhalil, Abla Kammoun, Romain Couillet, Tareq Y. Al-Naffouri, Mohamed Slim Alouini

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

11 Scopus citations

Abstract

This paper carries out a large dimensional analysis of the standard regularized quadratic discriminant analysis (QDA) classifier designed on the assumption that data arise from a Gaussian mixture model. The analysis relies on fundamental results from random matrix theory (RMT) when both the number of features and the cardinality of the training data within each class grow large at the same pace. Under some mild assumptions, we show that the asymptotic classification error converges to a deterministic quantity that depends only on the covariances and means associated with each class as well as the problem dimensions. Such a result permits a better understanding of the performance of regularized QDA and can be used to determine the optimal regularization parameter that minimizes the misclassification error probability. Despite being valid only for Gaussian data, our theoretical findings are shown to yield a high accuracy in predicting the performances achieved with real data sets drawn from popular real data bases, thereby making an interesting connection between theory and practice.

Original languageEnglish (US)
Title of host publication2017 IEEE International Workshop on Machine Learning for Signal Processing, MLSP 2017 - Proceedings
EditorsNaonori Ueda, Jen-Tzung Chien, Tomoko Matsui, Jan Larsen, Shinji Watanabe
PublisherIEEE Computer Society
Pages1-6
Number of pages6
ISBN (Electronic)9781509063413
DOIs
StatePublished - Dec 5 2017
Event2017 IEEE International Workshop on Machine Learning for Signal Processing, MLSP 2017 - Tokyo, Japan
Duration: Sep 25 2017Sep 28 2017

Publication series

NameIEEE International Workshop on Machine Learning for Signal Processing, MLSP
Volume2017-September
ISSN (Print)2161-0363
ISSN (Electronic)2161-0371

Conference

Conference2017 IEEE International Workshop on Machine Learning for Signal Processing, MLSP 2017
Country/TerritoryJapan
CityTokyo
Period09/25/1709/28/17

Keywords

  • Classification
  • Deterministic equivalent
  • Machine learning
  • QDA
  • Random matrix theory

ASJC Scopus subject areas

  • Human-Computer Interaction
  • Signal Processing

Cite this