TY - JOUR

T1 - A bregman matrix and the gradient of mutual information for vector poisson and gaussian channels

AU - Wang, Liming

AU - Carlson, David Edwin

AU - Rodrigues, Miguel R.D.

AU - Calderbank, Robert

AU - Carin, Lawrence

N1 - Generated from Scopus record by KAUST IRTS on 2021-02-09

PY - 2014/1/1

Y1 - 2014/1/1

N2 - A generalization of Bregman divergence is developed and utilized to unify vector Poisson and Gaussian channel models, from the perspective of the gradient of mutual information. The gradient is with respect to the measurement matrix in a compressive-sensing setting, and mutual information is considered for signal recovery and classification. Existing gradient-of-mutual-information results for scalar Poisson models are recovered as special cases, as are known results for the vector Gaussian model. The Bregman-divergence generalization yields a Bregman matrix, and this matrix induces numerous matrix-valued metrics. The metrics associated with the Bregman matrix are detailed, as are its other properties. The Bregman matrix is also utilized to connect the relative entropy and mismatched minimum mean squared error. Two applications are considered: 1) compressive sensing with a Poisson measurement model and 2) compressive topic modeling for analysis of a document corpora (word-count data). In both of these settings, we use the developed theory to optimize the compressive measurement matrix, for signal recovery and classification. © 1963-2012 IEEE.

AB - A generalization of Bregman divergence is developed and utilized to unify vector Poisson and Gaussian channel models, from the perspective of the gradient of mutual information. The gradient is with respect to the measurement matrix in a compressive-sensing setting, and mutual information is considered for signal recovery and classification. Existing gradient-of-mutual-information results for scalar Poisson models are recovered as special cases, as are known results for the vector Gaussian model. The Bregman-divergence generalization yields a Bregman matrix, and this matrix induces numerous matrix-valued metrics. The metrics associated with the Bregman matrix are detailed, as are its other properties. The Bregman matrix is also utilized to connect the relative entropy and mismatched minimum mean squared error. Two applications are considered: 1) compressive sensing with a Poisson measurement model and 2) compressive topic modeling for analysis of a document corpora (word-count data). In both of these settings, we use the developed theory to optimize the compressive measurement matrix, for signal recovery and classification. © 1963-2012 IEEE.

UR - http://ieeexplore.ieee.org/document/6746209/

UR - http://www.scopus.com/inward/record.url?scp=84899632397&partnerID=8YFLogxK

U2 - 10.1109/TIT.2014.2307068

DO - 10.1109/TIT.2014.2307068

M3 - Article

SN - 0018-9448

VL - 60

SP - 2611

EP - 2629

JO - IEEE Transactions on Information Theory

JF - IEEE Transactions on Information Theory

IS - 5

ER -