Generalized Twin Gaussian processes using Sharma–Mittal divergence

Mohamed Elhoseiny, Ahmed Elgammal

Research output: Contribution to journalArticlepeer-review

5 Scopus citations


There has been a growing interest in mutual information measures due to their wide range of applications in machine learning and computer vision. In this paper, we present a generalized structured regression framework based on Sharma–Mittal (SM) divergence, a relative entropy measure, which is introduced to in the machine learning community in this work. SM divergence is a generalized mutual information measure for the widely used Rényi, Tsallis, Bhattacharyya, and Kullback–Leibler (KL) relative entropies. Specifically, we study SM divergence as a cost function in the context of the Twin Gaussian processes (TGP) (Bo and Sminchisescu 2010), which generalizes over the KL-divergence without computational penalty. We show interesting properties of Sharma–Mittal TGP (SMTGP) through a theoretical analysis, which covers missing insights in the traditional TGP formulation. However, we generalize this theory based on SM-divergence instead of KL-divergence which is a special case. Experimentally, we evaluated the proposed SMTGP framework on several datasets. The results show that SMTGP reaches better predictions than KL-based TGP, since it offers a bigger class of models through its parameters that we learn from the data.
Original languageEnglish (US)
JournalMachine Learning
Issue number2-3
StatePublished - Sep 17 2015
Externally publishedYes

Bibliographical note

Generated from Scopus record by KAUST IRTS on 2019-11-20


Dive into the research topics of 'Generalized Twin Gaussian processes using Sharma–Mittal divergence'. Together they form a unique fingerprint.

Cite this