TY - JOUR

T1 - Learning to Sample with Adversarially Learned Likelihood-Ratio

AU - Li, Chunyuan

AU - Li, Jianqiao

AU - Wang, Guoyin

AU - Carin, Lawrence

PY - 2018

Y1 - 2018

N2 - We link the reverse KL divergence with adversarial learning. This insight enables learning to synthesize realistic samples in two settings: (i) Given a set of samples from the true distribution, an adversarially learned likelihood-ratio and a new entropy bound are used to learn a GAN model, that improves synthesized sample quality relative to previous GAN variants. (ii) Given an unnormalized distribution, a reference-based framework is proposed to learn to draw samples, naturally yielding an adversarial scheme to amortize MCMC/SVGD samples. Experimental results show the improved performance of the derived algorithms. 1 BACKGROUND ON THE REVERSE KL DIVERGENCE Target Distribution Assume we are given a set of samples D = {x i } i=1,N , with each sample assumed drawn iid from an unknown distribution q(x). For x ∈ X , let S q ⊂ X represent the support of q, implying that S q is the smallest subset of X for which Sq q(x)dx = 1 (or Sq q(x)dx = 1 − , for → 0 +). Let S o q represent the complement set of S q , i.e., S q ∪ S o q = X and S q ∩ S o q = ∅.

AB - We link the reverse KL divergence with adversarial learning. This insight enables learning to synthesize realistic samples in two settings: (i) Given a set of samples from the true distribution, an adversarially learned likelihood-ratio and a new entropy bound are used to learn a GAN model, that improves synthesized sample quality relative to previous GAN variants. (ii) Given an unnormalized distribution, a reference-based framework is proposed to learn to draw samples, naturally yielding an adversarial scheme to amortize MCMC/SVGD samples. Experimental results show the improved performance of the derived algorithms. 1 BACKGROUND ON THE REVERSE KL DIVERGENCE Target Distribution Assume we are given a set of samples D = {x i } i=1,N , with each sample assumed drawn iid from an unknown distribution q(x). For x ∈ X , let S q ⊂ X represent the support of q, implying that S q is the smallest subset of X for which Sq q(x)dx = 1 (or Sq q(x)dx = 1 − , for → 0 +). Let S o q represent the complement set of S q , i.e., S q ∪ S o q = X and S q ∩ S o q = ∅.

UR - https://openreview.net/forum?id=S1eZGHkDM

UR - https://www.mendeley.com/catalogue/385dcfb3-1bf3-3fc4-9211-3561a398d27e/

M3 - Article

SP - 1

EP - 6

JO - Iclr 2018

JF - Iclr 2018

IS - 2

ER -