Abstract
We propose a stochastic extension of the primal-dual hybrid gradient algorithm studied by Chambolle and Pock in 2011 to solve saddle point problems that are separable in the dual variable. The analysis is carried out for general convex-concave saddle point problems and problems that are either partially smooth / strongly convex or fully smooth / strongly convex. We perform the analysis for arbitrary samplings of dual variables, and we obtain known deterministic results as a special case. Several variants of our stochastic method significantly outperform the deterministic variant on a variety of imaging tasks.
| Original language | English (US) |
|---|---|
| Pages (from-to) | 2783-2808 |
| Number of pages | 26 |
| Journal | SIAM Journal on Optimization |
| Volume | 28 |
| Issue number | 4 |
| DOIs | |
| State | Published - 2017 |
Bibliographical note
KAUST Repository Item: Exported on 2020-04-23Acknowledgements: The work of the first author was supported by the ANR, 'EANOI' project I1148 / ANR-12-IS01-0003 (joint with FWF); part of this work was done while he was hosted in Churchill College and DAMTP, Centre for Mathematical Sciences, University of Cambridge, thanks to support from the French Embassy in the UK and the Cantab Capital Institute for the Mathematics of Information. The work of the second and fourth authors was supported by Leverhulme Trust project ``Breaking the non-convexity barrier,"" EPSRC grant EP/M00483X/1, EPSRC centre grant EP/N014588/1, the Cantab Capital Institute for the Mathematics of Information, and from CHiPS (Horizon 2020 RISE project grant). The second author carried out initial work supported by the EPSRC platform grant EP/M020533/1. Moreover, the fourth author is thankful for support by The Alan Turing Institute. The work of the third author was supported by EPSRC Fellowship in Mathematical Sciences grant EP/N005538/1, entitled ``Randomized algorithms for extreme convex optimization.""