TY - JOUR
T1 - Exploiting low-rank covariance structures for computing high-dimensional normal and Student-t probabilities
AU - Cao, Jian
AU - Genton, Marc G.
AU - Keyes, David E.
AU - Turkiyyah, George M.
N1 - KAUST Repository Item: Exported on 2021-02-01
Acknowledgements: The authors thank Prof. Georgiy Stenchikov at KAUST for providing the WRF data and the two anonymous reviewers for valuable comments that improved this manuscript.
PY - 2021/1/12
Y1 - 2021/1/12
N2 - We present a preconditioned Monte Carlo method for computing high-dimensional multivariate normal and Student-t probabilities arising in spatial statistics. The approach combines a tile-low-rank representation of covariance matrices with a block-reordering scheme for efficient quasi-Monte Carlo simulation. The tile-low-rank representation decomposes the high-dimensional problem into many diagonal-block-size problems and low-rank connections. The block-reordering scheme reorders between and within the diagonal blocks to reduce the impact of integration variables from right to left, thus improving the Monte Carlo convergence rate. Simulations up to dimension 65,536 suggest that the new method can improve the run time by an order of magnitude compared with the hierarchical quasi-Monte Carlo method and two orders of magnitude compared with the dense quasi-Monte Carlo method. Our method also forms a strong substitute for the approximate conditioning methods as a more robust estimation with error guarantees. An application study to wind stochastic generators is provided to illustrate that the new computational method makes the maximum likelihood estimation feasible for high-dimensional skew-normal random fields.
AB - We present a preconditioned Monte Carlo method for computing high-dimensional multivariate normal and Student-t probabilities arising in spatial statistics. The approach combines a tile-low-rank representation of covariance matrices with a block-reordering scheme for efficient quasi-Monte Carlo simulation. The tile-low-rank representation decomposes the high-dimensional problem into many diagonal-block-size problems and low-rank connections. The block-reordering scheme reorders between and within the diagonal blocks to reduce the impact of integration variables from right to left, thus improving the Monte Carlo convergence rate. Simulations up to dimension 65,536 suggest that the new method can improve the run time by an order of magnitude compared with the hierarchical quasi-Monte Carlo method and two orders of magnitude compared with the dense quasi-Monte Carlo method. Our method also forms a strong substitute for the approximate conditioning methods as a more robust estimation with error guarantees. An application study to wind stochastic generators is provided to illustrate that the new computational method makes the maximum likelihood estimation feasible for high-dimensional skew-normal random fields.
UR - http://hdl.handle.net/10754/666234
UR - http://link.springer.com/10.1007/s11222-020-09978-y
UR - http://www.scopus.com/inward/record.url?scp=85099391205&partnerID=8YFLogxK
U2 - 10.1007/s11222-020-09978-y
DO - 10.1007/s11222-020-09978-y
M3 - Article
SN - 1573-1375
VL - 31
JO - Statistics and Computing
JF - Statistics and Computing
IS - 1
ER -