Automatic cross-validation in structured models: Is it time to leave out leave-one-out?

Aritz Adin*, Elias Teixeira Krainski, Amanda Lenzi, Zhedong Liu, Joaquín Martínez-Minaya, Håvard Rue

*Corresponding author for this work

Research output: Contribution to journalArticlepeer-review

3 Scopus citations

Abstract

Standard techniques such as leave-one-out cross-validation (LOOCV) might not be suitable for evaluating the predictive performance of models incorporating structured random effects. In such cases, the correlation between the training and test sets could have a notable impact on the model's prediction error. To overcome this issue, an automatic group construction procedure for leave-group-out cross validation (LGOCV) has recently emerged as a valuable tool for enhancing predictive performance measurement in structured models. The purpose of this paper is (i) to compare LOOCV and LGOCV within structured models, emphasizing model selection and predictive performance, and (ii) to provide real data applications in spatial statistics using complex structured models fitted with INLA, showcasing the utility of the automatic LGOCV method. First, we briefly review the key aspects of the recently proposed LGOCV method for automatic group construction in latent Gaussian models. We also demonstrate the effectiveness of this method for selecting the model with the highest predictive performance by simulating extrapolation tasks in both temporal and spatial data analyses. Finally, we provide insights into the effectiveness of the LGOCV method in modeling complex structured data, encompassing spatio-temporal multivariate count data, spatial compositional data, and spatio-temporal geospatial data.

Original languageEnglish (US)
Article number100843
JournalSpatial Statistics
Volume62
DOIs
StatePublished - Aug 2024

Bibliographical note

Publisher Copyright:
© 2024 The Author(s)

Keywords

  • Cross-validation
  • Hierarchical models
  • INLA
  • Spatial statistics

ASJC Scopus subject areas

  • Statistics and Probability
  • Computers in Earth Sciences
  • Management, Monitoring, Policy and Law

Fingerprint

Dive into the research topics of 'Automatic cross-validation in structured models: Is it time to leave out leave-one-out?'. Together they form a unique fingerprint.

Cite this