Sparse Reduced-Rank Regression for Simultaneous Dimension Reduction and Variable Selection

Lisha Chen; Jianhua Z. Huang

doi:10.1080/01621459.2012.734178

Sparse Reduced-Rank Regression for Simultaneous Dimension Reduction and Variable Selection

Lisha Chen, Jianhua Z. Huang

Research output: Contribution to journal › Article › peer-review

183 Scopus citations

Abstract

The reduced-rank regression is an effective method in predicting multiple response variables from the same set of predictor variables. It reduces the number of model parameters and takes advantage of interrelations between the response variables and hence improves predictive accuracy. We propose to select relevant variables for reduced-rank regression by using a sparsity-inducing penalty. We apply a group-lasso type penalty that treats each row of the matrix of the regression coefficients as a group and show that this penalty satisfies certain desirable invariance properties. We develop two numerical algorithms to solve the penalized regression problem and establish the asymptotic consistency of the proposed method. In particular, the manifold structure of the reduced-rank regression coefficient matrix is considered and studied in our theoretical analysis. In our simulation study and real data analysis, the new method is compared with several existing variable selection methods for multivariate regression and exhibits competitive performance in prediction and variable selection. © 2012 American Statistical Association.

Original language	English (US)
Pages (from-to)	1533-1545
Number of pages	13
Journal	Journal of the American Statistical Association
Volume	107
Issue number	500
DOIs	https://doi.org/10.1080/01621459.2012.734178
State	Published - Oct 8 2012
Externally published	Yes

Bibliographical note

KAUST Repository Item: Exported on 2020-10-01
Acknowledged KAUST grant number(s): KUS-CI-016-04
Acknowledgements: Lisha Chen is Assistant Professor, Department of Statistics, Yale University, New Haven, CT 06511 (E-mail: [email protected]). Jianhua Z. Huang is Professor, Department of Statistics, Texas A&M University, College Station, TX 77843-3143 (E-mail: [email protected]). Huang's work was partially supported by grants from the National Science Foundation (NSF; DMS-0907170, DMS-1007618, DMS-1208952) and Award Number KUS-CI-016-04, made by King Abdullah University of Science and Technology (KAUST). The authors thank Joseph Chang, Dean Foster, Zhihua Qiao, Lyle Ungar, and Lan Zhou for helpful discussions. They also thank two anonymous reviewers, an associate editor and the co-editor Jun Liu for constructive comments.
This publication acknowledges KAUST support, but has no KAUST affiliated authors.

Access to Document

10.1080/01621459.2012.734178

Cite this

@article{66af5a761941451ebae4464f2151af95,

title = "Sparse Reduced-Rank Regression for Simultaneous Dimension Reduction and Variable Selection",

abstract = "The reduced-rank regression is an effective method in predicting multiple response variables from the same set of predictor variables. It reduces the number of model parameters and takes advantage of interrelations between the response variables and hence improves predictive accuracy. We propose to select relevant variables for reduced-rank regression by using a sparsity-inducing penalty. We apply a group-lasso type penalty that treats each row of the matrix of the regression coefficients as a group and show that this penalty satisfies certain desirable invariance properties. We develop two numerical algorithms to solve the penalized regression problem and establish the asymptotic consistency of the proposed method. In particular, the manifold structure of the reduced-rank regression coefficient matrix is considered and studied in our theoretical analysis. In our simulation study and real data analysis, the new method is compared with several existing variable selection methods for multivariate regression and exhibits competitive performance in prediction and variable selection. {\textcopyright} 2012 American Statistical Association.",

author = "Lisha Chen and Huang, {Jianhua Z.}",

note = "KAUST Repository Item: Exported on 2020-10-01 Acknowledged KAUST grant number(s): KUS-CI-016-04 Acknowledgements: Lisha Chen is Assistant Professor, Department of Statistics, Yale University, New Haven, CT 06511 (E-mail: [email protected]). Jianhua Z. Huang is Professor, Department of Statistics, Texas A&M University, College Station, TX 77843-3143 (E-mail: [email protected]). Huang's work was partially supported by grants from the National Science Foundation (NSF; DMS-0907170, DMS-1007618, DMS-1208952) and Award Number KUS-CI-016-04, made by King Abdullah University of Science and Technology (KAUST). The authors thank Joseph Chang, Dean Foster, Zhihua Qiao, Lyle Ungar, and Lan Zhou for helpful discussions. They also thank two anonymous reviewers, an associate editor and the co-editor Jun Liu for constructive comments. This publication acknowledges KAUST support, but has no KAUST affiliated authors.",

year = "2012",

month = oct,

day = "8",

doi = "10.1080/01621459.2012.734178",

language = "English (US)",

volume = "107",

pages = "1533--1545",

journal = "Journal of the American Statistical Association",

issn = "0162-1459",

publisher = "Taylor and Francis Ltd.",

number = "500",

}

TY - JOUR

T1 - Sparse Reduced-Rank Regression for Simultaneous Dimension Reduction and Variable Selection

AU - Chen, Lisha

AU - Huang, Jianhua Z.

N1 - KAUST Repository Item: Exported on 2020-10-01 Acknowledged KAUST grant number(s): KUS-CI-016-04 Acknowledgements: Lisha Chen is Assistant Professor, Department of Statistics, Yale University, New Haven, CT 06511 (E-mail: [email protected]). Jianhua Z. Huang is Professor, Department of Statistics, Texas A&M University, College Station, TX 77843-3143 (E-mail: [email protected]). Huang's work was partially supported by grants from the National Science Foundation (NSF; DMS-0907170, DMS-1007618, DMS-1208952) and Award Number KUS-CI-016-04, made by King Abdullah University of Science and Technology (KAUST). The authors thank Joseph Chang, Dean Foster, Zhihua Qiao, Lyle Ungar, and Lan Zhou for helpful discussions. They also thank two anonymous reviewers, an associate editor and the co-editor Jun Liu for constructive comments. This publication acknowledges KAUST support, but has no KAUST affiliated authors.

PY - 2012/10/8

Y1 - 2012/10/8

N2 - The reduced-rank regression is an effective method in predicting multiple response variables from the same set of predictor variables. It reduces the number of model parameters and takes advantage of interrelations between the response variables and hence improves predictive accuracy. We propose to select relevant variables for reduced-rank regression by using a sparsity-inducing penalty. We apply a group-lasso type penalty that treats each row of the matrix of the regression coefficients as a group and show that this penalty satisfies certain desirable invariance properties. We develop two numerical algorithms to solve the penalized regression problem and establish the asymptotic consistency of the proposed method. In particular, the manifold structure of the reduced-rank regression coefficient matrix is considered and studied in our theoretical analysis. In our simulation study and real data analysis, the new method is compared with several existing variable selection methods for multivariate regression and exhibits competitive performance in prediction and variable selection. © 2012 American Statistical Association.

AB - The reduced-rank regression is an effective method in predicting multiple response variables from the same set of predictor variables. It reduces the number of model parameters and takes advantage of interrelations between the response variables and hence improves predictive accuracy. We propose to select relevant variables for reduced-rank regression by using a sparsity-inducing penalty. We apply a group-lasso type penalty that treats each row of the matrix of the regression coefficients as a group and show that this penalty satisfies certain desirable invariance properties. We develop two numerical algorithms to solve the penalized regression problem and establish the asymptotic consistency of the proposed method. In particular, the manifold structure of the reduced-rank regression coefficient matrix is considered and studied in our theoretical analysis. In our simulation study and real data analysis, the new method is compared with several existing variable selection methods for multivariate regression and exhibits competitive performance in prediction and variable selection. © 2012 American Statistical Association.

UR - http://hdl.handle.net/10754/599681

UR - https://www.tandfonline.com/doi/full/10.1080/01621459.2012.734178

UR - http://www.scopus.com/inward/record.url?scp=84871942172&partnerID=8YFLogxK

U2 - 10.1080/01621459.2012.734178

DO - 10.1080/01621459.2012.734178

M3 - Article

SN - 0162-1459

VL - 107

SP - 1533

EP - 1545

JO - Journal of the American Statistical Association

JF - Journal of the American Statistical Association

IS - 500

ER -

Sparse Reduced-Rank Regression for Simultaneous Dimension Reduction and Variable Selection

Abstract

Bibliographical note

Access to Document

Other files and links

Fingerprint

Cite this