Model Monitoring and Dynamic Model Selection in Travel Time-Series Forecasting

Rosa Candela; Pietro Michiardi; Maurizio Filippone; Maria A. Zuluaga

doi:10.1007/978-3-030-67667-4_31

Model Monitoring and Dynamic Model Selection in Travel Time-Series Forecasting

Rosa Candela, Pietro Michiardi, Maurizio Filippone, Maria A. Zuluaga^*

^*Corresponding author for this work

Statistics

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

Abstract

Accurate travel products price forecasting is a highly desired feature that allows customers to take informed decisions about purchases, and companies to build and offer attractive tour packages. Thanks to machine learning (ML), it is now relatively cheap to develop highly accurate statistical models for price time-series forecasting. However, once models are deployed in production, it is their monitoring, maintenance and improvement which carry most of the costs and difficulties over time. We introduce a data-driven framework to continuously monitor and maintain deployed time-series forecasting models’ performance, to guarantee stable performance of travel products price forecasting models. Under a supervised learning approach, we predict the errors of time-series forecasting models over time, and use this predicted performance measure to achieve both model monitoring and maintenance. We validate the proposed method on a dataset of 18K time-series from flight and hotel prices collected over two years and on two public benchmarks.

Original language	English (US)
Title of host publication	Machine Learning and Knowledge Discovery in Databases
Subtitle of host publication	Applied Data Science Track - European Conference, ECML PKDD 2020, Proceedings
Editors	Yuxiao Dong, Dunja Mladenic, Craig Saunders
Publisher	Springer Science and Business Media Deutschland GmbH
Pages	513-529
Number of pages	17
ISBN (Print)	9783030676667
DOIs	https://doi.org/10.1007/978-3-030-67667-4_31
State	Published - 2021
Event	European Conference on Machine Learning and Knowledge Discovery in Databases, ECML PKDD 2020 - Virtual, Online Duration: Sep 14 2020 → Sep 18 2020

Publication series

Name	Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume	12460 LNAI
ISSN (Print)	0302-9743
ISSN (Electronic)	1611-3349

Conference

Conference	European Conference on Machine Learning and Knowledge Discovery in Databases, ECML PKDD 2020
City	Virtual, Online
Period	09/14/20 → 09/18/20

Bibliographical note

Publisher Copyright:
© 2021, Springer Nature Switzerland AG.

Keywords

Forecasting
Model maintenance
Model monitoring
Time-series

ASJC Scopus subject areas

Theoretical Computer Science
General Computer Science

Access to Document

10.1007/978-3-030-67667-4_31

Cite this

Candela, R., Michiardi, P., Filippone, M., & Zuluaga, M. A. (2021). Model Monitoring and Dynamic Model Selection in Travel Time-Series Forecasting. In Y. Dong, D. Mladenic, & C. Saunders (Eds.), Machine Learning and Knowledge Discovery in Databases: Applied Data Science Track - European Conference, ECML PKDD 2020, Proceedings (pp. 513-529). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 12460 LNAI). Springer Science and Business Media Deutschland GmbH. https://doi.org/10.1007/978-3-030-67667-4_31

Candela, Rosa ; Michiardi, Pietro ; Filippone, Maurizio et al. / Model Monitoring and Dynamic Model Selection in Travel Time-Series Forecasting. Machine Learning and Knowledge Discovery in Databases: Applied Data Science Track - European Conference, ECML PKDD 2020, Proceedings. editor / Yuxiao Dong ; Dunja Mladenic ; Craig Saunders. Springer Science and Business Media Deutschland GmbH, 2021. pp. 513-529 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)).

@inproceedings{4cf71685c1934f6ab9317329e54009b3,

title = "Model Monitoring and Dynamic Model Selection in Travel Time-Series Forecasting",

abstract = "Accurate travel products price forecasting is a highly desired feature that allows customers to take informed decisions about purchases, and companies to build and offer attractive tour packages. Thanks to machine learning (ML), it is now relatively cheap to develop highly accurate statistical models for price time-series forecasting. However, once models are deployed in production, it is their monitoring, maintenance and improvement which carry most of the costs and difficulties over time. We introduce a data-driven framework to continuously monitor and maintain deployed time-series forecasting models{\textquoteright} performance, to guarantee stable performance of travel products price forecasting models. Under a supervised learning approach, we predict the errors of time-series forecasting models over time, and use this predicted performance measure to achieve both model monitoring and maintenance. We validate the proposed method on a dataset of 18K time-series from flight and hotel prices collected over two years and on two public benchmarks.",

keywords = "Forecasting, Model maintenance, Model monitoring, Time-series",

author = "Rosa Candela and Pietro Michiardi and Maurizio Filippone and Zuluaga, {Maria A.}",

note = "Publisher Copyright: {\textcopyright} 2021, Springer Nature Switzerland AG.; European Conference on Machine Learning and Knowledge Discovery in Databases, ECML PKDD 2020 ; Conference date: 14-09-2020 Through 18-09-2020",

year = "2021",

doi = "10.1007/978-3-030-67667-4_31",

language = "English (US)",

isbn = "9783030676667",

series = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",

publisher = "Springer Science and Business Media Deutschland GmbH",

pages = "513--529",

editor = "Yuxiao Dong and Dunja Mladenic and Craig Saunders",

booktitle = "Machine Learning and Knowledge Discovery in Databases",

address = "Germany",

}

Candela, R, Michiardi, P, Filippone, M & Zuluaga, MA 2021, Model Monitoring and Dynamic Model Selection in Travel Time-Series Forecasting. in Y Dong, D Mladenic & C Saunders (eds), Machine Learning and Knowledge Discovery in Databases: Applied Data Science Track - European Conference, ECML PKDD 2020, Proceedings. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 12460 LNAI, Springer Science and Business Media Deutschland GmbH, pp. 513-529, European Conference on Machine Learning and Knowledge Discovery in Databases, ECML PKDD 2020, Virtual, Online, 09/14/20. https://doi.org/10.1007/978-3-030-67667-4_31

Model Monitoring and Dynamic Model Selection in Travel Time-Series Forecasting. / Candela, Rosa; Michiardi, Pietro; Filippone, Maurizio et al.
Machine Learning and Knowledge Discovery in Databases: Applied Data Science Track - European Conference, ECML PKDD 2020, Proceedings. ed. / Yuxiao Dong; Dunja Mladenic; Craig Saunders. Springer Science and Business Media Deutschland GmbH, 2021. p. 513-529 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 12460 LNAI).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - Model Monitoring and Dynamic Model Selection in Travel Time-Series Forecasting

AU - Candela, Rosa

AU - Michiardi, Pietro

AU - Filippone, Maurizio

AU - Zuluaga, Maria A.

PY - 2021

Y1 - 2021

N2 - Accurate travel products price forecasting is a highly desired feature that allows customers to take informed decisions about purchases, and companies to build and offer attractive tour packages. Thanks to machine learning (ML), it is now relatively cheap to develop highly accurate statistical models for price time-series forecasting. However, once models are deployed in production, it is their monitoring, maintenance and improvement which carry most of the costs and difficulties over time. We introduce a data-driven framework to continuously monitor and maintain deployed time-series forecasting models’ performance, to guarantee stable performance of travel products price forecasting models. Under a supervised learning approach, we predict the errors of time-series forecasting models over time, and use this predicted performance measure to achieve both model monitoring and maintenance. We validate the proposed method on a dataset of 18K time-series from flight and hotel prices collected over two years and on two public benchmarks.

AB - Accurate travel products price forecasting is a highly desired feature that allows customers to take informed decisions about purchases, and companies to build and offer attractive tour packages. Thanks to machine learning (ML), it is now relatively cheap to develop highly accurate statistical models for price time-series forecasting. However, once models are deployed in production, it is their monitoring, maintenance and improvement which carry most of the costs and difficulties over time. We introduce a data-driven framework to continuously monitor and maintain deployed time-series forecasting models’ performance, to guarantee stable performance of travel products price forecasting models. Under a supervised learning approach, we predict the errors of time-series forecasting models over time, and use this predicted performance measure to achieve both model monitoring and maintenance. We validate the proposed method on a dataset of 18K time-series from flight and hotel prices collected over two years and on two public benchmarks.

KW - Forecasting

KW - Model maintenance

KW - Model monitoring

KW - Time-series

UR - http://www.scopus.com/inward/record.url?scp=85103248382&partnerID=8YFLogxK

U2 - 10.1007/978-3-030-67667-4_31

DO - 10.1007/978-3-030-67667-4_31

M3 - Conference contribution

AN - SCOPUS:85103248382

SN - 9783030676667

T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

SP - 513

EP - 529

BT - Machine Learning and Knowledge Discovery in Databases

A2 - Dong, Yuxiao

A2 - Mladenic, Dunja

A2 - Saunders, Craig

PB - Springer Science and Business Media Deutschland GmbH

T2 - European Conference on Machine Learning and Knowledge Discovery in Databases, ECML PKDD 2020

Y2 - 14 September 2020 through 18 September 2020

ER -

Candela R, Michiardi P, Filippone M, Zuluaga MA. Model Monitoring and Dynamic Model Selection in Travel Time-Series Forecasting. In Dong Y, Mladenic D, Saunders C, editors, Machine Learning and Knowledge Discovery in Databases: Applied Data Science Track - European Conference, ECML PKDD 2020, Proceedings. Springer Science and Business Media Deutschland GmbH. 2021. p. 513-529. (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)). doi: 10.1007/978-3-030-67667-4_31

Model Monitoring and Dynamic Model Selection in Travel Time-Series Forecasting

Abstract

Publication series

Conference

Bibliographical note

Keywords

ASJC Scopus subject areas

Access to Document

Other files and links

Fingerprint

Cite this