TideWatch: Fingerprinting the cyclicality of big data workloads

Daniel W. Williams; Shuai Zheng; Xiangliang Zhang; Hani T. Jamjoom

doi:10.1109/INFOCOM.2014.6848144

TideWatch: Fingerprinting the cyclicality of big data workloads

Daniel W. Williams, Shuai Zheng, Xiangliang Zhang, Hani T. Jamjoom

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

7 Scopus citations

Abstract

Intrinsic to 'big data' processing workloads (e.g., iterative MapReduce, Pregel, etc.) are cyclical resource utilization patterns that are highly synchronized across different resource types as well as the workers in a cluster. In Infrastructure as a Service settings, cloud providers do not exploit this characteristic to better manage VMs because they view VMs as 'black boxes.' We present TideWatch, a system that automatically identifies cyclicality and similarity in running VMs. TideWatch predicts period lengths of most VMs in Hadoop workloads within 9% of actual iteration boundaries and successfully classifies up to 95% of running VMs as participating in the appropriate Hadoop cluster. Furthermore, we show how TideWatch can be used to improve the timing of VM migrations, reducing both migration time and network impact by over 50% when compared to a random approach. © 2014 IEEE.

Original language	English (US)
Title of host publication	IEEE INFOCOM 2014 - IEEE Conference on Computer Communications
Publisher	Institute of Electrical and Electronics Engineers (IEEE)
Pages	2031-2039
Number of pages	9
ISBN (Print)	9781479933600
DOIs	https://doi.org/10.1109/INFOCOM.2014.6848144
State	Published - Apr 2014

Bibliographical note

KAUST Repository Item: Exported on 2020-10-01

Access to Document

10.1109/INFOCOM.2014.6848144

Cite this

@inproceedings{f0e157120013428d8bd34145e854c5c0,

title = "TideWatch: Fingerprinting the cyclicality of big data workloads",

abstract = "Intrinsic to 'big data' processing workloads (e.g., iterative MapReduce, Pregel, etc.) are cyclical resource utilization patterns that are highly synchronized across different resource types as well as the workers in a cluster. In Infrastructure as a Service settings, cloud providers do not exploit this characteristic to better manage VMs because they view VMs as 'black boxes.' We present TideWatch, a system that automatically identifies cyclicality and similarity in running VMs. TideWatch predicts period lengths of most VMs in Hadoop workloads within 9% of actual iteration boundaries and successfully classifies up to 95% of running VMs as participating in the appropriate Hadoop cluster. Furthermore, we show how TideWatch can be used to improve the timing of VM migrations, reducing both migration time and network impact by over 50% when compared to a random approach. {\textcopyright} 2014 IEEE.",

author = "Williams, {Daniel W.} and Shuai Zheng and Xiangliang Zhang and Jamjoom, {Hani T.}",

note = "KAUST Repository Item: Exported on 2020-10-01",

year = "2014",

month = apr,

doi = "10.1109/INFOCOM.2014.6848144",

language = "English (US)",

isbn = "9781479933600",

pages = "2031--2039",

booktitle = "IEEE INFOCOM 2014 - IEEE Conference on Computer Communications",

publisher = "Institute of Electrical and Electronics Engineers (IEEE)",

}

TY - GEN

T1 - TideWatch: Fingerprinting the cyclicality of big data workloads

AU - Williams, Daniel W.

AU - Zheng, Shuai

AU - Zhang, Xiangliang

AU - Jamjoom, Hani T.

N1 - KAUST Repository Item: Exported on 2020-10-01

PY - 2014/4

Y1 - 2014/4

N2 - Intrinsic to 'big data' processing workloads (e.g., iterative MapReduce, Pregel, etc.) are cyclical resource utilization patterns that are highly synchronized across different resource types as well as the workers in a cluster. In Infrastructure as a Service settings, cloud providers do not exploit this characteristic to better manage VMs because they view VMs as 'black boxes.' We present TideWatch, a system that automatically identifies cyclicality and similarity in running VMs. TideWatch predicts period lengths of most VMs in Hadoop workloads within 9% of actual iteration boundaries and successfully classifies up to 95% of running VMs as participating in the appropriate Hadoop cluster. Furthermore, we show how TideWatch can be used to improve the timing of VM migrations, reducing both migration time and network impact by over 50% when compared to a random approach. © 2014 IEEE.

AB - Intrinsic to 'big data' processing workloads (e.g., iterative MapReduce, Pregel, etc.) are cyclical resource utilization patterns that are highly synchronized across different resource types as well as the workers in a cluster. In Infrastructure as a Service settings, cloud providers do not exploit this characteristic to better manage VMs because they view VMs as 'black boxes.' We present TideWatch, a system that automatically identifies cyclicality and similarity in running VMs. TideWatch predicts period lengths of most VMs in Hadoop workloads within 9% of actual iteration boundaries and successfully classifies up to 95% of running VMs as participating in the appropriate Hadoop cluster. Furthermore, we show how TideWatch can be used to improve the timing of VM migrations, reducing both migration time and network impact by over 50% when compared to a random approach. © 2014 IEEE.

UR - http://hdl.handle.net/10754/564894

UR - http://ieeexplore.ieee.org/document/6848144/

UR - http://www.scopus.com/inward/record.url?scp=84904438498&partnerID=8YFLogxK

U2 - 10.1109/INFOCOM.2014.6848144

DO - 10.1109/INFOCOM.2014.6848144

M3 - Conference contribution

SN - 9781479933600

SP - 2031

EP - 2039

BT - IEEE INFOCOM 2014 - IEEE Conference on Computer Communications

PB - Institute of Electrical and Electronics Engineers (IEEE)

ER -

TideWatch: Fingerprinting the cyclicality of big data workloads

Abstract

Bibliographical note

Access to Document

Other files and links

Fingerprint

Cite this