SAVE THE DATE Tapia 2018 Orlando, FL September 19-22, 2018

2017 Tapia Conference

Thermal-aware High Performance Clusters

Contributors

Shubbhi Taneja, Auburn University

Abstract

We have developed a thermal-aware job scheduling strategy called tDispatch tailored for MapReduce applications running on Hadoop clusters. The scheduling idea of tDispatch is motivated by a profiling study of CPU-intensive and I/O-intensive jobs from the perspective of thermal efficiency. More specifically, we investigate the thermal behaviors of these two types of jobs running on a Hadoop cluster by stress testing data nodes through extensive experiments. We show that CPU-intensive and I/O-intensive jobs exhibit various thermal and performance impacts on multicore processors and hard drives of Hadoop clusters. After we quantify the thermal behaviors of Hadoop jobs on the master and data nodes of a cluster, we propose our scheduler to alternatively dispatch CPU-intensive and I/O-intensive jobs. We apply our strategy to several MapReduce applications with different resource consumption profiles. Our experimental results show that tDispatch is conducive of creating opportunities to cool down multicore processors and disks in Hadoop clusters deployed in modern data centers. Currently, we are extending these experiments on a Spark cluster installed in a HPC room. For the ongoing experiments, along with the temperatures, we are also considering parameters like energy consumption of the worker nodes, height of the nodes in the rack, thermal patterns in HPC room and the number of nodes. For the same purpose, we are using a big data benchmark suite called HiBench. We will be developing thermal models based on the utilization patterns of these nodes. Our findings can be applied in other thermal-efficient job schedulers that are aware of thermal behaviors of CPU-intensive and I/O-intensive applications submitted to Hadoop and Spark clusters