Login | Register

Task Scheduling in Big Data Platforms: A Systematic Literature Review


Task Scheduling in Big Data Platforms: A Systematic Literature Review

Soualhia, Mbarka, Khomh, Foutse and Tahar, Sofiène (2017) Task Scheduling in Big Data Platforms: A Systematic Literature Review. Journal of Systems and Software . ISSN 01641212 (In Press)

[thumbnail of Tahar-JSS-2017.pdf]
Text (application/pdf)
Tahar-JSS-2017.pdf - Accepted Version
Available under License Spectrum Terms of Access.

Official URL: http://dx.doi.org/10.1016/j.jss.2017.09.001


Context: Hadoop, Spark, Storm, and Mesos are very well known frameworks in both research and industrial communities that allow expressing and processing distributed computations on massive amounts of data. Multiple scheduling algorithms have been proposed to ensure that short interactive jobs, large batch jobs, and guaranteed-capacity production jobs running on these frameworks can deliver results quickly while maintaining a high throughput. However, only a few works have examined the effectiveness of these algorithms.

Objective: The Evidence-based Software Engineering (EBSE) paradigm and its core tool, i.e., the Systematic Literature Review (SLR), have been introduced to the Software Engineering community in 2004 to help researchers systematically and objectively gather and aggregate research evidences about different topics. In this paper, we conduct a SLR of task scheduling algorithms that have been proposed for big data platforms.

Method: We analyse the design decisions of different scheduling models proposed in the literature for Hadoop, Spark, Storm, and Mesos over the period between 2005 and 2016. We provide a research taxonomy for succinct classification of these scheduling models. We also compare the algorithms in terms of performance, resources utilization, and failure recovery mechanisms.

Results: Our searches identifies 586 studies from journals, conferences and workshops having the highest quality in this field. This SLR reports about different types of scheduling models (dynamic, constrained, and adaptive) and the main motivations behind them (including data locality, workload balancing, resources utilization, and energy efficiency). A discussion of some open issues and future challenges pertaining to improving the current studies is provided.

Divisions:Concordia University > Gina Cody School of Engineering and Computer Science > Electrical and Computer Engineering
Item Type:Article
Authors:Soualhia, Mbarka and Khomh, Foutse and Tahar, Sofiène
Journal or Publication:Journal of Systems and Software
Date:5 September 2017
Digital Object Identifier (DOI):10.1016/j.jss.2017.09.001
Keywords:Task Scheduling; Hadoop; Spark; Storm; Mesos; Systematic Literature Review
ID Code:982995
Deposited By: Danielle Dennie
Deposited On:07 Sep 2017 20:45
Last Modified:01 Sep 2018 00:01
All items in Spectrum are protected by copyright, with all rights reserved. The use of items is governed by Spectrum's terms of access.

Repository Staff Only: item control page

Downloads per month over past year

Research related to the current document (at the CORE website)
- Research related to the current document (at the CORE website)
Back to top Back to top