Dssouli, Rachida, El-Kassabi, Hadeel T., Adel Serhani, M. and Navaz, Alramzana N. (2019) Trust enforcement through self-adapting cloud workflow orchestration. Future Generation Computer Systems, 97 . pp. 462-481. ISSN 0167739X (In Press)
Preview |
Text (application/pdf)
3MBDssouli 2019.pdf - Accepted Version Available under License Spectrum Terms of Access. |
Official URL: https://doi.org/10.1016/j.future.2019.03.004
Abstract
Providing runtime intelligence of a workflow in a highly dynamic cloud execution environment is a challenging task due the continuously changing cloud resources. Guaranteeing a certain level of workflow Quality of Service (QoS) during the execution will require continuous monitoring to detect any performance violation due to resource shortage or even cloud service interruption. Most of orchestration schemes are either configuration, or deployment dependent and they do not cope with dynamically changing environment resources. In this paper, we propose a workflow orchestration, monitoring, and adaptation model that relies on trust evaluation to detect QoS performance degradation and perform an automatic reconfiguration to guarantee QoS of the workflow. The monitoring and adaptation schemes are able to detect and repair different types of real time errors and trigger different adaptation actions including workflow reconfiguration, migration, and resource scaling. We formalize the cloud resource orchestration using state machine that efficiently captures different dynamic properties of the cloud execution environment. In addition, we use validation model checker to validate our model in terms of reachability, liveness, and safety properties. Extensive experimentation is performed using a health monitoring workflow we have developed to handle dataset from Intelligent Monitoring in Intensive Care III (MIMICIII) and deployed over Docker swarm cluster. A set of scenarios were carefully chosen to evaluate workflow monitoring and the different adaptation schemes we have implemented. The results prove that our automated workflow orchestration model is self-adapting, self-configuring, react efficiently to changes and adapt accordingly while supporting high level of Workflow QoS.
| Divisions: | Concordia University > Gina Cody School of Engineering and Computer Science > Concordia Institute for Information Systems Engineering |
|---|---|
| Item Type: | Article |
| Refereed: | Yes |
| Authors: | Dssouli, Rachida and El-Kassabi, Hadeel T. and Adel Serhani, M. and Navaz, Alramzana N. |
| Journal or Publication: | Future Generation Computer Systems |
| Date: | 2019 |
| Digital Object Identifier (DOI): | 10.1016/j.future.2019.03.004 |
| Keywords: | Cloud, QoS; Reconfiguration; Self-adapt system; State machine; Trust assessment; Workflow |
| ID Code: | 985080 |
| Deposited By: | ALINE SOREL |
| Deposited On: | 08 Apr 2019 15:44 |
| Last Modified: | 03 Mar 2021 02:00 |
References:
Weerasiri D. Configuration and Orchestration Techniques for Federated Cloud Resources(Ph.D. thesis) The University of New South Wales (2016)
Lemos A.L., Daniel F., Benatallah B. Web service composition: a survey of techniques and tools ACM Comput. Surv., 48 (3) (2016), p. 33
deltacloud, http://deltacloud.apache.org, accessed: 2018-05-01.
Apache libcloud, http://libcloud.apache.org, accessed: 2018-05-01.
jclouds, http://www.jclouds.org, accessed: 2018-05-01.
openstack, http://www.openstack.org, accessed: 2018-05-01.
Yu B., Singh M.P. An evidential model of distributed reputation management. Proceedings of the first international joint conference on Autonomous Agents and Multiagent Systems: Part 1, ACM (2002), pp. 294-301
Li X., Hu W., Ding T., Ruiz R. Trust constrained workflow scheduling in cloud computing Systems, Man, and Cybernetics (SMC), 2017 IEEE International Conference on, IEEE (2017), pp. 164-169
Bernstein D., Vij D.Intercloud security considerations. Cloud Computing Technology and Science (CloudCom), 2010 IEEE Second International Conference on, IEEE (2010), pp. 537-544
Abawajy J.Determining service trustworthiness in intercloud computing environments. Pervasive Systems, Algorithms, and Networks (ISPAN), 2009 10th International Symposium on, IEEE (2009), pp. 784-788
Keahey K., Tsugawa M., Matsunaga A., Fortes J. Sky computing. IEEE Internet Comput., 13 (5) (2009), pp. 43-51[12]
Bernstein D., Ludvigson E., Sankar K., Diamond S., Morrow M.Blueprint for the intercloud–protocols and formats for cloud computing. internet and web applications and services, 2009. ICIW’09. Fourth International Conference (2009), pp. 1-3
Toosi A.N., Calheiros R.N., Buyya R.Interconnected cloud computing environments: challenges, taxonomy, and survey, ACM Comput. Surv., 47 (1) (2014), p. 7
Weerasiri D., Barukh M.C., Benatallah B., Sheng Q.Z., Ranjan R.A taxonomy and survey of cloud resource orchestration techniques, ACM Comput. Surv., 50 (2) (2017), p. 26
Barth W.Nagios: System and Network Monitoring. No Starch Press (2008)
Zadrozny P., Kodali R.Big Data Analytics Using Splunk: Deriving Operational Intelligence from Social Media, Machine Data, Existing Data Warehouses, and Other Real-Time Streaming Sources Apress (2013)
Ganglia monitoring system, http://ganglia.sourceforge.net/, accessed: 2018-04-01.
Apache chukwa, http://chukwa.apache.org/, accessed: 2018-04-01.
Sematext, https://sematext.com/, accessed: 2018-04-01.
Sequenceiq, http://sequenceiq.com/, accessed: 2018-04-01.
Alhamazani K., Ranjan R., Mitra K., Rabhi F., Jayaraman P.P., Khan S.U., Guabtni A., Bhatnagar V.An overview of the commercial cloud monitoring tools: research dimensions, design issues, and state-of-the-art, Computing, 97 (4) (2015), pp. 357-377
K. Alhamazani, R. Ranjan, P.P. Jayaraman, K. Mitra, F. Rabhi, D. Georgakopoulos, L. Wang, Cross-layer multi-cloud real-time application qos monitoring and benchmarking as-a-service framework, IEEE Transactions on Cloud Computing.
Clayman S., Galis A., Chapman C., Toffetti G., Rodero-Merino L., Vaquero L.M., Nagin K., Rochwerger B .Monitoring service clouds in the future internet., Future Internet Assembly, Valencia, Spain (2010), pp. 115-126
Romano L., De Mari D., Jerzak Z., Fetzer C. A novel approach to qos monitoring in the cloud, Data Compression, Communications and Processing (CCP), 2011 First International Conference on, IEEE (2011), pp. 45-51
De Chaves S.A., Uriarte R.B., Westphall C.B. Toward an architecture for monitoring private clouds, IEEE Commun. Mag., 49 (12) (2011), pp. 130-137
Ranjan R., Garg S., Khoskbar A.R., Solaiman E., James P., Georgakopoulos D. Orchestrating bigdata analysis workflows, IEEE Cloud Comput., 4 (3) (2017), pp. 20-28
Yarn, https://yarnpkg.com/en/, accessed: 2018-04-01.
Apache mesos, http://mesos.apache.org/, accessed: 2018-04-01.
Amazon emr, https://aws.amazon.com/emr/, accessed: 2018-04-01.
Abouzeid A., Bajda-Pawlikowski K., Abadi D., Silberschatz A., Rasin A.Hadoopdb: an architectural hybrid of mapreduce and dbms technologies for analytical workloads,Proc. VLDB Endow., 2 (1) (2009), pp. 922-933
Castro Fernandez R., Migliavacca M., Kalyvianaki E., Pietzuch P. Integrating scale out and fault tolerance in stream processing using operator state management, Proceedings of the 2013 ACM SIGMOD international conference on Management of data, ACM (2013), pp. 725-736
Castiglione A., Gribaudo M., Iacono M., Palmieri F. Exploiting mean field analysis to model performances of big data architectures, Future Gener. Comput. Syst., 37 (2014), pp. 203-211
Bruneo D., Longo F., Ghosh R., Scarpa M., Puliafito A., Trivedi K.S. Analytical modeling of reactive autonomic management techniques in iaas clouds, Cloud Computing (CLOUD), 2015 IEEE 8th International Conference on, IEEE (2015), pp. 797-804
Zeng L., Benatallah B., Ngu A.H., Dumas M., Kalagnanam J., Chang H. Qos-aware middleware for web services composition, IEEE Trans. Softw. Eng., 30 (5) (2004), pp. 311-327
Hani A.F.M., Paputungan I.V., Hassan M.F. Renegotiation in service level agreement management for a cloud-based system, ACM Comput. Surv., 47 (3) (2015), p. 51
Kim H., Parashar M. Cometcloud: an autonomic cloud engine, Cloud Computing: Principles and Paradigms (2011), pp. 275-297
Nasridinov A., Byun J.-Y., Park Y.-H. A QoS-aware performance prediction for self-healing web service composition, Cloud and Green Computing (CGC), 2012 Second International Conference on, IEEE (2012), pp. 799-803
Schulte S., Janiesch C., Venugopal S., Weber I., Hoenisch P. Elastic business process management: state of the art and open challenges for BPM in the cloud, Future Gener. Comput. Syst., 46 (2015), pp. 36-50
Singh S., Chana I. Qos-aware autonomic resource management in cloud computing: a systematic review, ACM Comput. Surv., 48 (3) (2016), p. 42
Ferretti S., Ghini V., Panzieri F., Pellegrini M., Turrini E. Qos-aware clouds, Cloud Computing (CLOUD), 2010 IEEE 3rd International Conference on, IEEE (2010), pp. 321-328
Nathuji R., Kansal A., Ghaffarkhah A. Q-clouds: managing performance interference effects for qos-aware clouds, Proceedings of the 5th European conference on Computer systems, ACM (2010), pp. 237-250
Li X., Li K., Pang X., Wang Y. An orchestration based cloud auto-healing service framework, Edge Computing (EDGE), 2017 IEEE International Conference on, IEEE (2017), pp. 190-193
Nepal S., Malik Z., Bouguettaya A. Reputation propagation in composite services, Web Services, 2009. ICWS 2009. IEEE International Conference on, IEEE (2009), pp. 295-302
Qu L., Wang Y., Orgun M.A., Liu L., Liu H., Bouguettaya A. CCCloud: context-aware and credible cloud service selection based on subjective assessment and objective assessment, IEEE Trans. Serv. Comput., 8 (3) (2015), pp. 369-383
Huang J., Liu G., Duan Q., Yan Y.Qos-aware service composition for converged network-cloud service provisioning, Services Computing (SCC), 2014 IEEE International Conference on, IEEE (2014), pp. 67-74
Tools for monitoring compute, storage, and network resources, https://kubernetes.io/docs/tasks/debug-application-cluster/resource-usage-monitoring/, accessed: 2018-05-01.
Kassabi H.T.E., Serhani M.A., Dssouli R., Benatallah B. A multi-dimensional trust model for processing big data over competing clouds, IEEE Access, 6 (2018), pp. 39989-40007, 10.1109/ACCESS.2018.2856623
Serhani M.A., Kassabi H.A., Taleb I. Towards an efficient federated cloud service selection to support workflow big data requirements, Adv. Sci. Technol. Eng. Syst. J., 3 (5) (2018), pp. 235-247, 10.25046/aj030529
Adriyendi A. Multi-attribute decision making using simple additive weighting and weighted product in food choice, Int. J. Inf. Eng. Electron. Bus., 6 (2015), pp. 8-14
Yet another markup language (yaml) 1.0, http://yaml.org/spec/history/200-12-10.html, accessed: 2018-05-01 (2001).
Lomuscio A., Qu H., Raimondi F. MCMAS: a model checker for the verification of multi-agent systems, International Conference on Computer Aided Verification, Springer (2009), pp. 682-688
Clarke E.M., Grumberg O., Peled D.Model Checking, MIT press (1999)
Swarm mode key concepts, docker doc, https://docs.docker.com/engine/swarm/key-concepts/, accessed: 2018-05-01 (2017).
S. Prodan, Docker swarm instrumentation with prometheus, https://stefanprodan.com/2017/docker-swarm-instrumentation-with-prometheus/, accessed: 2018-05-01.
Grafana - the open platform for analytics and monitoring, https://grafana.com/, accessed: 2018-05-01 (2017).
Prometheus - monitoring system & time series database, https://prometheus.io/, accessed: 2018-05-01 (2017).
Slack features, https://slack.com/features, accessed: 2018-05-01 (2017).
The mimic-iii clinical database, https://www.physionet.org/physiobank/database/mimic3cdb/, accessed: 2018-05-01 (2017).
Mit-lcp/mimic-code, https://github.com/MIT-LCP/mimic-code/tree/master/buildmimic/docker, accessed: 2018-05-01 (2017).
All items in Spectrum are protected by copyright, with all rights reserved. The use of items is governed by Spectrum's terms of access.
Repository Staff Only: item control page
Download Statistics
Download Statistics