Personal tools
You are here: Home Publications Batch Queue Resource Scheduling for Workflow Applications
Document Actions

Yang Zhang, Charles Koelbel, and Keith Cooper (2009)

Batch Queue Resource Scheduling for Workflow Applications

In: 2009 IEEE International Conference on Cluster Computing (Cluster 2009, New Orleans, LA.

Workflow computations have become a major programming paradigm for scientific applications. However, acquiring enough computational resources to execute a workflow poses a challenge in a batch queue controlled resource due to the space-sharing nature of the resource management policy. This paper introduces a scheduling technique that aggregates a workflow application into several subcomponents. It then uses the batch queue to acquire resources for each subcomponent, overlapping resource provisioning overhead (wait time) of one with the execution of others. We implemented a prototype of this technique and tested it using five high performance computing centers’ job submission logs. The results show that our approach can eliminate as much as 70% of the wait time over more traditional techniques that request resources for individual workflow nodes or that acquire all the resources for the whole workflow at once.

(Accepted)
by Charles Koelbel last modified 2009-08-11 11:47
« May 2010 »
Su Mo Tu We Th Fr Sa
1
2345678
9101112131415
16171819202122
23242526272829
3031
 

VGrADS Collaborators include:

Rice University UCSD UH UCSB UTK ISI UTK

Powered by Plone