Optimal Checkpoint Scheduling using Automatic Resource Characterization

Abstract The architectures supported by the VGrADS initiative include high performance clusters, large scale multi-processor machines, workstation/server pools and even individual machines. For the abstraction and virtualization purposes of the VGrADS project, the components of these architectures must be statistically characterized. Although many component characteristics have been studied, one that has remained difficult to understand is that of machine availability. To serve the needs of the VGrADS infrastructure, we have developed a set of tools which automatically models and make predictions on machine availability data. Currently, we have tested our modeling and prediction techniques on three distinct resource types including student lab workstations, Internet hosts, and Condor machines. The ability to accurately model and predict resource availability durations is useful to the VGrADS software to both implement and optimize robustness mechanisms. To demonstrate, we have developed a generalized checkpoint scheduler that automatically determines a checkpoint schedule based on dynamic easurement taken from an individual resource. As Condor represents the most difficult resource pool, we have empirically tested our checkpoint scheduler on a real world Condor resource pool. Our scheduler achieves slightly better performance than previous techniques with regards to time efficiency, but has drastically improved network performance characteristics. In addition to results from our checkpointing experiments, we were able to verify a software simulation of our checkpointing system that can be used in the future to perform tests on different data sets and model parameters. In future work, we plan to enhance the entire system by employing non-parametric modeling techniques which require less data and are often more accurate. Poster Contributors Daniel Nurmi John Brevik Rich Wolski Poster Presented by Daniel Nurmi

Click here to get the file