Study of dynamic, adaptive grid applications and middleware depends on the ability to experiment with a wide range of realistic resource environments and to do detailed, performance and behavior-accurate simulations. Both to allow our own experiments within VGrADS, and as a research topic in its own right, we have developed methods of allowing these studies.

As part of our activity in grid simulation, which began with the GrADS project, we have completed several new releases of the MicroGrid system.  These new MicroGrid releases include automatic synthesis of internet-like topologies using well-established tools such as BRITE, and they enable large-scale simulation of large-scale networks (20,000 routers) by efficiently exploiting parallel resources such as the TeraGrid.  Recent advances include the generation of realistic multi-AS networks, enabling realistic modeling of Internet routing structures.   In addition, we have developed new load balancing techniques that improve our previous profile-based techniques with a new hierarchical approach to improve parallel simulation efficiency dramatically.  Parallel efficiencies of 40% on 128 nodes enable experiments with networks comprising large fractions of the Internet (for example a 20,000 router network, similar in scale to AT&T's network).  These capabilities enable the study of large-scale grid systems and applications, and serve as a controllable testbed for VGrADS experiments.

Simulations, including grid simulations, are only as good as the realism of their underlying model system.  However, to date little work has addressed the problem of realistic grid resource generation.  Leveraging the observation that the majority of high capability grid resources are Linux clusters, we are building a general-purpose grid resource configuration generator. 

Using several databases of Linux cluster configurations to populate a statistical model of grid resources and an array of statistical tools, we have designed and validated statistically a grid resource generator, one for which the output has been shown to be statistically the same as other sample data sets (which are as large as 10,000 processors!).  Further, by analyzing historical sample data and technology trends, we have extended our model to generate representative resource structures for grids of the future.  We will use this grid resource generator to study the efficacy of our virtual grid abstractions and their implementation techniques, and make it generally available to the research community to enable better-grounded study of software and application behavior in grid resource environments.