A central element of the VG approach is a resource description language (vgDL) based on application-centric resource abstractions. The initial vgDL design was based on detailed study of real-world grid applications conducted early in the project. In the study, we identified the high-level resource abstractions used by our grid applications, as well as a range of typical preferences and attributes for the resources ultimately used to implement these resource abstractions. The key insight is that sophisticated application developers of grid applications typically use a high-level resource abstraction to organize their application management for performance, reliability, etc.  The vgDL design provides application developers a convenient high-level means to describe the high-level abstractions explicitly, and then allows the application to use a concrete realization of that resource abstraction in the form of a VG. The vgDL language is designed to be easily readable for programmability, and it enables specification of both qualitative and quantitative resource requirements. The key strength of vgDL is that it not only uses simple resource abstractions, supporting simple specifications, but that it is also a rich, expressive language that enables experts to control resource specification with precision. The full description of vgDL can be found in its associated technical report. Key ideas in the design of the vgDL language are discussed in the following sections.

Resource Aggregators

Our study of grid applications showed that most applications use simple aggregations of resources with simple shared properties (e.g., all the same CPU type, all >1GB memory, etc.) as their primary resource abstractions. This reflects a desire for applications to simplify their interaction with the resources and resource environment, focusing instead on effective realization of the already complex application. To meet this need, vgDL includes only three aggregators:  “LooseBags, TightBags” (loosely-connected and tightly-connected collections of heterogeneous resources, respectively), and “Clusters” (tightly-connected homogeneous collections of nodes). Applications can compose these aggregators  (e.g., a “LooseBag” of “Clusters”). Aggregator properties, and many of the characteristics of vgDL descriptions are qualitative. Our study found that for many applications, detailed quantitative specifications are a distraction, making the resource specifications and resulting application programs quite fragile. We believe our qualitative approach will enable many applications to construct simple, robust, and enduring vgDL specifications.

Network Connectivity

Our study also found that inter-node and inter-aggregate network connectivity is critical for performance in many applications. vgDL includes four operators that define network connectivity: “close”, “far”, “highBW”, and “lowBW”. These composers indicate coarse, qualitative notions of network proximity in (intentionally vague) terms of latency and bandwidth. Again, our study suggests that over-reliance on quantitative specifications leads to lack of robustness in the resulting programs. Each specific vgDL implementation uses specific default values for defining quantitative meanings. We believe that appropriate defaults are likely to be an ongoing research topic, and may not only change as technology evolves, but also be a function of the resource environment in which a vgDL request is viewed. Applications that require more precise tuning can query the VG attribute system once a VG has been instantiated for quantitative performance characteristics and use that information to apply quantitatively optimized scheduling algorithms and other optimizations.

Approximate Selection and Scalar Ranking

By designing vgDL to qualitatively specify the needed resources, our intent was also to reflect the difficulty of achieving good, much less optimal selection and binding of resources in the open, shared, competitive resource environments that typify most production grids. In addition, future resource environments are likely to be much larger than the thousands of resources that comprise grids today. One future research focus is the design of resource selection and binding algorithms and mechanisms that achieve provably good results in the face of resource competition and in very large resource environments (millions of resources). Because applications will generally only provide approximate resource requests, vgDL also allows applications to express preference using a scalar ranking function. The ranking function is a user-defined expression consisting of basic arithmetic operations, which produces a scalar value that represents desirability. The ranking function is used by the vgES to rank the candidates that meet the vgDL specification constraints to determine which one the application would prefer. If no ranking function is provided, or the ranking function does not distinguish between the candidates, a set of default preferences is used.