Sharc: Managing Resources in Shared Clusters

In this paper, we argue the need for effective resource control mechanisms for sharing resources in commodity clusters. To address this issue, we present the design of Sharc---a system that enables resource sharing among applications in such clusters. Sharc depends on single node resource control mechanisms such as reservations or shares and extends the benefits of such mechanisms to clustered environments. Our Sharc prototype (i) supports resource reservation for applications, (ii) enables dynamic resource allocation based on resource usages, (iii) provides performance isolation to applications, and (iv) handles a variety of failure scenarios. Our experimental evaluation has shown that Sharc can scale to 256 node clusters running 100,000 applications. Our results demonstrate that Sharc can be an effective approach for sharing resources among competing applications in moderate size clusters.