Utility-based Resource Allocation and Pricing for Serverless Computing with Dependencies

Vipul Gupta, Soham Phade, Yigit Efe Erginbas, Thomas Courtade and Kannan Ramchandran
preprint

Abstract: Serverless computing platforms currently employ static pricing schemes that often lead to inefficiencies. To address this, our prior work introduced a novel market-based scheduler that utilizes user utility functions to optimize resource allocation and maximize social welfare based on delay-sensitivity. This paper extends that framework to tackle a critical challenge in cloud computing: the efficient scheduling of jobs with inter-dependencies. We propose an enhanced scheduler capable of allocating resources for serverless computing tasks with finish-to-start dependencies, ensuring that the overall system utility is maximized while respecting these constraints. Our approach retains the dynamic pricing mechanism derived from the dual problem and the decentralized feedback mechanisms for handling private user information, now incorporating the complexities introduced by job dependencies. Simulations demonstrate that our extended framework can effectively manage dependent tasks, track market demand, and achieve significantly higher social welfare compared to existing schemes that do not account for these dependencies.

Vipul Gupta, Soham Phade, Yigit Efe Erginbas, Thomas Courtade and Kannan Ramchandran, "Utility-based Resource Allocation and Pricing for Serverless Computing with Dependencies," preprint.

This paper can be downloaded here.