Adapting Kubernetes controllers to the edge: on-demand control planes using Wasm and WASI

arXiv:2209.01077 · cs.DC, cs.PF · Submitted 2022-09-02 · Read on arXiv

Listen

Radio episode about this paper

Transcript

Introduction to the show: ident: Computer Science. The day in computer science: software, systems and theory on arXiv.

Tom: We're looking at the paper: "Adapting Kubernetes controllers to the edge".

Title and authors: Tom: So we're looking at this paper called "Adapting Kubernetes controllers to the edge: on-demand control planes using Wasm and WASI". It sounds like they’re tackling a huge problem with Kubernetes running on low-resource hardware, specifically the edge stuff.

Jane: Right, and what I hear from the title is that they're trying to make those long-running controllers run only when they actually need to, instead of always being on. It seems like they’re focusing on how WebAssembly and WASI can help them achieve this lightweight execution.

Lu: The core idea is moving away from those always-on processes—the controllers—and making them run on demand using WebAssembly modules that are only loaded when necessary <ref:2209.01077#pg1>. It’s about making the control plane much more efficient for places with very limited power and memory.

Meng: From my perspective, I'm thinking about how this translates to deployment costs on real hardware, because right now, we’re paying for that overhead constantly. If we can scale down those controllers dynamically, it means less persistent compute resources needed for the control plane itself <ref:2209.01077#pg1>.

Lalam: I see the main thrust is using a framework that lets them swap idle controllers to disk and only activate them when an event happens, which sounds like a big win for scalability <ref:2209.01077#pg1>.

The paper's summary: Tom: So the paper explains this idea of having these lightweight controllers that live on demand, and they’re using this WebAssembly-based framework built on WASI to run them. They are extending the WebAssembly System Interface, which is key for letting these controllers interact with the system in a standardized way <ref:2209.01077#pg1>.

Jane: It seems like they’re using a specific runtime called Wasmtime, and they modify it so that instead of just running the modules constantly, it swaps them out to disk when they are idle and brings them back up when something triggers a change <ref:2209.01077#pg1>.

Lu: The architecture involves a parent operator that manages an event loop, and this loop is what handles calling host functions from inside the WebAssembly module, which is crucial for making these child operators asynchronous <ref:2209.01077#pg3>.

Meng: As an engineer shipping things in production, I’m curious about that parent operator loop—how does it manage the work queue so we know exactly when to unload or reload a controller? That sounds like a complex scheduling problem to get right at scale <ref:2209.01077#pg3>.

Lalam: They mention that inactive operators are unloaded after a long period of inactivity, which is essentially the scale-to-zero capability they’re aiming for <ref:2209.01077#pg3>.

The paper's improvements: Tom: One of the major things they highlight is how this framework achieves a sixty-four percent memory reduction compared to traditional container-based controller frameworks, which is a huge number for resource savings <ref:2209.01077#pg1>.

Jane: That memory saving is directly tied to how they structure the architecture, focusing on isolation overhead and modularity, so you can add or remove child operators without messing up the others <ref:2209.01077#pg1>.

Lu: They also move the low-level operator logic into the parent operator, which simplifies things for those client operators and reduces their complexity <ref:2209.01077#pg1>. And they chose Rust for the child operators because of its support for WASI host functions and better interoperability than Golang IMPROVEMENT three.

Meng: Rust sounds promising from a deployment standpoint if it means less memory overhead per operator, especially when dealing with garbage collection compared to other languages IMPROVEMENT three. But what about the performance hit? Assembly runtimes usually have some performance cost compared to native code, which is something I need to check on <ref:2209.01077#pg2>.

Lalam: They also mention that the parent operator manages the asynchronous runtime for a child operator and unloads it if it hasn't received any asynchronous results over a long time, which optimizes resource utilization IMPROVEMENT two <ref:2209.01077#pg3,any asynchronous results over a long>.

Conclusion: Tom: So to wrap up, the main point of "Adapting Kubernetes controllers to the edge: on-demand control planes using Wasm and WASI" is that this framework lets you replace those heavy, always-on controllers with lightweight WebAssembly modules that only run when needed <ref:2209.01077#pg1>.

Jane: It really boils down to achieving a sixty-four percent memory reduction compared to the old container methods, and they’ve built an asynchronous runtime in the parent operator to handle those calls efficiently so things don't block IMPROVEMENT two <ref:2209.01077#pg3>.

Lu: The modularity is important too, allowing you to swap out components easily, and using Rust for the child operators helps keep the per-operator memory footprint low IMPROVEMENT three. It’s about making Kubernetes control planes viable in those constrained edge environments where resources are tight <ref:2209.01077#pg1>.

Meng: I still have to be concerned about that performance hit they mentioned regarding assembly runtimes versus native code, and how reliable that swap-to-disk and reload cycle is under real traffic load <ref:2209.01077#pg2>.

Lalam: The authors admit a limitation, which is that even with the optimizations, there's still some overhead associated with the runtime itself when you’re constantly cycling modules in and out IMPROVEMENT four <ref:2209.01077#pg2>. But overall, it shows a path toward running these kinds of management tasks on very small devices <ref:2209.01077#pg1>.

Tom: So that's the big picture here, using this paper to look at how we can actually make Kubernetes control planes fit where they need to be deployed, which is a really tangible goal for edge computing <ref:2209.01077#pg1>.

Merlijn Sebrechts, Tim Ramlot, Sander Borny, Tom Goethals, Bruno Volckaert, Filip De Turck

IDLab Department of Information Technology Ghent University imec Belgium

cs.DC, cs.PF

Submitted: 2022-09-02

Updated: 2022-09-02

Comments: Pre-print of article accepted to IEEE CloudNet 2022

DOI: 10.1109/CloudNet55617.2022.9978884

Code: https://github.com/kubernetes/kube-controller-manager

Project page: https://rust-lang.github.io/async-book/02_execution/02_future.html

License: http://creativecommons.org/licenses/by/4.0/

Importance score: 80/100

The gist: Kubernetes’ high resource requirements hamper its adoption in constrained environments such as the edge and fog This paper presents a WebAssembly-based framework for running lightweight controllers…

Key concepts

Kubernetes Controllers
These are long-lived processes within Kubernetes that constantly monitor the cluster state and react to changes. They require continuous resources even when there is no immediate activity, making them costly in low-resource settings.
WebAssembly (WASM)
WASM allows running code securely and efficiently outside of a traditional container. It enables the framework to run small, isolated operator logic directly within a process with minimal overhead, facilitating on-demand execution.
Scale-to-zero Architecture
This design allows inactive child operators to be dynamically unloaded when they have been idle for a long time. This feature ensures that resources are only consumed when needed, optimizing memory usage in environments where resources are scarce.

Terminology

Summary

Kubernetes’ high resource requirements hamper its adoption in constrained environments such as the edge and fog This paper presents a WebAssembly-based framework for running lightweight controllers on-demand, only when they are needed A thorough evaluation shows this framework achieves a 64% memory reduction compared to traditional container-based controller frameworks

Problem Statement

Kubernetes’ extensible control plane is a significant contributor to this, consisting of long-lived processes called controllers that constantly listen for state changes and use resources even when they are not needed Running Kubernetes on low-resource clusters suffers from relatively high control plane overhead costs, which hinders adaptation in the edge market segment The primary orchestration targets of Kubernetes, however, are high-resource cloud clusters Operators are one of the main cost drivers of the Kubernetes control plane because they have to run as long-living processes to react to changes in the Kubernetes cluster state This is especially problematic for low-resource deployments

Solution Architecture

The WASM operator architecture presented in this paper and visualized in Figure 1 attempts to realize all the beneficial aspects that solutions in prior work have achieved The main parts of the architecture are the parent operator and the child operators

** Isolation overhead: All child operators run in the same process as the parent operator, this process runs inside a single container in a single Kubernetes pod Isolation is provided by the WASM engine, eliminating the overhead due to container isolation**

** Modularity: The WASM runtime makes it possible to add or remove child operators without interfering with the other active child operators**

** Simple child operator: Low-level operator logic is moved to the parent operator, which reduces the complexity and overhead of the client operators**

** Scale-to-zero: To limit the overhead of inactive operators, our architecture allows to dynamically unload inactive operators**

Implementation Details

The parent operator extends the Wasmtime WASM runtime Wasmtime was chosen over other WASM runtimes because it is the flagship WASM engine from the Bytecode Alliance The implementation configures Wasmtime to compile ahead of time (AOT) new WASM modules to machine code to eliminate the compiler memory overhead at runtime To initiate these compiled modules, Wasmtime only has to map the file to memory and provide the necessary tools to communicate with this initiated module

** Parent operator asynchronous runtime: The parent operator manages the asynchronous operations of a child operator and unloads an inactive child operator after a long period of inactivity**

** Child operator asynchronous client: All client operators run as single-threaded asynchronous WASM instances**

Resource Utilization and Results

The memory utilization measure that we use is determined by limiting the memory usage, as determined by cgroup v2 Figure 5b shows that, as expected, all operator types utilize less memory in case of idle workloads The WASM-based operator still used 83.

Green — the paper says it, and the page is printed. Red — no passage carries it, so treat that sentence as unverified. The same check runs on your own text: check a summary against its source, $5.

Improvements for AI systems

  1. Replace long-lived processes for Kubernetes controllers with on-demand, WebAssembly based modules to achieve a 64% memory reduction compared to traditional container-based controller frameworks. This allows Kubernetes control planes to run in a Function as a Service (FaaS) platform, enabling fast on-demand scaling and scaleto-zero capabilities suitable for resource-constrained edge environments.

  2. Implement an asynchronous runtime within the parent operator that supports host functions and handles work queues to allow child operators to perform Kubernetes API requests without blocking the execution loop. This design ensures that inactive operators can be unloaded when they have not received any asynchronous results over a long period, optimizing resource utilization in memory-constrained settings.

  3. Utilize Rust for child operators due to its support for WASI host function calls and better interoperability with the parent operator compared to Golang, which lacks native WASI support and has higher memory usage due to garbage collection. This choice contributes to lower per-operator memory overhead and better overall resource efficiency.

  4. Leverage a predictive scheduler as future work to optimize automatic WASM module unloading, ensuring that modules are only unloaded when beneficial for end-to-end latency without introducing significant disk swap overhead during worst-case unload and reload patterns.

Abstract

Kubernetes' high resource requirements hamper its adoption in constrained environments such as the edge and fog. Its extensible control plane is a significant contributor to this, consisting of long-lived processes called "controllers" that constantly listen for state changes and use resources even when they are not needed. This paper presents a WebAssembly-based framework for running lightweight controllers on-demand, only when they are needed. This framework extends the WebAssembly System Interface (WASI), in order to run Kubernetes controllers as lightweight Wasm modules. The framework runs these Wasm controllers in a modified version of Wasmtime, the reference WebAssembly (Wasm) runtime, that swaps idle controllers to disk and activates them when needed. A thorough evaluation shows this framework achieves a 64% memory reduction compared to traditional container-based controller frameworks.

Related papers