Shedding Old Code with Ecdysis: Graceful Restarts for Rust Services

Introduction to Ecdysis

Upgrading a network service without disrupting connections is challenging. At Cloudflare, we use ecdysis, a Rust library for graceful process restarts. After five years of production use, we open-sourced ecdysis for public use.

Why Graceful Restarts are Hard

The naive approach to restarting a service involves stopping the old process and starting a new one. However, this creates a window where connections are refused and requests are dropped.

Stopping the old process closes listening sockets, causing the OS to refuse new connections.
Established connections are killed, disconnecting clients mid-operation.

How Ecdysis Works

Ecdysis satisfies four key goals: old code shutdown, new process initialization, crash safety, and parallel upgrade prevention. The library uses a forking approach, where the parent process forks a new child process, and the child inherits the socket file descriptors via a named pipe.

The parent process fork()s a new child process.
The child process replaces itself with a new version of the code with execve().
The child process inherits the socket file descriptors via a named pipe shared with the parent.
The parent process waits for the child process to signal readiness before shutting down.

Security Considerations

Ecdysis addresses security concerns through its design, using fork-then-exec and explicit inheritance to ensure the child process starts with a clean slate and only explicitly-passed file descriptors cross the boundary.

Conclusion and Call to Action

Ecdysis has proven itself in production use at Cloudflare, enabling zero-downtime upgrades across our critical Rust infrastructure. Try ecdysis in your own projects to achieve graceful restarts and improve the reliability of your services.

FAQs: