How Agoda Built a Latency-Aware Reverse Proxy for S3 Load Balancing

When Agoda’s engineering team faced uneven traffic distribution in their object storage infrastructure, they turned to a clever solution: a custom reverse proxy called Storefront. This tool not only solved their load balancing challenges but also became a cornerstone of their data platform. Let’s explore how they did it.

Why DNS Round-Robin Fails at Scale

Agoda relies heavily on S3-compatible object storage for data pipelines and analytics workloads. Their storage provider used DNS round-robin to distribute traffic across backend nodes. However, this approach had a critical flaw: many application clients cached DNS responses, leading to traffic hotspots where certain nodes became overloaded while others sat idle.

“DNS-based load balancing works well in theory,” explains Desmond Xu, Agoda’s Technical Lead. “But in practice, client caching breaks the even distribution we needed for high-volume workloads.”

Storefront: A Reverse Proxy with Smarts

Agoda’s solution was Storefront, a latency-aware reverse proxy built on Pingora (Cloudflare’s open-source framework) and written in Rust. Instead of relying on DNS, Storefront dynamically evaluates backend health and request load in real time. Key features include:

• Latency-aware routing: Routes requests to the least busy backend node
• Smart timeouts: Prevents connection pool exhaustion from unresponsive clients
• Cross-data center isolation: Segments traffic to avoid latency spikes
• Credential-less auth: Uses Kubernetes pod identity for secure access

From Simple Proxy to Infrastructure Core

Initially designed as a basic reverse proxy, Storefront evolved into a critical infrastructure component. The team implemented:

1. Least-in-flight algorithm: Early version for basic load balancing
2. Latency scoring: Optimized for production workloads
3. Telemetry integration: Exposes metrics via OpenTelemetry

Operational Benefits That Matter

Storefront delivers measurable improvements:

• Reduces backend hotspots by 70%
• Eliminates credential management for 1,200+ services
• Handles 10x more concurrent requests than DNS-based routing

“Storefront isn’t just a proxy – it’s our data infrastructure’s nervous system,” says Xu. The team also added HTTP 100-continue optimizations for uploads and Kubernetes-native authentication to simplify security management.

Lessons for Your Architecture

Agoda’s experience offers valuable insights for any team managing large-scale storage:

1. Don’t trust DNS for critical traffic distribution
2. Build latency-aware routing into your proxy layer
3. Centralize authentication to reduce operational complexity

For developers facing similar challenges, Storefront demonstrates how a custom reverse proxy can transform infrastructure performance. The full technical details are available in Agoda’s engineering blog.

Key Takeaways

Agoda’s Storefront proves that custom solutions can solve complex infrastructure problems. By moving beyond DNS-based routing and adding smart proxy features, they achieved:

• Even traffic distribution across storage nodes
• Improved reliability through smart timeouts
• Centralized security with Kubernetes integration

When your data pipelines demand consistent performance, sometimes the best solution is building your own tools. Storefront shows how a latency-aware reverse proxy can become a strategic asset.