A former Atlassian engineer’s retrospective on building a global Envoy-based edge platform reveals something much deeper than “load balancing.” Beneath the proxies, control planes, and AWS infrastructure sat a programmable behavioural layer that centralised routing, security, governance, observability, and policy enforcement at hyperscale. The real product was not the infrastructure itself, but the institutional cognition embedded within it: a distributed nervous system governing how the organisation operated at runtime.

Executive Summary

This architecture demonstrates the broader direction of modern infrastructure engineering: the transition from static infrastructure toward programmable behavioural systems operating at organisational scale.

The key innovation was not the use of Envoy itself, but the construction of a centralised control plane capable of dynamically synthesising routing, policy, security, and operational behaviour across thousands of distributed edge proxies.

By centralising:

• authentication,
• authorization,
• rate limiting,
• observability,
• logging,
• and request governance,

the platform transformed the edge layer into a force multiplier for the entire organisation.

The result was:

• reduced duplicated engineering effort,
• stronger governance,
• improved operational consistency,
• faster product delivery,
• and significantly lower infrastructural entropy.

Architecturally, the system reflects several defining modern patterns:

• programmable control planes,
• immutable infrastructure,
• runtime configuration synthesis,
• asynchronous provisioning,
• sidecar-based capability extension,
• and policy-driven distributed systems.

The deeper implication is that infrastructure is increasingly evolving beyond passive request transport.

It is becoming:

• a governance layer,
• a behavioural abstraction system,
• an operational coordination mechanism,
• and a repository of institutional intelligence.

The edge is no longer merely where traffic enters the system.

It is increasingly where organisational cognition resides.

Architecturally, the system also reflects a modern continuation of the pragmatic scaling philosophy described by Cal Henderson in Building Scalable Web Sites: pressure-driven abstraction, asynchronous decomposition, and operationally evolved infrastructure rather than ideologically imposed architecture.

Contents

1. Introduction

Recently, I watched a former Atlassian engineer discuss the systems he spent nearly a decade building inside Atlassian Cloud.

On the surface, it sounds like a story about load balancing.

It is not.

It is a story about programmable infrastructure, distributed policy enforcement, edge cognition, and the gradual transformation of infrastructure teams into organisational nervous systems.

What struck me most was not the specific technologies involved, though the stack itself was extremely solid, but the architectural direction of travel.

Because beneath the discussion of Envoy, CloudFormation, SQS, AMIs, and FastAPI was something much more important:

the construction of a centralised programmable behavioural layer for a hyperscale software platform.

That is where the real architectural significance sits.

The discussion analysed here comes from infrastructure engineer Vasilios Syrakis, whose recent technical retrospectives on Atlassian Cloud architecture generated significant interest within engineering circles following a widely shared X post highlighting his background, engineering philosophy, and large-scale systems work. He described the design and evolution of Atlassian’s Envoy-based edge platform, dynamic control plane architecture, and distributed provisioning systems. His technical breakdowns and upcoming control-plane build series are available on his YouTube channel (@vsyrakis).

2. The Initial Problem Was Simple

The original requirement was relatively straightforward.

Atlassian needed a self-service mechanism allowing internal product teams to provision edge infrastructure without directly involving platform engineers.

In practical terms:

• developers wanted public ingress,
• routing,
• load balancing,
• certificates,
• DNS,
• and internet-facing services,

without needing tickets, manual intervention, or central bottlenecks.

The initial implementation used an Open Service Broker pattern.

At a high level:

Developer Request
        ↓
Broker API
        ↓
Async Provisioning Workers
        ↓
AWS Infrastructure Changes

The broker itself evolved through:

• Connexion,
• Flask,
• and eventually FastAPI.

Provisioning operations were handled asynchronously using SQS workers and DynamoDB state tracking.

This is already reasonably mature platform engineering.

But this is not where things become interesting.

3. The Important Decision: Replacing Enterprise Load Balancers

The major architectural shift came when the team decided to replace expensive enterprise load balancers with Envoy Proxy.

This matters because Envoy is not merely a reverse proxy.

Envoy is effectively a programmable distributed middleware runtime.

Once you understand that, the rest of the architecture makes sense.

Traditional load balancers are largely static appliances.

Envoy, by contrast, is dynamic.

It exposes APIs for runtime configuration.

That changes everything.

Because once infrastructure becomes dynamically programmable:

• policy can become centralised,
• behaviour can become templated,
• routing becomes software-defined,
• and infrastructure stops being static plumbing.

It becomes a distributed execution environment.

That is the real transition that occurred here.

4. The Control Plane Was The Real Product

The engineer describes building an Envoy management server called Sovereign.

This is the actual core of the system.

Not the proxies.

Not the AWS infrastructure.

The control plane.

The architecture worked approximately like this:

Provisioning Data + Runtime Context
                ↓
Dynamic Templates
                ↓
Envoy Configuration Generation
                ↓
xDS APIs
                ↓
Thousands of Envoy Proxies

The proxies themselves became relatively generic execution nodes.

The intelligence lived centrally.

This is one of the defining patterns of modern cloud infrastructure:

move complexity upward into the control plane.

Kubernetes does this.

Service meshes do this.

Cloud providers do this.

And increasingly, AI systems do this too.

The edge nodes become interchangeable.

The behavioural logic becomes centralised.

5. Infrastructure As Runtime Synthesis

One of the more sophisticated aspects of the platform was the templating system.

Developers did not configure Envoy directly.

That would have been operationally catastrophic.

Instead:

• developers supplied constrained parameters,
• the platform validated them,
• templates generated valid Envoy resources,
• and the resulting configuration was distributed dynamically.

This is effectively infrastructure synthesis.

The platform transformed intent into executable distributed infrastructure.

That distinction matters.

Because once a platform reaches this stage, infrastructure engineering stops being primarily about machines.

It becomes about abstraction management.

The difficult problem is no longer provisioning compute.

The difficult problem becomes:

how do you safely expose behavioural capability to thousands of teams simultaneously without allowing them to destroy each other?

That is fundamentally a governance problem masquerading as engineering.

6. The Sidecar Architecture Is The Real Story

The most important architectural insight in the entire discussion appears later.

Once the programmable edge infrastructure existed, Atlassian realised they could centralise cross-cutting concerns.

Instead of every service implementing:

• authentication,
• authorization,
• rate limiting,
• DDoS handling,
• observability,
• logging,
• and request governance,

those capabilities could instead be handled once at the edge.

Architecturally:

Client
  ↓
CloudFront
  ↓
NLB
  ↓
Envoy Edge Layer
   ├── Authentication Sidecar
   ├── Authorization Sidecar
   ├── Rate Limiting Sidecar
   ├── Logging
   └── Observability
  ↓
Backend Services

This is a profound architectural optimisation.

Not merely technically.

Organisationally.

Because the moment you centralise these concerns:

• security consistency improves,
• delivery velocity increases,
• duplicated effort collapses,
• operational governance strengthens,
• and platform capability compounds over time.

The edge layer becomes a force multiplier for the entire organisation.

This is what mature platform engineering actually looks like.

Not dashboards.

Not CI pipelines.

Centralised behavioural infrastructure.

7. The Hidden Cognitive Layer

What particularly interested me was how closely this mirrors broader systemic trends I have been writing about elsewhere.

Increasingly, modern infrastructure systems are not merely transporting requests.

They are governing behaviour.

The edge infrastructure described here effectively became:

• a policy execution layer,
• a trust enforcement layer,
• an organisational coordination mechanism,
• and a behavioural governance system.

The services behind the edge became progressively less important than the infrastructural cognition surrounding them.

This is increasingly true across the internet generally.

Infrastructure is no longer passive.

It interprets.

It shapes.

It constrains.

It routes.

It authorises.

It classifies.

It filters.

It decides.

The interesting thing about the Atlassian architecture is not simply that it solved scaling problems.

It is that it demonstrates how modern software organisations increasingly externalise operational cognition into programmable infrastructure.

The infrastructure itself begins carrying institutional intelligence.

8. Immutable Infrastructure And Long-Lived Stability

Another strong architectural decision was the use of immutable infrastructure.

The proxies were provisioned from AMIs generated through:

• Packer,
• SaltStack,
• automated provisioning pipelines,
• and CloudFormation.

This matters operationally because large-scale distributed systems become unstable very quickly when hosts drift.

By baking:

• Envoy,
• observability tooling,
• logging agents,
• tracing,
• security hardening,
• network tuning,
• and sidecars

into immutable machine images, the platform reduced entropy significantly.

Again, the important point is not the specific tools.

The important point is architectural philosophy.

Large systems survive through constraint.

Not flexibility.

Too much configurability at the wrong layer destroys operational coherence.

The platform deliberately constrained where complexity was allowed to exist.

That is good systems engineering.

9. The Cal Henderson Lineage

One of the reasons this architecture feels so familiar is because it strongly echoes the engineering philosophy articulated by Cal Henderson in Building Scalable Web Sites during the Flickr era of web architecture.

Not necessarily in the specific technologies.

But in the underlying instincts.

The architecture described here feels less like centrally planned “enterprise architecture” and more like systems evolving organically under operational pressure.

That distinction matters.

Many modern architectures are imposed ideologically:

• microservices because microservices,
• service meshes because service meshes,
• distributed systems because they are fashionable.

This feels different.

It feels pressure-derived.

The abstractions emerged because:

• operational entropy demanded them,
• duplicated effort demanded them,
• scaling demanded them,
• and governance demanded them.

That is much closer to the older Flickr-era engineering mindset:
pragmatic decomposition,
asynchronous separation,
runtime abstraction,
and incremental architectural evolution around real bottlenecks rather than theoretical purity.

You can see this lineage clearly in:

• asynchronous provisioning via SQS,
• separation of provisioning from request handling,
• immutable infrastructure pipelines,
• dynamic runtime configuration,
• sidecar-based capability extension,
• and centralisation of repeated behavioural concerns at the edge layer.

The result is architecture that feels operationally “alive.”

Not chaotic.

Not over-designed.

But evolved.

The best large-scale systems often emerge this way:
through sensible engineering contrarianism against unnecessary complexity while still accepting the realities of scale.

In many ways, this architecture feels like a modern hyperscale continuation of the same engineering philosophy Henderson was describing nearly two decades ago.

10. The Real Difficulty Was Never Building It

One of the most accurate observations made in the discussion was this:

building systems is relatively easy.

Maintaining them is difficult.

This is something younger engineers often underestimate.

Most systems look elegant at the beginning.

Complexity accumulates over time through:

• onboarding,
• operational drift,
• ownership transitions,
• edge cases,
• product exceptions,
• regulatory requirements,
• scaling pressures,
• and organisational entropy.

The engineer describes noticing where churn accumulated in codebases over time.

This is an important operational instinct.

Areas of persistent churn are usually indicators of unresolved abstraction failure.

Something structurally incorrect exists underneath.

Good engineers eventually stop merely fixing churn.

They begin identifying the architectural conditions producing it.

That transition, from implementation thinking to systemic thinking, is where senior engineering capability begins emerging.

11. What This Architecture Actually Represents

The deeper lesson here is not really about Atlassian.

Nor is it specifically about Envoy.

It is about the broader trajectory of software infrastructure.

The internet increasingly operates through:

• programmable policy layers,
• centralised control planes,
• behavioural enforcement systems,
• runtime synthesis,
• and distributed governance infrastructure.

The edge is no longer merely where traffic enters.

It is where organisational intelligence increasingly lives.

Which means modern infrastructure engineering is progressively converging with:

• systems governance,
• behavioural architecture,
• distributed cognition,
• trust systems,
• and institutional coordination.

That is the actual direction of travel.

And once you notice it, you begin seeing the same pattern everywhere.

The load balancer was never really the product.

The programmable behavioural infrastructure surrounding it was.

12. Conclusion: Final Thought

One of the reasons I found this discussion particularly interesting is because it reflects a broader shift occurring across technology generally.

Increasingly, value is migrating away from isolated applications and toward the systems coordinating them.

The infrastructure layer is becoming:

• more intelligent,
• more behavioural,
• more policy-driven,
• and more organisationally consequential.

Which means the future competitive advantage of many organisations may not primarily sit inside their applications.

It may sit inside:

• the control planes they build,
• the behavioural abstractions they expose,
• the governance layers they centralise,
• and the infrastructural cognition they accumulate over time.

That is where the real leverage increasingly appears to exist.

And once infrastructure reaches that point, it stops being plumbing.

It becomes institutional nervous tissue.

In many ways, this architecture feels less like a radical reinvention and more like the continued evolution of the pragmatic scaling philosophy articulated by engineers such as Cal Henderson during the early Flickr era: only now applied to programmable governance infrastructure operating at organisational scale.