Skip to content
Akwannya Hub
All tech roadmaps

Cloud & Infrastructure

Site Reliability Engineering

SRE applies software engineering to operations. Instead of asking for perfect availability, it defines how much unreliability is acceptable, measures it, and spends the difference on shipping features. It is the most quantitative of the infrastructure disciplines.

5 stages2 projectsAdvanced6 to 9 months, part time

Start here

SRE is a senior path and hiring reflects that. If you have not yet run something in production that other people depended on, start with DevOps Engineering or Backend Development and return here.

Before you begin

  • Can write real code, not only scripts
  • Have operated a production system
  • Comfortable with containers and cloud infrastructure
  • Understand distributed systems at least in outline
01

The engineering foundation

SRE is a software role. The distinguishing skill is writing code to remove operational work.

Required

Software engineering for operations

The line between SRE and sysadmin is whether you automate the problem away or perform the fix repeatedly.

What to learn

  • A real language, usually Go or Python
  • Testing your automation
  • APIs and integration
  • Reading someone else's service code

Next: Distributed systems.

Required

Distributed systems thinking

Almost every serious outage is a distributed systems failure wearing a different hat.

What to learn

  • Latency, throughput and saturation
  • Partial failure
  • Retries, timeouts and backoff
  • Idempotency
  • Cascading failure and backpressure
  • The CAP trade-off in practice

Practice

Take a service and write down every way one of its dependencies could fail, then what happens to it in each case.

Next: Measuring reliability.

02

Defining and measuring reliability

The core idea of SRE, and the part that makes it a discipline rather than a job title.

Required

SLIs, SLOs and error budgets

Without a number, 'reliable' is an argument. With one, it is a decision anyone in the business can participate in.

What to learn

  • Choosing indicators that reflect user experience
  • Setting realistic objectives
  • Error budgets
  • Burn rate
  • Using a budget to decide between shipping and stabilising

Practice

Write an SLO for a service you know. Then check whether last month would have met it. Most first attempts are far too strict.

Project

intermediate

An SLO for a real service

Define two indicators and one objective for a service you run, instrument them, build a dashboard, and report on a month of data.

  • Prometheus
  • Grafana

You can state your service's reliability as a number and defend the number.

Next: Observability, to make those numbers real.

03

Observability

Monitoring tells you something is wrong. Observability lets you find out why without shipping new code.

Required

Metrics, logs and traces

Debugging a distributed system without tracing is guesswork with extra steps.

What to learn

  • Instrumentation and cardinality
  • Structured logging
  • Distributed tracing
  • Correlating across all three
  • Dashboards designed for an incident, not a demo

Tools

  • Prometheus
  • Grafana
  • OpenTelemetry
  • A tracing backend

Next: Alerting.

Required

Alerting that respects people

Alert fatigue is a reliability problem in itself. An ignored pager is worse than no pager.

What to learn

  • Alerting on symptoms not causes
  • Burn-rate alerts
  • Severity and routing
  • Runbooks
  • Deleting alerts nobody acts on

Practice

Audit an existing alert set. Delete every alert that has never once led to an action.

Next: Incident response.

04

Incident response

How an organisation behaves in its worst hour, practised in advance.

Required

Running an incident

Structure under pressure is what turns a two-hour outage into a twenty-minute one.

What to learn

  • Incident command roles
  • Communicating while degraded
  • Mitigate first, diagnose second
  • Timelines
  • Handing over cleanly

Next: Learning from it.

Required

Blameless post-incident review

Systems fail because of conditions, not carelessness. Blame stops people reporting problems, which makes the next outage worse.

What to learn

  • Contributing factors rather than a root cause
  • Writing a timeline honestly
  • Action items with owners
  • Tracking whether they happen

Practice

Write a post-incident review for something that broke in your own project. Include what made it hard to diagnose.

Next: Testing failure on purpose.

05

Engineering for failure

Assume every dependency will fail, then decide what should happen when it does.

Required

Resilience patterns

Reliability is designed in. It cannot be monitored in afterwards.

What to learn

  • Timeouts and circuit breakers
  • Graceful degradation
  • Load shedding
  • Redundancy and failover
  • Capacity planning

Next: Proving it.

Advanced

Chaos and load testing

A failover you have never tested is a theory. Testing it on a Tuesday is much cheaper than discovering it on a Saturday.

What to learn

  • Game days
  • Fault injection
  • Load and soak testing
  • Practising failover
  • Disaster recovery drills

Project

advanced

A tested disaster recovery plan

Document a recovery plan for a system you run, then execute it end to end in a game day, timing each step and recording where reality differed from the plan.

  • Your infrastructure
  • A load testing tool

You have a recovery time you measured rather than estimated, which is the difference between a plan and a hope.

Next: Removing the work entirely.

Required

Eliminating toil

The defining SRE metric. Work that is manual, repetitive and automatable should be shrinking every quarter.

What to learn

  • Identifying and measuring toil
  • Automating away repeated tasks
  • Capping operational load
  • Making the case for engineering time

Where this leads

You do not have to pick one now. These are the directions this path opens up once you are working.

Platform Engineering

Turn reliability practice into a product other teams consume.

View roadmap

Systems & Infrastructure Engineering

Go deeper into the layer beneath the platform.

View roadmap

Solutions Architecture

Influence reliability at design time rather than at runtime.

View roadmap

Continue your journey

The paths closest to this one. Skills overlap more than the job titles suggest.

DevOps Engineering

Automate the path from a commit to something running in production.

View

Platform Engineering

Build the internal product that lets every other engineer ship safely.

View

Cloud Engineering

From never opening a cloud console to running real workloads on one.

View

Systems & Infrastructure Engineering

Understand and run the machines everything else is built on.

View

You do not have to do this alone

Our programs are free, taught live, and built around the same progression. Join one and work through it alongside other people.

See our programsCreate a free account