Rate Limiting Basics and the Baseline

Start with the real problem

Imagine an API that normally receives 200 requests per second. A new integration deploys a bug and begins sending 20,000 requests per second with the same API key.

The service now has two responsibilities:

1. protect healthy customers from one noisy client;
2. tell the noisy client exactly what happened and when it may try again.

A rate limiter is a traffic gate. Before a request enters the service, the gate asks:

Has this caller used too much of its request budget?

If the answer is no, the request continues. If the answer is yes, the request is rejected for a short time.

A complete design still needs a few precise choices: why we are limiting, who owns the budget, which work spends it, and what the caller sees after the budget is empty.

What are you protecting?

Do not choose an algorithm yet. First decide what problem the limiter should solve.

The same request count can mean very different risk. Ten fast cache reads may be harmless while ten report exports hold database connections for minutes.

Rate and concurrency answer different questions

Rate limit        → How often may work begin?Concurrency limit → How much work may be active now?

A rate limit is like allowing ten people through a door every minute. A concurrency limit is like allowing only five people inside the room at once.

The first controls arrival speed. The second controls active work. Production systems often use both.

Define the policy boundary

Every rate limit must answer two beginner-friendly questions:

1. Who owns the budget?
2. What action spends the budget?

Together they complete this sentence:

Allow N units per [identity] for [operation].

Choose a trustworthy identity

The identity is simply the caller that owns the budget. Common choices, from strongest to weakest, are:

1. API key or access token;
2. authenticated user ID;
3. tenant or organization ID;
4. client IP address.

An IP address is convenient but ambiguous. Hundreds of people may share one corporate NAT address, while an attacker may rotate through many addresses. On login, however, a trusted user ID may not exist yet. That endpoint might combine a normalized account identifier with a trusted client IP.

Scope the budget to an operation

A caller alone is not enough. We also need to know which action is spending the budget:

tenant=acme + GET /catalogtenant=acme + POST /reports

If those routes share one global budget, cheap reads can consume quota intended for an expensive report. A readable internal key might begin as:

rl:fixed-window:v1:POST/reports:tenant=acme

For now, remember this smaller rule:

limiter key = caller + action

Production systems later normalize route templates and hide sensitive identity values.

HTTP request
    │
    ├── authenticate ───────► trusted user or tenant
    │
    ├── normalize route ────► POST /reports/:reportID
    │
    └── choose policy ──────► 10 operations / minute
                                │
                                ▼
              tenant:acme | POST /reports/:reportID

Design the HTTP response

A caller needs to know whether it used too much quota or the whole service is unavailable.

Typical 503 causes are an overloaded database, a failing dependency, maintenance, or deliberate load shedding. None of those are the caller's fault, which is exactly why the caller must be able to tell the two responses apart.

Both responses may contain Retry-After, but they communicate different operational facts. That distinction affects client behavior, dashboards, and incident triage.

Calculate Retry-After safely

HTTP supports either a date or a non-negative integer number of seconds:

HTTP/1.1 429 Too Many RequestsRetry-After: 3

Suppose the limiter says 2.1 seconds remain. The HTTP layer must round up to 3 seconds. Rounding down sends the caller back before the window resets.

340 ms → Retry-After: 12.1 s  → Retry-After: 3

The worst version of this mistake is Retry-After: 0. It tells the client to retry immediately, the retry is rejected again, and a well-behaved client turns into a hot retry loop.

Keep sub-second precision inside the algorithm. Only the transport layer should convert it to HTTP's whole-second format.

Describe the traffic shape

“Ten requests per second” sounds complete, but it is not. Can all ten arrive at the same instant, or must they be spread across the second? That pattern is called the traffic shape.

This course starts with fixed window because its small state makes correctness problems easy to see. Lesson 3 will preserve its boundary burst as a documented algorithm property rather than hiding it.

Decide where state lives

Suppose the policy is 100 requests per minute and the service has three replicas.

                   load balancer
                  /      |      \
                 ▼       ▼       ▼
             replica A replica B replica C
              count 100 count 100 count 100

            effective cluster admission ≈ 300/minute

Each process-local limiter is behaving correctly, but each owns an independent budget.

If “replica” is new vocabulary, think of it as another running copy of the same application.

We will build locally first so the algorithm and contract are observable without a networked state system. The shared Limiter boundary in Lesson 2 is designed so a distributed implementation can later replace it.

Build a trustworthy control endpoint

The first code is intentionally not rate limited. A control endpoint lets you prove that later 429 responses come from the limiter rather than routing, deployment, or networking.

Create the repository:

mkdir go-rate-limitercd go-rate-limitergit init -b maingo mod init github.com/TonmoyTalukder/go-rate-limitergo mod edit -go=1.22mkdir -p cmd/servertouch cmd/server/main.go

Two of these lines deserve a closer look:

• The module path (github.com/TonmoyTalukder/go-rate-limiter) is the project's permanent import identity. Keep this exact path while following the course because every later import uses it. If you publish the project under your own account, replace the module path and every project import together after the course.
• The go 1.22 line declares the minimum Go version the project requires, not the version installed on your machine. Your local toolchain may be newer. Declaring the lowest version you actually need keeps the module usable by more people. Lesson 3 will raise this minimum when a dependency demands it—an honest, deliberate change rather than an accident.

Keep the initial structure small:

go-rate-limiter/├── cmd/│   └── server/│       └── main.go└── go.mod

Do not create algorithm, store, or middleware packages yet. A folder is an architectural promise; create it when a real responsibility exists.

This checkpoint contains only files created by the commands above. Add repository-hygiene files such as .gitignore, LICENSE, and README.md when you prepare the project for publishing; they are intentionally outside this implementation lesson.

Implement the server

Create cmd/server/main.go:

package mainimport (	"fmt"	"log"	"net/http"	"time")func main() {	mux := http.NewServeMux()	// This endpoint stays unrestricted as the course control case.	mux.HandleFunc("GET /api/demo", func(w http.ResponseWriter, _ *http.Request) {		w.Header().Set("Content-Type", "application/json")		w.WriteHeader(http.StatusOK)		_, _ = fmt.Fprint(w, `{"status":"ok","endpoint":"demo"}`)	})	server := &http.Server{		Addr:              ":8080",		Handler:           mux,		ReadHeaderTimeout: 5 * time.Second,	}	log.Println("listening on http://localhost:8080")	if err := server.ListenAndServe(); err != nil && err != http.ErrServerClosed {		log.Fatalf("server failed: %v", err)	}}

The method-aware pattern GET /api/demo requires Go 1.22 and makes the route contract visible. ReadHeaderTimeout bounds how long a slow client may occupy a connection while sending headers.

One ordering rule inside the handler matters: headers are set before WriteHeader. Once the status line is written, the headers may already be on the wire, and later Header().Set calls are silently ignored. Lesson 3 relies on this when it adds Retry-After to denied responses.

Verify the baseline

Run the engineering gate before manual testing:

gofmt -w cmd/server/main.gogo mod tidygo test ./...go build ./...go vet ./...go run ./cmd/server

In a second terminal:

curl -i http://localhost:8080/api/democurl -i -X POST http://localhost:8080/api/democurl -i http://localhost:8080/unknown

The expected contract is:

Finally, prove the endpoint has no limiter:

for i in $(seq 1 100); do  curl -s -o /dev/null -w "%{http_code}\n" \    http://localhost:8080/api/demodone | sort | uniq -c

Expected:

100 200

Write the first decision record

Before continuing, record the choices that later lessons will change.

Lesson 1 checkpoint

You are ready for Lesson 2 when you can explain all of these without rereading:

• what fairness, cost, abuse, and overload policies protect;
• why rate and concurrency limits are different controls;
• why identity and operation both belong in a limiter key;
• why 429 is not the same as 503;
• why Retry-After rounds upward;
• how burst capacity differs from sustained rate;
• why a local limit is multiplied across replicas;
• why /api/demo must remain unrestricted.

Next, you will design the common contracts that let local and distributed algorithms answer the same admission question without leaking HTTP or storage details into the algorithm.