Chapter 13

Async Rust & Tokio

Threads work but don't scale. Async lets our server handle thousands of connections on a handful of threads - the same model that powers Node.js, Go, and every modern web framework.

Why Async: The Limits of Thread-per-Connection

Our thread pool from Chapter 12 works well for moderate load. But it has a hard ceiling: if the pool has 4 threads and all 4 are handling slow clients, the 5th connection waits. Scaling up the pool helps, but each OS thread costs 1–8 MB of stack memory:

Threads	Stack memory	Concurrent connections
4	~32 MB	4
100	~800 MB	100
10,000	~80 GB	Not feasible

The fundamental problem: most of a webserver's time is spent waiting. Waiting for network reads, waiting for database queries, waiting for file I/O. A blocked thread sitting idle while waiting for bytes from a slow client is wasted capacity.

Async solves this by letting one thread handle many connections. When a task needs to wait (for I/O, a timer, etc.), it yields control and the thread picks up another task. Thousands of tasks can share a small pool of threads:

Thread pool (threads):     Async runtime (tasks on few threads):

Thread 0: [conn A]         Thread 0: [A] [C] [A] [D] [B] [A]
Thread 1: [conn B]         Thread 1: [B] [D] [C] [A] [C] [D]
Thread 2: [    idle   ]
Thread 3: [conn C]         4 threads handle 10,000+ tasks
                           by multiplexing during I/O waits

async/.await Syntax and Future Fundamentals

Rust's async model is built on one concept: the Future trait. A Future represents a value that will be available eventually. An async fn is syntactic sugar for a function that returns a Future:

// This async function...
async fn greet(name: &str) -> String {
    format!("Hello, {}!", name)
}

// ...is roughly equivalent to:
fn greet(name: &str) -> impl Future<Output = String> {
    // returns a state machine that, when polled, produces a String
}

The critical detail: calling an async function does nothing. It returns a Future, which is inert until someone drives it. The .await keyword is how you drive a future to completion:

async fn fetch_data() -> String {
    // This call returns a Future but doesn't run it:
    let future = greet("Rust");

    // .await drives the future, suspending this task if it's not ready:
    let greeting = future.await;

    // Typically you chain them:
    let greeting = greet("Rust").await;

    greeting
}

When you .await a future, two things can happen:

The future is ready - it returns immediately with the value.
The future is not ready (waiting for I/O) - the current task is suspended, and the thread picks up other work. When the I/O completes, the runtime wakes the task and resumes from the .await point.

Note

Unlike JavaScript, Rust futures are lazy. In JS, fetch(url) starts the request immediately. In Rust, fetch(url) returns a future that does nothing until awaited. This gives you control over when and whether work happens.

No Built-in Runtime

Rust provides the async/.await syntax and the Future trait, but not a runtime to execute futures. You need an external crate. The ecosystem standard is Tokio.

This is different from Go (goroutines + built-in scheduler) or JavaScript (built-in event loop). Rust lets you choose your runtime, which is more flexible but means one more dependency to learn.

Introduction to the Tokio Runtime

Tokio is Rust's most widely used async runtime. It provides:

A multi-threaded task scheduler that runs thousands of tasks on a pool of OS threads.
Async I/O - TCP, UDP, files, timers - all non-blocking.
Synchronization primitives - async mutexes, channels, semaphores.

Add it to your project:

cargo add tokio --features "rt-multi-thread,net,io-util,macros"

The features we're using:

rt-multi-thread

Multi-threaded scheduler. Tasks are distributed across all available CPU cores.

net

Async TCP and UDP. Replaces std::net.

io-util

Helpers like AsyncReadExt and AsyncWriteExt for reading/writing async streams.

macros

The #[tokio::main] macro that sets up the runtime.

A minimal Tokio program:

#[tokio::main]
async fn main() {
    println!("Hello from Tokio!");
}

// The #[tokio::main] macro expands to roughly:
// fn main() {
//     tokio::runtime::Runtime::new().unwrap()
//         .block_on(async {
//             println!("Hello from Tokio!");
//         });
// }

Tip

#[tokio::main] creates a multi-threaded runtime by default (one thread per CPU core). For a single-threaded runtime, use #[tokio::main(flavor = "current_thread")]. For tests, use #[tokio::test].

Rewriting the Server with tokio::net::TcpListener

Let's convert our server from blocking I/O to async. The structure is almost identical - the main change is adding async and .await to I/O operations and replacing std::net with tokio::net:

src/main.rs

use tokio::io::{AsyncReadExt, AsyncWriteExt};
use tokio::net::{TcpListener, TcpStream};
use std::sync::Arc;

mod error;
mod handler;
mod request;
mod response;
mod router;

use error::ServerError;
use request::Request;
use response::Response;

async fn process_request(
    stream: &mut TcpStream,
    router: &router::Router,
) -> Result<Response, ServerError> {
    let mut buffer = [0u8; 4096];
    let n = stream.read(&mut buffer).await?;  // .await - non-blocking read

    if n == 0 {
        return Err(ServerError::ParseError("empty request".into()));
    }

    let raw = String::from_utf8_lossy(&buffer[..n]);
    let request = Request::parse(&raw)?;
    println!("{} {}", request.method, request.path);
    router.route(&request)
}

async fn handle_connection(mut stream: TcpStream, router: Arc<router::Router>) {
    let response = match process_request(&mut stream, &router).await {
        Ok(resp) => resp,
        Err(e) => {
            eprintln!("Error: {}", e);
            e.to_response()
        }
    };

    if let Err(e) = stream.write_all(&response.to_bytes()).await {
        eprintln!("Write failed: {}", e);
    }
    let _ = stream.flush().await;
}

#[tokio::main]
async fn main() {
    let router = Arc::new(handler::build_router());

    let addr = "127.0.0.1:8080";
    let listener = TcpListener::bind(addr).await.unwrap();
    println!("Listening on http://{}", addr);

    loop {
        match listener.accept().await {
            Ok((stream, addr)) => {
                let router = Arc::clone(&router);
                tokio::spawn(async move {
                    handle_connection(stream, router).await;
                });
            }
            Err(e) => eprintln!("Accept failed: {}", e),
        }
    }
}

Let's compare the changes from the synchronous version:

Sync (Chapter 12)	Async (Tokio)
std::net::TcpListener	tokio::net::TcpListener
stream.read(&mut buf)	stream.read(&mut buf).await
stream.write_all(bytes)	stream.write_all(bytes).await
for stream in listener.incoming()	loop { listener.accept().await }
thread::spawn(move \|\| ...)	tokio::spawn(async move ...)
fn main()	#[tokio::main] async fn main()
ThreadPool (manual)	Built into Tokio runtime

The code structure is nearly identical. The key difference is what happens when a .await would block:

Sync: the thread sleeps until I/O completes. No other work happens on that thread.
Async: the task yields, the thread picks up another task. When I/O completes, the runtime resumes the original task.

This means our async server can handle thousands of connections on just 4–8 threads. No thread pool to manage, no stack memory to worry about.

Note

We no longer need our custom ThreadPool. Tokio provides a work-stealing scheduler that's far more sophisticated - it dynamically balances tasks across threads, handles I/O polling via epoll/kqueue, and supports timers, cancellation, and more.

Spawning Tasks with tokio::spawn

tokio::spawn is the async equivalent of thread::spawn. It takes an async block or future and schedules it on the runtime. The task runs concurrently with other tasks, potentially on different threads:

// Spawn a background task
let handle = tokio::spawn(async {
    // This runs concurrently
    expensive_computation().await
});

// Do other work while the task runs...

// Optionally await the result
let result = handle.await.unwrap();

tokio::spawn has the same requirements as thread::spawn - the future must be Send and 'static. This means it must own its data (no borrowed references to the spawning scope). That's why we use Arc for the router and move to transfer ownership of the stream.

Tasks vs Threads

OS Threads

1–8 MB stack each
OS-managed scheduling
Heavy context switches
Hundreds practical, thousands expensive
Can run blocking code

Tokio Tasks

~few hundred bytes each
Runtime-managed scheduling
Cheap context switches (just swapping state machines)
Millions practical
Must not block - use .await for I/O

Concurrent Operations

With async, running multiple operations concurrently is straightforward:

use tokio::time::{sleep, Duration};

// Sequential - takes 3 seconds
async fn sequential() {
    let a = fetch_page("/a").await;   // 1 second
    let b = fetch_page("/b").await;   // 1 second
    let c = fetch_page("/c").await;   // 1 second
}

// Concurrent with tokio::join! - takes 1 second
async fn concurrent() {
    let (a, b, c) = tokio::join!(
        fetch_page("/a"),
        fetch_page("/b"),
        fetch_page("/c"),
    );
}

// Select the first to complete
async fn race() {
    tokio::select! {
        result = fetch_page("/fast") => {
            println!("Fast returned: {:?}", result);
        }
        _ = sleep(Duration::from_secs(5)) => {
            println!("Timeout - 5 seconds exceeded");
        }
    }
}

Three composition tools:

tokio::join! - run all futures concurrently, wait for all to complete. Like Promise.all in JavaScript.
tokio::select! - run all futures concurrently, return when the first completes, cancel the rest. Like Promise.race. Great for timeouts.
tokio::spawn - fire and forget. The task runs in the background, independently.

Warning

Never do CPU-heavy work in an async task without yielding. It blocks the entire thread, starving other tasks. For CPU-bound work, use tokio::task::spawn_blocking - it runs the closure on a dedicated thread pool designed for blocking operations.

Common Async Pitfalls

Async Rust has some rough edges. Here are the most common issues beginners hit:

1. Forgetting .await

async fn handle(stream: &mut TcpStream) {
    stream.write_all(b"hello"); // Warning: unused Future!
    // Nothing was written - the future wasn't awaited

    stream.write_all(b"hello").await; // Actually writes
}

The compiler warns about unused futures. Always check for this - a missing .await means the operation never runs.

2. Holding a MutexGuard Across .await

use std::sync::Mutex;

let data = Arc::new(Mutex::new(vec![]));

// BAD - holds std::sync::Mutex lock across .await
async fn bad(data: &Mutex<Vec<String>>, stream: &mut TcpStream) {
    let mut lock = data.lock().unwrap();
    let n = stream.read(&mut [0u8; 1024]).await; // blocks other tasks!
    lock.push("data".into());
}

// GOOD - drop the lock before .await
async fn good(data: &Mutex<Vec<String>>, stream: &mut TcpStream) {
    let n = stream.read(&mut [0u8; 1024]).await;
    let mut lock = data.lock().unwrap();
    lock.push("data".into());
    // lock dropped immediately
}

// ALSO GOOD - use tokio::sync::Mutex for async-aware locking
use tokio::sync::Mutex as AsyncMutex;

3. Blocking in Async Context

// BAD - blocks the async thread
async fn bad() {
    std::thread::sleep(Duration::from_secs(5)); // blocks!
    std::fs::read_to_string("big_file.txt");    // blocks!
}

// GOOD - use async equivalents
async fn good() {
    tokio::time::sleep(Duration::from_secs(5)).await;
    tokio::fs::read_to_string("big_file.txt").await;
}

// If you MUST block (e.g., CPU-bound work):
async fn compute() {
    let result = tokio::task::spawn_blocking(|| {
        heavy_cpu_work() // runs on a separate thread pool
    }).await.unwrap();
}

Tip

Rule of thumb: if a function comes from std and does I/O, it blocks. Replace it with the tokio equivalent. If there's no async equivalent, wrap it in spawn_blocking.

Putting It Together

Here's our updated Cargo.toml and the complete async server. The module files (error.rs, request.rs, response.rs, etc.) are unchanged - parsing and routing don't involve I/O, so they stay synchronous:

Cargo.toml

[package]
name = "webserver"
version = "0.1.0"
edition = "2021"

[dependencies]
tokio = { version = "1", features = ["rt-multi-thread", "net", "io-util", "macros"] }

src/main.rs

mod error;
mod handler;
mod request;
mod response;
mod router;

use std::sync::Arc;
use tokio::io::{AsyncReadExt, AsyncWriteExt};
use tokio::net::{TcpListener, TcpStream};

use error::ServerError;
use request::Request;
use response::Response;

async fn process_request(
    stream: &mut TcpStream,
    router: &router::Router,
) -> Result<Response, ServerError> {
    let mut buffer = [0u8; 4096];
    let n = stream.read(&mut buffer).await?;

    if n == 0 {
        return Err(ServerError::ParseError("empty request".into()));
    }

    let raw = String::from_utf8_lossy(&buffer[..n]);
    let request = Request::parse(&raw)?;
    println!("{} {}", request.method, request.path);
    router.route(&request)
}

async fn handle_connection(mut stream: TcpStream, router: Arc<router::Router>) {
    let response = match process_request(&mut stream, &router).await {
        Ok(resp) => {
            println!("  → {}", resp.status);
            resp
        }
        Err(e) => {
            eprintln!("  → Error: {}", e);
            e.to_response()
        }
    };

    if let Err(e) = stream.write_all(&response.to_bytes()).await {
        eprintln!("Write failed: {}", e);
    }
    let _ = stream.flush().await;
}

#[tokio::main]
async fn main() {
    let router = Arc::new(handler::build_router());

    let addr = "127.0.0.1:8080";
    let listener = TcpListener::bind(addr).await.unwrap();
    println!("Listening on http://{}", addr);

    loop {
        match listener.accept().await {
            Ok((stream, _addr)) => {
                let router = Arc::clone(&router);
                tokio::spawn(async move {
                    handle_connection(stream, router).await;
                });
            }
            Err(e) => eprintln!("Accept failed: {}", e),
        }
    }
}

Run it with cargo run and test the same way as before. The behavior is identical, but this server can now handle thousands of concurrent connections on a handful of threads. No manual thread pool needed.

Exercise

Add a request timeout: use tokio::select! to race the request processing against a tokio::time::sleep(Duration::from_secs(5)). If the timeout wins, respond with 408 Request Timeout.
Add a /slow route that calls tokio::time::sleep(Duration::from_secs(3)).await. Hit it from two terminals at the same time and verify both complete in ~3 seconds (not ~6). This proves they're running concurrently.
Replace std::sync::Mutex with tokio::sync::Mutex for the request counter from Chapter 12's exercises. Why might you choose one over the other?
Use tokio::join! to fetch data from two different handlers concurrently and combine the results into a single response.
Try running with a single-threaded runtime: #[tokio::main(flavor = "current_thread")]. Does the server still handle concurrent connections? Why?

What's Next

Our server is now fully async - accepting connections, reading requests, and writing responses all without blocking. Tokio handles scheduling, I/O polling, and thread management. We can handle thousands of connections on a few threads.

In the next chapter, we add JSON support with Serde - deserializing request bodies and serializing response bodies. This will turn our HTML-only server into a proper API server.

Chapter 12: Concurrency Chapter 14: JSON & Serde