Tokio Under the Hood

I’ve been writing async Rust for a while now, and at some point I realized I was just sprinkling #[tokio::main] and .await everywhere without really understanding what happens beneath. So I went digging.

This post is what I found.

First, Rust’s async model

Before Tokio makes sense, we need to understand what Rust actually gives us.

Futures are state machines

At the core of async Rust is the Future trait:

pub trait Future {
    type Output;
    fn poll(self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll<Self::Output>;
}

When you write an async fn, the compiler doesn’t generate threads or callbacks. It transforms your function into a state machine that implements Future. Each .await point becomes a state transition.

async fn fetch_data(id: u32) -> String {
    delay_for(Duration::from_millis(100)).await;  // state boundary
    format!("Data for id {}", id)
}

The compiler turns this into something like an enum with variants for “before the delay” and “after the delay”, plus the logic to resume from either state.

The critical thing: Futures in Rust are lazy. They do absolutely nothing until someone calls poll() on them. You can create a future and drop it — no work happens.

So who calls poll()?

That’s where Tokio comes in

Tokio is the thing that actually drives your futures to completion. Without it (or another runtime), your async code just sits there.

It does three things:

1. Polls your futures when they’re ready to make progress
2. Provides non-blocking I/O (network, timers, signals)
3. Gives you async-friendly primitives (channels, locks, semaphores)

The Reactor

At Tokio’s core is an event loop based on the reactor pattern.

Think of it like a restaurant host. Instead of having a waiter stand at each table waiting for customers to decide, the host keeps a list of tables. When a table is ready to order, the host gets notified and sends a waiter over.

The reactor does the same thing with I/O:

1. Tasks register interest in events -> “tell me when this socket has data”
2. The reactor waits efficiently using OS primitives (epoll on Linux, kqueue on macOS, IOCP on Windows)
3. When an event fires, the reactor wakes up the corresponding task

No busy-waiting. No thread-per-connection. Just event notification.

The Scheduler

Once a task is woken up, someone needs to actually run it. That’s the scheduler.

Tokio has two flavors:

• Multi-thread (default): A work-stealing thread pool. Multiple worker threads, each with their own task queue. If one thread runs out of work, it steals from another. This keeps all cores busy.
• Current-thread: Everything runs on one thread. Useful for simpler workloads or when you need deterministic execution.

The scheduler maintains queues of “ready” tasks and distributes them across worker threads. It also ensures no single task can hog a thread forever. Tokio uses cooperative scheduling, where tasks voluntarily yield at .await points.

Tracing a request through the system

Let’s follow what actually happens with a simple TCP server:

use tokio::net::{TcpListener, TcpStream};
use tokio::io::{AsyncReadExt, AsyncWriteExt};

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    let listener = TcpListener::bind("127.0.0.1:8080").await?;

    loop {
        let (socket, _) = listener.accept().await?;

        tokio::spawn(async move {
            handle_connection(socket).await
        });
    }
}

async fn handle_connection(mut socket: TcpStream) -> Result<(), Box<dyn std::error::Error>> {
    let mut buffer = [0; 1024];
    let n = socket.read(&mut buffer).await?;

    let response = b"HTTP/1.1 200 OK\r\nContent-Length: 12\r\n\r\nHello world!";
    socket.write_all(response).await?;

    Ok(())
}

Here’s the play-by-play:

1. #[tokio::main] sets up the runtime -> spawns worker threads, initializes the reactor
2. TcpListener::bind().await -> registers with the OS for connection events
3. listener.accept().await -> no connection yet, so the task suspends. The reactor is now watching for incoming connections
4. A client connects → the OS notifies the reactor → the reactor wakes our task → the scheduler picks it up → accept() returns the socket
5. tokio::spawn() wraps our handler in a Task (a future + scheduling metadata) and puts it on the queue
6. socket.read().await -> no data yet, task suspends again. Reactor watches the socket
7. Data arrives → reactor wakes the task → scheduler runs it → read() returns
8. socket.write_all().await -> same dance. Register interest, suspend, get woken when writable
9. Response sent. Task completes and gets dropped

All of this with maybe 4 threads handling thousands of connections. No thread blocks. No thread waits.

The pieces inside the runtime

Drivers

Tokio has specialized drivers for different event sources:

• I/O Driver — wraps epoll/kqueue/IOCP for network and file I/O
• Time Driver — manages timers and tokio::time::sleep efficiently (uses a timer wheel, not one OS timer per sleep)
• Signal Driver — handles OS signals like SIGTERM

Each driver feeds events back to the scheduler through the waker mechanism.

Work-stealing

The multi-thread scheduler uses a work-stealing algorithm. Each worker thread has a local queue. New tasks go to the local queue first (cheap, no contention). When a thread’s queue is empty, it steals from other threads’ queues.

This is the same approach used by Go’s goroutine scheduler and Java’s ForkJoinPool. It balances load without centralized coordination.

Tasks are cheap

A Tokio task is not an OS thread. It’s a future plus some metadata which is around a few hundred bytes. You can spawn millions of them. The overhead is the state machine the compiler generates, not a full thread stack.

Configuring the runtime

You don’t have to use the #[tokio::main] macro. You can configure things manually:

let runtime = tokio::runtime::Builder::new_multi_thread()
    .worker_threads(4)
    .enable_io()
    .enable_time()
    .build()
    .unwrap();

This is useful when you need to control thread count or selectively enable drivers.

One gotcha: CPU-bound work

Cooperative scheduling only works if tasks actually yield. If you have a tight loop with no .await points, that task monopolizes its thread and starves everything else.

Two options:

// Option 1: manually yield
tokio::task::yield_now().await;

// Option 2: run CPU-bound work on a blocking thread
tokio::task::spawn_blocking(|| {
    heavy_computation()
}).await;

spawn_blocking moves work to a separate thread pool so it doesn’t interfere with the async workers.

Key takeaways

The thing that clicked for me: Tokio isn’t magic. It’s a loop that polls futures when the OS says there’s work to do. The elegance is in how efficiently it does this — work-stealing, minimal allocations, and zero-cost abstractions from the compiler.

Understanding this has made me much better at reasoning about where time is actually spent in async Rust code. Hopefully it does the same for you.