Rust Async Runtime Internals: How It Works Under the Hood

Rust’s async/await syntax looks simple on the surface, but there is a carefully designed system underneath. Unlike languages with built-in runtimes, Rust ships no runtime at all. The language gives you the Future trait and the async/await keywords, and the runtime is provided by libraries like Tokio and async-std. This tutorial peels back the layers to show how it all fits together.

The Future Trait

At the core of Rust’s async system is the Future trait:

use std::pin::Pin;
use std::task::{Context, Poll};

pub trait Future {
    type Output;

    fn poll(self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll<Self::Output>;
}

A future is something that can be polled. Each call to poll returns either Poll::Ready(value) when the computation is complete, or Poll::Pending when it is not ready yet. The key insight is that the caller (the executor) drives the future by calling poll, and the future does not block.

Here is a minimal future that counts down:

use std::pin::Pin;
use std::task::{Context, Poll};
use std::future::Future;

struct Countdown {
    remaining: u32,
}

impl Future for Countdown {
    type Output = String;

    fn poll(mut self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll<Self::Output> {
        if self.remaining == 0 {
            Poll::Ready("Done!".to_string())
        } else {
            self.remaining -= 1;
            cx.waker().wake_by_ref(); // Schedule another poll immediately
            Poll::Pending
        }
    }
}

What async/await Compiles To

When you write an async fn, the compiler transforms it into a state machine that implements Future. Each .await point becomes a state transition.

async fn fetch_and_process() -> String {
    let data = fetch_data().await;    // State 0 -> State 1
    let result = process(data).await; // State 1 -> State 2
    result
}

The compiler generates something conceptually like this:

enum FetchAndProcess {
    State0 { /* initial captures */ },
    State1 { data: Vec<u8>, /* sub-future for process() */ },
    State2, // complete
}

impl Future for FetchAndProcess {
    type Output = String;

    fn poll(self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll<String> {
        // Match on current state, poll the relevant sub-future,
        // transition to next state when sub-future is Ready
        todo!()
    }
}

This is why async in Rust has zero-cost abstractions: no heap allocation is required for the state machine itself (unless you box the future), and there is no implicit scheduling overhead.

Wakers: The Notification System

The Context passed to poll contains a Waker. When a future returns Poll::Pending, it must arrange for the waker to be called when progress can be made. The waker tells the executor that the task should be polled again.

use std::task::{Waker, RawWaker, RawWakerVTable};

// A minimal waker that does nothing (for demonstration)
fn dummy_waker() -> Waker {
    fn no_op(_: *const ()) {}
    fn clone(data: *const ()) -> RawWaker {
        RawWaker::new(data, &VTABLE)
    }

    static VTABLE: RawWakerVTable = RawWakerVTable::new(clone, no_op, no_op, no_op);

    unsafe { Waker::from_raw(RawWaker::new(std::ptr::null(), &VTABLE)) }
}

In a real runtime, the waker is connected to the executor’s task queue. When an I/O event completes or a timer fires, the reactor calls waker.wake(), which pushes the task back onto the executor’s run queue.

Building a Minimal Executor

An executor is just a loop that polls futures until they are ready. Here is a simplified single-threaded executor:

use std::collections::VecDeque;
use std::future::Future;
use std::pin::Pin;
use std::sync::{Arc, Mutex};
use std::task::{Context, Poll, Wake};

struct Task {
    future: Mutex<Pin<Box<dyn Future<Output = ()> + Send>>>,
    queue: Arc<Mutex<VecDeque<Arc<Task>>>>,
}

impl Wake for Task {
    fn wake(self: Arc<Self>) {
        // When woken, put the task back on the queue
        self.queue.lock().unwrap().push_back(self.clone());
    }
}

struct MiniExecutor {
    queue: Arc<Mutex<VecDeque<Arc<Task>>>>,
}

impl MiniExecutor {
    fn new() -> Self {
        MiniExecutor {
            queue: Arc::new(Mutex::new(VecDeque::new())),
        }
    }

    fn spawn(&self, future: impl Future<Output = ()> + Send + 'static) {
        let task = Arc::new(Task {
            future: Mutex::new(Box::pin(future)),
            queue: Arc::clone(&self.queue),
        });
        self.queue.lock().unwrap().push_back(task);
    }

    fn run(&self) {
        loop {
            let task = {
                let mut queue = self.queue.lock().unwrap();
                queue.pop_front()
            };

            match task {
                Some(task) => {
                    let waker = task.clone().into();
                    let mut cx = Context::from_waker(&waker);
                    let mut future = task.future.lock().unwrap();
                    let _ = future.as_mut().poll(&mut cx);
                }
                None => break, // No more tasks
            }
        }
    }
}

This executor is trivially simple. Real executors like Tokio add work-stealing across threads, I/O polling, timer management, and more. But the core loop is the same: pop a task, poll it, and if it returns Pending, wait for the waker to re-queue it.

The Reactor Pattern

An executor needs a counterpart: the reactor. The reactor monitors I/O resources (sockets, files, timers) and wakes tasks when events occur. On Linux, this is typically built on epoll; on macOS, kqueue; on Windows, IOCP.

The flow works like this:

1. A task calls an async I/O operation (e.g., reading from a socket).
2. The I/O future registers interest with the reactor and stores the waker.
3. The future returns Poll::Pending.
4. The reactor uses OS-level APIs to wait for readiness events.
5. When the socket is readable, the reactor calls waker.wake().
6. The executor re-polls the task, which now reads data and returns Poll::Ready.

// Pseudocode showing the reactor's role
async fn handle_connection(stream: TcpStream) {
    let mut buf = [0u8; 1024];

    // Under the hood:
    // 1. Register stream fd with epoll
    // 2. Store waker
    // 3. Return Pending
    // ... later, epoll signals readiness ...
    // 4. Waker fires, executor re-polls
    // 5. read() succeeds, returns Ready
    let n = stream.read(&mut buf).await.unwrap();

    println!("Read {n} bytes");
}

How Tokio Puts It Together

Tokio, the most popular async runtime, combines all these pieces:

• Multi-threaded executor with work-stealing. Worker threads each have a local queue, and they steal tasks from other workers when idle.
• I/O reactor built on mio, which abstracts epoll/kqueue/IOCP.
• Timer wheel for managing sleep() and timeout futures.
• Task budgeting to prevent a single task from monopolizing a thread.

#[tokio::main]
async fn main() {
    // tokio::main sets up the runtime:
    // 1. Creates a thread pool (default: one thread per CPU core)
    // 2. Starts the I/O reactor on each thread
    // 3. Initializes the timer system
    // 4. Spawns the main future as the first task

    let handle = tokio::spawn(async {
        tokio::time::sleep(std::time::Duration::from_millis(100)).await;
        println!("Task completed");
    });

    handle.await.unwrap();
}

Key Takeaways

Futures are lazy. Creating a future does nothing. It only runs when an executor polls it. This is different from JavaScript promises, which start executing immediately.

The compiler generates state machines. Each async fn becomes an enum of states. This is what makes Rust async zero-cost: no heap allocations per state machine, no garbage collector, and no hidden threads.

Wakers are the glue. Without wakers, the executor would have to busy-loop, polling every future continuously. Wakers let futures signal when they are worth polling again.

The runtime is pluggable. Because the Future trait is in std but the executor is not, you can choose Tokio, async-std, smol, or write your own. Your async code stays the same.

Wrapping Up

Rust’s async system splits into four layers: the Future trait defines the interface, the compiler generates state machines from async/await, the executor drives futures by polling them, and the reactor monitors I/O to wake tasks when events arrive. Understanding these layers helps you debug async code, choose the right runtime, and write efficient concurrent programs.