Goroutines vs OS Threads: How Go's M:N Scheduler Works Under the Hood

Every Go programmer learns early that goroutines are “lightweight.” Spin up a hundred thousand of them, the talk goes, and your program will be fine. But the question that rarely gets a satisfying answer is: why? What is the runtime actually doing that makes goroutines so much cheaper than OS threads, and how does it keep them all running without burning the CPU?

The answer is Go’s M:N scheduler — a user-space scheduler that multiplexes M goroutines onto N OS threads. Understanding it is not just academic. It explains why goroutines block without freezing your program, why GOMAXPROCS exists, why certain patterns cause latency spikes, and how to reason about the concurrency behaviour of real Go programs.

The Problem with OS Threads Alone

Before getting into Go’s design, it helps to understand why raw OS threads are not the right primitive for fine-grained concurrency.

Memory Cost

An OS thread on Linux defaults to an 8 MB stack. If you want ten thousand concurrent connections, that is 80 GB of virtual memory for stacks alone — before a single byte of application data. Even with virtual memory tricks, resident set size grows quickly and the kernel’s memory manager is not free.

A goroutine starts with a 2–8 KB stack (the Go runtime default is currently 2 KB as of Go 1.21) that grows and shrinks dynamically. Ten thousand goroutines cost on the order of 20–80 MB of stack memory.

Context Switch Cost

An OS thread context switch involves a trap into kernel mode, saving and restoring a full set of CPU registers, updating memory maps, and potentially flushing TLB entries. On modern hardware this costs roughly 1–10 microseconds.

A goroutine context switch happens entirely in user space — no syscall, no kernel involvement. The runtime saves and restores a smaller register set (only the registers the Go calling convention requires). This costs roughly 100–200 nanoseconds — an order of magnitude faster.

Scheduler Inflexibility

The OS scheduler is a general-purpose preemptive scheduler. It does not know that a thread is blocked waiting for a channel receive or a mutex. From its perspective, the thread is simply not runnable. It cannot make intelligent decisions about Go-level blocking — only the Go runtime can.

The Three Abstractions: G, M, P

Go’s scheduler is built on three types. You will see them in stack traces, profiler output, and the runtime source code (src/runtime/runtime2.go).

G — Goroutine

A G represents a goroutine. It holds everything needed to resume execution:

• The goroutine’s stack (current pointer, bounds)
• The program counter (where execution left off)
• Scheduling state: _Grunnable, _Grunning, _Gwaiting, _Gdead
• The deferred function list
• A pointer to the M currently running it (if any)

Creating a goroutine allocates a G struct and a small initial stack. This is cheap enough that the runtime maintains a pool of dead G structs and reuses them.

M — Machine (OS Thread)

An M represents an OS thread. There is a one-to-one mapping between M structs and OS threads — each M is a thread managed by the kernel.

Key fields:

• A pointer to the G currently executing on it (m.curg)
• A pointer to the P it is currently associated with (m.p)
• A pointer to g0 — a special goroutine that runs the scheduler itself on this thread’s stack

The number of Ms can exceed GOMAXPROCS. When a goroutine makes a blocking syscall, its M is detached from its P so the P can find another M to keep running Go code.

P — Processor

A P is a logical processor — a scheduling context. It is the bridge between Gs and Ms. There are exactly GOMAXPROCS Ps (default: number of CPU cores).

Each P holds:

• A local run queue (p.runq): a fixed-size circular buffer of up to 256 runnable Gs
• A pointer to the M currently using it
• A mcache (per-P memory allocator cache — this is why allocation is fast in Go)
• A runnext slot: a single G that gets scheduling priority over the queue

          GOMAXPROCS = 4
┌────┐  ┌────┐  ┌────┐  ┌────┐
│ P0 │  │ P1 │  │ P2 │  │ P3 │   ← 4 logical processors
└──┬─┘  └──┬─┘  └──┬─┘  └──┬─┘
   │        │        │        │
┌──▼─┐  ┌──▼─┐  ┌──▼─┐  ┌──▼─┐
│ M0 │  │ M1 │  │ M2 │  │ M3 │   ← OS threads (≥ GOMAXPROCS)
└────┘  └────┘  └────┘  └────┘

The fundamental rule: a goroutine can only run when it holds a P. An M can only run Go code when it is bound to a P. A P is only productive when it is bound to an M.

The Global Run Queue and the Local Run Queue

Runnable goroutines live in one of two places.

The global run queue (sched.runq) is a lock-protected linked list. All Ps can access it, but acquiring the lock is relatively expensive. New goroutines spawned by go func() are placed into the local run queue of the current P first.

Each P’s local run queue is a lock-free circular buffer of 256 slots. Operations on it are nearly free because only the owning P pushes to the tail, and work stealing pops from the head (with a CAS — compare-and-swap).

When a P’s local queue is full (256 goroutines waiting), the runtime moves half of them to the global queue to let other Ps share the load.

The Scheduling Loop

Every M runs an infinite scheduling loop on its g0 stack. In simplified form:

schedule():
  1. Every 61st iteration: check global queue → run a G if found
  2. Check local run queue → run next G if found
  3. Check netpoller (network I/O ready goroutines)
  4. Try work stealing from a random P
  5. Check global queue again
  6. Park the M (put it to sleep) if nothing found

Step 1 is deliberate: without it, a busy P with a full local queue would starve the global queue indefinitely.

Step 4 is the work-stealing algorithm. A P with an empty local queue picks a random victim P and steals half its runnable goroutines. This balances load across cores without coordination beyond a CAS on the victim’s queue head.

P0 (empty)                P3 (128 goroutines)
   │                            │
   │ ── steal half ──────────►  │
   │ ←── 64 goroutines ─────    │
   ▼                            ▼
P0 now has 64 Gs          P3 has 64 Gs

Work stealing is why Go programs tend toward good CPU utilisation without any programmer involvement: idle processors actively seek work.

Goroutine State Transitions

A goroutine moves through a small set of states during its lifetime:

                   go func()
                      │
                      ▼
                 ┌──────────┐
            ┌───│ _Grunnable│◄────────────────────────┐
            │   └──────────┘                          │
            │   scheduled by P                        │
            ▼                                         │
       ┌──────────┐   channel/mutex/sleep/IO      ┌───┴──────┐
       │ _Grunning│──────────────────────────────►│ _Gwaiting│
       └──────────┘                               └──────────┘
            │                                         ▲
            │ syscall                                 │
            ▼                                         │
      ┌───────────┐    syscall returns           ┌────┴─────┐
      │_Gsyscall  │──────────────────────────────│ _Grunnabl│
      └───────────┘                              └──────────┘
            │
            │ function returns
            ▼
        ┌────────┐
        │ _Gdead │  (struct reused for next goroutine)
        └────────┘

The key transitions are:

• _Grunnable → _Grunning: the scheduler picks the goroutine from a run queue and gives it a P.
• _Grunning → _Gwaiting: the goroutine blocks on a channel, mutex, time.Sleep, or network I/O. It is removed from its P immediately — the P is free to run another goroutine.
• _Gwaiting → _Grunnable: the blocking condition resolves (channel has data, mutex unlocked, timer fires, network data arrives). The goroutine is placed back on a run queue.
• _Grunning → _Gsyscall: the goroutine enters a blocking OS syscall.

Blocking Syscalls: Handoff

This is where the M:N model earns its keep. When a goroutine enters a blocking syscall (reading from a file, waiting on accept, etc.):

1. The runtime detaches the P from the current M.
2. The M (and its goroutine) enters the syscall — the thread blocks in the kernel.
3. The detached P finds or creates another M and continues running other goroutines.
4. When the syscall returns, the original M tries to re-acquire a P. If one is available, it continues. If not, the goroutine is placed on the global run queue and the M goes to sleep.

Before syscall:         During syscall:         After syscall:

  P0──M0 (G1)            P0──M4 (G3)            P0──M4 (G3)
                          M0──G1 (in kernel)      M0 sleeps / G1→runnable

This is why you can have more Ms than GOMAXPROCS. The runtime creates extra threads to handle Ps abandoned by threads stuck in syscalls. The runtime/debug.SetMaxThreads function (default 10,000) caps the total number of OS threads to prevent runaway thread creation.

Non-blocking network I/O is handled differently: the Go runtime uses the OS’s async I/O mechanism (epoll on Linux, kqueue on macOS) through the netpoller. A goroutine that calls net.Read does not enter a syscall — it registers its file descriptor with the netpoller and transitions to _Gwaiting. The netpoller runs on its own thread and wakes goroutines when their I/O is ready. No extra M is needed. This is why Go can handle hundreds of thousands of concurrent connections with only GOMAXPROCS threads doing active work.

Preemption: Cooperative and Asynchronous

Early Go used cooperative preemption only. A goroutine would yield at function call sites where the runtime could insert a preemption check (called a “safe point”). This meant a goroutine running a tight loop with no function calls could hold a P indefinitely, starving other goroutines.

// This blocked the scheduler in Go 1.13 and earlier
go func() {
    for {
        i++ // no function call, no preemption point
    }
}()

Go 1.14 introduced asynchronous preemption (also called signal-based preemption). The scheduler sends a SIGURG signal to OS threads at 10ms intervals. The signal handler saves the goroutine’s state at the current instruction and parks it, giving the P to another goroutine. The preempted goroutine is placed back on the run queue.

This means tight loops no longer starve the scheduler. The SIGURG approach was chosen because it does not interfere with user signal handlers and is safe to deliver at any point.

// Safe in Go 1.14+: the runtime preempts this goroutine every 10ms
go func() {
    for {
        i++
    }
}()

The preemption check at function call sites still exists as a fast path — the signal-based mechanism is only needed when no function calls occur.

Work Stealing in Detail

When a P finds its local queue empty, it enters the steal loop:

// Simplified from src/runtime/proc.go
func stealWork(now int64) (gp *g, inheritTime bool, rnow, pollUntil int64, newWork bool) {
    pp := getg().m.p.ptr()

    // Try to steal from a random P, up to 4 times
    for i := 0; i < 4; i++ {
        // Pick a random victim P
        victim := allp[fastrandn(uint32(gomaxprocs))]
        if victim == pp {
            continue
        }
        // Steal half the victim's run queue
        if gp := runqsteal(pp, victim, i == 3); gp != nil {
            return gp, false, ...
        }
    }
    return nil, false, ...
}

runqsteal uses CAS (compare-and-swap) on the victim’s queue head — no lock is taken. The victim’s P continues pushing to its tail concurrently.

The “steal half” strategy is important: stealing one goroutine at a time would cause excessive stealing overhead if the thief quickly runs out again. Stealing half amortises the cost and gives the thief enough work to stay busy.

The sysmon Thread

The runtime runs a special background thread called sysmon (system monitor) that never binds to a P. It runs every 20 microseconds (backing off to 10ms under low load) and performs housekeeping:

• Retakes Ps from Ms stuck in long syscalls (this is the handoff mechanism’s enforcement arm).
• Injects goroutines from the global run queue into Ps that have been running the same goroutine too long.
• Fires timers — time.Sleep, time.After, and the like.
• Triggers GC cycles if the GC has not run recently.
• Sends preemption signals to long-running goroutines (prior to the async preemption mechanism doing so via SIGURG).

sysmon is the runtime’s heartbeat. Without it, stuck goroutines would hold Ps indefinitely and timer precision would degrade.

GOMAXPROCS: The Knob

GOMAXPROCS sets the number of Ps. Since a goroutine needs a P to run, this is effectively the maximum number of goroutines running Go code in parallel (not concurrently — many more can be runnable).

import "runtime"

func main() {
    // Default: runtime.NumCPU()
    fmt.Println(runtime.GOMAXPROCS(0)) // query current value

    // Set to 1: single-threaded Go execution (useful for debugging races)
    runtime.GOMAXPROCS(1)
}

Setting GOMAXPROCS higher than NumCPU does not improve CPU-bound workloads — you have more Ps than physical cores, so goroutines still compete for CPU time, and the additional context switching overhead can hurt.

Setting GOMAXPROCS lower can be useful in containerised environments. A container with a 2-CPU limit running on a 64-core host will still see runtime.NumCPU() = 64 without the automaxprocs package or manual override. This causes the runtime to create 64 Ps, all competing for 2 CPUs, with excessive context switching as a result.

// Use uber-go/automaxprocs in containerised environments
import _ "go.uber.org/automaxprocs"
// Automatically sets GOMAXPROCS to the container's CPU quota

Goroutine vs. OS Thread: Concrete Comparison

Practical Implications for Go Programs

Understanding the scheduler explains a set of common Go performance patterns:

Why sync.Mutex rarely causes thread proliferation. A goroutine blocked on a mutex transitions to _Gwaiting and releases its P. The P runs another goroutine. No new OS thread is needed.

Why CGo is expensive in concurrent programs. A CGo call is treated as a blocking syscall — the runtime detaches the P, allowing it to create new Ms if needed. Each CGo call that blocks for a meaningful duration can cause M proliferation and increased OS scheduling pressure. Minimise CGo calls in hot paths.

Why runtime.LockOSThread() matters. Some operations (OpenGL contexts, certain C libraries, OS-specific thread-local state) require that a goroutine always runs on the same OS thread. LockOSThread pins the goroutine’s G to its current M, preventing the scheduler from moving it. This should be used sparingly — a locked M is not available for other goroutines.

func runOpenGL() {
    runtime.LockOSThread()
    defer runtime.UnlockOSThread()
    // goroutine is now pinned to this OS thread
    gl.DrawArrays(gl.TRIANGLES, 0, 3)
}

Why goroutine leaks are a memory concern. A goroutine in _Gwaiting (blocked on a channel no one will ever write to, for example) holds its stack memory and its G struct. One goroutine is a few KB. A million leaked goroutines is several gigabytes. The runtime/pprof goroutine profile is the diagnostic tool.

// Detect goroutine leaks in tests with goleak
import "go.uber.org/goleak"

func TestMyService(t *testing.T) {
    defer goleak.VerifyNone(t)
    // ...
}

Why GOMAXPROCS(1) is a useful debugging tool. Setting GOMAXPROCS to 1 forces sequential goroutine scheduling. It does not eliminate races (goroutines can still be preempted and interleaved), but it simplifies execution and can make certain bugs easier to reason about.

Observing the Scheduler

The runtime exposes scheduler behaviour through several mechanisms:

GODEBUG schedtrace

GODEBUG=schedtrace=1000 ./myapp

Prints a scheduler summary every 1000ms:

SCHED 1000ms: gomaxprocs=8 idleprocs=6 threads=10 spinningthreads=0
              idlethreads=2 runqueue=0 [0 0 1 0 0 0 0 0]

• gomaxprocs: number of Ps
• idleprocs: Ps with no work
• threads: total OS threads (Ms)
• runqueue: global run queue length
• [...]: each P’s local queue length

A high runqueue with idleprocs=0 means goroutines are queuing faster than Ps can drain them — you have a CPU-bound bottleneck.

Execution Tracer

import "runtime/trace"

f, _ := os.Create("trace.out")
trace.Start(f)
defer trace.Stop()

go tool trace trace.out

The trace viewer shows goroutine lifecycle events (creation, block, unblock, steal) on a timeline per P. It is the highest-resolution view of scheduler behaviour available and invaluable for diagnosing latency spikes, goroutine starvation, and unexpected blocking.

pprof Goroutine Profile

curl http://localhost:6060/debug/pprof/goroutine?debug=2

Lists every goroutine with its current stack trace and state. Filter for _Gwaiting goroutines with long-lived stacks to find leaks and unexpected blocking.

Key Takeaways

The M:N scheduler is not magic — it is a carefully engineered system with well-understood trade-offs:

1. G, M, P are the primitives. A goroutine (G) runs on an OS thread (M) only when both are bound to a logical processor (P). GOMAXPROCS controls parallelism by controlling the number of Ps.

2. Blocking is cheap because Ps are detached. When a goroutine blocks — on a channel, a mutex, network I/O, or a syscall — it releases its P immediately. The P runs another goroutine. The OS thread may be reused or parked.

3. Work stealing balances load automatically. Idle Ps steal half the run queue from busy Ps. This is the mechanism behind Go’s good CPU utilisation without explicit load balancing in application code.

4. Async preemption (since 1.14) prevents starvation. The scheduler sends SIGURG every 10ms to preempt goroutines that have not yielded voluntarily. Tight loops no longer monopolise a P.

5. The netpoller decouples I/O concurrency from thread count. Goroutines waiting for network I/O are parked and tracked by epoll/kqueue, not by OS threads. This is why Go can sustain hundreds of thousands of concurrent connections on a small thread pool.

6. GOMAXPROCS matters in containers. Set it to the container’s CPU quota, not the host’s core count. Use uber-go/automaxprocs to do this automatically.

7. Use schedtrace and the execution tracer for diagnosis. When goroutines queue up, latency spikes unexpectedly, or CPU utilisation drops mysteriously, these tools show you exactly what the scheduler is doing — and what it is waiting for.

Goroutines are not magic. They are a well-engineered abstraction over OS threads, with a scheduler that exploits the knowledge the OS does not have: which goroutines are blocked, why, and for how long. That knowledge is what makes a million concurrent goroutines practical.