Local is faster

Sometimes quantity has a quality all its own.

Every task has a speed requirement and a latency. At some threshold, the time it takes to complete the task exceeds the time available, and the task becomes impossible.

For example, how much typing latency do you think you'd tolerate? A fifth of a second (200 ms)? A fiftieth (20 ms)? Make your guess and then find out: https://aresluna.org/keyboard-secrets/typing-delay/

Local operations tend to be faster than those requiring communication with a remote resource. Here's Peter Norvig's famous list of times every beginning programmer should know (circa 2012):

Operation Latency
execute typical instruction 1 nanosec
fetch from L1 cache memory 0.5 nanosec
branch misprediction 5 nanosec
fetch from L2 cache memory 7 nanosec
Mutex lock/unlock 25 nanosec
fetch from main memory 100 nanosec
send 2K bytes over 1Gbps network 20,000 nanosec
read 1MB sequentially from memory 250,000 nanosec
fetch from new disk location (seek) 8,000,000 nanosec
read 1MB sequentially from disk 20,000,000 nanosec
send packet US to Europe and back 150,000,000 nanosec

Recall Douglas Engelbart's 1968 collaborative editing demo, the "mother of all demos", in which multiple editors at terminals connected to a mainframe worked on the same document.

Imagine that the mainframe was in Europe while the terminals were in the California SRI building. Each keystroke would be one round trip: terminal sends signal to Europe (75ms), mainframe processes and sends back (75ms). Total: 150ms per keystroke.

In the typing latency demo above, my threshold of perception is about 10ms; my threshold of tolerability is about 30ms. 150ms feels like typing in sand.

In Engelbart's actual setup, which had a dumb terminal and the mainframe in the same building, latency was imperceptible — likely under 10ms. The collaborative editing task worked because operations were local.

This principle is ubiquitous. Tasks that work at one latency become impossible at another.

Consider a program manager in a software company who learns to code not to write production software, but to enable real-time thinking. Suppose he's in a conversation with an engineer, and his goal is to bring clarity to a plan for building a component. The engineer is being vague. If the program manager can think based on his own local knowledge about how such a component will work, he can ask focused questions and propose reasonable approximations. If not, he can only depend on what he gets from the engineer. The software engineering knowledge is available to him, just not locally. To utilize it he'd have to say something like, "let me spend a couple weeks reading up on this, then I can make some reasonable approximations." The latency makes the collaborative planning impossible.

Local is faster. Sometimes that's nice to have, sometimes it's a sine qua non.

2025-11-04

Home