Selection as the bottleneck of the record¶

Definition¶

Selection as the bottleneck of the record is the diagnosis that the limit on humanity's use of its accumulated knowledge is not producing or storing that knowledge but consulting it: the record can be extended cheaply and without bound, while the prime action of use is selection, so a scholar who reaches one pertinent item a week by diligent search cannot form syntheses fast enough to keep up with what is being published.

Explanation¶

The cause named is not scale but the artificiality of the indexes we impose on it. Material placed in storage is filed alphabetically or numerically and found, when it is found, by tracing down from subclass to subclass; an item can sit in only one place unless it is duplicated; the rules governing which path locates it are cumbersome and must be learned; and having arrived at one item, the searcher has to leave the system entirely and re-enter on a fresh path to reach the next. Human recall works the other way — one item in mind snaps to the next by association along a web of trails — which is slower to build and prone to fade but never requires exiting and re-entering. The essay separates two mechanical selection strategies that are still the two available: simple selection examines every item in a set and picks out those with specified characteristics, while class-narrowing selection, exemplified by an automatic telephone exchange, attends only to a class given by the first digit, then a subclass given by the second, and never touches most of the collection. Vannevar Bush wrote this in 1945 as a magazine essay, directed at scientists leaving wartime research, and its illustrations are microfilm, photocells and dry photography — machinery that is entirely obsolete. The diagnosis is not: every generation of catalogs, search engines, wikis and retrieval systems since has moved where the bottleneck sits without dislodging the claim that consultation, not accumulation, is what constrains use of the record.

Key Properties¶

The record can be extended far faster than it can be consulted; extension was never the binding constraint
The named cause is artificial indexing — one item in one place, hierarchical descent, cumbersome path rules
Every lookup terminates by leaving the system and re-entering on a new path
Simple selection scans a whole set for matching characteristics; class-narrowing selection discards most of it unexamined
A 1945 magazine essay whose microfilm machinery is obsolete and whose diagnosis has held for eighty years

Relationships¶

Memex — is the device proposed against this diagnosis — every one of its design commitments exists to make consultation, rather than storage, the cheap operation
Associative trail — is the mechanism by which that proposal attacks this bottleneck, replacing descent through an index with a link the reader laid down themselves
Context rot — is the same bottleneck relocated inside a model — the input can be extended far more cheaply than the reliability with which any given part of it is actually consulted

Applications¶

Diagnosing why a knowledge base with everything in it still fails its users — the question is what selection costs, not what storage holds; arguing for retrieval and linking investment over further capture; framing search, tagging and link schemes as competing answers to one durable constraint.

Sources¶

https://www.theatlantic.com/magazine/archive/1945/07/as-we-may-think/303881/