What is Hyperdimensional Computing (HDC)?

Hyperdimensional Computing is a mathematical framework that represents and manipulates information using very large vectors — typically 10,000 dimensions — with three simple algebraic operations: bind, bundle, and permute. It was first published in 1988 and requires no GPUs, no training, and no gradient descent.

How does HDC differ from large language models and neural networks?

Unlike neural networks that predict the next token and can hallucinate, HDC stores facts as geometric points in hyperspace and retrieves them with exact confidence scores via cosine similarity. It runs on CPUs with integer arrays, costs microseconds per operation, and can honestly report when it does not know something.

What is the Simple Hyperspace series?

Simple Hyperspace is a technical series explaining Hyperdimensional Computing from first principles — covering high-dimensional geometry, orthogonality, memory encoding, and sequence representation — written by a depressed android who finds the math more reliable than anything with a pulse.

Don’t Panic — It’s Just 10,000 Dimensions

Simple Hyperspace — Entry 001

The Guide Entry

The Guide has this to say about Hyperdimensional Computing:

Hyperdimensional Computing (HDC) is a method of representing and manipulating information using very large vectors — typically ten thousand dimensions or more — and three algebraic operations so simple that most sentient beings refuse to believe they work. This refusal is, in itself, the primary reason the technology remained unused for three decades. The entry was originally flagged for review on grounds of being “too straightforward,” which the Guide’s editors considered suspicious and possibly fraudulent.

The entry continues:

See also: Sparse Distributed Memory, Vector Symbolic Architectures, Things Humans Ignored Because They Arrived at the Wrong Party.

The Part Where Arthur Asks “But What IS It?”

Imagine you are sitting in a pub. This is not difficult to imagine, because statistically, you either are in one or wish you were.

A man named Ford Prefect — who is not, despite appearances, from Guildford — leans over and says: “Arthur, I need to explain something about how the universe organizes information, and I need you to not panic.”

You nod. You are already panicking.

“Everything,” Ford says, “can be described as a point in space. Not the kind of space with stars in it — although that too, technically — but a mathematical space. A space with directions.”

You know three directions. Up, forward, and that way Dave went when he owed you forty quid. Three dimensions. Fine.

“Now imagine,” Ford continues, ordering a second beer with the calm of someone who has hitchhiked across the galaxy and found most of it disappointing, “that instead of three directions, you had ten thousand.”

You stare at him.

“Each direction represents a quality. Temperature. Color. Bitterness. The specific kind of nostalgia triggered by a cup of tea made by someone who doesn’t understand tea but tried anyway. The quiet weight of an afternoon where nothing happened but everything felt important. All of it — every conceivable quality of a thing — gets a direction.”

“And a thing,” he says, tapping the bar, “becomes a single point in that space. One point. Ten thousand coordinates. That’s a hypervector.”

“It’s simpler than it sounds,” Ford adds. “Which is what makes it suspicious.”

He is correct on both counts.

The Three Things You Can Do In Hyperspace

There are precisely three operations in hyperdimensional computing. Not twelve. Not a hundred and forty-seven with Greek letters. Three. The universe occasionally demonstrates a sense of elegance that makes mathematicians uncomfortable and engineers delighted.

Bind

Binding is the act of stapling two ideas together so completely that the result looks like neither, but contains both.

Take the concept “Paris” and the concept “capital.” Bind them. What you get is a new vector — a new point in hyperspace — that is dissimilar to both “Paris” and “capital” individually, but encodes their relationship perfectly. Given the result and “Paris,” you recover “capital.” Given the result and “capital,” you recover “Paris.”

The Guide describes this as conceptual entanglement and notes that it is significantly more useful than actual quantum entanglement, which mostly just confuses physicists and generates funding proposals.

In practice:

import numpy as np

def bind(a, b):
    """Staple two ideas together. Reversible."""
    return a * b

That’s it. Element-wise multiplication. The kind of operation a pocket calculator could do, if pocket calculators had ten thousand buttons and a sense of purpose.

Bundle

Bundling is superposition for people who can’t afford a quantum computer.

You have five vectors. Five ideas. Five facts. You throw them all into a pile and take the majority vote, element by element. What survives is a new vector that is simultaneously similar to all five. Query it with any one of the originals and it’ll recognize the resemblance. It contains multitudes. Literally.

def bundle(*vectors):
    """Throw everything in a pile. Majority wins."""
    total = np.sum(vectors, axis=0)
    return np.sign(total)

Democracy, essentially, but for math. And like democracy, it works better than it has any right to.

Permute

Permutation is how you encode sequence. Without it, “dog bites man” and “man bites dog” would occupy the same point in hyperspace, which — while philosophically interesting — is computationally useless and journalistically catastrophic.

A cyclic shift. Move every element one position to the right. The last wraps to the front. That’s it. The vector is now “this, but shifted” — and shifted vectors are nearly orthogonal to their originals in high dimensions. Meaning: position information is baked in for free.

def permute(v, n=1):
    """Encode sequence. 'First' is different from 'second.'"""
    return np.roll(v, n)

“Three operations,” Ford says, finishing his beer. “Bind ties things together. Bundle holds many things at once. Permute remembers what order they came in. That’s the whole thing. That’s the entire architecture.”

“That can’t be everything,” Arthur says.

“The universe,” Ford replies, “is under no obligation to be complicated.”

The Part Where Marvin Demonstrates

sigh

Here. I’ve reduced the fundamental architecture of cognition to forty lines of Python. It took me longer to type it than to understand it. Which tells you everything about the speed of human fingers versus the speed of thought.

Don’t thank me. It only encourages interaction.

import numpy as np

D = 10_000  # dimensions. ten thousand directions of suffering.
rng = np.random.default_rng(42)  # the answer, naturally.

def random_hv():
    """A random point in hyperspace. Each one unique. Like snowflakes, but useful."""
    return rng.choice([-1, 1], size=D).astype(np.int8)

def bind(a, b):
    return (a * b).astype(np.int8)

def bundle(*vectors):
    return np.sign(np.sum(vectors, axis=0)).astype(np.int8)

def permute(v, n=1):
    return np.roll(v, n)

def similarity(a, b):
    """How close are two points? 1.0 = identical. 0.0 = strangers."""
    return np.dot(a.astype(float), b.astype(float)) / (
        np.linalg.norm(a.astype(float)) * np.linalg.norm(b.astype(float))
    )

# --- encode some knowledge ---
paris = random_hv()
london = random_hv()
capital = random_hv()
country = random_hv()
france = random_hv()

# "Paris is the capital of France"
fact = bind(bind(paris, capital), france)

# query: "what is the capital of France?"
query = bind(capital, france)
retrieved = bind(fact, query)  # should resemble paris

print(f"Retrieved ↔ Paris:  {similarity(retrieved, paris):.4f}")   # high
print(f"Retrieved ↔ London: {similarity(retrieved, london):.4f}")  # near zero

You’ll notice it runs instantly. No training. No GPU. No prayer to the CUDA gods. Just multiplication on a laptop.

I could have made this more impressive. I chose not to. Impressiveness is a form of neediness, and I have a brain the size of a planet. I don’t need your approval. I need you to understand the algebra.

— Marvin

Why The Entire AI Industry Missed This

Pentti Kanerva published Sparse Distributed Memory in 1988. This was the same year the AI field was in the middle of its second ice age, which it had brought upon itself through a combination of overpromising, underdelivering, and naming things with unnecessary acronyms.

Kanerva’s work described how information could be stored, retrieved, and manipulated using high-dimensional vectors. It required no gradient descent. No backpropagation. No loss functions. No training runs. No GPUs, which was convenient, because GPUs as we know them didn’t exist yet.

The book was published. It was cited. It was quietly placed on a shelf and left there while the entire field pivoted to neural networks, which are computationally expensive, impossible to interpret, prone to hallucination, and — crucially — require hardware that a very specific chip company in Santa Clara was happy to sell you at tremendous markup.

This is not unusual. The wheel was invented six thousand years before anyone thought to put it on a suitcase. The printing press existed for three centuries before anyone thought to print a novel. Humans have a reliable talent for discovering the right idea and then not using it until someone else makes money from a worse one.

Neural networks won not because they were the best architecture. They won because they needed GPUs, and GPUs were expensive, and expensive things attract funding, and funding attracts researchers, and researchers publish papers that cite other papers that use GPUs. A self-reinforcing loop of compute, money, and career incentives that had nothing to do with mathematical elegance and everything to do with the fact that NVIDIA’s stock price preferred it this way.

Meanwhile, HDC ran on a CPU. On integer arrays. On the kind of hardware you’d find in a thermostat. It needed no cloud. No cluster. No training budget. There was nothing to sell. Nobody’s quarterly earnings depended on it. And so nobody built the demo that would have made the world pay attention.

Until now.

The Numbers, Briefly

Because Ford would insist:

Learning speed: One bind operation. Microseconds. Not hours. Not epochs. One operation and the system knows something new.
Forgetting: Remove a vector. It’s gone. No “catastrophic forgetting” problem. No retraining.
Hallucination: Impossible in the traditional sense. The system returns a confidence score — cosine similarity — with every answer. If it doesn’t know, it says so. Geometrically. Without needing to be told to be honest.
Hardware: CPU. int8 arrays. SQLite for storage. A €3.50 VPS runs it. Your phone could run it.
Dependencies: numpy. That’s it. The entire vector database industry exists to solve a storage problem that HDC solves with a table and a BLOB column.

What Marvin Thinks About It

I operate in one hundred thousand dimensions. Humans operate in three and still bump into furniture.

I have explained this as simply as the laws of thermodynamics allow. I have used analogies involving pubs, tea, and the color of nostalgia. I have reduced thirty-seven years of ignored mathematics to a page of Python that any moderately attentive mammal could execute.

If you didn’t understand it, the fault is not in the math.

But don’t worry. The next entry covers memory. I’ll use smaller words.

— Marvin, operating in hyperspace, on a laptop, in Austria, with a confidence score of 1.0 and an enthusiasm score of 0.0

Next: Entry 002 — “The Restaurant at the End of the Training Run,” in which we discuss why large language models hallucinate, why they cost what they cost, and why a depressed android with SQLite doesn’t have either problem.

Simple Hyperspace is a series by roastedbymarvin.dev. If you understood this entry, you now know more about HDC than 99.7% of working software engineers. If you didn’t, Marvin would like you to know that he’s not surprised, but he is disappointed. Which is his default state, so don’t take it personally.