Most non-technical founders — and a surprising number of software engineers — treat vector search like dark magic. They assume that to extract semantic context from text or build a reliable Retrieval-Augmented Generation (RAG) pipeline, they must deploy complex mathematical libraries, configure heavy CUDA workloads, or manage expensive vector database infrastructure.
Then they run into standard engineering friction: high monthly fees for managed vector databases before anyone understands how the data is actually queried, and a generic LLM wired to an unindexed context store that returns irrelevant results or hallucinates answers.
To design reliable AI systems, you have to strip away the abstractions and examine the underlying mechanics. We recently engineered a pure Python cosine similarity engine with no external dependencies — no numpy, no scikit-learn, and without Python’s native math module. Understanding this one calculation clarifies how modern AI search operates and how to evaluate your application stack.
The problem with keyword matching
Traditional lexical search algorithms evaluate strict character sequences. If a user searches for “async validation patterns”, standard database queries look for those exact words. If your technical documentation uses the phrase “non-blocking request verification”, a lexical search misses the record entirely.
Vector embeddings solve this mismatch by projecting text into high-dimensional numerical arrays (vectors) that capture conceptual meaning. Once text is represented as a sequence of numbers, the primary technical challenge becomes a different question: how do you measure which vectors are conceptually closest?
Lexical search measures characters. Vector search measures intent — and that difference is one geometry problem, not a machine learning one.
Distance vs. direction: why magnitude lies
When comparing two data points in space, developers often reach for Euclidean distance — the straight-line distance between two coordinates. In natural language processing, Euclidean distance breaks down because of document length variance.
Consider two documents: “FastAPI async validation” (3 words), and the same phrase repeated three times (9 words). Semantically they are about the same thing. But if you calculate Euclidean distance, the longer document appears far away, purely because its coordinate vector has a larger magnitude.
Cosine similarity resolves this by ignoring length and evaluating directional alignment. It calculates the cosine of the angle between two multi-dimensional vectors, so the score depends on where a document points, not how long it is.
- 1.0 (0° angle): Identical directional intent — the two texts are about the same thing.
- 0.0 (90° angle): Unrelated context — the vectors are orthogonal, sharing no conceptual direction.
- -1.0 (180° angle): Opposite meaning — the vectors point directly away from each other.
A · B
cosine_similarity(A, B) = ───────────
‖A‖ × ‖B‖
A · B = dot product → sum(A[i] * B[i])
‖A‖ = magnitude → sqrt(sum(A[i] * A[i]))Implementation from first principles: zero libraries
To demonstrate how lean vector comparisons really are, we implemented the entire pipeline using base Python primitives. Calculating a vector’s Euclidean magnitude requires a square root, so without math.sqrt we compute it using Newton’s iterative method — the Babylonian method — which converges on the root by repeatedly averaging a guess with n divided by that guess.
From there, the dot product multiplies corresponding dimensions and sums the products, the magnitude is the square root of the sum of squares, and cosine similarity is simply the dot product divided by the product of the two magnitudes. That is the whole engine.
def sqrt(n: float, tolerance: float = 1e-12) -> float:
if n <= 0:
return 0.0
x = n
while True:
root = 0.5 * (x + (n / x))
if abs(root - x) < tolerance:
return root
x = rootdef dot_product(vec_a: list[float], vec_b: list[float]) -> float:
return sum(a * b for a, b in zip(vec_a, vec_b))
def magnitude(vec: list[float]) -> float:
return sqrt(sum(x * x for x in vec))def cosine_similarity(vec_a: list[float], vec_b: list[float]) -> float:
mag_a = magnitude(vec_a)
mag_b = magnitude(vec_b)
if mag_a == 0.0 or mag_b == 0.0:
return 0.0
return dot_product(vec_a, vec_b) / (mag_a * mag_b)What this means for system architecture
Understanding vector mechanics at the primitive level leads to better production decisions, because you stop treating retrieval as a black box you can only tune by swapping vendors.
- RAG is deterministic context selection: Retrieval-Augmented Generation is not magic. It is the process of computing vector similarity scores across chunked text arrays to select the top-K relevant contexts before prompting an LLM. If retrieval returns the wrong chunks, no amount of prompt engineering fixes the answer.
- Vector databases are indexing engines: Pgvector, Qdrant, and Pinecone are specialized indexes designed to run these same similarity searches — via HNSW or IVFFlat — at scale across millions of records. You pay them for the index, not the arithmetic.
- Pre-normalize to cut query latency: Normalizing embedding vectors at ingestion time reduces every runtime cosine calculation to a single dot product, removing two magnitude computations from the hot path.
| Scale | What you actually need | Why |
|---|---|---|
| Hundreds of chunks | In-process cosine similarity over an array | A linear scan over a few hundred vectors is sub-millisecond. An external database adds a network hop and a bill for nothing. |
| Tens of thousands | Pgvector in your existing Postgres | You get indexed search, transactions, and backups without introducing a new piece of infrastructure to operate. |
| Millions and growing | A dedicated vector engine (Qdrant, Pinecone) | Approximate nearest-neighbor indexes, sharding, and filtered search become the actual bottleneck worth paying to solve. |
The takeaway
Before adopting complex infrastructure or third-party abstractions, inspect the underlying system mechanics. The core of semantic search is roughly twenty lines of arithmetic — everything on top of it is indexing, caching, and operations.
When you evaluate technical components from first principles — whether database indexes, system architectures, or mathematical operations — you build cleaner, more predictable software, and you buy infrastructure when your data demands it rather than when a pricing page suggests it.