Once meaning has been turned into numbers, those numbers have to live somewhere, and you have to be able to ask that somewhere a particular kind of question: not “find me the row with this id”, but “find me the handful of things most like this”. A vector database is storage built around that second question.
Picture a library where nothing is filed alphabetically. Instead, every book sits near the books it is most like. You walk in holding a book, say “something along these lines”, and the librarian walks you to the right shelf without ever reading the title you came in with.
In short
- What
- A store for embeddings and the text they came from, built to answer “what are the nearest few to this?” across very large collections.
- Why
- Finding nearest neighbours by checking every item works fine for thousands and falls apart for millions. These databases are the index that makes it fast.
- When
- When you have a lot of embeddings, they change often, and something user-facing is waiting on the answer.
- How
- Store the source text and where it came from beside every vector, filter on ordinary fields as well as distance, and keep it in step with the source.
Why an ordinary database struggles
Regular databases are superb at exact and ranged questions: this id, this date range, this status. They get their speed from indexes that rely on being able to sort things into a sensible order.
Nearest-neighbour questions break that assumption. There is no single order that puts every point near all of its neighbours when a point has hundreds of dimensions. So the obvious approach is to measure the distance to every stored item and keep the closest — which is exactly right, and exactly as slow as it sounds once there are millions.
Vector databases use indexes designed for this shape of problem. Most of them are approximate: they give up a very small amount of accuracy — occasionally missing a result that was technically the fifth-closest — in exchange for being orders of magnitude faster. For search, that is almost always the right trade. It is worth knowing you made it, though.
When you actually need one
Honestly: later than people think. For a few thousand items, holding the vectors in memory and comparing against all of them is simple, exact, fast enough, and has no extra system to run. Plenty of useful features never outgrow that.
Several ordinary databases can now store vectors and search them directly, which is often the better next step — one system instead of two, and your vectors sit beside the data they describe rather than in a separate place that can drift out of step.
A dedicated vector database earns its place when the collection is genuinely large, when it changes constantly, when many queries arrive at once, or when you need the operational things a database gives you: backups, replication, access control, someone else’s problem at three in the morning.
How to use one well
Store the text and its source beside every vector. You will need to show people where an answer came from, and a bare vector cannot tell you.
Use ordinary fields as well as distance. “Nearest to this question, in this city, from a listing that is still live” is a far better query than nearest alone, and it is usually the filter, not the similarity, that saves you from an embarrassing result.
Keep it in step with the source. A vector store is a copy, and copies go stale. When a document changes, its vectors have to change; when it is deleted, they have to go. Deciding how that happens is part of building the thing, not an afterthought.
Measure how often the right answer actually comes back. Write down thirty real questions and the passages that should be found, and check. Approximate indexes have settings that trade speed against accuracy, and without a measurement you are tuning blind.
Where it stops
A vector database returns things that are similar. It has no opinion on whether they are true, current or relevant to what the person actually meant, and it will return its nearest matches just as cheerfully when nothing useful exists at all.
Deciding what to do with those results — which to keep, how to present them, whether to answer at all — belongs to the system around it. When they are being fed to a model to answer a question, that arrangement has a name: retrieval-augmented generation.
