Vectors are a key part of AI search. What are vectors exactly? At a high-level, vectors are a list of numbers that represent the meaning of something, like a word, sentence, or even a whole page. Okay, but what exactly are all of those numbers? Where do they come from? How do they work?
Think of a vector as a coordinate in a high-dimensional space. Instead of 2D (like a map with latitude and longitude) or 3D (like physical space), these vectors might live in 100, 300, or even 1,000+ dimensions. That space is called vector space. Vector space is where machine learning models organize information and where they extract patterns.
Each number can be thought of as relating to a learned feature. Conceptually, it is helpful to think of one number loosely correlating with a feature like “is this content about a product?” or “does this content relate to travel?”
Of course, the features are not this literal or human-readable. Each feature is an abstract concept that the model has learned from the training data. This is part of why AI systems are considered a black box. We don’t typically understand what features the model has selected or why. The model learns what matters most to distinguish meaning and selects features accordingly.
So, for example, a model represents words as a vector with 100 dimensions. Let’s say we pass the phrase “search engine” to that model. The model will convert the phrase into a vector with 100 different numbers. Each number describes some aspect of that phrase. So, we’ll end up with a vector that could look something like this:
[0.12, -0.8, 0.45, 0.0, …, 0.33]Different models will select different features and result in different vectors. The following screenshot shows two different vectors from two different models of the same phrase. One vector has 100 dimensions and the other has 384 dimensions.
So how are vectors used in AI search? How do vectors change AI search? I wrote a separate post digging into how vector search differs from traditional search.