Vector Databases Explained: How AI Stores and Searches Embeddings
Modern AI applications increasingly work with embeddings.
An embedding converts information such as text, images, or audio into a numerical vector that represents useful patterns and semantic relationships.
For example:
"How do I reset my password?"
can be converted into a vector such as:
[0.12, -0.43, 0.71, 0.08, ...]
If an application has thousands or millions of these vectors, it needs an efficient way to store them and find the vectors most similar to a user's query.
This is where vector databases become important.
A vector database is a database system designed to store, index, and search high-dimensional vectors efficiently.
In this article, we will explore:
What vector databases are
Why traditional databases are different
How vectors are stored
Similarity search
Distance metrics
Vector indexes
PostgreSQL with pgvector
Python examples
Semantic search
Metadata filtering
RAG systems
Hybrid search
Vector database architecture
Performance considerations
Practical limitations
What Is a Vector Database?
A vector database stores numerical vectors and provides efficient similarity search.
A simplified record might look like:
Document:
"Python is a programming language."
Embedding:
[0.12, -0.43, 0.71, 0.08, ...]
Instead of searching only for exact words, the database can search for vectors that are close to a query vector.
The basic workflow is:
User Query
↓
Embedding Model
↓
Query Vector
↓
Vector Database
↓
Similarity Search
↓
Relevant Results
This is the foundation of many AI-powered search systems.
Why Do We Need Vector Databases?
Traditional databases are excellent for structured queries.
For example:
SELECT *
FROM products
WHERE category = 'laptop';
This searches for an exact value.
But imagine asking:
"Find products similar to a lightweight computer
for programming."
The database needs to understand semantic relationships.
An embedding model can convert the query into a vector:
Query
↓
Embedding Model
↓
[0.21, -0.14, 0.72, ...]
The vector database then searches for nearby vectors.
This is called vector similarity search.
Traditional Database vs Vector Database
A traditional relational database stores structured information such as:
ID
Name
Price
Category
Date
A vector database stores information such as:
ID
Text
Embedding
Metadata
The two approaches solve different problems.
Traditional Search
"python tutorial"
may look for matching words.
Vector Search
"How can I start learning Python?"
can find content such as:
"Beginner's guide to Python programming"
even though the wording is different.
What Is a Vector?
A vector is simply a sequence of numbers.
For example:
[0.12, 0.45, -0.32]
Real embeddings are usually much larger:
[
0.123,
-0.452,
0.781,
0.091,
...
]
The number of values is called the vector dimension.
For example:
VECTOR(384)
means the vector contains 384 numerical values.
The dimension depends on the embedding model.
How Vector Search Works
Suppose we have three document embeddings:
Document A → [0.1, 0.2, 0.7]
Document B → [0.8, 0.1, 0.1]
Document C → [0.2, 0.3, 0.6]
The user query becomes:
Query → [0.15, 0.25, 0.65]
The vector database compares the query against the stored vectors.
Conceptually:
Query
|
+---- Document A → Similar
|
+---- Document B → Less Similar
|
+---- Document C → Very Similar
The database returns the closest vectors.
Similarity and Distance
Vector databases use mathematical metrics to determine how close vectors are.
Common metrics include:
Cosine similarity
Euclidean distance
Dot product
Manhattan distance in some specialized applications
The appropriate metric depends on the embedding model and use case.
Cosine Similarity
Cosine similarity measures the angle between two vectors.
The formula is:
cosine_similarity(A, B)
=
(A · B) / (||A|| × ||B||)
Conceptually:
Similarity close to 1
↓
Very similar direction
Similarity close to 0
↓
Less similar
Cosine similarity is widely used for text embeddings.
Euclidean Distance
Euclidean distance measures the straight-line distance between vectors.
For two vectors:
A = [a1, a2, a3]
B = [b1, b2, b3]
the distance is:
√((a1-b1)² + (a2-b2)² + (a3-b3)²)
Smaller distance generally means the vectors are closer.
Dot Product
The dot product is:
A · B
For:
A = [1, 2, 3]
B = [4, 5, 6]
we get:
1×4 + 2×5 + 3×6
which equals:
32
Some embedding systems are designed to work efficiently with dot-product similarity.
What Is a Vector Index?
Searching every vector individually becomes expensive when a database contains millions of vectors.
A simple approach would be:
Query
↓
Compare with Vector 1
↓
Compare with Vector 2
↓
Compare with Vector 3
↓
...
↓
Compare with Vector 1,000,000
This is called a brute-force or exact search approach.
It can become expensive at large scale.
Vector indexes improve search performance by organizing vectors so that nearby candidates can be found more efficiently.
Approximate Nearest Neighbor Search
Many vector databases use Approximate Nearest Neighbor (ANN) algorithms.
Instead of guaranteeing an exhaustive comparison against every vector, ANN methods efficiently search for highly similar candidates.
Common indexing approaches include:
HNSW
IVF
Product Quantization
Disk-based ANN techniques
The exact implementation varies between databases.
The trade-off is generally:
More search accuracy
↕
More computation
versus:
Faster search
↕
Potentially approximate results
HNSW
One popular vector indexing technique is Hierarchical Navigable Small World, commonly abbreviated as HNSW.
HNSW organizes vectors into graph-like layers.
Conceptually:
Layer 2
A -------- B
\ /
\ /
\ /
C
Layer 1
A -- B -- C -- D -- E
The search can move through the graph rather than comparing against every vector.
HNSW is widely used for approximate nearest-neighbor search.
Vector Database Architecture
A simplified vector database looks like:
Application
|
↓
Search Request
|
↓
┌────────────────────┐
│ Vector Database │
└────────────────────┘
|
┌────────────┼────────────┐
↓ ↓ ↓
Vector Data Metadata Index
|
↓
Similarity Search
|
↓
Top-K Results
The vector database manages both vector storage and efficient retrieval.
PostgreSQL + pgvector
You do not always need a separate specialized vector database.
PostgreSQL can support vector search through the pgvector extension.
This is especially useful when your application already uses PostgreSQL.
The architecture becomes:
Application
↓
PostgreSQL
├── Normal relational data
├── Metadata
└── Embeddings
This can simplify application architecture.
Creating a Vector Table
With pgvector installed, a table can be created like:
CREATE EXTENSION IF NOT EXISTS vector;
Then:
CREATE TABLE documents (
id BIGSERIAL PRIMARY KEY,
content TEXT NOT NULL,
embedding VECTOR(384)
);
Here:
VECTOR(384)
means that each embedding must contain 384 dimensions.
The dimension must match the embedding model being used.
Insert a Vector
A vector can be stored alongside document content.
INSERT INTO documents (
content,
embedding
)
VALUES (
'Python is a popular programming language.',
'[0.12, -0.43, 0.71, ...]'
);
In production applications, embeddings are normally generated programmatically rather than manually entered.
Generate Embeddings With Python
Install:
pip install sentence-transformers
Then:
from sentence_transformers import SentenceTransformer
model = SentenceTransformer(
"all-MiniLM-L6-v2"
)
text = "Python is a programming language."
embedding = model.encode(text)
print(embedding)
print("Dimensions:", len(embedding))
The generated vector can then be stored in a vector database.
Store Embeddings With Python
For PostgreSQL, you can use a database driver such as psycopg.
Install:
pip install psycopg
A simplified example:
import psycopg
from sentence_transformers import SentenceTransformer
model = SentenceTransformer(
"all-MiniLM-L6-v2"
)
text = "Python is a programming language."
embedding = model.encode(text).tolist()
connection = psycopg.connect(
"postgresql://user:password@localhost/mydb"
)
with connection.cursor() as cursor:
cursor.execute(
"""
INSERT INTO documents (
content,
embedding
)
VALUES (%s, %s)
""",
(text, embedding)
)
connection.commit()
connection.close()
In a real application, use environment variables for database credentials rather than putting passwords directly into source code.
Searching a Vector Database
Suppose the user asks:
How can I learn Python?
The query is converted into an embedding.
Then PostgreSQL can search for nearby vectors.
With pgvector, a query can look conceptually like:
SELECT
id,
content
FROM documents
ORDER BY embedding <=> '[QUERY_VECTOR]'
LIMIT 5;
The <=> operator is used by pgvector for cosine distance.
The exact operator should match the distance metric and index you have chosen.
What Does Top-K Mean?
Vector searches commonly return the top K results.
For example:
LIMIT 5
means:
Return the 5 best matching results.
The workflow is:
Query
↓
Vector Search
↓
Rank Results
↓
Top 5
For example:
1. Python beginner guide
2. Python programming tutorial
3. Introduction to Python
4. Python syntax reference
5. Python development guide
Metadata Filtering
Vector search becomes more useful when combined with metadata.
Imagine storing articles with:
title
category
language
date
embedding
A search can be restricted to a category.
For example:
SELECT
id,
content
FROM documents
WHERE category = 'programming'
ORDER BY embedding <=> '[QUERY_VECTOR]'
LIMIT 5;
This combines:
Metadata Filtering
+
Vector Similarity
Why Metadata Matters
Imagine an e-commerce website containing:
10 million products
A user searches:
"lightweight laptop for programming"
The application could filter by:
category = laptop
price < 1500
availability = true
and then perform semantic vector search among the remaining products.
This can improve both relevance and efficiency.
Semantic Search With a Vector Database
A complete semantic search system looks like:
Documents
↓
Chunking
↓
Embedding Model
↓
Vector Database
↓
Vector Index
↓
User Query
↓
Query Embedding
↓
Similarity Search
↓
Top-K Results
This architecture can power AI search systems.
Vector Databases and RAG
Vector databases are especially important for Retrieval-Augmented Generation, or RAG.
RAG allows an LLM to retrieve relevant information before generating an answer.
The architecture is:
Documents
↓
Chunking
↓
Embeddings
↓
Vector Database
↓
Stored Vectors
User Question
↓
Question Embedding
↓
Vector Search
↓
Relevant Documents
↓
Prompt + Retrieved Context
↓
LLM
↓
Generated Answer
The vector database acts as the retrieval layer.
Example RAG System
Suppose a company has:
Employee Handbook
Product Documentation
Technical Manuals
Support Articles
Company Policies
A user asks:
How many vacation days do employees receive?
The system does not need to send every document to the LLM.
Instead:
Question
↓
Embedding
↓
Vector Search
↓
Vacation Policy
↓
LLM
↓
Answer
This can reduce unnecessary context and improve retrieval.
Chunking Documents
Large documents are usually divided into smaller chunks before creating embeddings.
For example:
Large PDF
↓
Page extraction
↓
Paragraph splitting
↓
Chunk 1
Chunk 2
Chunk 3
...
Each chunk receives an embedding.
A record may contain:
{
"content": "Employees receive 20 days...",
"embedding": [...],
"document_id": 42,
"page": 15
}
Metadata makes it possible to identify where the retrieved information came from.
Example Chunking Code
A simple token-based chunker can be created like this:
def chunk_text(text, chunk_size=500):
words = text.split()
chunks = []
for i in range(
0,
len(words),
chunk_size
):
chunk = " ".join(
words[i:i + chunk_size]
)
chunks.append(chunk)
return chunks
text = """
This is a large document containing
many paragraphs and sections.
"""
chunks = chunk_text(text)
for chunk in chunks:
print(chunk)
print("---")
Production systems often use smarter chunking strategies based on paragraphs, sentences, headings, token counts, or semantic boundaries.
Storing Metadata
A useful vector database record might contain:
id
document_id
title
content
embedding
category
author
page_number
created_at
For example:
document = {
"document_id": 1001,
"title": "Python Tutorial",
"content": "Python is...",
"category": "programming",
"page_number": 5,
"embedding": embedding
}
This information can be returned together with search results.
Vector Search in Python
A simple in-memory vector search can be built before introducing a database.
from sentence_transformers import SentenceTransformer
from sklearn.metrics.pairwise import cosine_similarity
model = SentenceTransformer(
"all-MiniLM-L6-v2"
)
documents = [
"Python is a programming language.",
"FastAPI is a framework for APIs.",
"PostgreSQL is a relational database.",
"React is used for web interfaces."
]
document_embeddings = model.encode(
documents
)
query = "How can I build a Python API?"
query_embedding = model.encode(
[query]
)
scores = cosine_similarity(
query_embedding,
document_embeddings
)[0]
results = sorted(
zip(documents, scores),
key=lambda x: x[1],
reverse=True
)
for document, score in results:
print(
f"{score:.4f} - {document}"
)
This demonstrates the basic idea behind vector database search.
A production database adds persistent storage, indexes, filtering, concurrency, and scalable retrieval.
Exact Search vs Approximate Search
There are two broad approaches to vector search.
Exact Search
Compare the query against every stored vector.
Query
↓
Vector 1
Vector 2
Vector 3
...
Vector N
Advantages:
Exact
Simple
Good for smaller datasets
Disadvantages:
Can become expensive at very large scale
Approximate Search
Use an index to efficiently find likely nearest neighbors.
Query
↓
Vector Index
↓
Candidate Vectors
↓
Top Results
Advantages:
Much faster at large scale
Disadvantages:
Results may be approximate
Index configuration affects performance
Choosing a Vector Database
Popular options include:
PostgreSQL + pgvector
Pinecone
Qdrant
Weaviate
Milvus
Chroma
The best option depends on your project.
For a small application already using PostgreSQL, pgvector can be an attractive choice.
For specialized large-scale vector workloads, dedicated vector databases may provide more specialized capabilities.
PostgreSQL vs Dedicated Vector Database
PostgreSQL + pgvector
Advantages:
Existing PostgreSQL ecosystem
Relational data and vectors in one database
SQL queries
Metadata filtering
Convenient for many applications
Potential limitation:
Very large vector workloads may require specialized architecture and tuning
Dedicated Vector Database
Advantages:
Designed specifically around vector workloads
Specialized indexing and retrieval
Can provide scalable vector-search infrastructure
Potential limitation:
Adds another service to your architecture
May increase operational complexity and cost
There is no universal winner.
Hybrid Search
Vector search is powerful, but keyword search is still useful.
Consider a query containing an exact product code:
"Product ABC-123"
A keyword search may be extremely useful because the exact identifier matters.
A semantic search system may focus more on meaning.
A hybrid search system combines both.
User Query
↓
┌────────┴────────┐
↓ ↓
Keyword Search Vector Search
↓ ↓
└────────┬────────┘
↓
Result Ranking
↓
Final Results
This can provide stronger retrieval for many real-world applications.
Reranking
A vector database may return the top 20 candidate documents.
A reranking model can then evaluate those candidates more carefully.
User Query
↓
Vector Search
↓
Top 20 Candidates
↓
Reranker
↓
Top 5 Results
This two-stage architecture is common in sophisticated search systems.
Vector Database Performance
Performance depends on several factors:
Number of vectors
Vector dimensions
Index type
Distance metric
Hardware
Filtering
Query volume
Database configuration
Number of returned results
For example:
10,000 vectors
may be easy to search with brute force.
But:
100,000,000 vectors
requires much more careful architecture.
Memory Usage
Vectors can consume significant storage.
Suppose you have:
1,000,000 vectors
and each vector has:
1,536 dimensions
If each dimension is stored as a 32-bit floating-point number:
1,536 × 4 bytes
=
6,144 bytes
per vector, ignoring additional index and metadata overhead.
For one million vectors:
≈ 6.14 GB
just for the raw vector values.
Indexes and metadata require additional storage.
This demonstrates why vector storage needs to be planned carefully.
Reducing Vector Storage
Some systems use techniques such as:
Quantization
Lower-precision data types
Product quantization
Dimensionality reduction
These can reduce storage and improve performance.
However, reducing precision or dimensionality can affect retrieval quality.
There is usually a trade-off between:
Storage
Performance
Accuracy
Vector Database Security
Vector databases should be treated like any other production database.
Important considerations include:
Authentication
Authorization
Encryption
Network security
Access control
Tenant isolation
Backup
Monitoring
This becomes especially important when embeddings contain information derived from private documents.
An embedding should not automatically be treated as harmless just because it is represented as numbers.
Multi-Tenant Vector Databases
SaaS applications may have multiple customers.
For example:
Company A
├── Documents
└── Embeddings
Company B
├── Documents
└── Embeddings
A search request from Company A must never accidentally retrieve Company B's data.
Metadata filtering can help:
SELECT *
FROM documents
WHERE tenant_id = 123
ORDER BY embedding <=> '[QUERY_VECTOR]'
LIMIT 5;
Proper authorization must still be enforced at the application and database levels.
Common Vector Database Mistakes
Mistake 1: Using the Wrong Embedding Dimension
If the model produces 384 dimensions:
VECTOR(384)
must be used.
Do not mix vectors from incompatible models without a deliberate strategy.
Mistake 2: Ignoring Metadata
Only storing:
embedding
makes it difficult to understand where results came from.
Store useful metadata such as:
document_id
title
category
page
source
Mistake 3: Poor Chunking
Huge chunks may contain too much unrelated information.
Tiny chunks may lose important context.
Chunk size should be tested against your data and retrieval task.
Mistake 4: Returning Too Many Results
Retrieving 100 irrelevant documents does not necessarily improve an LLM response.
Start with a reasonable top-K value and evaluate retrieval quality.
Mistake 5: Treating Similarity as Truth
A high similarity score means the vectors are close according to the chosen metric.
It does not guarantee that the retrieved content is correct.
Practical Vector Search Project
Let's build a small knowledge-search system.
Install:
pip install sentence-transformers scikit-learn
Create:
vector_search.py
Then:
from sentence_transformers import SentenceTransformer
from sklearn.metrics.pairwise import cosine_similarity
model = SentenceTransformer(
"all-MiniLM-L6-v2"
)
documents = [
{
"title": "Python",
"content": "Python is a popular programming language."
},
{
"title": "FastAPI",
"content": "FastAPI is a framework for building APIs with Python."
},
{
"title": "PostgreSQL",
"content": "PostgreSQL is an open-source relational database."
},
{
"title": "React",
"content": "React is a JavaScript library for building user interfaces."
}
]
texts = [
document["content"]
for document in documents
]
embeddings = model.encode(
texts
)
query = input(
"Enter your question: "
)
query_embedding = model.encode(
[query]
)
scores = cosine_similarity(
query_embedding,
embeddings
)[0]
results = sorted(
zip(documents, scores),
key=lambda x: x[1],
reverse=True
)
print("\nSearch Results:\n")
for document, score in results[:3]:
print(
f"Score: {score:.4f}"
)
print(
f"Title: {document['title']}"
)
print(
f"Content: {document['content']}"
)
print()
Try:
How can I build an API using Python?
The FastAPI document should be among the highest-ranked results.
From Prototype to Production
The previous example stores vectors in memory.
A production system can replace the in-memory list with:
PostgreSQL + pgvector
or another vector database.
The architecture becomes:
Web / Mobile App
↓
Backend
↓
Embedding Model
↓
Vector Database
↙ ↘
Vector Index Metadata
↓
Search Results
↓
LLM
↓
Response
This architecture can scale far beyond a simple Python script.
Complete AI Search Architecture
A production AI knowledge platform might use:
┌───────────────┐
│ Documents │
└───────┬───────┘
↓
Text Parser
↓
Chunking
↓
Embedding Model
↓
┌────────────────┐
│ Vector Database│
└───────┬────────┘
│
│
User → Query → Embedding → Search
↓
Retrieved Chunks
↓
Reranker
↓
Context Builder
↓
LLM
↓
AI Response
This architecture is commonly used for enterprise search, documentation assistants, customer-support systems, and RAG applications.
When Should You Use a Vector Database?
A vector database is useful when your application needs to:
Search by semantic meaning
Store large numbers of embeddings
Find similar documents
Build RAG systems
Build recommendation systems
Search images or multimedia
Cluster related content
Perform similarity matching
You may not need one for a simple application containing only a few hundred vectors.
For small datasets, an in-memory index or a regular database extension may be sufficient.
Final Summary
Vector databases provide the storage and search infrastructure required for many modern AI applications.
The basic process is:
Text
↓
Embedding Model
↓
Vector
↓
Vector Database
↓
Vector Index
↓
Similarity Search
↓
Relevant Results
For RAG:
Documents
↓
Chunking
↓
Embeddings
↓
Vector Database
↓
User Question
↓
Query Embedding
↓
Similarity Search
↓
Relevant Context
↓
LLM
↓
Answer
The most important concepts to remember are:
Vector
↓
Numerical representation of data
Vector Database
↓
Database designed to store and search vectors
Similarity Search
↓
Find vectors close to a query vector
Vector Index
↓
Makes large-scale vector search faster
Metadata
↓
Additional information used for filtering and results
RAG
↓
Retrieval + Context + LLM
Hybrid Search
↓
Keyword Search + Vector Search
Vector databases are an important part of the modern AI stack. They provide the connection between embeddings and useful applications, allowing developers to build semantic search engines, document assistants, recommendation systems, AI knowledge bases, and RAG-powered applications.
Once you understand embeddings, vector similarity, indexing, and retrieval, you have the foundation needed to build many of today's practical AI search systems.