Embeddings Explained: How AI Understands Meaning
Traditional computer programs work very well with exact values and predefined rules. However, human language is much more complicated.
Consider these two sentences:
I bought a new laptop.
and:
I purchased a brand-new computer.
The words are different, but the meaning is very similar.
A traditional keyword search may treat them as completely different because they do not contain the same words.
AI systems can solve this problem using embeddings.
Embeddings convert text, images, audio, or other data into numerical vectors that capture useful patterns of meaning and relationships.
In this article, we will explore:
What embeddings are
How embeddings represent meaning
Vectors and dimensions
Text embeddings
Semantic similarity
Cosine similarity
Vector databases
Semantic search
Retrieval-Augmented Generation (RAG)
Creating embeddings with Python
Building a simple semantic search engine
Embeddings for documents
Embeddings for recommendations
Common embedding use cases
Practical limitations
What Is an Embedding?
An embedding is a numerical representation of data.
For example, a sentence such as:
Python is a programming language.
can be converted into a vector:
[
0.021,
-0.183,
0.742,
0.091,
...
]
The actual embedding usually contains many dimensions.
Instead of processing the sentence directly as text, an AI application can work with the numerical representation.
Conceptually:
Text
↓
Embedding Model
↓
Numerical Vector
↓
[0.21, -0.42, 0.73, ...]
This vector can then be compared with other vectors.
Why Are Embeddings Important?
Consider these sentences:
The car is very fast.
and:
The vehicle has a high speed.
They use different words but have similar meanings.
An embedding model can produce vectors that are relatively close to each other.
Conceptually:
Sentence A
↓
[0.21, 0.48, -0.11, ...]
↘
Similar
↗
Sentence B
↓
[0.20, 0.46, -0.09, ...]
This allows AI systems to perform semantic search instead of relying only on exact keyword matches.
What Is a Vector?
A vector is simply a list of numbers.
For example:
[0.2, 0.7, -0.1]
is a three-dimensional vector.
An embedding could contain hundreds or thousands of numerical values.
For example:
[
0.123,
-0.452,
0.781,
0.092,
0.314,
...
]
Each dimension contributes to the representation learned by the embedding model.
However, it is important not to interpret one individual dimension as a simple human-readable concept such as "happiness" or "technology." Meaning is generally distributed across many dimensions.
Embeddings and Meaning
The key idea behind embeddings is that semantically related data tends to have similar representations.
For example:
Dog
Puppy
Canine
are related concepts.
Their embeddings may be relatively close in vector space.
Likewise:
Car
Automobile
Vehicle
may form another group.
Conceptually:
Dog
/ \
Puppy Canine
Car
/ \
Automobile Vehicle
The actual embedding space is much higher-dimensional than this simple diagram.
Embedding Space
Imagine a two-dimensional coordinate system.
Y
↑
| Dog
| Puppy
|
|
| Car
| Vehicle
|
|________________________________→ X
Related concepts tend to occupy nearby regions.
In a real embedding model, there may be hundreds or thousands of dimensions rather than just two.
We can visualize only two or three dimensions by applying dimensionality-reduction techniques.
Text Embeddings
Text embeddings convert text into vectors.
For example:
"I love programming."
becomes:
[0.13, -0.24, 0.81, ...]
Another sentence:
"I enjoy writing code."
might produce another vector:
[0.15, -0.21, 0.79, ...]
The vectors may be close because the sentences express similar ideas.
Similarity Between Embeddings
Once text has been converted into vectors, we can calculate how similar two vectors are.
One commonly used method is cosine similarity.
The cosine similarity between two vectors is:
cosine_similarity(A, B)
=
(A · B) / (||A|| ||B||)
The result is commonly interpreted as:
Closer to 1
↓
More similar direction
Closer to 0
↓
Less similar
Negative values
↓
Opposite directions in some embedding spaces
The exact interpretation depends on the embedding model and how the vectors are used.
Calculate Cosine Similarity With Python
Install NumPy:
pip install numpy
Then:
import numpy as np
vector_a = np.array([1, 2, 3])
vector_b = np.array([1, 2, 3])
similarity = np.dot(
vector_a,
vector_b
) / (
np.linalg.norm(vector_a) *
np.linalg.norm(vector_b)
)
print("Similarity:", similarity)
Because the vectors point in the same direction, the similarity will be very close to:
1.0
Compare Different Vectors
import numpy as np
vector_a = np.array([1, 2, 3])
vector_b = np.array([1, 2, 2])
dot_product = np.dot(
vector_a,
vector_b
)
norm_a = np.linalg.norm(vector_a)
norm_b = np.linalg.norm(vector_b)
similarity = dot_product / (
norm_a * norm_b
)
print(similarity)
The result will be lower than 1 because the vectors are not identical.
Using Scikit-Learn
Cosine similarity can also be calculated using Scikit-learn.
Install:
pip install scikit-learn
Then:
from sklearn.metrics.pairwise import cosine_similarity
vector_a = [[1, 2, 3]]
vector_b = [[1, 2, 2]]
similarity = cosine_similarity(
vector_a,
vector_b
)
print(similarity)
This is useful when working with many vectors.
Creating Text Embeddings
An embedding model takes text as input and returns a vector.
Conceptually:
Text
↓
Embedding Model
↓
Vector
For example:
"Machine learning is a branch of AI."
might become:
[
0.012,
-0.143,
0.721,
0.332,
...
]
The actual vector values depend on the embedding model.
Sentence Transformers
One popular approach for generating embeddings locally is the sentence-transformers library.
Install it:
pip install sentence-transformers
Then:
from sentence_transformers import SentenceTransformer
model = SentenceTransformer(
"all-MiniLM-L6-v2"
)
text = "Machine learning is useful."
embedding = model.encode(text)
print(embedding)
You can check its size:
print(
"Embedding dimensions:",
len(embedding)
)
The exact dimensionality depends on the selected model.
Generate Multiple Embeddings
We can generate embeddings for multiple sentences.
from sentence_transformers import SentenceTransformer
model = SentenceTransformer(
"all-MiniLM-L6-v2"
)
sentences = [
"I love programming.",
"I enjoy writing code.",
"The weather is sunny today."
]
embeddings = model.encode(sentences)
print(
"Number of embeddings:",
len(embeddings)
)
print(
"Embedding dimensions:",
len(embeddings[0])
)
Each sentence receives its own vector.
Compare Sentence Meaning
Now let's compare semantic similarity.
from sentence_transformers import SentenceTransformer
from sklearn.metrics.pairwise import cosine_similarity
model = SentenceTransformer(
"all-MiniLM-L6-v2"
)
sentences = [
"I love programming.",
"I enjoy writing code.",
"The weather is sunny today."
]
embeddings = model.encode(sentences)
similarity = cosine_similarity(
embeddings
)
print(similarity)
The first two sentences should generally have a stronger semantic relationship than the first and third sentences.
This demonstrates an important advantage of embeddings.
The system does not need the sentences to use exactly the same words.
Keyword Search vs Semantic Search
Traditional keyword search:
Query:
"How to fix my computer?"
may prioritize documents containing:
computer
fix
A semantic search system can also identify documents containing:
How to troubleshoot a laptop
even though the word "computer" does not appear.
This is possible because embeddings represent semantic relationships.
Semantic Search
A semantic search system works like this:
User Query
↓
Embedding Model
↓
Query Vector
↓
Vector Database
↓
Similarity Search
↓
Relevant Documents
For example:
Query:
"How can I repair my laptop?"
The system might retrieve:
1. Laptop troubleshooting guide
2. Computer repair instructions
3. Hardware diagnostic documentation
even if those documents do not contain the exact words from the query.
Building a Simple Semantic Search Engine
Let's create a small semantic search application.
Install:
pip install sentence-transformers scikit-learn
Create:
semantic_search.py
Use:
from sentence_transformers import SentenceTransformer
from sklearn.metrics.pairwise import cosine_similarity
model = SentenceTransformer(
"all-MiniLM-L6-v2"
)
documents = [
"Python is a popular programming language.",
"JavaScript is commonly used for web development.",
"Machine learning allows computers to learn from data.",
"Neural networks are used in many AI applications.",
"SQL is used to work with relational databases."
]
document_embeddings = model.encode(
documents
)
query = "How do computers learn from information?"
query_embedding = model.encode(
[query]
)
scores = cosine_similarity(
query_embedding,
document_embeddings
)[0]
ranked_results = sorted(
zip(documents, scores),
key=lambda x: x[1],
reverse=True
)
for document, score in ranked_results:
print(
f"{score:.4f} - {document}"
)
The machine-learning document should rank highly because its meaning is related to the query.
Understanding the Search Process
The application performs several steps.
First, the documents are converted into vectors:
Document
↓
Embedding Model
↓
Vector
Then the query is converted into a vector:
Query
↓
Embedding Model
↓
Vector
Then the query vector is compared with document vectors.
Query
↓
Query Vector
↓
┌──────────┼──────────┐
↓ ↓ ↓
Vector A Vector B Vector C
↓ ↓ ↓
0.91 0.32 0.75
The highest similarity results are returned.
Ranking Search Results
Instead of returning documents randomly, we sort them by similarity.
ranked_results = sorted(
zip(documents, scores),
key=lambda x: x[1],
reverse=True
)
Then:
for document, score in ranked_results:
print(
f"Score: {score:.4f}"
)
print(document)
print()
The most relevant documents appear first.
Embeddings for Documents
Embeddings become especially useful when working with large collections of documents.
Imagine a company has:
10,000 PDF files
50,000 support articles
100,000 product descriptions
Searching every document using exact keywords may not provide the best results.
Instead:
Documents
↓
Chunking
↓
Embedding Model
↓
Vectors
↓
Vector Database
When a user asks a question:
Question
↓
Embedding
↓
Vector Search
↓
Relevant Documents
This architecture is widely used in modern AI applications.
What Is a Vector Database?
A vector database stores embeddings and allows applications to search for similar vectors efficiently.
Instead of storing only:
Document Text
a vector database can store:
Document
+
Embedding
+
Metadata
For example:
{
"text": "Python is a programming language.",
"embedding": [0.12, -0.43, ...],
"category": "programming"
}
Popular Vector Database Options
Common technologies used for vector search include:
PostgreSQL with pgvector
Pinecone
Weaviate
Qdrant
Milvus
Chroma
The right choice depends on the application's scale, deployment environment, cost, and existing architecture.
Embeddings and RAG
Embeddings are one of the key components of Retrieval-Augmented Generation (RAG).
A simplified RAG architecture looks like:
Documents
↓
Chunking
↓
Embeddings
↓
Vector Database
↓
Search
↑
|
User Question → Embedding
↓
Relevant Chunks
↓
LLM
↓
Answer
The embedding system finds relevant information.
The LLM uses that information to generate the response.
Example RAG Workflow
Imagine a company has a document:
Employee Handbook
It contains:
Vacation Policy
Remote Work Policy
Expense Policy
Security Policy
A user asks:
How many vacation days can I take?
The system converts the question into an embedding.
It searches the vector database.
The relevant section might be:
Employees receive 20 days
of annual vacation leave.
That information is then passed to the LLM.
The LLM generates:
Employees receive 20 days
of annual vacation leave.
This is much more efficient than sending the entire employee handbook to the LLM.
Embeddings for Recommendations
Embeddings are not limited to search.
They can also be used for recommendation systems.
Suppose users like these products:
Laptop
Wireless Mouse
Mechanical Keyboard
Monitor
Products can be converted into embeddings.
Products with similar characteristics can occupy nearby regions in vector space.
For example:
Laptop
/ \
Monitor Keyboard
|
Mouse
A recommendation engine can find products similar to the user's interests.
Image Embeddings
Embeddings can also represent images.
For example:
Image
↓
Vision Embedding Model
↓
Vector
Similar images may produce similar vectors.
This can be used for:
Image search
Image recommendations
Duplicate detection
Visual classification
Content discovery
Audio Embeddings
Audio can also be converted into embeddings.
For example:
Audio
↓
Audio Embedding Model
↓
Vector
Possible applications include:
Audio search
Speaker analysis
Music recommendation
Sound classification
Voice similarity
The same general concept applies:
Data
↓
Embedding Model
↓
Vector Representation
↓
Similarity Search
Multimodal Embeddings
Some AI systems can represent different types of data in compatible embedding spaces.
For example:
Text
↓
Embedding
and:
Image
↓
Embedding
If the model is designed for multimodal alignment, related text and images can be compared in the same or compatible vector space.
For example:
Text:
"Golden retriever playing outside"
could be matched with a related image.
This enables powerful applications such as multimodal search.
Embeddings for Duplicate Detection
Embeddings can help identify documents with similar meanings.
Suppose a website contains:
Article A:
How to learn Python
Article B:
A beginner's guide to learning Python
Keyword matching may show some overlap.
Embedding similarity can provide a more semantic comparison.
Example:
similarity = cosine_similarity(
[embedding_a],
[embedding_b]
)
print(similarity)
A high similarity score can indicate that the documents discuss closely related content.
However, similarity scores should be treated as signals rather than absolute proof that two documents are duplicates.
Embeddings for Question Matching
Embeddings are useful for customer support systems.
Suppose the database contains:
How do I reset my password?
How can I change my account password?
Where can I update my password?
A user asks:
I forgot my password. How can I change it?
Even though the exact wording is different, the semantic meaning may be similar.
An embedding-based system can retrieve the relevant support article.
Embeddings for Chatbots
A chatbot can use embeddings to search a knowledge base.
Architecture:
USER
↓
QUESTION
↓
EMBEDDING
↓
VECTOR SEARCH
↓
RELEVANT DATA
↓
LLM
↓
ANSWER
This is one of the most common practical applications of embeddings.
Store Embeddings in a Database
A simple database record might contain:
id
title
content
embedding
category
created_at
For example:
document = {
"id": 1,
"title": "Python Tutorial",
"content": "Python is a programming language...",
"embedding": [0.12, -0.45, 0.73],
"category": "programming"
}
In a real application, the vector would contain many more dimensions.
Example With PostgreSQL and pgvector
PostgreSQL can be extended with pgvector to store and search embeddings.
A simplified table might look like:
CREATE TABLE documents (
id BIGSERIAL PRIMARY KEY,
content TEXT,
embedding VECTOR(384)
);
The dimension:
384
must match the output dimension of the embedding model being used.
For example, a model producing 384-dimensional embeddings requires:
VECTOR(384)
Insert an Embedding
Conceptually:
INSERT INTO documents (
content,
embedding
)
VALUES (
'Python is a programming language.',
'[0.12, -0.43, 0.72, ...]'
);
The exact syntax depends on the database client and vector representation.
Search Similar Embeddings
A vector database can calculate distances between vectors.
Conceptually:
SELECT
content
FROM documents
ORDER BY embedding <=> '[query vector]'
LIMIT 5;
The exact operator and interpretation depend on the vector index and similarity metric being used.
The result is a list of documents whose vectors are closest to the query vector.
Similarity Metrics
Different applications can use different distance or similarity metrics.
Common choices include:
Cosine Similarity
Measures the angle between vectors.
cosine(A, B)
Euclidean Distance
Measures straight-line distance:
distance(A, B)
Dot Product
Calculates:
A · B
The best metric depends on the embedding model and application.
Cosine Similarity Example
from sklearn.metrics.pairwise import cosine_similarity
query = [[0.2, 0.4, 0.7]]
documents = [
[0.2, 0.4, 0.6],
[0.9, 0.1, 0.2],
[0.1, 0.3, 0.8]
]
scores = cosine_similarity(
query,
documents
)[0]
for i, score in enumerate(scores):
print(
f"Document {i + 1}: "
f"{score:.4f}"
)
The document with the highest similarity score is considered the closest according to this metric.
Embedding Dimensions
An embedding model may produce vectors with dimensions such as:
128
384
512
768
1024
1536
The number depends on the model.
For example:
Sentence
↓
Embedding Model
↓
384-dimensional vector
The dimension is not a direct measure of how much "meaning" the model understands.
A higher-dimensional vector is not automatically better.
Model quality, training, domain fit, and evaluation are also important.
Embeddings Are Not Human-Readable
Consider:
[
0.124,
-0.582,
0.193,
0.774,
...
]
A human cannot easily look at these numbers and determine the meaning of the sentence.
The useful information comes from the relationships between vectors.
For example:
Vector A ↔ Vector B
may indicate high semantic similarity.
This relational property is what makes embeddings useful.
Embedding Models vs LLMs
Embedding models and generative language models have different primary purposes.
Embedding Model
Input:
Text
Output:
Vector
Primary use:
Search
Similarity
Retrieval
Clustering
Recommendations
Generative LLM
Input:
Prompt
Output:
Generated Text
Primary use:
Generation
Reasoning
Summarization
Conversation
Coding
They can work together in the same application.
Embeddings + LLM Architecture
A modern AI application might use:
USER
↓
QUERY
↓
EMBEDDING
↓
VECTOR SEARCH
↓
RELEVANT DOCUMENTS
↓
LLM
↓
RESPONSE
The embedding model finds information.
The LLM generates the final response.
Build a Mini Semantic Search Application
Here is a complete example:
from sentence_transformers import SentenceTransformer
from sklearn.metrics.pairwise import cosine_similarity
model = SentenceTransformer(
"all-MiniLM-L6-v2"
)
documents = [
"Python is a programming language used for software development.",
"FastAPI is a Python framework for building APIs.",
"PostgreSQL is a relational database system.",
"React is a JavaScript library for building user interfaces.",
"Machine learning allows computers to learn patterns from data."
]
document_embeddings = model.encode(
documents,
normalize_embeddings=True
)
while True:
query = input(
"\nSearch (type exit to quit): "
)
if query.lower() == "exit":
break
query_embedding = model.encode(
[query],
normalize_embeddings=True
)
scores = cosine_similarity(
query_embedding,
document_embeddings
)[0]
ranked = sorted(
zip(documents, scores),
key=lambda item: item[1],
reverse=True
)
print("\nTop Results:\n")
for document, score in ranked[:3]:
print(
f"{score:.4f} - {document}"
)
Now try queries such as:
How can I create an API with Python?
or:
What technology is used for databases?
The system can retrieve semantically related documents.
Applications of Embeddings
Embeddings are used in many AI systems.
Semantic Search
Find documents based on meaning rather than exact keywords.
RAG
Retrieve relevant information before asking an LLM to generate an answer.
Recommendation Systems
Find products, articles, or videos similar to user interests.
Document Similarity
Compare documents based on semantic content.
Duplicate Detection
Identify potentially similar or duplicate content.
Classification
Use embeddings as features for downstream machine-learning models.
Clustering
Group related documents or content automatically.
Question Matching
Match user questions with existing answers.
Image Search
Find visually or semantically related images.
Personalization
Represent users and content in vector space for recommendation systems.
Embeddings and Clustering
Embeddings can also be used to group similar content.
Suppose a website has thousands of articles.
After generating embeddings:
Articles
↓
Embeddings
↓
Clustering
↓
Topic Groups
The system may discover groups such as:
Cluster 1 → Python / FastAPI / Django
Cluster 2 → React / JavaScript / Frontend
Cluster 3 → Machine Learning / AI
Cluster 4 → Databases / SQL
Algorithms such as K-Means can be used for clustering.
Simple K-Means Example
Install:
pip install scikit-learn
Then:
from sentence_transformers import SentenceTransformer
from sklearn.cluster import KMeans
model = SentenceTransformer(
"all-MiniLM-L6-v2"
)
documents = [
"Python programming tutorial",
"FastAPI backend development",
"React frontend development",
"JavaScript web applications",
"Machine learning with Python",
"Deep learning neural networks"
]
embeddings = model.encode(
documents
)
kmeans = KMeans(
n_clusters=2,
random_state=42
)
labels = kmeans.fit_predict(
embeddings
)
for document, label in zip(
documents,
labels
):
print(
f"Cluster {label}: {document}"
)
The exact clusters depend on the data and model.
Limitations of Embeddings
Embeddings are powerful, but they are not perfect.
Similarity Is Not Truth
Two documents can have similar meanings while containing incorrect information.
A high similarity score does not mean the information is factually correct.
Domain-Specific Language
General-purpose embedding models may not perform equally well in highly specialized domains.
Examples include:
Medicine
Law
Finance
Scientific research
Technical engineering
Domain-specific evaluation may be necessary.
Context Matters
A short sentence can be ambiguous.
For example:
Apple
could refer to:
Fruit
Technology company
The surrounding context can affect the embedding.
Similarity Thresholds Require Testing
There is no universal threshold such as:
0.80 = always similar
The appropriate threshold depends on the model, data, language, and application.
How to Improve Semantic Search
A production semantic search system can use several techniques.
Better Embedding Model
Choose a model appropriate for the language and domain.
Better Chunking
Split documents into meaningful sections instead of arbitrary pieces.
Metadata Filtering
Filter results using metadata such as:
Category
Date
Author
Language
Department
Product
Hybrid Search
Combine:
Keyword Search
+
Vector Search
This can improve retrieval when exact terms are important.
Reranking
Retrieve several candidates with embeddings and then use a reranking model to improve the final ordering.
Embedding Pipeline for a Production Application
A complete document search system can look like:
DOCUMENTS
↓
Text Extraction
↓
Chunking
↓
Embedding Model
↓
Vector Database
↓
Similarity Index
↓
User Query
↓
Query Embedding
↓
Vector Search
↓
Top Candidates
↓
Reranking
↓
Relevant Information
↓
LLM
↓
Answer
This architecture forms the foundation of many modern AI knowledge-search applications.
Final Summary
Embeddings transform data into numerical vectors that allow AI systems to work with semantic relationships.
The basic process is:
Text
↓
Embedding Model
↓
Vector
↓
Similarity Calculation
↓
Related Content
For semantic search:
User Query
↓
Query Embedding
↓
Vector Search
↓
Relevant Documents
For RAG:
User Question
↓
Embedding
↓
Vector Database
↓
Relevant Context
↓
LLM
↓
Answer
The most important concepts to remember are:
Embedding
↓
Numerical representation of data
Vector
↓
List of numerical values
Similarity
↓
Measure of how closely vectors relate
Semantic Search
↓
Search based on meaning
Vector Database
↓
Efficient storage and retrieval of embeddings
RAG
↓
Embeddings + Retrieval + LLM
Embeddings are one of the most important technologies behind modern AI search and retrieval systems. They allow applications to move beyond simple keyword matching and work with the semantic relationships between pieces of information.
Once embeddings are combined with vector databases and LLMs, developers can build powerful systems such as AI document search, knowledge assistants, recommendation engines, semantic search platforms, and RAG-based chatbots.