Embedding Models Compared: OpenAI, Cohere, and Open-Source

Your choice of embedding model determines how well your RAG system retrieves relevant documents. A weak model misses relevant passages. An expensive model burns budget on every query. This guide compares the leading options across dimensions, cost, benchmark performance, and practical use.

The contenders

We are comparing five models across three providers:

Model	Provider	Dimensions	Max Tokens	Pricing (per 1M tokens)
text-embedding-3-small	OpenAI	1536	8191	$0.02
text-embedding-3-large	OpenAI	3072	8191	$0.13
embed-v3 (english)	Cohere	1024	512	$0.10
e5-large-v2	Open-source	1024	512	Free (self-hosted)
bge-large-en-v1.5	Open-source	1024	512	Free (self-hosted)

MTEB benchmark scores

The Massive Text Embedding Benchmark (MTEB) evaluates models across retrieval, classification, clustering, and semantic textual similarity tasks. Higher is better.

Model	Retrieval (avg)	Classification	STS	Overall
text-embedding-3-small	59.2	65.1	78.0	61.6
text-embedding-3-large	62.4	68.3	81.2	64.6
embed-v3	64.5	69.7	82.1	66.3
e5-large-v2	60.1	65.8	79.4	62.2
bge-large-en-v1.5	61.4	66.9	80.3	63.5

Cohere embed-v3 leads on retrieval, which is the metric that matters most for RAG. OpenAI’s 3-large is close behind. The open-source models (E5, BGE) are competitive, especially considering they are free to run.

OpenAI: text-embedding-3-small and 3-large

OpenAI’s embedding API is the easiest to start with. One API call, no GPU needed.

from openai import OpenAI

client = OpenAI()

def embed_openai(texts: list[str], model: str = "text-embedding-3-small") -> list[list[float]]:
    """Generate embeddings using OpenAI's API."""
    response = client.embeddings.create(input=texts, model=model)
    return [item.embedding for item in response.data]

# Embed documents
chunks = ["RAG retrieves relevant documents...", "Chunking splits text into..."]
vectors = embed_openai(chunks)
print(f"Dimensions: {len(vectors[0])}")  # 1536 for 3-small

A useful feature of the 3-series models is dimension reduction. You can request fewer dimensions to shrink your vector index without re-embedding:

response = client.embeddings.create(
    input=["example text"],
    model="text-embedding-3-large",
    dimensions=1024,  # reduce from 3072 to 1024
)

This uses Matryoshka Representation Learning — the first N dimensions retain most of the information. Going from 3072 to 1024 typically costs less than 1% retrieval accuracy.

When to choose OpenAI: You want zero infrastructure, fast integration, and your budget allows API costs. Start with 3-small; upgrade to 3-large only if retrieval recall on your eval set justifies the 6.5x cost increase.

Cohere: embed-v3

Cohere’s embed-v3 currently leads the MTEB retrieval leaderboard. It also introduces input types, letting you tell the model whether the text is a search_document or a search_query, which improves asymmetric search.

import cohere

co = cohere.ClientV2("your-api-key")

def embed_cohere(
    texts: list[str],
    input_type: str = "search_document",
    model: str = "embed-english-v3.0",
) -> list[list[float]]:
    """Generate embeddings using Cohere's API."""
    response = co.embed(
        texts=texts,
        model=model,
        input_type=input_type,
        embedding_types=["float"],
    )
    return response.embeddings.float_

# Embed documents with document type
doc_vectors = embed_cohere(
    ["RAG retrieves relevant documents..."],
    input_type="search_document",
)

# Embed queries with query type
query_vector = embed_cohere(
    ["How does RAG work?"],
    input_type="search_query",
)

The input_type distinction is important. Documents get embedded differently from queries because the model learns that queries are short and documents are longer. Mixing these up can hurt retrieval by 3-5%.

When to choose Cohere: You need the best retrieval quality and are willing to learn a slightly more complex API. Cohere also offers excellent multilingual support with embed-multilingual-v3.0.

Open-source: sentence-transformers

If you want to self-host — for privacy, cost control, or offline use — the sentence-transformers library gives you access to hundreds of models.

from sentence_transformers import SentenceTransformer

# BGE-large is one of the best open-source options
model = SentenceTransformer("BAAI/bge-large-en-v1.5")

def embed_local(texts: list[str]) -> list[list[float]]:
    """Generate embeddings locally with sentence-transformers."""
    embeddings = model.encode(
        texts,
        normalize_embeddings=True,
        show_progress_bar=True,
        batch_size=32,
    )
    return embeddings.tolist()

vectors = embed_local(["RAG retrieves relevant documents..."])
print(f"Dimensions: {len(vectors[0])}")  # 1024

For production, you will want a GPU. On an A10G, BGE-large embeds roughly 500 passages per second. On CPU, expect 20-50 per second.

The top open-source models to consider:

Model	Parameters	Dimensions	MTEB Retrieval
bge-large-en-v1.5	335M	1024	61.4
e5-large-v2	335M	1024	60.1
gte-large	335M	1024	60.8
all-MiniLM-L6-v2	22M	384	49.5

all-MiniLM-L6-v2 is worth noting — it is 15x smaller and much faster. For prototypes or latency-sensitive applications, it is a reasonable trade-off.

When to choose open-source: Your data cannot leave your infrastructure, you are embedding millions of documents (API costs would be prohibitive), or you need to fine-tune the model on domain-specific data.

Practical comparison: cost at scale

Suppose you have 1 million documents averaging 500 tokens each. Here is the cost to embed them all:

Model	Cost to embed 1M docs	Monthly query cost (100k queries)
text-embedding-3-small	$10	$0.10
text-embedding-3-large	$65	$0.65
embed-v3	$50	$0.50
bge-large (self-hosted, A10G)	~$30/month GPU	$0 marginal

At small scale, API-based models are cheaper (no GPU rental). At large scale, self-hosting wins because the marginal cost per embedding approaches zero.

Decision framework

Use this flowchart:

1. Can your data leave your infrastructure? If no, use open-source (BGE-large or E5-large).
2. Do you need multilingual support? If yes, Cohere embed-multilingual-v3 is the best option.
3. Is retrieval quality the top priority? Cohere embed-v3 leads on retrieval benchmarks.
4. Is simplicity the top priority? OpenAI 3-small is the easiest to integrate.
5. Embedding millions of documents? Self-host to avoid API costs.
6. Prototyping? Start with OpenAI 3-small. It is cheap and fast to integrate. Switch later if recall is insufficient.

Tips for production

1. Always normalize embeddings — use cosine similarity, not dot product, unless you are sure your model was trained for dot product.
2. Batch your API calls — OpenAI allows up to 2048 inputs per call. Cohere allows 96. Batching reduces latency and avoids rate limits.
3. Cache embeddings — never re-embed the same text twice. Store vectors alongside your chunks.
4. Evaluate on your data — MTEB scores are averages across many datasets. Your domain may favor a different model. Build a test set of 50-100 queries with known relevant documents and measure recall@10.

What’s next

You have chunks and vectors. The next question is how to search them effectively. The next article, Hybrid Search for RAG, covers combining dense vector search with sparse keyword search for better retrieval.