Build RAG with ASP.NET Core and Qdrant

In this tutorial we build a practical RAG pipeline using ASP.NET Core, an embedding model, and Qdrant as the vector store.

Architecture

The system contains:

• ASP.NET Core Web API
• An embedding model (local via Ollama, or a hosted API)
• Qdrant for vector storage and similarity search
• An LLM for final answer generation
• A document ingestion pipeline that chunks and embeds source content

Step 1 — Ingest and chunk documents

Documents are split into smaller, overlapping chunks before embedding. Chunking too coarsely loses precision; chunking too finely loses context — this is a tuning decision, not a fixed rule.

public record DocumentChunk(string Text, string SourceId, int Index);
 
public IEnumerable<DocumentChunk> Chunk(string text, string sourceId, int chunkSize = 800)
{
    for (var i = 0; i < text.Length; i += chunkSize)
    {
        var length = Math.Min(chunkSize, text.Length - i);
        yield return new DocumentChunk(text.Substring(i, length), sourceId, i / chunkSize);
    }
}

Step 2 — Embed and store in Qdrant

Each chunk is converted into a vector and upserted into a Qdrant collection alongside its source metadata, so retrieved results can be traced back to their origin.

Step 3 — Retrieve at query time

var queryVector = await embeddings.EmbedAsync(userQuestion);
var results = await qdrantClient.SearchAsync(
    collectionName: "foundry-docs",
    vector: queryVector,
    limit: 5);

Step 4 — Construct the prompt

The retrieved chunks are assembled into a context block and passed to the LLM together with the original question, with instructions to answer only from the supplied context.

Common errors

• Forgetting to normalize embeddings before comparing cosine similarity.
• Chunking mid-sentence, which hurts retrieval quality more than it seems like it should.
• Sending the full document set to the model instead of the top-k retrieved chunks.

What's next

The SQL + AI discipline picks this pattern back up — instead of retrieving documents, the system retrieves schema and query intent before generating SQL.