Run Ollama Locally and Connect It to .NET

Running an LLM locally is useful for development, experimentation, and privacy-sensitive workloads where sending data to a third-party API isn't an option.

In this guide we connect Ollama to an ASP.NET Core application.

Architecture

ASP.NET Core
     ↓
   Ollama
     ↓
 Local LLM

Step 1 — Install and run Ollama

ollama pull llama3.1
ollama serve

Ollama exposes a local HTTP API, by default on http://localhost:11434.

Step 2 — Call it from ASP.NET Core

public class OllamaClient(HttpClient http)
{
    public async Task<string> GenerateAsync(string prompt)
    {
        var response = await http.PostAsJsonAsync("/api/generate", new
        {
            model = "llama3.1",
            prompt,
            stream = false
        });
 
        response.EnsureSuccessStatusCode();
        var result = await response.Content.ReadFromJsonAsync<OllamaResponse>();
        return result!.Response;
    }
}
 
public record OllamaResponse(string Response);

Step 3 — Register the client

builder.Services.AddHttpClient<OllamaClient>(client =>
{
    client.BaseAddress = new Uri("http://localhost:11434");
});

Why this matters for the Foundry

Every article in the AI + .NET discipline builds toward the same flagship project — this local Ollama connection is the same building block the AI Database Agent uses before it ever touches a real SQL Server instance.

What's next

Swap GenerateAsync for a streaming implementation over Server-Sent Events or SignalR once the API needs to support a live chat interface.