Local AI Development with Ollama

Introduction

Local AI development is useful when developers want to experiment with models without sending every prompt to a hosted provider.

Ollama provides a convenient local runtime for downloading and running supported models.

This article focuses specifically on local workstation development. It is intentionally different from the ASP.NET Core LLM integration article, which focuses on application/backend integration.

Why Run Models Locally?

Common reasons include:

• development experimentation
• privacy-sensitive prototypes
• offline development
• avoiding provider API costs during experimentation
• testing local inference workflows

Local inference also introduces trade-offs:

• hardware requirements
• model size
• inference speed
• memory usage
• model quality

Local does not automatically mean better or faster.

Installing Ollama

Install Ollama using the official installation process for the operating system.

After installation, verify the CLI:

ollama --version

Then list locally available models:

ollama list

Running a Model

A model can be started using:

ollama run <model>

For example, when a compatible DeepSeek model is available:

ollama run <deepseek-model>

The exact model name should be taken from the currently available Ollama model catalog rather than hard-coded into documentation that may become outdated.

DeepSeek and Llama Models

Model availability and model naming change over time.

When choosing a local model, consider:

• parameter size
• quantization
• RAM/VRAM requirements
• context length
• coding performance
• general reasoning performance
• license

Do not assume the largest model is automatically the best choice for a development machine.

Local Model Workflow

A typical developer workflow is:

Developer
   ↓
IDE / Terminal
   ↓
Ollama
   ↓
Local Model
   ↓
Response

This is useful for experimentation before deciding whether a hosted provider is appropriate for production.

Calling Ollama from .NET

Ollama exposes a local HTTP API.

A .NET application can communicate with the local endpoint using HttpClient or an appropriate SDK.

Keep the local endpoint behind an abstraction:

public interface ILocalAiClient
{
    Task<string> GenerateAsync(
        string prompt,
        CancellationToken cancellationToken);
}

The implementation can call the local Ollama API.

This keeps the rest of the application independent of the local runtime.

Configuration

Use configuration rather than hard-coding the local endpoint:

{
  "Ollama": {
    "BaseUrl": "http://localhost:11434"
  }
}

Bind it through ASP.NET Core configuration/options where appropriate.

IDE Integration

Local models can also be useful with development tools that support Ollama-compatible endpoints.

Keep IDE integration separate from production application architecture.

The local workstation setup is useful for experimentation, but production applications should still have explicit authentication, observability, resource controls, and provider abstractions.

Resource Planning

Local inference can consume significant system resources.

Consider:

• available RAM
• GPU memory
• CPU performance
• concurrent requests
• model size
• quantization

For a development workstation, start with a model that fits comfortably within available resources rather than maximizing model size.

Security Considerations

A locally running model can still expose sensitive data if the machine or network is not protected.

Do not assume:

“Local = automatically secure.”

Protect the machine, local API access, source code, and model inputs.

Local vs Hosted AI

Consideration	Local Ollama	Hosted Provider
Setup	Developer-managed	Provider-managed
Hardware	Local resources	Provider infrastructure
Data path	Can remain local	Sent to provider
Scaling	Limited by local resources	Provider-dependent
Maintenance	Local responsibility	Provider responsibility
Experimentation	Convenient	Convenient
Production suitability	Depends on workload	Depends on provider and requirements

Conclusion

Ollama provides a practical environment for local AI experimentation.

Use it to understand model behavior, prototype AI workflows, and test integrations without immediately committing to a hosted provider.

For production systems, evaluate the model, infrastructure, security, observability, latency, and operational requirements independently of the development experience.