Running Local LLMs with Laravel: Privacy-First AI

Why Local LLMs?

External AI APIs raise privacy concerns for sensitive data. Local LLMs keep data on your infrastructure while still providing AI capabilities. The trade-off is compute requirements and potentially lower quality than frontier models.

Ollama Integration

class OllamaService
{
    public function __construct(
        private string $baseUrl = 'http://localhost:11434'
    ) {}

    public function generate(string $prompt, string $model = 'llama2'): string
    {
        $response = Http::post("{$this->baseUrl}/api/generate", [
            'model' => $model,
            'prompt' => $prompt,
            'stream' => false,
        ]);

        return $response->json('response');
    }

    public function chat(array $messages, string $model = 'llama2'): string
    {
        $response = Http::post("{$this->baseUrl}/api/chat", [
            'model' => $model,
            'messages' => $messages,
            'stream' => false,
        ]);

        return $response->json('message.content');
    }
}

Model Selection for Tasks

class LocalModelSelector
{
    private array $models = [
        'general' => 'llama2',
        'code' => 'codellama',
        'fast' => 'mistral',
        'embedding' => 'nomic-embed-text',
    ];

    public function forTask(string $task): string
    {
        return $this->models[$task] ?? $this->models['general'];
    }
}

Hybrid Approach

class HybridAIService
{
    public function generate(string $prompt, array $options = []): string
    {
        $sensitivity = $options['data_sensitivity'] ?? 'normal';

        if ($sensitivity === 'high' || $options['use_local'] ?? false) {
            return $this->localAI->generate($prompt, $options);
        }

        return $this->cloudAI->generate($prompt, $options);
    }
}

Docker Deployment

# docker-compose.yml
services:
  ollama:
    image: ollama/ollama
    ports:
      - "11434:11434"
    volumes:
      - ollama_data:/root/.ollama
    deploy:
      resources:
        reservations:
          devices:
            - driver: nvidia
              count: 1
              capabilities: [gpu]

volumes:
  ollama_data:

Performance Optimization

class LocalLLMOptimizer
{
    public function warmUp(array $models): void
    {
        // Pre-load models into memory
        foreach ($models as $model) {
            $this->ollama->generate('Hello', $model);
        }
    }

    public function configureContext(int $maxTokens): void
    {
        // Adjust context window for memory constraints
        $this->contextSize = min($maxTokens, $this->availableMemory() / 4);
    }
}

Conclusion

Local LLMs provide AI capabilities with full data privacy. Consider a hybrid approach: local for sensitive operations, cloud for quality-critical tasks. Monitor performance and adjust model selection based on your requirements.