Software Architecture

AI Gateway Setup with Laravel: OpenAI + Ollama (Local AI) Single-Stop Management

AI Gateway Setup with Laravel: OpenAI + Ollama (Local AI) Single-Stop Management

AI integrations have now become an indispensable part of modern web applications. However, the following problem arises in most projects:

"Which AI provider should I use? OpenAI or local (Ollama)?"

In this article, we describe an architecture that fundamentally solves this problem:

👉 A system that can use both OpenAI and local AI (Ollama) through a single API (ai.kitmote.com).

Thanks to this structure:

  • All AI operations are managed through a single endpoint.
  • Changing the provider is simply a setting change.
  • Switching to local AI becomes easier when traffic increases.
  • Multi-tenant AI infrastructure is created for SaaS systems.

🧠 Why AI Gateway?

The classic approach:

 
Frontend → OpenAI API
 

This approach:

  • API key creates security problems.
  • The provider creates a dependency.
  • It is not flexible.

Our approach:

 
Client → ai.kitmote.com → AI Provider
 

Well:

  • All AI requests pass through a single central point.
  • Provider selection is done by the backend.
  • Security and logging are brought under control.

🏗️ System Architecture

https://miro.medium.com/v2/resize%3Afit%3A875/0%2A0YiHfsf_hWAQ9Kw8.png
https://sgnl.ai/images/blog/3-sgnl-aws-api.png
https://media.licdn.com/dms/image/v2/D5612AQGEcPvpteKVOw/article-inline_image-shrink_400_744/B56ZWdPIlIHoAc-/0/1742099746253?e=2147483647&t=FkLkduUgprBZpXMnpJdUfO4y6QkANIafoHJ1YMMQ82A&v=beta
 

The general flow is as follows:

 
Client Domain
↓
ai.kitmote.com (Laravel API)
↓
Middleware (Security Layer)
↓
AI Manager
↓
Provider Resolver
↓
[OpenAI | Ollama | Fallback ]
↓
Response + Logging
 

⚙️ Technologies Used

  • Laravel 13 → API & orchestration
  • OpenAI API → cloud AI
  • Ollama → local AI (can run without GPU)
  • MySQL → settings + log
  • PEST → testing infrastructure

🔌 Provider Logic (Core Idea)

The heart of the system is this principle:

It doesn't know the controller provider; it simply calls the AI Manager.

Interface

 
interface AiProviderInterface
{
public function generate(AiRequestData $request): AiResponseData;
}
 

Provider examples

  • OpenAiProvider
  • LocalAiProvider (Ollama)
  • NullProvider (fallback)

🔄 Provider Switching Algorithm

The most critical part of the system:

 
1. Active provider = read from the settings table.
2. Check if the provider is available.
3. If that's not the case, switch to the fallback provider.
4. If none of these are found, a controlled error is returned.
 

Pseudo code

 
if (provider == local) {
if (local_available) {
return local;
}

if (fallback == openai) {
return openai;
}

return nullProvider;
}

if (provider == openai) {
return openai;
}
 

⚡ OpenAI Integration (Cloud AI)

 
$response = Http::withToken($apiKey)
->post('https://api.openai.com/v1/responses', [
'model' => 'gpt-4.1-mini',
'input' => $prompt,
]);
 

Advantages:

  • high accuracy
  • stable performance
  • zero infrastructure cost (initially)

🖥️ Local AI with Ollama

 
 
 

Thanks to Ollama:

  • You can run the local model even without a GPU.
  • Provides access via API endpoint.
 
llama run llama3
 

On the Laravel side:

 
Http::post('http://localhost:11434/api/generate', [
'model' => 'llama3',
'prompt' => $prompt,
]);
 

Advantages:

  • No cost (no API required)
  • No data will be leaked.
  • It can work offline.

🧩 Settings-Based Management

All system settings are managed through the menu:

ai_provider = openai | local
ai_fallback_provider = openai
ai_local_enabled = true/false
ai_openai_api_key = xxx
 

In this way:

👉 You can change the provider without changing the code.

🔐 Security Layer

The critical issue in this system is:

"Not everyone should be able to use the API."

Therefore, the middleware layer:

  • API Token verification
  • Domain whitelist
  • IP whitelist
  • Rate limit
Request → Token check → Domain check → IP check → AI
 

📊 Logging and Cost Tracking

Every AI request is logged:

  • provider
  • model
  • input/output token
  • response time
  • cost
  • domain

In this way:

👉 You'll get a clear answer to the question, "How much did I spend?"

🚀 Real-World Use Case Study

This structure is particularly strong in the following systems:

  • Creating a CV (KitResume style)
  • Blog content creation
  • SEO description generation
  • Chat systems
  • SaaS AI services

🔁 Hybrid AI Strategy (The most critical part)

The most accurate approach:

Stage Use
Beginning OpenAI
When traffic increases Hybrid
Large scale Local AI

Well:

Today → OpenAI
Tomorrow → OpenAI + Local
Then → Locally focused
 

🎯 Result

In this article, we learned the following:

  • Single endpoint management with AI Gateway
  • Provider abstraction with Laravel
  • OpenAI + Ollama used together
  • fallback and provider switching logic
  • Scalable AI architecture for SaaS

👉 The most important takeaway:

AI integration isn't just about "calling APIs," it's about building the right architecture.

Keywords: laravel ai integration, ollama laravel, openai laravel api, ai gateway nedir, local ai nasıl kurulur, laravel yapay zeka entegrasyonu

01.04.2026 2

Frequently Asked Questions

What is an AI Gateway?

An AI Gateway is a centralized API layer that routes requests to different AI providers such as OpenAI or local models.

Can I use Ollama without GPU?

Yes, Ollama can run on CPU, but performance improves significantly with GPU.

Why not call OpenAI directly from frontend?

Because API keys must be kept secure and backend control allows logging, rate limiting, and provider switching.

Can Laravel manage multiple AI providers?

Yes, using a provider pattern and service layer, Laravel can dynamically switch between AI providers.

More in this category

Comments

Log in or sign up to write a comment
Continue with Google
Giriş
Sign Up