Modern AI agents are more than a language model. If you want sovereignty, you have to check every building block – an overview of European and self-hosted options.
Matthias Allitsch-Wutte · GEKI founder
27 October 2026 · 8 min read
You want to build AI agents, but your data must not leave Europe – and ideally you also don’t want to depend on US providers for critical components? Then simply running a language model in Europe is not enough. A production AI agent today consists of far more building blocks:
Once a company runs many agents, orchestration adds a central control layer on top.
Every tool call is part of your data chain.
Your language model can run entirely in a European data center. But if your agent then sends the full context to a US web search, an external MCP server or a cloud provider’s monitoring tool, your data has left the controlled environment anyway. “Our language model runs in Europe” is therefore not yet a sovereign agent architecture.
In this article we show what to look out for with sovereign AI agents, which European and self-hosted building blocks already exist and how to keep many agents under control. The list is deliberately not complete. It is meant to show that a large part of a modern agent architecture can now be built in Europe or fully self-hosted.
The most important question for an agent is not “Which language model do we use?” but: “Where can our data go while an agent is working?” Because every external step can create a new data recipient.
With the language model, the first question is location: where does the model run, who operates the GPUs and where are requests processed? Just as important is what happens to the data – whether inputs or responses are stored and whether they are used for training or product improvement. And finally: which other providers sit behind it?
Web search is one of the most frequently overlooked data flows. A badly built agent could take an internal request like
“Analyze the contract between our customer Max Mustermann and ACME GmbH and check whether there are any recent insolvency notices.”
and simply send large parts of the full context to a search provider – even though “ACME GmbH insolvency” would probably have been enough as a search query. This leads to a core principle of the architecture:
Send every tool only the context it actually needs.
The same applies to tools and connectors such as MCP servers. Just because a tool is technically available, an agent should not automatically be allowed to use it. For every tool, ask what data it receives, where it runs and which provider is behind it. Also check what permissions it has, whether it stores inputs and whether it can call further systems in turn.
Internal knowledge bases are part of the data chain too: the documents themselves, their vectors and metadata, the queries against the knowledge base, but also the conversation history and the agent’s long-term memory. This is often where particularly critical information lives – internal documents, customer data or details from confidential conversations. Ideally, all of these components stay within your own controlled infrastructure.
This layer is especially easy to forget. The record of a single agent run can contain:
That can quickly turn the monitoring system into one of the most sensitive databases in the company.
Inference service. If you need GPU capacity and the complete inference platform, a full AI Factory is the natural choice – for example the GEKI AI Factory with dedicated GPU infrastructure and a complete inference platform. Examples of other providers are the Raiffeisen Rechenzentrum AI Factory with AI infrastructure from Austria (aifactory.at) and STACKIT AI Model Serving, which runs models as a service in Europe (stackit.de).
Model router. Not every company wants to commit to a single model or inference provider. This is where a model router becomes interesting: Cortecs offers a European routing layer between applications and different models or inference providers (cortecs.ai). That brings an important sovereignty advantage, because the application does not depend directly on a single model provider. Today an agent might use model A, tomorrow model B or your own AI Factory. Models must be replaceable.
A language model on its own is not yet an agent. It only becomes one through the runtime, often called the harness: it runs the model in a loop, supplies it with context, tools and memory, and controls permissions, actions and handoffs between agents. Model and harness together make the agent – which makes the runtime one of the most important components for data sovereignty.
The language model thinks. The runtime decides what it is allowed to do.
Modern agents go far beyond a chat with a few connected tools. It is becoming increasingly important for an agent to have its own working environment, such as an isolated Linux environment with tools, where it runs programs, edits files and checks results (often called computer use). Such agents increasingly configure themselves and extend their own capabilities, for example by setting up new tools or writing their own scripts. This is exactly where the air in Europe gets thin: runtimes of this kind barely exist in Europe so far.
What is established are AI chat systems with simple agents. Here there are European providers such as Langdock (langdock.com) or open-source solutions to self-host such as LibreChat (librechat.ai) and Open WebUI (openwebui.com).
With GEKI Companybots we take the modern route: with us, everything is an agent, and every agent has its own isolated environment. Its capabilities are therefore no longer limited to predefined tools. Everything runs in data centers in Europe.
Of course you can also build a runtime entirely yourself. But then the company also takes on security, tool governance, permissions, memory, tracing, lifecycle and updates itself.
Running one agent is relatively easy. But what happens with 20, 100 or 1,000 agents? Then a company needs another layer next to the runtime: orchestration. It answers questions like: which agents exist, who owns them and which version is in production? Which models, tools and MCP servers may they use, and which agents talk to each other? Which actions need human approval, what does an agent cost and which agents fail frequently? And can an agent be switched off centrally – with a complete record of all changes and actions?
Camunda is an example of orchestration. It becomes interesting when agents are part of larger business processes, for example:
This is less about a single agent and more about the interplay of agents, people and classic business systems (camunda.com).
One thing matters here: runtime and orchestration are not the same.
| Layer | Job | Examples |
|---|---|---|
| Runtime | Runs an agent. | GEKI Companybots, Langdock, your own runtime |
| Orchestration | Steers all of the company’s agents. | GEKI Companybots, Camunda |
GEKI Companybots covers both, because as the number of bots grows, central management becomes decisive. All bots live in one place, each with a draft and a live version, and earlier versions can be restored. Models, tools, MCP connectors and permissions are managed centrally, every bot can be paused with one click, and monitoring plus an audit log show who changed which bot when.
Tools are what make an agent truly useful – and every one of them is a potential data recipient. For the most important tools there are European or self-hosted solutions.
Internet access and web search. Many enterprise agents need access to the internet. A European alternative for web search is Staan, a search API built for AI applications and agents with its own European search index (staan.ai); our Companybots use Staan for web search as well. The same rule applies as for any other tool (see the ACME example above): the agent derives a minimal search query from its internal context and does not send the full context, including customer data, to the search provider.
Knowledge base. For many purposes you don’t need a dedicated database at all: agents have become so good at working with files that simple Markdown files in storage are often enough. Where a vector database makes sense, Qdrant, self-hosted, is an obvious choice (qdrant.tech). This keeps internal documents, vectors, metadata and queries within your own infrastructure.
Storage. For storage we deliberately don’t prescribe a single provider. Use the data center or storage provider of your choice and check data location and jurisdiction, encryption, backups and access control, retention periods and the ability to take your data to another provider. The agent architecture should adapt to your infrastructure – not your infrastructure to the agent vendor.
Computer use. If an agent gets its own working environment, such as an isolated Linux environment, it processes documents there, runs programs and stores intermediate results. That environment must therefore be just as much under your control as the model itself: isolated per agent, in your own infrastructure or in a European data center, and with clearly governed network access.
No special AI vendor is needed here: even a classic, proven tool like Keycloak, self-hosted, does the job perfectly (keycloak.org). What matters, though, is thinking identity through the entire agent process: it is not enough for the user to sign in to the agent. Permissions must be passed all the way through to the tool and the resource the agent accesses. An agent should never automatically have more permissions than the respective user or process needs.
Running agents in production without monitoring is hard. But monitoring also creates a new sensitive data set. With Langfuse, self-hosted, agent records stay within your own infrastructure (langfuse.com). Still: self-hosted doesn’t mean recording everything. Sensitive inputs, documents and tool results should only be stored when they are actually needed.
These very questions are why we built GEKI Companybots. From the start we had the whole chain in view – from the model through runtime, orchestration and tools to identity and monitoring – and wanted a European solution that brings all of it.
With GEKI Companybots we provide these building blocks from a single source. You can use the platform flexibly as infrastructure: to build your own agents or entire applications on top of it, or to resell it to your customers.
Register, test and get starting credits.
