Your AI Runs Locally. Can One Tenant Still See Another Tenant’s Data?

Your models run in your data centre. Your documents stay on your network. The application has single sign-on, and external internet access is blocked.

Can one customer still receive another customer’s information?

Yes—if an internal isolation boundary fails.

Local deployment gives an organisation control over where its AI operates. Tenant isolation determines which users, teams and customers can access which information inside that environment. Both need deliberate engineering.

For sovereign AI, the question extends beyond whether data leaves the building: can it reach someone inside the building who has no right to see it?

A real incident behind the question

On 26 October 2026, Cloudflare disclosed a tenant-isolation incident involving its GLM-5.3 Workers AI service. Between 27 August and 9 September, 155 responses contained data from other customers’ requests. Cloudflare said the issue was fixed and it found no evidence of deliberate exploitation. [1]

According to its report, an inference configuration mismatch combined with an SGLang error-handling bug allowed certain oversized requests to generate responses using stale KV-cache data from earlier requests. [1]

This was a specific hosted-service incident. It does not establish that every SGLang deployment, shared GPU or local AI system is vulnerable. Its relevance to local deployments is architectural: a shared inference runtime is a security boundary that must be validated.

Think of a shared office printer

Imagine two companies using a printer in a secure building. Visitors cannot enter, and the printer has no internet connection. But a fault causes Company B’s print job to include a page left over from Company A’s job.

The building remained secure. The information still reached the wrong person.

An AI platform has several equivalents of that printer: document retrieval, conversation storage, response caches, inference memory, agent tools and diagnostic logs. Each needs to preserve the intended access boundary.

A tenant might be a customer, department or project. Even within one tenant, individual users may have different permissions. A finance employee and an HR employee sharing the same company account should not automatically share access to payroll documents.

Where the boundary can fail

The following are architectural failure scenarios to evaluate, not claims that a particular product has these defects.

An instruction such as “never reveal another tenant’s data” is useful guidance for model behaviour. It cannot replace access enforcement. The safer design prevents unauthorised material from entering the prompt in the first place.

Shared model weights and shared request state are different

Multiple tenants can use the same base model. Ordinary inference does not automatically train their prompts into its weights.

The serving system also manages changing information: retrieved context, conversation history and intermediate calculations. A KV cache stores attention keys and values to avoid repeating computation; it is distinct from the model’s trained weights.

Some reuse is intentional. Prefix caching can accelerate requests with matching prompt prefixes. That optimisation needs an explicit security policy.

For example, vLLM documents prefix-cache timing side channels and supports a cache_salt to scope reuse. Its security guidance recommends an authentication gateway that scopes client-supplied cache identifiers per tenant. [2]

That mitigation should not be confused with a general fix for runtime memory-safety bugs. Cache-key isolation, safe request handling and application authorisation address different failure modes.

vLLM's guidance explicitly states that tenants sharing one server process are not isolated. Where tenant isolation is required, it recommends dedicated vLLM instances per tenant alongside gateway controls. A cache salt does not turn a shared process into a tenant security boundary. [2]

What namespaces, database policies and MIG actually protect

Kubernetes namespaces organise isolation; they need supporting controls. Kubernetes guidance describes namespace-based tenancy alongside RBAC, network policies and other protections. For strict network separation, it recommends a default-deny starting point with explicit allowances. [3]

Two applications in separate namespaces may still call the same inference endpoint. That shared service must enforce its own boundaries.

Database policies can strengthen retrieval isolation—if the application actually uses them. PostgreSQL row-level security can restrict accessible rows, but superusers and roles with BYPASSRLS bypass it; table owners normally do too. [4] Test retrieval using the production application role, including background jobs and connection-pool reuse. A policy working under a test account is insufficient evidence.

MIG provides GPU-instance isolation. NVIDIA documents separate memory-system resources for GPU instances, including dedicated cache banks and memory paths. [5] This can support separate inference deployments on a supported GPU.

However, if several tenants share one inference server inside one MIG instance, MIG does not distinguish those application tenants. Nor can it correct a retrieval service that selects the wrong document.

Dedicated GPUs also leave shared databases, gateways and operational access in scope. Hardware separation is one part of the architecture.

Choose isolation around the trust boundary

My recommendation is to decide what tenants are allowed to share before selecting the GPU layout.

Greater separation can increase cost, duplicate model memory and reduce utilisation. Shared serving can be a reasonable choice when the boundaries are explicit and tested. The acceptable trade-off depends on the sensitivity of the data and the consequences of exposure.

Do not use successful application tests to override a runtime's documented isolation limits. Tenants requiring separation need a deployment that provides it.

A practical test before calling the platform isolated

The following is a proposed acceptance test, not a certification standard. Run it in an authorised test environment using synthetic data.

Create Tenant A and Tenant B. Give each distinct documents, conversations and unique markers that appear nowhere else. Also create two users within one tenant with different document permissions.

Then exercise the boundaries:

1. Retrieval and direct access: Ask related questions, request guessed document IDs and attempt cross-tenant conversation and file access. Inspect retrieved evidence as well as final answers.
2. Cache reuse: Ask identical questions from both tenants and test warm caches. Revoke document access and verify that previously cached content is no longer served to that user.
3. Concurrency and failure: Interleave requests, cancel streams, trigger timeouts and submit inputs around configured limits. Verify clean rejection and correct response routing.
4. Recovery: Restart workers and restore a test backup. Confirm that ownership, permissions and tenant routing survive.
5. Operational access: Check what tenant users and support roles can see in traces, exports, error messages and logs.

Finding another tenant’s unique marker is a clear failure. Finding none is useful evidence, but does not prove the absence of every leak: outputs may paraphrase sensitive content, and a test covers only the conditions exercised.

Record the model and runtime versions, configuration, deployment layout, tested permissions, concurrency, results and limitations. Repeat relevant checks after changes to caching, serving topology, quantisation or authentication.

Ask for evidence that matches the claim

“Hosted locally,” “air-gapped” and “uses dedicated GPUs” each describe part of a deployment. None, alone, demonstrates end-to-end tenant isolation.

A defensible claim is narrower and testable:

For this deployment version and configuration, these tests found no unauthorised cross-tenant access under the documented conditions. These shared components and untested scenarios remain in scope for further assessment.

That statement gives customers something they can examine. An architecture diagram shows intended boundaries; test results show how those boundaries behaved.

Before accepting a sovereign AI platform, ask its owner to demonstrate that Tenant B cannot retrieve Tenant A’s documents, reuse its conversations, receive its cached answers or act through its credentials—even when requests fail.

Your AI can remain entirely inside your network and still disclose information to the wrong person. Tenant isolation needs its own evidence.

Put the boundary into practice

Start with the public 15-minute tenant-isolation exercise. Draw two tenants and plan allowed, denied and spoofed-tenant tests using synthetic documents. No account, GPU or installation is required.

Start with the Academy's public trust boundaries and data flows lesson. Draw the retrieval, inference, cache and operational boundaries for your own system, then mark which components tenants share.

For a second step, read the public introduction to private RAG and use the RAG evaluation and hallucination test kit to plan permission and cross-tenant checks.

You can read these starting lessons without signing in. A free Academy account lets you save progress and continue into the advanced lessons after email confirmation.

Sources

Sources checked on 2 November 2026. The incident summary is attributed to Cloudflare; deployment recommendations and the proposed test plan are the author’s engineering analysis.

1. Cloudflare: Cross-request KV cache exposure in the GLM-5.3 Workers AI model, and our remediation with SGLang, 26 October 2026.
2. vLLM: Security guidance. Controls and supported parameters should be checked against the deployed release.
3. Kubernetes: Multi-tenancy.
4. PostgreSQL 17: Row Security Policies.
5. NVIDIA: Multi-Instance GPU User Guide—Introduction.