Sample deliverable

What an architecture review report looks like.

A synthetic example of the written findings document delivered after the review session. The client, figures and findings are invented to show the format and level of detail.

Request a review How the review works

Synthetic sample, not a customer engagement. The organisation, numbers and findings are fictional and illustrative. A real report is written for your system, from your intake and session, and is not published.

1. Summary

Private assistant for a 400-person engineering consultancy

The firm plans an internal assistant that answers questions from technical standards, past project reports and client deliverables. Client-confidential projects must stay visible only to their project teams. The prototype works in demos, and the team wants to know whether the design will survive real use and an audit before ordering hardware.

Overall assessment: the approach is sound and can run fully inside the firm's boundary, likely on modest hardware once a load test confirms the candidate configuration. Two high findings must be fixed before any wider pilot: retrieval can cross client boundaries, and the gateway can send requests to a public API.

Top priorities

1. Enforce project permissions before ranking (F1)
2. Remove the public-API fallback (F2)
3. Gate releases on an evaluation set (F4)

2. Recommended architecture

Self-hosted, deny-by-default, with evidence at each decision

Staff (SSO)Assistant UIAI gatewayPermission filterPrivate indexLocal modelEvidence log

Every request carries the user's identity. Permissions are applied before ranking, the model is reachable only through the gateway, and each decision is logged with versions, not content.

Model stack

One 7–9B-parameter open-weight instruct model, 4-bit quantized, served by an OpenAI-compatible local runtime. A second model is not justified until the evaluation set shows a quality gap.

Deployment boundary

Self-hosted in the firm's own data centre, operated by its platform team. No runtime egress; model and package updates enter through an internal registry.

Hardware sizing (candidate, pending load test)

Candidate: one 48 GB data-centre GPU plus a second for failover. Memory arithmetic suggests the 4-bit model weights and an FP16 cache for 25 sequences at 8k context fit with room to spare, but memory fit does not establish latency or throughput. Approve the purchase only after a load test at 25 concurrent users meets the agreed time-to-first-token and tokens-per-second targets.

Runtime topology

Gateway and runtime on the existing Kubernetes cluster in a dedicated namespace with deny-by-default egress. Only the gateway may reach the model endpoint.

RAG design

Index per access domain, filtering before ranking, version-exact citations and one approved no-answer message. Each request checks the user's current group membership with the identity provider; document permissions are synchronised every 15 minutes, and revocations are pushed immediately and invalidate cached results for the affected scope. If permission freshness cannot be confirmed, restricted collections are excluded and the assistant answers only from content open to all staff.

3. Findings

Six findings, ranked by consequence

F1High

Client-confidential projects share one retrieval collection

Observed: The prototype indexes all project folders together and filters results after ranking.
Recommendation: Carry project ACLs onto every chunk, filter before ranking, check current membership per request, invalidate cached results on revocation, and add paired allowed/denied and revoked-access test questions per project.

F2High

Silent fallback to a public API when the GPU is busy

Observed: The gateway prototype retries on a public endpoint after a timeout.
Recommendation: Remove the fallback. Queue or return a clear capacity message; any external route needs its own approval and data classification rule.

F3Medium

Model weights are downloaded at container start

Observed: Pods pull from a public model hub using a mutable name.
Recommendation: Import once into the internal registry, record the digest and licence, and deploy by digest only.

F4Medium

Acceptance is based on demos, not an evaluation set

Observed: No fixed questions, expected sources or no-answer cases exist.
Recommendation: Build a 60-question set with expected sources, permission pairs and no-answer cases, and gate releases on it.

F5Medium

Logs keep full prompts and retrieved text for a year

Observed: Debug logging was left at the prototype setting.
Recommendation: Record identifiers, versions and decisions by default; keep content only when justified, with a 30-day retention class.

F6Low

Hardware request has not been justified by measurements

Observed: The budget request assumes four 80 GB GPUs for a single 8B-class model, without a load test or a stated latency target.
Recommendation: Agree latency and throughput targets, load test the candidate configuration, and size the order from the results and three months of usage data.

4. Compliance posture notes

How the design maps to likely obligations

EU AI Act

An internal knowledge assistant for staff is unlikely to fall in a high-risk category. Tell users they are interacting with an AI system, and reassess if it is ever used for decisions about employees or clients.

GDPR

Project reports contain names and contact details. Record the purpose and lawful basis, minimise personal data in logs (F5), and consider a DPIA before the pilot widens.

Client contracts

Confidentiality clauses require project-level separation. F1 is the control that evidences it; keep the paired permission tests as audit evidence.

Compliance notes are technical orientation, not legal advice. Confirm obligations with your counsel.

5. 30-day action plan

What the team does next, in order

Week 1

Remove the public fallback (F2)
Design per-project ACL propagation and filtering before ranking (F1)

Week 2

Import the model into the internal registry and deploy by digest (F3)
Change logging to identifiers and versions with 30-day content retention (F5)

Week 3

Write the 60-question evaluation set and record a baseline (F4)
Implement the permission filter and revocation handling, and test paired and revoked-access questions (F1)

Week 4

Load test the candidate configuration at 25 concurrent users against the agreed latency targets, then size the GPU order (F6)
Go/no-go review against the evaluation and permission gates

Assumptions and limits: sizing and timings are planning estimates based on the intake answers. The review does not include a penetration test, a legal opinion, or vendor selection on price.

Your system

Want this for your own architecture?

A real review starts with a short question by email or the contact form. You get a fixed quote before committing, a 90-minute session, and a findings document like this one within 5 business days.

Discuss your AI project