Private AI inference for businesses
Scope and deploy model execution with defined data controls. Architecture, testing and operational responsibilities agreed for your business.
A service shaped by your data and operating requirements
Private inference runs models under access, data-handling and operating conditions defined for your organisation. Work starts with a bounded use case and produces evidence for deciding whether the solution can be used in that context. Model location, integrations and support are agreed in the scope. An on-premises installation, dedicated hosting and confidential computing are different arrangements. Establish the property you need to verify before choosing an architecture or accepting a privacy claim.
- Use case, inputs, outputs and accountable business reviewer.
- Data, connectivity and administrative access requirements.
- Quality, performance and pilot acceptance criteria.
- Responsibilities and conditions for moving into production.
What the proposal needs to define
The proposal specifies deliverables and boundaries. Inference is model execution; a business chat interface, document retrieval, connectors or model tuning may require additional work. Making those boundaries explicit allows suppliers to be compared on the same task and economic basis. IT, security and the process owner should review the design together so that technical capability translates into a useful, supportable service.
| Area | Documented decision | Acceptance evidence |
|---|---|---|
| Architecture | Environment and included components | Architecture and data-flow map |
| Integration | API, interface and connected systems | Agreed workflow test |
| Models | Version, licence and configuration | Representative task evaluation |
| Access | Users, services and administrators | Permission and revocation tests |
| Operation | Support, changes and recovery | Owners and tested procedures |
Choose where the model runs
Deployment depends on your restrictions and operating capability. A customer-premises environment requires a review of hardware, network, maintenance and recovery. Dedicated hosted infrastructure requires agreement on isolation, location and supplier access. Each option is assessed before it is included in the proposal; availability and suitability are not assumed for every engagement. The on-premises versus private cloud guide explains these trade-offs.
- Available capacity and expected peak workload.
- External dependencies, including identity, telemetry and support.
- Continuity requirements and the response to an outage.
- Configuration export, deletion and service exit conditions.
Privacy supported by evidence
The data boundary includes prompts, outputs, logs, caches and backups, as well as information passed to connected tools. Review who can access each component, for what purpose and for how long. Private models still need controls against malicious inputs, inappropriate access and excessive resource use. The private LLM security guide explains the checks to request. Where governance and risk assessment are also needed, AI Assurance has a separate scope. Implementation is not described as an independent assessment of our own work.
A pilot that ends with a decision
A pilot should demonstrate usefulness for your task, capacity under a stated workload and working controls. Agree the test cases, exclusion criteria and acceptance owners before testing begins. Record the model version and configuration so the findings can be reproduced. A short demonstration is not a substitute for that evaluation. The private AI pilot guide sets out how to decide whether to proceed, correct the design or stop.
- Prepare representative inputs and a quality reference.
- Measure response times, errors and concurrency under recorded conditions.
- Verify permissions, limits and failure behaviour.
- Document findings, restrictions and the next decisions.
How the commercial scope is calculated
Separate implementation from recurring expenditure. Implementation may cover design, configuration, integration and testing. Recurring cost depends on infrastructure, reserved capacity, support and maintenance included in the agreement. A token price that omits operation or a universal quotation without a workload definition would not provide a useful comparison. The private inference cost guide provides a method with explicit assumptions. Evaluate cost per useful task, including retries and any human review that remains necessary.
Prepare for a technical scoping conversation
Bring the use case, expected users and frequency, data categories and response requirements. Synthetic or anonymised examples are sufficient for the first conversation; sensitive production information is not needed. These inputs establish what must be evaluated, which dependencies exist and what is missing before a proposal can be prepared. The private AI vendor assessment checklist helps procurement, IT and security agree their requirements before requesting bids.
Frequently asked questions
Make the decision with evidence
Define the task, controls and workload. A pilot should establish what works, where the limits are and who will operate it.