Is locally hosted AI necessary for legal work?

For most law firms, locally hosted AI is not necessary. Cloud-hosted AI with enterprise data processing agreements, UK or US data residency, and proper encryption meets the requirements of the SRA, ABA Model Rules, and UK GDPR. Local hosting costs 3 to 10 times more, requires specialist infrastructure skills most firms do not have, and is often less secure than enterprise cloud environments because firms lack the resources to maintain it properly.

Short answer: No, for most firms. Cloud AI with enterprise contracts and UK data residency is sufficient. Local hosting only makes sense for national security work or specific regulatory mandates.

Why this question keeps coming up

The instinct to keep everything on-premises is understandable. Lawyers are trained to protect confidentiality, and sending client data to a third-party API feels risky. Partners who remember the early days of cloud computing, when it genuinely was less secure, carry that wariness forward.

The AI vendor landscape has not helped. Some legal AI companies market local hosting as a premium feature, implying that cloud deployment is inherently unsafe. This is marketing, not technical reality.

The actual question is not “local or cloud?” but “what controls do we need around our AI deployment to meet our regulatory and ethical obligations?” Once you frame it that way, the answer usually points to properly contracted cloud services rather than local hardware.

What the regulators actually say

UK: SRA and ICO

The Solicitors Regulation Authority requires firms to act in clients’ best interests and protect confidential information. Its 2023 technology guidance and subsequent updates make clear that firms can use cloud services provided they conduct proper due diligence, have appropriate data processing agreements, and ensure data is handled in compliance with UK GDPR.

The Information Commissioner’s Office (ICO) requires a lawful basis for processing, appropriate security measures, and compliance with data subject rights. It does not require local hosting. It does require you to know where your data is, who can access it, and what safeguards are in place.

Key requirements: UK data residency (or adequate safeguards for international transfers), a data processing agreement with your AI provider, encryption in transit and at rest, and documented data protection impact assessments for high-risk processing.

US: ABA and state bars

ABA Model Rule 1.1 (Competence) requires lawyers to understand the technology they use. Model Rule 1.6 (Confidentiality) requires reasonable efforts to prevent unauthorised disclosure. Comment 18 to Rule 1.6 lists factors to consider, including the sensitivity of information, the likelihood of disclosure, and the cost of additional safeguards.

No state bar has mandated local hosting. Several ethics opinions, including from California, New York, and Florida, have addressed cloud computing and concluded that cloud services are acceptable with appropriate due diligence. The same reasoning applies to AI services.

The real security comparison

Here is what most firms miss: a properly managed cloud environment is almost always more secure than a self-hosted one.

Cloud provider security. Azure, AWS, and Google Cloud spend billions on security. They employ thousands of security engineers. They hold SOC 2 Type II, ISO 27001, and numerous other certifications. Their data centres have physical security, redundancy, and disaster recovery that no law firm can match.

Enterprise AI provider security. OpenAI’s Enterprise tier, Anthropic’s API, and Azure OpenAI Service all offer zero data retention, contractual commitments not to train on your data, SOC 2 compliance, and enterprise data processing agreements. These are contractually enforceable protections, not marketing promises.

Self-hosted reality. A firm running AI on local hardware needs to manage GPU drivers, model updates, security patches, access controls, backup, monitoring, and incident response. Most firms lack the staff to do this properly. An unpatched local GPU server is a bigger security risk than a properly contracted cloud API.

The exception is firms with existing, well-resourced IT security teams who already manage on-premises infrastructure to government or defence standards. For these firms, local hosting is a natural extension of existing capability.

When local hosting genuinely makes sense

Government and defence work

Firms handling matters subject to government security classifications (Official-Sensitive, Secret, or equivalents under US ITAR or CMMC) may have contractual or regulatory requirements that preclude cloud processing. Local hosting on appropriately accredited infrastructure is the right answer here.

Ultra-sensitive M&A and litigation

Hostile takeover defence, major competition investigations, or litigation where even metadata (the fact that documents were processed through an AI system) could be discoverable. These are edge cases, but they exist.

Firms with dedicated AI engineering teams

Large firms with 5+ AI engineers who can properly manage local GPU infrastructure, keep models updated, and maintain security controls may find local hosting cost-effective at scale. The threshold is typically 100+ fee earners with heavy AI usage.

The cost reality

A practical comparison for a mid-market firm running a document analysis system processing 500 documents per month:

Cloud-hosted (Azure OpenAI or equivalent):

• API costs: £400 to £1,200 per month
• Hosting for application layer: £100 to £300 per month
• Total annual cost: £6,000 to £18,000
• Setup time: 4 to 8 weeks

Locally hosted (Llama 3.1 on own hardware):

• GPU hardware (2x A100 or equivalent): £30,000 to £60,000
• Server, networking, and installation: £5,000 to £15,000
• Electricity: £200 to £500 per month
• IT staff time for maintenance: £500 to £1,500 per month (allocated)
• Total year-one cost: £43,000 to £99,000
• Total annual cost from year two: £8,400 to £24,000
• Setup time: 8 to 16 weeks

The cloud option costs less in year one and roughly the same or less in subsequent years, with none of the hardware risk, maintenance burden, or staffing requirements. The cloud option also gives you access to the best models (GPT-4, Claude, Gemini) rather than limiting you to open-source alternatives that are good but not yet equivalent.

The pragmatic middle ground

Most firms we work with end up in one of three configurations:

Cloud-only with enterprise contracts. Use Azure OpenAI or Anthropic’s API with enterprise agreements, UK data residency, and zero data retention. This suits 80 percent of firms.

Cloud with sensitive data filtering. Route most work through cloud AI but strip or redact personally identifiable information and ultra-sensitive content before it reaches the API. The AI works on anonymised data and the firm re-attaches identifiers locally. This adds complexity but addresses many confidentiality concerns.

Hybrid deployment. Run a local model for document classification and initial triage (where the model sees raw client data) and use cloud AI for drafting and analysis (where input can be controlled). This gives the security benefits of local processing where they matter most while preserving access to superior cloud models for other tasks.

What we build at Formulaic

Across 30 production systems, we have deployed 27 on cloud infrastructure and 3 on hybrid configurations. None of our mid-market law firm clients have required fully local hosting once they understood the contractual protections available from enterprise cloud providers.

Our standard deployment uses Azure OpenAI with UK South data residency, zero data retention, and enterprise data processing agreements. This meets every SRA and data protection requirement we have encountered.

We have built hybrid systems for clients in financial services where regulatory requirements demanded local processing of certain data categories. These work well but cost roughly three times more to build and maintain.

Our advice: do not let the local hosting question delay your AI programme. Start with properly contracted cloud hosting, measure the actual risk against the theoretical risk, and upgrade to local hosting only if specific regulatory or client requirements demand it. The firms that moved fastest on AI are the ones that made practical security decisions rather than perfect ones.