Your SaaS product embeds an LLM for customer-facing features. During procurement, an enterprise prospect requests your SOC 2 Type II report. The auditor’s scope must now include the GPU infrastructure running AI inference — and most SOC 2 guides were written before AI workloads existed. This guide maps the Trust Services Criteria to self-hosted GPU infrastructure running AI models, with practical controls you can implement before the auditor arrives.
Trust Services Criteria for AI Systems
SOC 2 evaluates five Trust Services Criteria. For AI hosting, each applies differently than traditional web applications:
| Criteria | Traditional Application | AI Inference Addition |
|---|---|---|
| Security | Firewalls, access controls | Model weight protection, GPU memory isolation |
| Availability | Uptime SLAs, failover | Inference latency SLAs, GPU health monitoring |
| Processing Integrity | Input validation | Model versioning, deterministic inference, output validation |
| Confidentiality | Encryption, access logs | Prompt/response logging controls, model memorisation risks |
| Privacy | PII handling, consent | Training data provenance, inference data retention |
Most AI companies include Security and Availability at minimum. Add Confidentiality and Privacy if your inference pipeline handles customer data. Private AI hosting simplifies the confidentiality narrative — customer data never touches third-party infrastructure.
Security Controls for GPU Infrastructure
CC6 (Logical and Physical Access Controls) requires demonstrable access restrictions. On a dedicated GPU server, implement SSH key-only authentication with certificate authority signing, role-based access through sudo groups (infra-admin, ml-engineer, app-developer), network segmentation placing the inference server on a private subnet, and API authentication using short-lived JWT tokens rather than static API keys.
CC7 (System Operations) covers monitoring and incident response. GPU-specific monitoring must track VRAM utilisation (unexpected spikes may indicate model extraction attempts), inference request rates (anomalies suggest credential compromise), GPU temperature and ECC error counts (hardware reliability), and model version deployed (drift detection). Ship metrics to a centralised monitoring stack separate from the GPU server. See infrastructure best practices for monitoring architecture.
Availability Controls
CC9 covers risk mitigation. AI inference availability requires different thinking than web application uptime. Define your inference SLA: maximum acceptable latency per request, maximum queue depth before requests are rejected, recovery time objective after GPU failure, and model reload time after server restart.
Document your recovery procedures. vLLM can restart and reload a model in under 60 seconds on fast NVMe storage. Keep model weights on the server’s local NVMe rather than network storage — this is both a performance and availability decision. Maintain automated configuration management so a replacement GPU server can be provisioned from scripts, not tribal knowledge.
Processing Integrity for AI
CC8 addresses change management and processing accuracy. This is where AI workloads diverge most from traditional applications. Model updates are deployments — treat them with the same rigour as code releases. Maintain a model registry with checksums (SHA-256) for every deployed version. Log which model version served each inference request. Validate model outputs against expected patterns before returning them to users.
For open-source models, document the provenance: where the weights were downloaded from, the exact version or commit hash, and the integrity verification performed at download time. Auditors want to see that you know precisely what code is running on your infrastructure.
Evidence Collection Strategy
SOC 2 Type II covers a review period (typically 6-12 months). Collect evidence continuously rather than scrambling before the audit. Automated evidence sources include infrastructure-as-code repositories with full commit history, centralised log aggregation with tamper-evident storage, automated access reviews exported monthly, vulnerability scan reports from weekly scans, and GPU server configuration snapshots captured daily.
Manual evidence includes quarterly access reviews with sign-off, annual risk assessments updated for new AI capabilities, incident response tabletop exercises (at least annually), and security awareness training completion records. GDPR compliance documentation often overlaps with SOC 2 privacy criteria evidence.
Getting Started
Choose a SOC 2 readiness platform (Vanta, Drata, Secureframe) to automate evidence collection. Map your AI-specific controls into the platform’s framework. Begin the Type I audit to validate control design, then run controls for 6 months minimum before the Type II audit. Budget 3-6 months for readiness, and select an auditor experienced with technology companies — ideally one who has audited AI-focused organisations. Teams running customer-facing AI chatbots or document processing should include these workloads in scope from day one rather than expanding scope later.
SOC 2-Ready AI Infrastructure
Dedicated GPU servers with the isolation, logging, and access controls your SOC 2 auditor expects. UK-hosted, fully documented.
Browse GPU Servers
