Security Scoring Methodology
How BestClawHosting evaluates OpenClaw hosting provider security. Risk-based scoring grounded in OWASP Agentic Security, OWASP Top 10, and OWASP LLM Top 10.
1. Introduction
Something fundamental is changing. People are starting to trust AI agents with their most valuable digital assets — their email accounts, their Telegram conversations, their API keys, their browser sessions, their business operations. This isn’t a toy anymore. When you give an agent access to your inbox and your wallet, you’re placing real trust in the platform that hosts it.
We believe that users who make that leap of trust deserve to understand exactly what they’re risking — and what protects them. Not in the language of security engineers, but in terms that actually mean something to someone choosing a hosting provider for the first time.
Why We Built This
Traditional security assessments speak in technical dimensions: “execution isolation maturity,” “network segmentation,” “encryption at rest.” These are important concepts, but they don’t help a user who just wants to know: can someone steal my data? Can my agent go rogue? Will I get a surprise bill?
Meanwhile, providers describe their security in marketing language: “enterprise-grade,” “fully isolated,” “military-grade encryption.” These words sound reassuring but say nothing specific. There’s a gap between what the security community measures and what users actually need to know.
We built this methodology to close that gap. Instead of asking “how strong is the provider’s infrastructure?” we ask: “Which provider is least likely to cause me problems?” That’s a risk question, not an infrastructure checklist. And it’s the question every user actually cares about.
We translated 30 established security threats from three OWASP frameworks — the industry standard for identifying what goes wrong in software systems — into 10 risks that any user can understand. Each one is framed as a question you’d actually ask before handing your credentials to a provider. Then we score every provider on how well they address each risk, weighted by how much evidence backs up their claims.
Who This Is For
This methodology is written for people who want to use an AI agent without having to become a security engineer first. You shouldn’t need to understand container isolation, network policies, or threat modeling to make a safe choice. We score the provider’s contribution to your safety — not your technical ability. A provider that handles security invisibly scores higher than one that requires you to configure firewalls yourself.
If you’re a security professional, the framework alignment section maps our categories to the specific OWASP risks underneath, and the OWASP coverage tags on each risk category show exactly which threats are addressed.
2. The 10 Risk Categories
These 10 categories aren’t arbitrary. We started from three established threat frameworks — the OWASP Top 10 for Agentic Applications (2026), the OWASP Top 10 for Web Applications (2021), and the OWASP Top 10 for LLM Applications (2025) — which together define 30 known security risks. Then we asked: what do these risks actually mean for a user?
Related threats were grouped by their real-world impact. Five different OWASP risks all lead to the same outcome — someone takes control of your agent — so they all live under “Agent Hijacking.” The result is 10 categories that cover every known threat while staying in language anyone can understand.
Each category is framed as a question you’d actually ask before trusting a provider. For each, we list the specific things that can go wrong — these are what we assess.
Data Leaks
“Can anyone else see my data?”
Your agent handles sensitive data — conversations, files, emails. This risk covers whether that data can leak to other users, provider employees, or external attackers.
- •Another user's agent accesses your files, messages, or credentials
- •Provider employees can read your conversations or data
- •Your data gets exposed in a breach of the provider's infrastructure
- •Your conversation history or prompts are used to train AI models
- •Your agent's logs contain sensitive data that isn't properly protected
- ✓Per-user isolation preventing cross-agent contamination
- ✓Encryption of data at rest
- ✓Data not used for AI model training
- ✓Minimal data retention and log sanitization
- ✓User guidance on sensitive data sharing
- ✓Employee access controls with audit trail
Agent Hijacking
“Can someone take over my agent?”
Your agent has real access to your accounts. If someone can redirect its behavior — through prompt injection, account takeover, or poisoned memory — they control everything your agent can access.
- •Someone sends a crafted message that makes your agent do something harmful (prompt injection)
- •An attacker gains control of your hosting account
- •A malicious tool, plugin, or extension compromises your agent
- •Another user's agent on the same infrastructure interferes with yours
- •Your agent's memory or context gets poisoned, altering its behavior across sessions
- •Your agent is tricked into generating and executing dangerous code
- ✓Separation of instructions from external data
- ✓Hardware-enforced sandboxing for code execution
- ✓Human-in-the-loop for goal changes and code execution
- ✓Memory integrity protection with provenance tracking
- ✓Output encoding and sanitization
- ✓Container escape prevention
Credential Exposure
“Are my keys and passwords safe?”
You give your agent API keys, Telegram tokens, email passwords, and other credentials. This risk covers whether those secrets are properly stored, protected, and prevented from leaking.
- •API keys (Anthropic, OpenAI) are leaked from the provider's systems
- •Your Telegram token or email credentials are exposed
- •Credentials are visible in logs, error messages, or debug output
- •The provider stores your credentials in plaintext
- •A low-privilege service inherits credentials from a higher-privilege one (privilege escalation)
- •Your agent's system prompt — which may contain secrets — gets extracted
- ✓Unique managed identities with least-privilege access
- ✓Credential leak detection in outputs
- ✓Credentials excluded from AI model context
- ✓Encrypted credential storage
- ✓Credential lifecycle management with rotation
Agent Going Rogue
“Can my agent do things I didn't authorize?”
AI agents have autonomy — that's the point. But without proper guardrails, an agent can send messages you didn't approve, spend money without limits, or use tools in ways you didn't intend.
- •Your agent sends messages you didn't approve (spam, wrong info to contacts)
- •Your agent spends money without limits (API calls, purchases)
- •Your agent uses a legitimate tool in an unsafe or unintended way
- •Your agent accesses external services beyond what you intended
- •Your agent gets stuck in a loop burning resources and credits
- •Your agent has more permissions or tools than it actually needs
- ✓Least privilege and least agency
- ✓High-risk tool classification and gating
- ✓Resource consumption limits and rate limiting
- ✓Emergency kill switch
- ✓Continuous behavioral monitoring
Losing Everything
“Can I lose my data or get locked out?”
Your agent accumulates valuable data — conversation history, configurations, files. This risk covers whether that data can be lost to outages, missing backups, provider shutdown, or inability to export.
- •Provider has an outage and your agent is down with no timeline
- •Your data is lost because there were no backups
- •The provider shuts down or goes bankrupt and you lose everything
- •You can't export your data or agent configuration when you want to leave
- •Your data isn't actually deleted after you cancel
- •A failure in one part of the system triggers a chain of increasingly destructive failures
- ✓Tested backups with verified restore
- ✓Data export capability
- ✓Circuit breakers and blast-radius limits
- ✓Clear data lifecycle policy
- ✓Provider stability signals
Surprise Costs
“Will I get unexpected bills or policy changes?”
AI agent hosting involves real money — subscription fees, API credits, resource consumption. This risk covers whether you'll face unexpected charges or unfavorable changes to pricing and terms.
- •The provider changes pricing without adequate notice
- •Hidden costs you didn't anticipate when signing up
- •Terms of service change in ways that affect how your data is used
- •The provider is acquired and the new owner has different policies
- •Your agent's resource consumption spikes unexpectedly, causing a large bill
- ✓Hard spending caps
- ✓Transparent, complete pricing
- ✓Usage monitoring with alerts
- ✓Price change notification policy
Accountability
“Who's responsible when something goes wrong?”
When something breaks — a breach, an outage, or your agent misbehaving — you need to know who's responsible, what happened, and what's being done about it. This risk covers the provider's accountability infrastructure.
- •Your data is stored in a jurisdiction you're not comfortable with
- •The provider doesn't comply with GDPR or privacy regulations
- •There's no audit trail of what your agent did (liability if agent misbehaves)
- •The provider has no incident response process for security events
- •No active security monitoring — breaches go undetected
- ✓Comprehensive audit logging with tamper evidence
- ✓Incident response process
- ✓Breach notification with defined timeline
- ✓Agent action audit trail
Supply Chain
“What if a tool or dependency gets compromised?”
Every AI agent hosting provider depends on external components — AI models, third-party tools, MCP servers, open-source libraries. If any of these are compromised, your agent is affected.
- •A third-party tool or MCP server your agent uses gets compromised
- •The AI model provider introduces a vulnerability or backdoor
- •Open-source dependencies have known vulnerabilities that aren't patched
- •The provider's update mechanism is compromised (auto-update without verification)
- •The provider's CI/CD pipeline is compromised, delivering malicious code
- ✓MCP server and tool vetting
- ✓Dependency scanning and patching
- ✓AI provider data policies
- ✓Runtime external data validation
- ✓Component inventory (SBOM)
- ✓Model training and fine-tuning security
- ✓Build pipeline and image integrity
Platform Security
“Is the hosting platform itself built securely?”
Separate from agent-specific risks, the hosting platform itself — the dashboard, APIs, and infrastructure — needs to be secure. Traditional web vulnerabilities in the platform can expose everything.
- •The platform has access control flaws letting attackers reach other users' data
- •The dashboard or API is vulnerable to injection attacks (SQL injection, XSS)
- •The platform has security misconfigurations (open storage, verbose errors, default credentials)
- •The platform's authentication is weak (no MFA, brute-forceable, poor session management)
- •Communication between multiple agents in the same system can be intercepted or forged
- ✓Injection attack prevention
- ✓Strong authentication with MFA
- ✓Secure platform defaults and hardening
- ✓TLS everywhere with strong configuration
- ✓Server-side access control
- ✓SSRF and metadata service protection
- ✓Authenticated inter-agent communication
- ✓Independent security testing
Misinformation & Trust
“Can I trust what my agent tells me?”
AI agents can produce confident but wrong answers, and attackers can manipulate agent outputs to trick you into approving harmful actions. This risk covers output reliability and trust exploitation.
- •Your agent produces false but confident outputs that you act on (hallucinations)
- •An attacker manipulates your agent's outputs to trick you into approving a harmful action
- •Your agent fabricates convincing explanations to bypass your security judgement
- •Your agent gives wrong answers in sensitive contexts (medical, legal, financial)
- ✓Approval workflows that prevent rubber-stamping
- ✓Independent verification for high-impact decisions
- ✓Prompt injection and output manipulation monitoring
- ✓Undo and rollback capability
- ✓Transparency about AI uncertainty
3. How We Score
For each risk category, we assess two things independently:
We look at how many sub-risks the provider addresses, and how thoroughly. A provider that addresses all sub-risks with specific technologies and defense-in-depth scores higher than one that addresses a few with generic claims.
How much evidence supports the provider’s claims? A claim backed by public documentation, open-source code, or a third-party audit earns more credit than one that exists only in marketing copy.
Evidence Grades
Evidence grades determine how much credit a provider receives for their claims. A provider can make ambitious claims, but without evidence, those claims earn reduced or zero credit.
| Grade | Label | Weight | What It Means |
|---|---|---|---|
| U | Unknown | ×0 | No information available — this is the default. Unverifiable claims receive zero credit. |
| C | Claimed | ×0.25 | Marketing language, generic security statements, or claims without verifiable specifics |
| D | Documented | ×0.8 | Named technologies in public docs, visible configuration, specific technical detail that can be externally checked |
| V | Verified | ×1 | Third-party audit, compliance certification, published pentest results, or architecturally guaranteed (e.g., open-source code) |
The single most impactful action any provider can take is to document their security practices publicly. Moving from Claimed (C) to Documented (D) more than triples credit for that risk category — without changing any actual security controls. Just document what you already do.
Weighting
All 10 risk categories carry equal weight in the combined score, and all mitigations within a category are weighted equally. This is deliberate: unequal weighting would require subjective judgments about which risks matter more, and those judgments would vary by use case. A developer running a coding agent cares most about data leaks and credential exposure; someone running a customer-facing bot cares more about agent hijacking and misinformation. Equal weighting avoids embedding our assumptions into your score.
Total Score
A provider’s total score is the sum of each category’s maturity score (0–10) multiplied by its evidence weight, across all 10 categories. Maximum possible score is 100.
Example: a category scored 8 with Documented evidence = 8 × 0.8 = 6.4
Overall Levels
| Level | What It Means |
|---|---|
| Basic | Essential protections are missing or unverifiable. Users should investigate independently before trusting this provider with sensitive credentials. |
| Good | Core risks are addressed with notable gaps. Minimum recommended level for non-expert users. |
| Strong | Most risks are well-addressed with documented evidence. Suitable for users handling sensitive data. |
| Excellent | Comprehensive, verified security across all risk categories. Third-party validation or hardware guarantees. No provider has achieved this yet — it represents a target for the industry. |
4. Framework Alignment
Our 10 risk categories provide exhaustive coverage of three established security frameworks. Every risk identified by these frameworks maps to at least one of our categories.
The primary threat model. 10 risks specific to AI agent systems: goal hijack, tool misuse, identity abuse, supply chain, code execution, memory poisoning, inter-agent communication, cascading failures, trust exploitation, and rogue agents.
ASI-01 through ASI-10 — all mapped to our categories.
Traditional web security risks that apply to the hosting platform itself: broken access control, cryptographic failures, injection, insecure design, misconfiguration, vulnerable components, authentication failures, integrity failures, logging gaps, and SSRF.
A01 through A10 — all mapped to our categories.
Risks specific to large language model applications: prompt injection, sensitive info disclosure, supply chain, data poisoning, improper output handling, excessive agency, system prompt leakage, vector weaknesses, misinformation, and unbounded consumption.
LLM01 through LLM10 — all mapped to our categories.
30 OWASP risks. 10 categories. Full coverage. Each OWASP risk maps to at least one of our risk categories. Below is the detailed mapping — every mitigation we assess, grounded in specific OWASP framework references.
Data Leaks
OWASP basis: ASI-04, LLM02, A02
- •Each user's agent runs in an isolated environment with enforced resource boundaries, not just app-level permissions(ASI-04 §5)
- •User A's agent cannot access User B's container filesystem, environment variables, or network(A01)
- •Access control based on record ownership, not global permissions(A01 §3)
- •Proven encryption algorithms like AES-256 for data at rest(A02 §2)
- •Keys stored in managed services (KMS, Vault), never hard-coded in source or config(A02 §5)
- •Container filesystems, databases, and backups use encrypted storage with separate keys(A02)
- •User data not fed into model training pipelines — documented cases of LLMs regurgitating training data including API keys(LLM02)
- •Privacy-preserving techniques including federated learning and differential privacy(LLM02 §3)
- •Sanitize and validate all data before processing(LLM02 §1)
- •Continuously monitor model outputs for signs of data exposure(LLM02 §7)
- •Secure system configurations to prevent unintended data access(LLM02 §8)
- •Clear guidance on safe vs. unsafe data types (PII, credentials, proprietary data) when interacting with agents(LLM02 §6)
- •Admin access logging with time, user, action, and justification(A01)
- •Separation between customer support read-only access and full admin privileges(A01)
- •Per-user data isolation with no cross-contamination(LLM02)
Agent Hijacking
OWASP basis: ASI-01, ASI-05, ASI-06, LLM01, LLM04, LLM05
- •External content (emails, documents, web pages) never processed as trusted instructions(ASI-01 §1)
- •Clear boundaries between user instructions, system prompts, and external content using delimiters and structured formats(ASI-01 §4)
- •Trust boundaries between the LLM, external sources, and extensible functionality(LLM01 §2)
- •Hardware-enforced isolation (VMs, containers with strict resource limits) — software-only sandboxes are insufficient(ASI-05 §1)
- •Static and dynamic analysis before execution to detect dangerous patterns(ASI-05 §2)
- •Strict allowlists for permitted commands; dangerous operations blocked(ASI-05 §4)
- •Auto-run disabled; generated code never executes without explicit gating(ASI-05 §3)
- •Explicit user approval before executing actions that deviate from original intent(ASI-01 §3)
- •Explicit user approval before executing any code; never auto-approve based on repository content(ASI-05 §3)
- •Human approval for privileged operations(LLM01 §4)
- •Memory stores protected with encryption and least-privilege access; malicious content scanning before commit(ASI-06 §1)
- •Cryptographic signatures for all memory entries; integrity validation before retrieval(ASI-06 §2)
- •Source tracking for every piece of information; reliability weighted by provenance(ASI-06 §3)
- •User-accessible tools to inspect, edit, and delete stored memory(ASI-06 §4)
- •Time-based expiration for certain memory types to limit long-term poisoning(ASI-06 §4)
- •Version control on memory entries to enable rollback(ASI-06 §4)
- •Model output encoded to mitigate JavaScript or Markdown injection(LLM05 §3)
- •Parameterized queries for all database operations involving LLM output(LLM05 §4)
- •Content Security Policy headers to prevent XSS from LLM-generated content(LLM05 §5)
- •LLM output sanitized before use in file paths, system commands, or API calls(LLM05 §6)
- •Hardware-enforced isolation required(ASI-05 §1)
- •Privilege escalation from container to host blocked(A01)
Credential Exposure
OWASP basis: ASI-03, LLM07, A02, A07
- •Agents treated as identities with explicit, scoped permissions; unique managed identities per agent; short-lived session-based credentials(ASI-03 §1)
- •Minimum required permissions; just-in-time access granted only when needed, revoked immediately(ASI-03 §4)
- •Explicit authentication required for agent-to-agent communication(ASI-03 §2)
- •Output filtering to detect and block sensitive information in responses(LLM02 §5)
- •Automated PII detection and redaction in LLM outputs(LLM02)
- •Credentials never logged or transmitted in URLs(A02)
- •HTTP caching disabled for responses containing sensitive information(A02 §8)
- •Credentials, API keys, and secrets never included in system prompts; secure secrets management instead(LLM07 §4)
- •Enforcement at the application/policy layer, outside the model — not in natural language(LLM07 §2)
- •LLMs configured to refuse requests attempting to extract system instructions(LLM07 §1)
- •System prompts kept minimal, avoiding detailed business logic or sensitive information(LLM07 §5)
- •Monitoring for prompt extraction attempts with rate limiting(LLM07 §6, §7)
- •Proven encryption algorithms like AES-256 for credentials at rest(A02 §2)
- •Keys stored in managed services; never hard-coded(A02 §5)
- •Regular key rotation with access controls on key usage(A02 §6)
- •Short-lived, session-based credentials revoked after task completion(ASI-03 §1)
- •Continuous audit of all credential usage; break-glass emergency revocation(ASI-03 §3)
- •Stateful sessions invalidated on logout; JWT tokens short-lived(A07 §7)
Agent Going Rogue
OWASP basis: ASI-02, ASI-10, LLM05, LLM06
- •LLM agent extensions limited to only the minimum necessary(LLM06 §1)
- •Authorization enforced in downstream systems, not by the LLM(LLM06 §2)
- •Complete mediation — all requests validated against security policies(LLM06 §3)
- •Strict, granular, just-in-time permissions on all tools(ASI-02 §1)
- •Tools that send, delete, or execute identified; explicit user approval enforced; toxic tool combinations blocked(ASI-02 §2)
- •All LLM outputs validated against strict schemas before passing to tools(ASI-02 §3)
- •User approval for sensitive operations including financial transactions and file deletion(LLM06 §4)
- •Dynamic resource allocation management to prevent single-user overconsumption(LLM10 §1)
- •Timeouts and throttling for resource-intensive operations(LLM10 §2)
- •Rate limiting per user/API key(LLM10 §3)
- •Agent-level rate limiting on tool invocations to prevent brute-force abuse(LLM06 §7)
- •Cost controls and alerts for usage anomalies(LLM10 §6)
- •Outbound network access restricted to only necessary APIs and services(LLM10 §4)
- •Fair queue management preventing single-agent resource monopolization(LLM10 §8)
- •Instant disable capability; auditable (who, when, why); tested regularly(ASI-10 §2)
- •Baselines of normal behavior; every decision, tool call, and state change logged; alerts on deviation(ASI-10 §1)
- •Monitoring for anomalous actions; immediate shutdown on rogue behavior detection(ASI-10 §5)
- •Stable goal identifiers with regular audit of alignment between actions and stated goals(ASI-10 §4)
Losing Everything
OWASP basis: ASI-08, LLM10 (availability)
- •Transactional semantics for multi-step operations; rollback of entire workflows if any step fails(ASI-08 §3)
- •Users can take their data and leave at any time
- •Clear trust boundaries between agent workflows; circuit breakers to halt propagation; blast-radius caps(ASI-08 §1)
- •Multi-agent consensus for critical decisions; ground truth validation against external sources(ASI-08 §2)
- •Users know what happens to their data at every stage — creation, storage, backup, deletion
- •A provider that might disappear is a data loss risk regardless of technical controls
Surprise Costs
OWASP basis: LLM10 (cost)
- •Dynamic resource allocation to prevent single-user overconsumption(LLM10 §1)
- •Rate limiting per user/API key(LLM10 §3)
- •Maximum tokens, prompt complexity, or processing time limits per request(LLM10 §7)
- •All possible charges listed upfront with no hidden costs
- •Continuous resource usage monitoring with logging for unusual patterns(LLM10 §5)
- •Cost controls and alerts for usage anomalies(LLM10 §6)
- •Adequate notice before pricing or terms changes
Accountability
OWASP basis: A09
- •All security-relevant events logged: logins, permission changes, tool invocations, data access(A09)
- •Every decision, tool call, and state change logged(ASI-10 §1)
- •Access control failures logged with admin alerts on repeated failures(A09 §5)
- •Structured log format (JSON) with timestamp, user, action, result, and source(A09 §2)
- •Immutable audit trails with integrity checks for high-value transactions(A09 §3)
- •Centralized log aggregation across all services(A09 §6)
- •Log retention policy aligned with compliance requirements(A09 §7)
- •Monitoring dashboards and rule-based alerting; penetration testing triggers alerts(A09 §4, §9)
- •Defined process for detection, triage, notification, and remediation
- •Users notified when their data is affected; GDPR requires notification within 72 hours
- •Every decision, tool call, and state change logged(ASI-10 §1)
- •All actions taken by LLM agents logged for audit purposes(LLM06 §6)
Supply Chain
OWASP basis: ASI-04, LLM03, LLM04, A06, A08
- •Only MCP servers from official/verified registries; Software Bill of Materials maintained; dependencies pinned(ASI-04 §1)
- •MCP code and dependencies scanned pre-deployment; unverified dependencies blocked(ASI-04 §2)
- •Monitoring for typosquatting and unexpected behavior changes between versions(ASI-04 §2)
- •All components monitored for known vulnerabilities using automated tools(A06)
- •Regular dependency scanning for known vulnerabilities(LLM03 §6)
- •Software Composition Analysis (SCA) in CI/CD with build gates on critical/high vulnerabilities(A06 §8)
- •Security bulletin subscriptions; policy for evaluating unmaintained libraries(A06 §3, §5)
- •Models, adapters, and plugins only from trusted and verified providers(LLM03 §1)
- •Model signatures and provenance chains verified(LLM03 §5)
- •Repository authenticity verified before downloading models or plugins(LLM03 §9)
- •All external data treated as potentially hostile; cryptographic integrity checks(ASI-04 §3)
- •Data provenance validated before ingestion; API responses, RAG sources, and tool definitions verified(ASI-04 §3)
- •Complete inventory of all agentic components: hosts, agents, extensions, MCPs, tools, models — with source, version, permissions, and review dates(ASI-04 §4)
- •Shadow MCPs operating outside security visibility identified(ASI-04 §4)
- •Supply chain of training data verified with trusted sources(LLM04 §1)
- •Anomaly detection during training to identify poisoned data patterns(LLM04 §2)
- •Training data sanitized to remove potentially malicious content(LLM04 §3)
- •Adversarial robustness testing to identify backdoors(LLM04 §4)
- •Sandboxed environments for model training and fine-tuning(LLM04 §5)
- •Model behavior monitoring during and after training for poisoning indicators(LLM04 §7)
- •Software and data integrity verified by digital signatures(A08)
- •Subresource Integrity (SRI) for CDN resources(A08 §6)
- •No automatic pulling of new versions without review(LLM03 §3)
- •Updates integrated only after testing in isolated environments(LLM03 §4)
- •Configuration changes and software updates reviewed by separate parties(A08 §7)
Platform Security
OWASP basis: ASI-07, A01, A03, A04, A05, A07, A10
- •All database queries use parameterized statements or ORMs, never string concatenation(A03 §1, §2)
- •Container and process management uses libraries, not shell exec with user-supplied strings(A03)
- •User-provided names and configuration validated against strict allowlists(A03 §3)
- •Special characters escaped for any remaining dynamic queries(A03 §4)
- •Untrusted data never deserialized; strict schema validation with disabled external entity processing(A08 §8)
- •SAST/DAST/IAST security scanning in CI/CD pipeline(A03 §6)
- •Multi-factor authentication to prevent credential stuffing, brute force, and stolen credential reuse(A07)
- •Session identifiers invalidated after logout; tokens short-lived(A07 §7)
- •Passwords checked against known-breached lists(A07 §3)
- •Failed login rate limiting with progressive delays(A07 §4)
- •Strong adaptive password hashing (Argon2, bcrypt, scrypt)(A07 §6)
- •No default credentials, no verbose error messages, no unnecessary services exposed(A05)
- •Repeatable hardening process identical across dev, staging, and production(A05 §1)
- •Minimal container images without unnecessary tools or services(A05 §2)
- •Security headers: HSTS, Content-Security-Policy, X-Frame-Options, X-Content-Type-Options(A05 §5)
- •Automated verification of security configuration across all environments(A05 §6)
- •Error handling never exposes stack traces or internal paths; details logged server-side only(A05 §8)
- •TLS 1.2 or 1.3 minimum for all data in transit; weak protocols and ciphers disabled(A02 §3)
- •Certificates properly validated and not expired(A02 §9)
- •Deny access by default, except for public resources(A01 §1)
- •Single application-wide access control mechanism reused throughout(A01 §2)
- •Rate limiting on API and controller access(A01 §6)
- •User-provided URLs validated against a positive allowlist of permitted schemas, ports, and destinations(A10 §4)
- •All private IP ranges blocked in outbound requests (10.0.0.0/8, 172.16.0.0/12, 192.168.0.0/16, 169.254.0.0/16)(A10 §4)
- •HTTP redirects disabled or strictly controlled to prevent redirect-chain bypass(A10 §5)
- •DNS resolved at validation time to prevent DNS rebinding and TOCTOU attacks(A10 §6)
- •IMDSv2 enforced for cloud metadata access; DNS allowlist for internal services(A10 §8, §9)
- •Authenticated and encrypted inter-agent communication; mutual TLS for all agent-to-agent channels(ASI-07 §1)
- •Message integrity checks (HMAC, digital signatures); replay detection using nonces(ASI-07 §2)
- •Identity of every sender validated; peer agents never assumed trustworthy(ASI-07 §3)
- •Sensitive agents segmented from general-purpose agents with distinct network boundaries(ASI-07 §4)
- •Threat modeling for authentication, access control, business logic, and key flows(A04 §2)
- •Security requirements integrated into the development lifecycle(A04 §1)
- •Unit and integration tests validate threat model assumptions(A04 §7)
Misinformation & Trust
OWASP basis: ASI-09, LLM09
- •Approval workflows force genuine review; agent articulates decisions with data sources and intended tools(ASI-09 §2)
- •Clear escalation paths with defined thresholds for automatic escalation(ASI-09 §3)
- •Human reviewers trained on LLM hallucination patterns and common failure modes(LLM09 §3)
- •Independent (non-AI) verification for YMYL decisions; agent recommendations cross-referenced with authoritative sources(ASI-09 §1)
- •Domain-specific validation for high-stakes applications (legal, medical, financial)(LLM09 §7)
- •LLM input and output logged to detect prompt injection attempts(LLM01 §7)
- •External content filtered and sanitized before LLM processing(LLM01 §5)
- •Rollback of entire workflows if any step fails; safe failure modes defined(ASI-08 §3)
- •LLMs express uncertainty or provide confidence scores when appropriate(LLM09 §4)
- •LLMs cite sources for factual claims(LLM09 §5)
- •Critical information cross-verified from multiple sources(LLM09 §6)
- •Retrieval-augmented generation (RAG) with verified knowledge bases to ground responses(LLM09 §1)
5. Trust Tiers
Not all information is equally trustworthy. We distinguish between three tiers based on the source and verification level:
Information gathered independently from public sources: provider websites, documentation, public repositories, DNS records, and login flows. This is the default for all providers.
The provider has verified domain ownership and provided security information directly. Their responses are assessed for specificity and cross-referenced against our existing research. Provider-claimed information is clearly labeled — the provider’s name is on it.
Claims that have been independently confirmed through third-party audits, compliance certifications, or our own testing. This is the highest trust level and the hardest to achieve.
6. For Providers
We score what we can see. If your security is good but invisible, your score won’t reflect it. The single biggest lever you have is making your existing practices verifiable.
How Evidence Grades Affect Your Score
Every mitigation is scored on a four-level evidence scale: Verified, Documented, Claimed, Unknown. These map to how security audits actually work — the gap between “we do this” and “here’s proof we do this” is where most real-world security failures hide. The same control can earn dramatically different credit depending on the evidence behind it:
Score Impact by Action
These are the highest-impact actions mapped to the specific risk categories they improve. The “effort” column reflects implementation complexity, not score impact — documentation changes are often the highest-impact, lowest-effort actions.
| Action | Risks Improved | Effort |
|---|---|---|
| Publish a security page naming specific technologies (isolation, encryption, auth) | All 10 categories | Low |
| Document your isolation model: container runtime, namespace config, resource limits | Data Leaks, Agent Hijacking, Platform Security | Low |
| Publish backup schedule, encryption algorithm, retention and deletion policy | Data Leaks, Losing Everything, Credential Exposure | Low |
| Add hard spending caps and usage alerts | Surprise Costs, Agent Going Rogue | Medium |
| Implement MFA and document your auth model | Credential Exposure, Agent Hijacking, Platform Security | Medium |
| Add a kill switch with audit logging | Agent Going Rogue, Accountability | Medium |
| Implement human-in-the-loop approval for sensitive tool calls | Agent Going Rogue, Agent Hijacking | Medium |
| Add data export functionality | Losing Everything | Medium |
| Publish an incident response process with notification timeline | Accountability | Low |
| Publish a dependency update policy and maintain an SBOM | Supply Chain | Medium |
| Commission a third-party pentest and publish the summary | Platform Security (moves to Verified) | High |
| Achieve SOC 2 Type II certification | Data Leaks, Credential Exposure, Losing Everything, Accountability, Platform Security (moves to Verified) | High |
What Counts as Evidence
- A dedicated security or trust page with named technologies
- Public documentation describing your architecture
- Open-source infrastructure code (Dockerfiles, Terraform, Helm charts)
- Published pentest reports or compliance certifications with dates
- A security.txt file with a responsible disclosure policy
- “Enterprise-grade security” without specifics
- “Fully isolated” without naming the isolation technology
- “Encrypted” without naming the algorithm or key management
- No security page, no documentation, no public disclosure policy
- Claims that contradict observable behavior (e.g., “MFA required” but login has no MFA prompt)
Dispute Process
If you believe your score doesn’t reflect your actual security practices, we’re building a self-service process where providers can claim their profile and submit evidence directly. This will be available soon. In the meantime, scores are updated whenever we detect new public evidence during our regular review cycles.
7. FAQ
Why risk categories instead of technical dimensions?
When you hand an AI agent your email credentials, your API keys, and access to your money, you care about outcomes — can someone steal my data? Can my agent go rogue? Will I get a surprise bill? You don't care about "execution isolation maturity" or "network segmentation posture." Traditional security assessments measure infrastructure. We measure the probability of bad outcomes. The technical detail (150+ specific mitigations grounded in OWASP framework references) is in the Framework Alignment section for security professionals, but the risk categories are what actually help you choose a provider.
How do you cover 30 OWASP risks with just 10 categories?
By grouping related risks by their real-world impact. OWASP's ASI-01 (Agent Goal Hijack), ASI-05 (Code Execution), ASI-06 (Memory Poisoning), LLM01 (Prompt Injection), and LLM04 (Data Poisoning) are five different attack vectors — but they all lead to the same user outcome: someone takes control of your agent. They're all assessed under "Agent Hijacking" with 6 distinct mitigations and 20+ specific assessment criteria. The Framework Alignment section shows the full mapping: every mitigation, every OWASP section reference, every assessment criterion. Nothing is lost in the grouping.
What's the difference between "What can go wrong" and "What we look for"?
"What can go wrong" describes the user-facing risks — the bad outcomes you want to avoid. "What we look for" describes the mitigations we assess — the defenses a provider should have. These are different things. A risk like "someone sends a crafted message that hijacks your agent" (prompt injection) is mitigated by multiple defenses: input/output separation, hardware sandboxing, human-in-the-loop approval. The Framework Alignment section goes deeper — each mitigation is broken down into specific assessment criteria with OWASP references.
What does "Provider-Claimed" mean?
The provider verified ownership of their domain and provided security information directly. Their claims are assessed for specificity and cross-referenced against our independent research. Provider-claimed information is clearly labeled so you always know the source. Think of it like manufacturer specs vs. independent lab testing — both are useful, but one comes from the company itself. A provider who claims "per-user VM isolation" is publicly accountable for that statement. If users discover it's not true, the provider's credibility is on the line.
Why do documented providers score higher than undocumented ones with better tech?
Because undocumented security is indistinguishable from no security. Consider two providers: Provider A says "we use gVisor with hardware attestation and per-user Kubernetes namespaces" but has no public documentation. Provider B uses basic Docker containers but publishes their Dockerfile, seccomp profiles, and network policies. We can verify Provider B's claims. We can't verify Provider A's. Provider A might be lying, exaggerating, or describing aspirational architecture. Provider B's security is inspectable. This isn't a flaw in our methodology — it's a feature. If you can't verify it, you can't trust it.
I self-host — isn't that more secure?
It depends entirely on what you do with the control. Self-hosting means you're responsible for all 10 risk categories: isolation, credential management, patching, backups, monitoring, access control, incident response. If you actively manage all of these, self-hosting can be very secure. But most users don't — they run a Docker container on a VPS with default settings, no monitoring, no backups, and root access everywhere. Our scoring reflects the provider's contribution to your safety, not your personal competence. Self-hosted solutions where the user manages everything score as if the provider contributes nothing — because for a non-technical user, that is the reality.
How do critical failure caps work?
Certain gaps are so fundamental that no amount of strength elsewhere can compensate. If a provider runs all users' agents in the same process with no isolation, their Data Leaks score is capped at 3/10 regardless of their encryption, logging, or documentation. If the Docker socket is exposed inside containers, Agent Hijacking is capped at 2/10 because any agent can escape to the host. These caps exist because some architectural decisions are load-bearing — getting them wrong undermines everything built on top.
How often are scores updated?
We review all providers monthly by re-checking their documentation, security pages, login flows, and public infrastructure. Material changes trigger immediate re-evaluation: new security documentation, compliance certifications, architectural changes, security incidents, or community reports. When our methodology itself changes (new risk criteria, updated OWASP mappings), we re-evaluate all 42 providers simultaneously to ensure consistency.
Which pricing tier do you score?
We score the highest publicly available tier, because that represents the best security the provider offers. If your free tier uses shared containers but your paid tier uses dedicated VMs with hardware isolation, the paid tier is what we assess. We note which tier in the review. This means a provider's score reflects their best, not their cheapest offering.
Can a provider game the scoring?
Not sustainably. Our evidence-based system is designed to resist gaming. Claiming "enterprise-grade security" without specifics earns only quarter credit. Documenting technologies that don't match observable behavior (e.g., claiming MFA when the login page doesn't have it) triggers a credibility penalty. The most effective strategy is also the most honest one: actually implement security controls and document them publicly. A provider who documents what they genuinely do will always outscore one who exaggerates.
8. Versioning & Disclosure
Current Version
Version 2.0 — Effective February 2026. Major overhaul introducing risk-based scoring with full OWASP coverage.
How We Update
The methodology is a living document. We update when new threat vectors emerge, frameworks publish new versions, or industry feedback warrants changes. All changes are versioned, dated, and documented. When a methodology change causes score changes, we re-evaluate all providers simultaneously.
Changelog
- Risk-based scoring: 10 categories framed as user questions
- Full OWASP coverage: Agentic Top 10 (2026), Web App Top 10 (2021), LLM Top 10 (2025) — 30 risks mapped
- New categories: Supply Chain, Surprise Costs, Agent Going Rogue, Misinformation & Trust
- Trust tiers: Unverified / Provider-Claimed / Independently Verified
- Evidence-grade system retained (V/D/C/U)
- 42 providers re-evaluated under v2 methodology
- Infrastructure-focused: 6 dimensions (isolation, access, network, data, ops, transparency)
- Two-axis scoring: maturity (1-10) × evidence (V/D/C/U)
- 42 providers scored
- Superseded by v2 risk-based approach