CASE STUDY
Prompt Injection
Agent Tools
Severity: CRIT
Agent coerced into leaking customer data via “helpful” tool calls
A benign-looking prompt caused an agent to call internal tools with attacker-controlled arguments (unsafe schema + missing allowlist),
resulting in cross-tenant document access.
What failed
- No strict schema validation on tool arguments
- Tool allowlist missing (agent could call sensitive actions)
- Authorization handled “after the fact” instead of at the data boundary
Fix that held
- Tool allowlist + per-tool schemas (reject unknown/extra fields)
- Enforce tenant-scoped authorization at the data access layer
- Prompt/tool tracing + alerts for anomalous tool invocation patterns
Need this tested? →
CASE STUDY
RAG
Vector DB
Severity: HIGH
RAG document harvesting through weak metadata filters
Retrieval queries could be manipulated to enumerate documents across tenants due to inconsistent filter enforcement
between ingestion and retrieval.
What failed
- Tenant filter applied only in app logic—not enforced in the vector DB query layer
- Metadata fields leaked in responses and logs
- No rate limiting / anomaly detection on retrieval patterns
Fix that held
- Hard tenant isolation at the vector DB namespace/collection level
- Strict retrieval filter enforcement + output minimization
- Throttle and alert on high-cardinality retrieval patterns
Scope a RAG test →
CHECKLIST
Pre-Launch
Startups
Pre-launch AI security checklist (fast)
A minimal set of controls that remove the most common “LLM app surprise” failures.
Checklist
- Tools: allowlist + strict schemas + argument validation
- RAG: enforce tenant isolation at the DB/query layer
- Egress: restrict outbound network + DNS allowlist (stop SSRF chains)
- Authz: object-level authorization for docs, actions, and tools
- Telemetry: prompt/tool traces + anomaly alerts
- Abuse: rate limits + cost ceilings + prompt injection monitoring
- IR: LLM-specific incident playbook (revocation, rollback, containment)
# Example: tool allowlist (concept)
allowed_tools = {"search_docs", "summarize", "create_ticket"}
if tool_name not in allowed_tools:
reject("Tool not allowed")
Run this against my app →