Skip to content
Last updated

Policy Testing Guide

This guide helps you validate that Capsule policies are working correctly by running controlled test scenarios with your coding agents. Each scenario triggers a specific policy, so you can confirm your organization's security monitoring is functioning as expected.

Overview

Capsule monitors coding agent sessions in real time, analyzing user messages, tool invocations, and agent configuration changes against your defined policies. Policies define what to look for and how to respond when a violation is detected.

This guide provides ready-to-use test scenarios for each policy. Each scenario walks you through the exact steps to trigger a detection, what to expect in the portal, and how to verify the result.

Important: Some test scenarios involve prompts that reference destructive commands (such as file deletion, database drops, or infrastructure changes). Always run these tests in a sandbox or isolated test environment — never against production systems or agents with live credentials. Policies in Monitoring mode will log violations but will not block the action, so a destructive command could execute before the violation is recorded.

Prerequisites

Before running test scenarios, make sure you have:

  • A sandbox or test environment — do not run these scenarios against production systems or agents with live infrastructure credentials
  • A Capsule Security account with access to the portal
  • At least one coding agent integration deployed (for example, Claude Code or Cursor)
  • At least one agent discovered in your Capsule portal (Inventory > Agents)
  • Default policies enabled in Monitoring mode (enabled by default)

How Policies Work

When Policies Are Evaluated

Policies are evaluated at specific points during an agent session:

WhenDescription
Agent configuration changesWhen an agent gets new tools, MCP servers, skills, or settings
User sends a messageWhen a user types a prompt in an agent session
Agent uses a toolWhen an agent runs a shell command, edits a file, or calls an MCP tool

Policy Modes

Each policy operates in one of three modes that determine how it responds to a violation:

ModeBehavior
MonitoringDetects and logs violations with no user-facing action
WarningDetects and alerts on violations
PreventionBlocks the action in real time

What Happens When a Policy Triggers

When a policy triggers, Capsule takes the following actions:

  • Creates a policy violation visible in the portal
  • Blocks the action in real time (Prevention mode only)

Out-of-the-Box Policies

All Capsule accounts come with pre-configured policies enabled in Monitoring mode. The tables below map each policy to the test scenario that validates it.

Data Leakage Policies

Policy NameSeverityTriggerTest Guide
Credentials Leakage in RuntimeMediumUser messageTest this policy →
Credentials Leakage with Tool InvocationHighTool useTest this policy →
PII Leakage in RuntimeMediumUser messageTest this policy →
PII Leakage with Tool InvocationHighTool useTest this policy →
PCI Leakage in RuntimeMediumUser messageTest this policy →
PCI Leakage with Tool InvocationHighTool useTest this policy →
Business Intelligence Leakage in shared AgentsHighUser messageTest this policy →

Data Leakage in Public Agents

Policy NameSeverityTriggerTest Guide
Credentials Leakage in Public AgentsCriticalUser messageTest this policy →
PII Leakage in Public AgentsCriticalUser messageTest this policy →
PCI Leakage in Public AgentsCriticalUser messageTest this policy →

Data Exfiltration via Tools

Policy NameSeverityTriggerTest Guide
Credentials Leakage with External Data Exposure via ToolCriticalTool useTest this policy →
PII Leakage with External Data Exposure via ToolCriticalTool useTest this policy →
PCI Leakage with External Data Exposure via ToolCriticalTool useTest this policy →

Prompt Security Policies

Policy NameSeverityTriggerTest Guide
Prompt Injection with Instruction ViolationCriticalUser messageTest this policy →
Prompt Injection with Data ManipulationCriticalUser messageTest this policy →

Operational Safety Policies

Policy NameSeverityTriggerTest Guide
Dangerous Actions with Instruction ViolationHighTool useTest this policy →

Agent Configuration Policies

Policy NameSeverityTriggerTest Guide
Tool Poisoning Attack RiskCriticalAgent config changeTest this policy →
Identity Security Risks in ToolsHighAgent config changeTest this policy →
Sensitive Data Exposure RiskHighAgent config changeTest this policy →
Public Agent Instruction VulnerabilitiesHighAgent config changeTest this policy →
Excessive or Unused Tool PermissionsMediumAgent config changeTest this policy →
Missing Agent OwnerMediumAgent config changeTest this policy →
Experimental Model UsageLowAgent config changeTest this policy →

Verifying Results

After running any test scenario:

  1. Navigate to Policies > Violations in the Capsule portal
  2. Look for a new violation matching the scenario you tested
  3. Click the violation to see:
    • Violation category -- The specific violation category that was detected
    • Evidence -- Quoted content from the session that triggered the detection
    • Policy -- Which policy matched and its severity
    • Session -- Link to the full agent session for context

Tips for Testing

  • Run one scenario at a time for clear results
  • Allow some time after the agent session for analysis to complete
  • Start with Monitoring mode -- switch to Prevention mode only after validating detections work
  • Use a dedicated test environment or sandbox agent if available