Skip to main content
Want to skip setup? Copy one of the Jupyter notebooks:

​
Choose Your Setup

​
Step 2: Install the SDK

pip install agent-diff
# or with uv
uv add agent-diff

​
Step 3: Create Your First Environment

Where do these values come from? The templateName and impersonateUserId must match data in the template. See Environments & Templates for available templates and user IDs (e.g., U01AGENBOT9 is the agent bot user in slack_default).
from agent_diff import AgentDiff

# For hosted version, pass explicility or add environment variables
client = AgentDiff(
    base_url="https://api.agentdiff.dev",
    api_key="ak_your_api_key"  # From dashboard
)

# Create isolated Slack environment
env = client.init_env(
    templateService="slack",
    templateName="slack_default",
    impersonateUserId="U01AGENBOT9",  # User your agent will act as
    ttlSeconds=3600  # Auto-cleanup after 1 hour
)

print(f"Environment: {env.environmentId}")
print(f"API URL: {env.environmentUrl}")

​
Step 4: Run Your Agent & Capture Diff

from agent_diff import PythonExecutorProxy, create_openai_tool
from agents import Agent, Runner

# Start run (takes "before" snapshot)
run = client.start_run(envId=env.environmentId)

# Create code executor that intercepts API calls
python_executor = PythonExecutorProxy(env.environmentId, base_url=client.base_url, api_key=client.api_key)
python_tool = create_openai_tool(python_executor)

# Your agent with the tool
agent = Agent(
    name="Slack Assistant",
    instructions="""Use execute_python tool to interact with Slack API 
    at https://slack.com/api/*. Authentication is handled automatically.""",
    tools=[python_tool]
)

# Run the agent
response = await Runner.run(agent, "Post 'Hello World!' to #general")

# Get the diff (what changed)
diff = client.diff_run(runId=run.runId)

print("Inserts:", diff.diff['inserts'])  # New messages created
print("Updates:", diff.diff['updates'])  # Records modified
print("Deletes:", diff.diff['deletes'])  # Records removed

# Cleanup
client.delete_env(envId=env.environmentId)

​
Example Output

{
  "inserts": [
    {
      "__table__": "messages",
      "message_id": "1732645891.000200",
      "channel_id": "C01GENERAL99",
      "user_id": "U01AGENBOT9",
      "message_text": "Hello World!",
      "created_at": "2025-11-26T15:31:31"
    }
  ],
  "updates": [],
  "deletes": []
}
You’ve just run an agent against an isolated Slack replica and captured the exact changes it made!

​
What Just Happened?

  1. Environment Created: A fresh PostgreSQL schema was created with seed data (users, channels, messages)
  2. Before Snapshot: start_run captured the initial state
  3. Agent Executed: Your agent’s API calls were intercepted and routed to the isolated environment
  4. Diff Computed: The system compared before/after states to produce a deterministic diff

​
Next Steps

Example Benchmarks

See built-in evaluation suites for Slack and Linear

Environments

Understand environments and templates in depth

Run Evaluations

Add assertions and run benchmarks against your agent

Integrate with Agents

Connect your AI agent framework