Create Test Suite

Create a new test suite with tests

POST /api/platform/testSuites

Create Test Suite

cURL

curl --request POST \
  --url https://api.example.com/api/platform/testSuites \
  --header 'Content-Type: application/json' \
  --data '
{
  "name": "<string>",
  "description": "<string>",
  "templateService": "<string>",
  "templateName": "<string>",
  "impersonateUserId": "<string>",
  "visibility": "<string>"
}
'

200

{
  "suiteId": "<string>",
  "name": "<string>"
}

Request

POST /api/platform/testSuites

Body Parameters

name · string · required

Suite name

description · string

Suite description

templateService · string · required

Service type: "slack" or "linear"

templateName · string · required

Template to use for tests

impersonateUserId · string

Default user ID for tests

visibility · string · default: "private"

"public" or "private"

Example Request

curl -X POST https://api.agentdiff.dev/api/platform/testSuites \
  -H "X-API-Key: ad_live_sk_..." \
  -H "Content-Type: application/json" \
  -d '{
    "name": "My Agent Tests",
    "description": "Custom tests for my Slack agent",
    "templateService": "slack",
    "templateName": "slack_default",
    "impersonateUserId": "U01AGENBOT9",
    "visibility": "private"
  }'

Response

suiteId · string

New suite identifier

name · string

Suite name

Example Response

{
  "suiteId": "suite-789",
  "name": "My Agent Tests"
}

Adding Tests

After creating a suite, add tests:

curl -X POST https://api.agentdiff.dev/api/platform/testSuites/suite-789/tests \
  -H "X-API-Key: ad_live_sk_..." \
  -H "Content-Type: application/json" \
  -d '{
    "tests": [
      {
        "name": "Post welcome message",
        "prompt": "Post a welcome message to #general",
        "expectedOutput": {
          "assertions": [{
            "diff_type": "added",
            "entity": "messages",
            "where": {
              "channel_id": {"eq": "C01GENERAL99"},
              "message_text": {"contains": "welcome"}
            },
            "expected_count": 1
          }]
        }
      }
    ]
  }'

Complete Example

from agent_diff import AgentDiff

client = AgentDiff()

# 1. Create suite
suite = client.create_test_suite({
    "name": "My Slack Tests",
    "description": "Custom Slack agent tests",
    "templateService": "slack",
    "templateName": "slack_default",
    "impersonateUserId": "U01AGENBOT9"
})

# 2. Add tests
client.create_tests(suite.suiteId, {
    "tests": [
        {
            "name": "Post message",
            "prompt": "Post 'Hello' to #general",
            "expectedOutput": {
                "assertions": [{
                    "diff_type": "added",
                    "entity": "messages",
                    "where": {"message_text": {"contains": "Hello"}},
                    "expected_count": 1
                }]
            }
        },
        {
            "name": "Create channel",
            "prompt": "Create a new channel called 'test-channel'",
            "expectedOutput": {
                "assertions": [{
                    "diff_type": "added",
                    "entity": "channels",
                    "where": {"name": {"eq": "test-channel"}},
                    "expected_count": 1
                }]
            }
        }
    ]
})

# 3. Use the suite
suite = client.get_test_suite(suite.suiteId, expand=True)
print(f"Created suite with {len(suite.tests)} tests")

SDK Usage

Python

suite = client.create_test_suite({
    "name": "My Tests",
    "templateService": "slack",
    "templateName": "slack_default",
    "impersonateUserId": "U01AGENBOT9"
})
print(f"Created suite: {suite.suiteId}")