> ## Documentation Index
> Fetch the complete documentation index at: https://agentdiff.mintlify.site/llms.txt
> Use this file to discover all available pages before exploring further.

# Writing Assertions

> Define expected outcomes for your agent tests with the evaluation DSL

## Overview

Assertions describe what database changes you expect after the agent runs. They're written in a JSON DSL that gets evaluated against the computed diff.

## Schema

```json theme={null}
{
  "strict": false,
  "assertions": [
    {
      "diff_type": "added|changed|removed|unchanged",
      "entity": "table_name",
      "where": { /* predicates */ },
      "expected_count": 1,
      "expected_changes": { /* for 'changed' type */ }
    }
  ]
}
```

### Top-Level Fields

| Field        | Type    | Required | Description                                  |
| ------------ | ------- | -------- | -------------------------------------------- |
| `strict`     | boolean | No       | Fail if extra changes exist (default: false) |
| `assertions` | array   | Yes      | List of assertions to evaluate               |

### Assertion Fields

| Field              | Type          | Required | Description                             |
| ------------------ | ------------- | -------- | --------------------------------------- |
| `diff_type`        | string        | Yes      | Type of change to check                 |
| `entity`           | string        | Yes      | Database table name                     |
| `where`            | object        | No       | Conditions to match records             |
| `expected_count`   | number/object | No       | How many records should match           |
| `expected_changes` | object        | No       | Field changes to verify (for `changed`) |

***

## Diff Types

| Type        | Description                        |
| ----------- | ---------------------------------- |
| `added`     | New records were inserted          |
| `changed`   | Existing records were updated      |
| `removed`   | Records were deleted               |
| `unchanged` | Records exist and weren't modified |

### `added` - New Records

Check that new rows were inserted:

```json theme={null}
{
  "diff_type": "added",
  "entity": "messages",
  "where": {
    "message_text": {"contains": "Hello"}
  },
  "expected_count": 1
}
```

### `changed` - Modified Records

Check that existing rows were updated:

```json theme={null}
{
  "diff_type": "changed",
  "entity": "issues",
  "where": {
    "id": {"eq": "issue-123"}
  },
  "expected_changes": {
    "status": {"to": {"eq": "Done"}}
  }
}
```

### `removed` - Deleted Records

Check that rows were deleted:

```json theme={null}
{
  "diff_type": "removed",
  "entity": "messages",
  "where": {
    "message_id": {"eq": "1234567890.000100"}
  },
  "expected_count": 1
}
```

***

## Predicate Operators

### Equality Operators

| Operator | Description  | Example                           |
| -------- | ------------ | --------------------------------- |
| `eq`     | Equal to     | `{"status": {"eq": "Done"}}`      |
| `neq`    | Not equal to | `{"status": {"neq": "Archived"}}` |

### Comparison Operators

| Operator | Description           | Example                    |
| -------- | --------------------- | -------------------------- |
| `gt`     | Greater than          | `{"priority": {"gt": 2}}`  |
| `gte`    | Greater than or equal | `{"priority": {"gte": 2}}` |
| `lt`     | Less than             | `{"priority": {"lt": 5}}`  |
| `lte`    | Less than or equal    | `{"priority": {"lte": 5}}` |

### String Operators

| Operator       | Description                         | Example                               |
| -------------- | ----------------------------------- | ------------------------------------- |
| `contains`     | Contains substring (case-sensitive) | `{"text": {"contains": "bug"}}`       |
| `not_contains` | Does not contain substring          | `{"text": {"not_contains": "error"}}` |
| `starts_with`  | Starts with prefix                  | `{"title": {"starts_with": "Fix"}}`   |
| `ends_with`    | Ends with suffix                    | `{"name": {"ends_with": ".js"}}`      |

### Null Operators

| Operator   | Description       | Example                            |
| ---------- | ----------------- | ---------------------------------- |
| `is_null`  | Value is NULL     | `{"assignee": {"is_null": true}}`  |
| `not_null` | Value is not NULL | `{"assignee": {"not_null": true}}` |

### Set Operators

| Operator | Description         | Example                                  |
| -------- | ------------------- | ---------------------------------------- |
| `in`     | Value is in set     | `{"status": {"in": ["Done", "Closed"]}}` |
| `not_in` | Value is not in set | `{"status": {"not_in": ["Backlog"]}}`    |

### Array Operators

| Operator  | Description           | Example                                      |
| --------- | --------------------- | -------------------------------------------- |
| `has_any` | Array contains any of | `{"labels": {"has_any": ["bug", "urgent"]}}` |
| `has_all` | Array contains all of | `{"labels": {"has_all": ["bug", "p1"]}}`     |

***

## Combining Predicates

### AND (implicit)

Multiple conditions in `where` are ANDed:

```json theme={null}
{
  "where": {
    "channel_id": {"eq": "C01GENERAL99"},
    "message_text": {"contains": "hello"},
    "user_id": {"eq": "U01AGENBOT9"}
  }
}
```

### AND (explicit)

```json theme={null}
{
  "where": {
    "and": [
      {"status": {"eq": "Done"}},
      {"priority": {"lt": 3}}
    ]
  }
}
```

### OR

```json theme={null}
{
  "where": {
    "or": [
      {"status": {"eq": "Done"}},
      {"status": {"eq": "Closed"}}
    ]
  }
}
```

### Nested Logic

```json theme={null}
{
  "where": {
    "and": [
      {"team_id": {"eq": "eng"}},
      {"or": [
        {"priority": {"eq": 1}},
        {"labels": {"has_any": ["urgent"]}}
      ]}
    ]
  }
}
```

***

## Expected Count

### Exact Count

```json theme={null}
{"expected_count": 1}           // Exactly 1 record
{"expected_count": 0}           // No records
{"expected_count": 5}           // Exactly 5 records
```

### Bounded Count

```json theme={null}
{"expected_count": {"min": 1}}           // At least 1
{"expected_count": {"max": 5}}           // At most 5
{"expected_count": {"min": 1, "max": 10}} // Between 1 and 10
```

***

## Expected Changes

For `diff_type: "changed"`, specify field transitions:

### Check New Value Only

```json theme={null}
{
  "expected_changes": {
    "status": {"to": {"eq": "Done"}}
  }
}
```

### Check Both Old and New

```json theme={null}
{
  "expected_changes": {
    "status": {
      "from": {"eq": "In Progress"},
      "to": {"eq": "Done"}
    }
  }
}
```

### Multiple Field Changes

```json theme={null}
{
  "expected_changes": {
    "status": {"to": {"eq": "Done"}},
    "completed_at": {"to": {"not_null": true}},
    "updated_at": {"to": {"not_null": true}}
  }
}
```

***

## Strict Mode

With `strict: true`, the evaluation fails if there are changes beyond what's asserted:

```json theme={null}
{
  "strict": true,
  "assertions": [{
    "diff_type": "changed",
    "entity": "issues",
    "expected_changes": {
      "status": {"to": {"eq": "Done"}}
    }
  }]
}
```

<Note>
  If `strict: true` and the agent also changed `priority`, the assertion fails even if `status` was correctly changed.
</Note>

***

## Ignore Fields

Ignore fields let you exclude certain fields when comparing diffs. This is useful for:

* Auto-generated fields (`id`, `created_at`, `updated_at`)
* Timestamps that change on every run
* Fields set by the system, not the agent

### Using Ignore Fields

```json theme={null}
{
  "assertions": [{
    "diff_type": "added",
    "entity": "messages",
    "where": {
      "message_text": {"contains": "Hello"}
    }
  }],
  "ignore_fields": {
    "global": ["created_at", "updated_at", "message_id", "ts"]
  }
}
```

### Common Ignore Fields by Service

**Slack:**

```json theme={null}
{
  "ignore_fields": {
    "global": ["ts", "message_id", "created_at", "updated_at", "edited_at"]
  }
}
```

**Linear:**

```json theme={null}
{
  "ignore_fields": {
    "global": ["id", "created_at", "updated_at", "sort_order", "number"]
  }
}
```

### Entity-Specific Ignore Fields

You can ignore fields for specific tables:

```json theme={null}
{
  "ignore_fields": {
    "global": ["created_at", "updated_at"],
    "messages": ["ts", "message_id"],
    "issues": ["number", "sort_order"]
  }
}
```

<Tip>
  Without ignore\_fields, auto-generated timestamps and IDs would cause assertion failures even when the agent performed the correct action.
</Tip>

***

## Complete Examples

### Slack: Post Message to Channel

```json theme={null}
{
  "assertions": [{
    "diff_type": "added",
    "entity": "messages",
    "where": {
      "channel_id": {"eq": "C01GENERAL99"},
      "message_text": {"contains": "Hello"},
      "user_id": {"eq": "U01AGENBOT9"}
    },
    "expected_count": 1
  }]
}
```

### Slack: Add Reaction

```json theme={null}
{
  "assertions": [{
    "diff_type": "added",
    "entity": "reactions",
    "where": {
      "message_id": {"eq": "1234567890.000100"},
      "reaction_name": {"eq": "thumbsup"}
    },
    "expected_count": 1
  }]
}
```

### Linear: Create Issue with Label

```json theme={null}
{
  "strict": true,
  "assertions": [
    {
      "diff_type": "added",
      "entity": "issues",
      "where": {
        "and": [
          {"title": {"contains": "Fix"}},
          {"team_id": {"eq": "team-eng"}},
          {"priority": {"lte": 2}}
        ]
      },
      "expected_count": 1
    },
    {
      "diff_type": "changed",
      "entity": "teams",
      "where": {
        "id": {"eq": "team-eng"}
      },
      "expected_changes": {
        "issue_count": {
          "from": {"eq": 5},
          "to": {"eq": 6}
        }
      }
    }
  ]
}
```

### Linear: Update Issue Priority

```json theme={null}
{
  "assertions": [{
    "diff_type": "changed",
    "entity": "issues",
    "where": {
      "title": {"contains": "login bug"}
    },
    "expected_changes": {
      "priority": {"to": {"eq": 1}}
    }
  }]
}
```

***

## Table Names by Service

### Slack

| Table             | Description        |
| ----------------- | ------------------ |
| `users`           | User accounts      |
| `channels`        | Channels and DMs   |
| `channel_members` | Channel membership |
| `messages`        | Messages           |
| `reactions`       | Emoji reactions    |

### Linear

| Table             | Description     |
| ----------------- | --------------- |
| `issues`          | Issues/tickets  |
| `teams`           | Teams           |
| `users`           | User accounts   |
| `comments`        | Issue comments  |
| `labels`          | Labels          |
| `workflow_states` | Workflow states |
| `projects`        | Projects        |
| `cycles`          | Cycles/sprints  |

***

## Next Steps

<CardGroup cols={2}>
  <Card title="Example Benchmarks" icon="chart-line" href="/test-suites/benchmarks">
    See built-in Slack and Linear test suites
  </Card>

  <Card title="Creating Test Suites" icon="folder" href="/test-suites/creating-suites">
    Organize and manage your tests
  </Card>
</CardGroup>