> For the complete documentation index, see [llms.txt](https://gdplabs.gitbook.io/catapa/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://gdplabs.gitbook.io/catapa/developer-documentation/digital-employee/advanced-examples/build-verification-tests-bvt.md).

# Build Verification Tests (BVT)

### Overview

**Build Verification Tests (BVT)** provide a pre-deployment validation step for `DigitalEmployee`. Before the underlying agent is deployed, BVT verifies that the resolved agent graph and its MCP integrations are usable.

BVT is implemented by `DeploymentVerifier` in `digital_employee_core.bvt.verifier` and is executed automatically by `DigitalEmployee.deploy()` unless you explicitly disable it.

The default verifier checks every MCP on the root agent and nested sub-agents. Custom tool checks are also supported, but are **opt-in**: tools without a registered check produce no BVT result.

This helps catch deployment issues early, before they fail at runtime.

### When BVT Runs

BVT runs during deployment:

```python
digital_employee.deploy()
```

By default, `deploy()` does this sequence:

1. Build the resolved `glaip_sdk.Agent` instance
2. Run `self.verifier.verify(self.agent)`
3. Raise `BuildVerificationError` if any check failed
4. Continue to `agent.deploy()` only when all checks passed

The implementation also supports skipping verification:

```python
digital_employee.deploy(run_bvt=False)
```

Use `run_bvt=False` only when you intentionally want to bypass pre-deployment validation.

### Key Concepts

#### What the default verifier checks

The built-in `DeploymentVerifier` performs checks on the fully resolved agent tree:

* **Recursive agent traversal**: walks the top-level agent and every nested sub-agent
* **MCP validation**: checks names and URLs, then creates an MCP session and calls `session.initialize()`
* **Authentication propagation check by execution**: auth headers are built from the resolved config and used during session creation
* **Registered tool checks**: invokes only checks explicitly registered for matching tools

This means BVT validates the final resolved configuration, not just the original constructor inputs.

#### Pass, fail, and skip semantics

Each check produces a `BVTCheckResult` with one of three statuses:

* `passed`: the check succeeded
* `failed`: the check failed and should block deployment
* `skipped`: the check was intentionally not run

Important behavior:

* `BVTResults.all_passed` only returns `False` when at least one check is `failed`
* `skipped` checks do **not** block deployment by themselves
* Tools are opt-in. An unregistered tool is not skipped and does not produce a result

For example, if an MCP exists on the agent but no matching config is found, the default verifier marks it as `skipped` with the message `No configuration found — skipped`.

#### What causes deployment to fail

Deployment is blocked when any BVT check returns `failed`. Common failure conditions include:

* MCP has no name, has a missing or malformed URL, cannot initialize, is unauthenticated, unreachable, or times out
* A registered tool-check callback raises an exception
* A tool-check callback returns something other than `BVTCheckResult`
* A registered tool check does not match any tool in the complete agent tree
* A `tool_configs` key cannot be resolved to a runtime tool, or multiple keys resolve to the same tool name

### Default Verification Flow

The built-in `DeploymentVerifier` follows this sequence:

1. Reset previous results and matched-tool state
2. Run `_pre_checks(agent)`
3. Traverse the resolved agent tree depth-first. For each agent node:
   1. Run `_check_agent_node(agent)`
   2. Verify every MCP on that node
   3. Run matching registered tool checks for every tool on that node
   4. Continue to the node's sub-agents
4. Run `_post_checks(agent)`
5. After traversal, record a failed result for each registered tool check that matched no tool anywhere in the tree

MCP verification resolves the MCP config from `agent.mcp_configs`, validates the URL, builds an `MCPConfiguration`, and attempts real session initialization. Tool checks receive the runtime tool and its resolved configuration (or `None`). Neither kind of check is schema-only validation.

### Verify Custom Tools

Register custom checks when a tool needs deployment-time validation beyond merely being present. The callback is called as `check(tool, config)` and must return a `BVTCheckResult`.

For example, this check verifies the registered `generate_interview_date` tool. This tool does not require a per-tool configuration, so the check only validates its runtime contract:

{% code lineNumbers="true" %}

```python
from digital_employee_core import BVTCheckResult, BVTStatus, DeploymentVerifier


def check_generate_interview_date(tool, config) -> BVTCheckResult:
    if tool.name != "generate_interview_date":
        return BVTCheckResult(
            name="generate_interview_date",
            item_type="tool",
            status=BVTStatus.FAILED,
            message="Unexpected tool name",
        )

    return BVTCheckResult(
        name=tool.name,
        item_type="tool",
        status=BVTStatus.PASSED,
        message="generate_interview_date is registered correctly",
    )


verifier = DeploymentVerifier(
    tool_checks={"generate_interview_date": check_generate_interview_date}
)
```

{% endcode %}

`tool_checks` is a mapping from a tool reference to one callback or a sequence of callbacks. The constructor usage is `DeploymentVerifier(tool_checks={...})`; it is not a mapping of configuration values. The callback's `config` argument is the resolved entry from the agent's `tool_configs` for that tool, or `None` when the tool has no matching configuration. Use it when a tool has deployment-specific configuration to validate.

Checks may be synchronous or asynchronous:

```python
async def check_generate_interview_date_async(tool, config):
    # Perform an asynchronous validation if needed.
    return BVTCheckResult(
        name=tool.name,
        item_type="tool",
        status=BVTStatus.PASSED,
        message="Async tool check passed",
    )
```

Tool references are accepted as:

* a string tool name, such as `"generate_interview_date"`
* a `glaip_sdk.Tool` object
* a tool class whose Pydantic `model_fields["name"]` has a non-empty default

The same registration can be added after construction with `register_tool_check`. It returns the verifier, so registration can be chained:

```python
verifier = DeploymentVerifier()
verifier.register_tool_check("generate_interview_date", check_generate_interview_date)
```

Checks run once for each matching tool occurrence on each agent node. A registered reference that matches no tool anywhere is reported as a failed `item_type="tool"` result after traversal, including the available tool names. A tool with no registered check is intentionally ignored.

### Custom Verifiers and Subclass Hooks

The verifier is injectable. `DigitalEmployee` accepts a `verifier` parameter:

{% code lineNumbers="true" %}

```python
from digital_employee_core import DeploymentVerifier, DigitalEmployee

verifier = DeploymentVerifier(timeout=15)

digital_employee = DigitalEmployee(
    identity=identity,
    mcps=[google_mail_mcp],
    configurations=configurations,
    verifier=verifier,
)
```

{% endcode %}

You can also subclass `DeploymentVerifier` to add organization-specific checks. These subclass hooks are separate from opt-in tool checks:

* `_pre_checks(agent)`: yields or returns results before agent traversal
* `_check_agent_node(agent)`: yields or returns results once for each visited agent node
* `_post_checks(agent)`: yields or returns results after traversal

The verifier's internal traversal should not be overridden. Use these hooks for environment, policy, or agent-level checks; use `tool_checks` or `register_tool_check` for checks that receive a specific tool and its configuration.

#### Example: add pre-deployment policy checks

The repository already includes a working example in `examples/custom_deployment_verifier_example.py`.

```python
from collections.abc import Iterable

from glaip_sdk import Agent

from digital_employee_core import BVTCheckResult, BVTStatus, DeploymentVerifier


class StrictDeploymentVerifier(DeploymentVerifier):
    def _pre_checks(self, agent: Agent) -> Iterable[BVTCheckResult]:
        yield BVTCheckResult(
            name="deployment_policy",
            item_type="policy",
            status=BVTStatus.PASSED,
            message="Custom policy passed",
        )
```

### BVT API and Results

#### `BVTStatus`

`BVTStatus` is a `StrEnum` with these values:

* `BVTStatus.PASSED`
* `BVTStatus.FAILED`
* `BVTStatus.SKIPPED`

#### `ToolCheck`

`ToolCheck` is the exported type alias for a synchronous or asynchronous callback:

```python
ToolCheck = Callable[
    [Tool, Mapping[str, Any] | None],
    BVTCheckResult | Awaitable[BVTCheckResult],
]
```

The callback receives `(tool, config)` and returns a `BVTCheckResult`, either directly or through an awaitable.

#### `BVTCheckResult`

Represents a single check result:

```python
BVTCheckResult(
    name="generate_interview_date",
    item_type="tool",
    status=BVTStatus.PASSED,
    message="Tool configuration is valid",
    details={},
)
```

Fields:

* `name`: item being checked
* `item_type`: category such as `mcp`, `tool`, `agent`, `env_var`, or `policy`
* `status`: one of the `BVTStatus` values
* `message`: human-readable result description
* `details`: optional diagnostic metadata

Tool callbacks' returned results are recorded as-is, including their `details`. If a callback raises, BVT records a failed tool result with message `Tool check raised an exception: ...` and `details` containing `error_type` and `error`. If it returns an invalid type, BVT records a failed tool result with `error_type="InvalidToolCheckResult"` and `actual_type` in `details`. Tool-config resolution failures are recorded as failed agent results with a `Failed to resolve tool_configs: ...` message. Unmatched registered checks are failed tool results with the available tool names in the message.

#### `BVTResults`

`BVTResults.checks` is the complete ordered list of `BVTCheckResult` instances, including MCP, tool, agent, and subclass-hook results:

```python
results = verifier.verify(agent)

print(results.checks)
print(results.all_passed)
print(results.failed_checks)
print(results.skipped_checks)
print(results.summary())
```

Available helpers are `checks`, `all_passed`, `failed_checks`, `skipped_checks`, and `summary()`.

### Transport and Timeout Notes

The default verifier supports MCP transports via the MCP definition on each agent node:

* If an MCP does not define a transport, the verifier uses `streamable_http`.
* For non-`stdio` transports, the generated session config includes `timeout=self.timeout`.
* For `stdio`, the verifier applies `asyncio.wait_for(..., timeout=self.timeout)` around `session.initialize()`.

Thus `DeploymentVerifier(timeout=30)` controls the MCP session's configured or initialization timeout according to the transport. It does not impose a timeout on custom tool callbacks; async tool callbacks are awaited by the verifier.

### Handling BVT Failures

Catch `BuildVerificationError` if you want to inspect or log the failure before exiting:

```python
from digital_employee_core import BuildVerificationError

try:
    digital_employee.deploy()
except BuildVerificationError as exc:
    print(exc)
    print(exc.results.failed_checks)
```

The exception message is generated from `BVTResults.summary()`, which includes total passed, failed, and skipped checks plus the names of failed and skipped checks.

### Best Practices

1. Keep BVT enabled in normal deployments.
2. Treat skipped MCP checks as configuration signals.
3. Register tool checks only for tools that require explicit validation.
4. Use subclass hooks for organization rules and agent-level checks.
5. Use the resolved built agent as the source of truth.
6. Log or surface `results.summary()` in CI and deployment output.

### API Summary

Most users only need these exported symbols:

```python
from digital_employee_core import (
    BuildVerificationError,
    BVTCheckResult,
    BVTResults,
    BVTStatus,
    DeploymentVerifier,
    ToolCheck,
)
```

These cover failure handling, individual and aggregate results, default or custom verifiers, and custom tool-check callbacks.