Skip to main content

Publishing Evaluators

This guide walks through building and publishing a Rust evaluator crate to the Agent Vigilo registry.

Need a build-first walkthrough before publishing? See Creating Evaluators, including the dedicated "Using Codex" section.

Prerequisites​

Install vigilo via Cargo:

cargo install vigilo

Verify the installation:

vigilo --version

Evaluator publishing currently reads package identity from Cargo.toml, validates the declared WIT contract, and expects a prebuilt WASI Preview 2 WebAssembly component artifact.

Setup​

Each evaluator requires a Vigilo.toml file at the evaluator crate root alongside Cargo.toml.

At minimum:

[package]
manifest = "Cargo.toml"

[wit]
path = "../../wit/evaluator.wit"
world = "evaluator-world"
package = "vigilo:evaluator"
version = "0.1.0"
interface = "evaluator"
strict = true

[profile.dev]
wasm = "wasm32-wasip2/debug/my_evaluator.wasm"

[profile.release]
wasm = "wasm32-wasip2/release/my_evaluator.wasm"

See the Vigilo.toml reference for all fields.

Build​

Build the evaluator for WASI Preview 2:

cargo build --manifest-path evaluators/my-evaluator/Cargo.toml --target wasm32-wasip2 --release

The wasm path in Vigilo.toml is resolved relative to Cargo's target directory for the evaluator package. For the default Cargo target directory, a release artifact normally lives under:

target/wasm32-wasip2/release/<crate_name_with_underscores>.wasm

If your package uses a custom target directory, keep the configured profile path aligned with the actual artifact location.

Publish​

Publish the release artifact:

vigilo evaluators publish ./evaluators/my-evaluator --release

Publish a non-release profile by name:

vigilo evaluators publish ./evaluators/my-evaluator --profile dev

If neither --release nor --profile is provided, vigilo uses the dev profile from Vigilo.toml.

What Happens at Publish Time​

When you run vigilo evaluators publish, the following steps occur:

  1. Vigilo.toml is read from the evaluator directory.
  2. The selected [profile.<name>] resolves the compiled wasm artifact path.
  3. name, version, description, keywords, and optional package.metadata.vigilo are read from Cargo.toml.
  4. Optional [package] fields in Vigilo.toml override description, tags, and registry metadata.
  5. Optional [wit] settings validate the evaluator WIT package, version, world, and exported interface.
  6. The wasm artifact timestamp is compared with the Cargo manifest timestamp to catch stale builds.
  7. The wasm component is tagged with embedded package metadata, compiled with Wasmtime, hashed, and inserted into the registry.

The published registry identity uses the strict evaluator identifier format:

<namespace>/<name>:<version>

Current CLI publishing inserts into the built-in vigilo namespace.

Duplicate Handling​

Publishing is idempotent only when the existing registry row has the same identity and the same content hash.

  • Same <namespace>/<name>:<version> and same content hash: skipped.
  • Same <namespace>/<name>:<version> and different content hash: rejected.
  • Different identity but duplicate content hash in the same namespace: rejected.

There is no --force overwrite flag. To publish changed evaluator content, bump the evaluator version in Cargo.toml, rebuild, and publish the new version.

Stale Build Detection​

vigilo checks whether the wasm output is older than the package manifest before publishing. This catches the common mistake of changing Cargo.toml without rebuilding.

If a stale build is detected, rebuild and publish again:

cargo build --manifest-path evaluators/my-evaluator/Cargo.toml --target wasm32-wasip2 --release
vigilo evaluators publish ./evaluators/my-evaluator --release

Workspaces and Multiple Evaluators​

Each evaluator in a workspace has its own Vigilo.toml and is published independently:

my-workspace/
  Cargo.toml
  evaluators/
    core/
      Cargo.toml
      Vigilo.toml
    sentiment/
      Cargo.toml
      Vigilo.toml

Build and publish each:

cargo build --manifest-path evaluators/core/Cargo.toml --target wasm32-wasip2 --release
cargo build --manifest-path evaluators/sentiment/Cargo.toml --target wasm32-wasip2 --release

vigilo evaluators publish ./evaluators/core --release
vigilo evaluators publish ./evaluators/sentiment --release

For a small workspace, publish all evaluator directories with:

for dir in evaluators/*/; do vigilo evaluators publish "$dir" --release; done

CI Integration​

A typical CI publish workflow:

name: Publish Evaluators

on:
  push:
    tags:
      - 'v*'

jobs:
  publish:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4

      - name: Install vigilo
        run: cargo install vigilo

      - name: Build evaluator
        run: cargo build --manifest-path evaluators/my-evaluator/Cargo.toml --target wasm32-wasip2 --release

      - name: Publish evaluator
        run: vigilo evaluators publish ./evaluators/my-evaluator --release

Reference​

vigilo evaluators publish​

vigilo evaluators publish <evaluator-path> [--release | --profile <PROFILE>]

Arguments:

  • <evaluator-path>: path to the evaluator crate directory containing Vigilo.toml

Options:

  • --release: select the release profile from Vigilo.toml
  • --profile <PROFILE>: select a named profile from Vigilo.toml
vigilo evaluators test 'vigilo/my-evaluator:0.1.0' --input-file evaluators/my-evaluator/example-input.json
vigilo evaluators show 'vigilo/my-evaluator:0.1.0'
vigilo evaluators search --namespace vigilo my-evaluator