Skip to main content

Publishing Evaluators

This guide walks through building and a Rust evaluator crate to the Agent Vigilo .

Need a build-first walkthrough before publishing? See Creating Evaluators, including the dedicated "Using Codex" section.

Prerequisites​

Install vigilo via Cargo:

cargo install vigilo

Verify the installation:

vigilo --version

Evaluator publishing currently reads package identity from Cargo.toml, validates the declared WIT contract, and expects a prebuilt .

Setup​

Each evaluator requires a Vigilo.toml file at the evaluator crate root alongside Cargo.toml.

At minimum:

[package]
manifest = "Cargo.toml"

[wit]
path = "../../wit/evaluator/v1.0.0/evaluator.wit"
world = "evaluator-world"
package = "vigilo:evaluator"
version = "1.0.0"
interface = "evaluator"
strict = true

[profile.dev]
wasm = "wasm32-wasip2/debug/my_evaluator.wasm"

[profile.release]
wasm = "wasm32-wasip2/release/my_evaluator.wasm"

See the Vigilo.toml reference for all fields.

Build​

Build the evaluator for :

cargo build --manifest-path evaluators/my-evaluator/Cargo.toml --target wasm32-wasip2 --release

The wasm path in Vigilo.toml is resolved relative to Cargo's target directory for the evaluator package. The selected chooses this path. For the default Cargo target directory, a release artifact normally lives under:

target/wasm32-wasip2/release/<crate_name_with_underscores>.wasm

If your package uses a custom target directory, keep the configured profile path aligned with the actual artifact location.

Publish​

Publish the release artifact:

vigilo evaluator publish ./evaluators/my-evaluator --release

Publish a non-release profile by name:

vigilo evaluator publish ./evaluators/my-evaluator --profile dev

If neither --release nor --profile is provided, vigilo uses the dev profile from Vigilo.toml.

What Happens at Publish Time​

When you run vigilo evaluator publish, the following steps occur:

  1. Vigilo.toml is read from the evaluator directory.
  2. The selected [profile.<name>] resolves the compiled wasm artifact path.
  3. name, version, description, keywords, and optional package.metadata.vigilo are read from Cargo.toml.
  4. Optional [package] fields in Vigilo.toml override description, tags, and registry metadata.
  5. Required [wit] settings select a supported versioned ABI; the referenced WIT declaration and compiled component are both verified against its host adapter.
  6. The wasm artifact timestamp is compared with the Cargo manifest timestamp to catch stale builds.
  7. The wasm component is tagged with embedded package metadata, compiled with Wasmtime, hashed, and stored with its WIT contract hash and adapter identity.

Publication never guesses compatibility from a version string. Typed component linking must succeed for the declared adapter. See Evaluator Compatibility.

The published registry identity uses the strict format:

<namespace>/<name>:<version>

Current CLI publishing inserts into the built-in vigilo namespace.

Duplicate Handling​

Publishing is idempotent only when the existing registry row has the same identity and the same content hash.

  • Same <namespace>/<name>:<version> and same content hash: skipped.
  • Same <namespace>/<name>:<version> and different content hash: rejected.
  • Different identity but duplicate content hash in the same namespace: rejected.

There is no --force overwrite flag. To publish changed evaluator content, bump the evaluator version in Cargo.toml, rebuild, and publish the new version.

Stale Build Detection​

vigilo checks whether the wasm output is older than the package manifest before publishing. This catches the common mistake of changing Cargo.toml without rebuilding.

If a stale build is detected, rebuild and publish again:

cargo build --manifest-path evaluators/my-evaluator/Cargo.toml --target wasm32-wasip2 --release
vigilo evaluator publish ./evaluators/my-evaluator --release

Workspaces and Multiple Evaluators​

Each evaluator in a workspace has its own Vigilo.toml and is published independently:

my-workspace/
  Cargo.toml
  evaluators/
    core/
      Cargo.toml
      Vigilo.toml
    sentiment/
      Cargo.toml
      Vigilo.toml

Build and publish each:

cargo build --manifest-path evaluators/core/Cargo.toml --target wasm32-wasip2 --release
cargo build --manifest-path evaluators/sentiment/Cargo.toml --target wasm32-wasip2 --release

vigilo evaluator publish ./evaluators/core --release
vigilo evaluator publish ./evaluators/sentiment --release

For a small workspace, publish all evaluator directories with:

for dir in evaluators/*/; do vigilo evaluator publish "$dir" --release; done

CI Integration​

A typical CI publish workflow:

name: Publish Evaluators

on:
  push:
    tags:
      - 'v*'

jobs:
  publish:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4

      - name: Install vigilo
        run: cargo install vigilo

      - name: Build evaluator
        run: cargo build --manifest-path evaluators/my-evaluator/Cargo.toml --target wasm32-wasip2 --release

      - name: Publish evaluator
        run: vigilo evaluator publish ./evaluators/my-evaluator --release

Reference​

vigilo evaluator publish​

vigilo evaluator publish <evaluator-path> [--release | --profile <PROFILE>]

Arguments:

  • <evaluator-path>: path to the evaluator crate directory containing Vigilo.toml

Options:

  • --release: select the release profile from Vigilo.toml
  • --profile <PROFILE>: select a named profile from Vigilo.toml
vigilo evaluator test 'vigilo/my-evaluator:0.1.0' --input-file evaluators/my-evaluator/example-input.json
vigilo evaluator show 'vigilo/my-evaluator:0.1.0'
vigilo evaluator search --namespace vigilo my-evaluator