Skip to content

VAMOS canonical v1 run-artifact contract

Status: implemented pre-release contract Document identity: vamos.run-manifest Schema version: 1.0.0

1. Scope and authority

This document defines the only supported VAMOS run-artifact format. It governs the top-level Python persistence API, CLI run output, analysis discovery, and Studio loading. A run artifact represents one execution attempt. It is not a study database or a general export format. Replay is an explicit service that consumes one verified run and publishes another through the same writer.

VAMOS 1.0.0 establishes the first public run-artifact baseline. The package supports one schema and one reader/writer path. Internal development inputs outside schema 1.0.0 must be regenerated with the current version.

Internal development run directories created before the canonical schema 1.0.0 implementation are unsupported and should be regenerated.

2. Canonical directory

A successful run directory contains exactly these required files:

<run>/
├── manifest.json
├── result.npz
└── environment.json

manifest.json is the authoritative envelope and artifact index. result.npz is the authoritative numerical bundle. environment.json is the authoritative bounded environment description. Additional files are not produced by the v1 writer. Optional future roles such as metrics or events require a schema-defined, nonduplicated responsibility before a writer may emit them.

The run directory is relocatable. Manifest artifact paths are normalized, relative POSIX paths confined to the run directory. Absolute paths, traversal, empty segments, backslashes, drive prefixes, URI forms, NUL bytes, and symlink escapes are rejected.

3. Writer ownership

There is one persistence implementation. These entry points all converge on it:

  • vamos.save_result;
  • the CLI StorageObserver adapter;
  • experiment, benchmark, zoo, quickstart, and ablation executions that use the CLI runner.

Study summaries may reference a run directory, but they never copy or rebuild its files. Analysis and Studio are readers only. Result helper and example code must use vamos.save_result; they do not implement their own persistence.

The writer never overwrites, merges, or repairs a destination. Every existing path is an OutputCollisionError, including an empty directory or a partial run. It snapshots supported arrays, writes a uniquely owned sibling staging directory, fsyncs files, commits the terminal manifest last, and publishes with one directory rename. Failure removes only staging state owned by that writer. It also fsyncs the staging and parent directories, except on Windows, which offers no directory fsync: there, a power loss just after publication can lose the new directory entry, though never leave a partial run under the final name.

4. Public Python API

The authoritative imports are top-level:

import vamos

stored = vamos.save_result(result, "runs/run-1")
run = vamos.load_run("runs/run-1")
result = vamos.load_result("runs/run-1")

The signatures are:

save_result(result, path, *, requested_spec=None, resolved_spec=None,
            labels=None, limits=None) -> StoredRun
load_run(path, *, verify="required", limits=None) -> StoredRun
load_result(path, *, verify="required", limits=None) -> OptimizationResult
verify_run(path, *, require_level=None, limits=None) -> VerificationReport
reproduce(path, *, output=None, limits=None) -> ReplayReport

There is no persistence export from vamos.ux.api.

An OptimizationResult returned by vamos.optimize carries captured requested and resolved specs. A manually constructed result must pass both requested_spec= and resolved_spec=. Supplying neither or only one raises IncompleteRunMetadataError; VAMOS never invents an algorithm, backend, termination, provenance, or seed from array shapes.

5. Requested and resolved specifications

requested_spec preserves user intent as JSON data. Omitted defaults remain omitted. A requested seed may be null when the user explicitly asks VAMOS to choose one.

VAMOS 1.2.0: for a CLI run, requested_spec is the effective request for that run, with the command-line overrides and the problem's problems.<name> block applied; the config file it came from is kept under config_file as provenance. The problem configuration holds only what built the problem (n_var, n_obj). results verify proves the resolved spec with the reconstruction reproduce uses, without running the optimization, so it no longer reports exact for a run that reproduce rejects.

resolved_spec has identity vamos.resolved-run-spec/1 and records the actual execution state:

  • problem identity, dimensions, encoding, constraint convention, provider, and configuration;
  • algorithm identity, provider, complete configuration, and result mode;
  • active/inactive operator descriptors and resolved parameters;
  • kernel and evaluation backend descriptors;
  • termination descriptor and effective hard budget;
  • actual integer seed;
  • population, offspring, and archive sizes;
  • defaults applied and their sources;
  • determinism declaration and RNG family.

VAMOS 1.3.0: the configuration of a registered problem also holds the options passed through problem_kwargs, such as capacity_ratio for bin_knapsack; a run without options records only the dimensions, the encoding and the constraint convention, as before. Verification and replay pass the options to the problem's registry factory. A field that the factory does not declare as a keyword-only parameter makes the run not reconstructable.

resolved_spec.seed is always an integer. Zero is valid. For seed=None, VAMOS generates an integer before constructing any stochastic execution object, exposes it in OptimizationResult.meta["seed"], and persists it in the resolved spec. No sentinel represents an unknown seed.

task_id is sha256: plus the SHA-256 of canonical JSON bytes for the complete resolved spec. run_id is a unique UUIDv4 for the attempt. Repeated executions of the same task share a task ID but have distinct run IDs.

VAMOS 1.3.0: a MOEA/D run's algorithm.resolution also holds weight_vectors, where its weight vectors come from: source is uniform (two objectives), das_dennis with its divisions, packaged_file or file with the file name, and vectors is their number, the population. It is derived from the configuration, so verification rebuilds it like the rest of the resolved spec; a MOEA/D run stored before it does not match its reconstruction and is not verified as exact.

6. Run manifest

The manifest requires:

  • document_type, exactly vamos.run-manifest;
  • schema_version, exactly 1.0.0;
  • run_id and content-derived task_id;
  • status and RFC 3339 timestamps;
  • requested_spec and resolved_spec;
  • provenance and replayability;
  • outcome for a terminal run;
  • artifacts descriptors;
  • integrity.manifest_sha256 for a terminal run.

Statuses are running, succeeded, failed, partial, and cancelled. completed_at is required only for terminal states. A succeeded run requires both result_bundle and environment descriptors. A failed run requires a structured failure object and has no implied numerical result.

outcome records counters, runtime, termination reason, interruption and usability flags, result mode, array-derived solution/dimension counts, and bounded JSON metrics. Cached counts never override result array shapes.

Every artifact descriptor records role, relative path, media_type, exact bytes, lowercase SHA-256, required_for, and whether it is canonical. The result descriptor also records the name, shape, and exact NumPy dtype string of each array.

The manifest self-hash is computed from canonical JSON with integrity.manifest_sha256 omitted. Object keys are sorted, UTF-8 is used, duplicate keys are rejected, and non-finite JSON numbers are forbidden. Whitespace-only reformatting does not change the semantic hash.

7. Numerical ResultBundle

result.npz is an NPZ of independent NPY members loaded with allow_pickle=False. F is required for a succeeded run. Supported names are:

  • F, X, G, CV, and reference_directions;
  • population/F, population/X, population/G, population/CV;
  • archive/F, archive/X, archive/G, archive/CV;
  • external_archive/F, external_archive/X, external_archive/G, external_archive/CV.

VAMOS 1.3.0: external_archive/ holds the external archive of a SPEA2 run whose X and F do not: those of a final subset of it (ExternalArchiveConfig.final_selection) or of the population (result_mode="population"). SPEA2's archive/ is its environmental archive. No other run writes these members, so a VAMOS 1.2.1 reader, which rejects unknown members, still loads every other bundle. Their column and row checks are those of archive/.

V1 accepts only fixed-width boolean, signed integer, unsigned integer, and floating arrays. Object, string, structured, complex, datetime, and pickle-backed arrays are rejected. Shape relationships and manifest array contracts are validated before values are exposed. Loaded arrays are defensive copies and are never executable objects.

VAMOS 1.2.0: the shape check includes columns. F, population/F, archive/F, and reference_directions have resolved_spec.problem.config.n_obj columns, and X, population/X, and archive/X have its n_var columns, whenever the configuration records that dimension as an integer. load_result and verify_run reject a bundle that disagrees with MalformedResultBundleError, and save_result refuses such a result before writing anything.

8. Environment and provenance

environment.json has identity vamos.environment and version 1.0.0. It contains Python, operating-system/architecture, installed distributions, backend package, BLAS, allowlisted thread controls, locale, and timezone. It does not store hostnames, account names, arbitrary environment variables, secrets, or personal paths.

Provenance records VAMOS/distribution identity, Git/source evidence when available, the environment artifact role, and the same timestamps as the manifest. Python saves identify vamos.optimize; CLI saves identify a sanitized CLI entry point. A caller-supplied manual context is marked as such.

VAMOS 1.2.0: saving a result that load_result returned is a manual copy. It has entry point vamos.save_result with arguments_source loaded_result, the timestamps of the new save, runtime 0, and replayability manual with the reason loaded_result_saved_again. No new schema field records the source run; replay lineage stays reserved for reproduce. A problem is a built-in component only when it was requested by registry name, or when it is an instance of the exact class the registry builds for its name. Other problems, including every make_problem problem, get provider custom_python and a custom.problem:<module>.<qualified name> component ID, never one derived from their free name. Their resolution records module, qualified_name and label, and their replayability is manual.

Replayability levels are exact, compatible, best_effort, manual, and unavailable. They are evidence declarations, not an execution API. An exact claim requires the same resolved spec, implementation, backend, materially equivalent environment, and a deterministic path. Loading does not test replay equivalence.

VAMOS 1.3.0: a run stopped by ("max_time", seconds) has no deterministic path: its termination is vamos.termination:max_time@1, its determinism.declared is false, its replayability is best_effort with the reason wall_clock_termination, and verification reports that reason and no exact replay. outcome.termination_reason is max_time, or max_evaluations when the optional evaluation cap stopped it first.

Exact verification compares only material evidence: VAMOS version and implementation fingerprint, source kind, Python implementation and major/minor version, operating system and architecture, NumPy/SciPy, selected backend and backend package, captured capabilities, BLAS, and allowlisted thread controls. The complete installed-package inventory is not material. Missing material evidence blocks exact replay. A dirty checkout qualifies only with a matching reproducible content fingerprint. Current evidence capture uses no Git command, shell, network, installation, or mutation.

VAMOS 1.2.0: saving and verification classify the source kind with one rule.

  • Checkout: the nearest Git repository above the imported package counts only when its src/vamos is that package, or when the distribution is an editable install. Otherwise the installed distribution decides:
  • a regular install is a wheel;
  • an editable install without Git is a checkout with reason: "git_checkout_unavailable". It has no revision or dirty state, and its fingerprints identify it.
  • Packaged data: checkouts and editable installs also record source.package_data_sha256, a SHA-256 of the data files under vamos/resources. The implementation fingerprint does not cover these files: for those installs it hashes *.py files only. Verification compares this field when the stored run has it, and a difference blocks exact replay.

9. Loading, verification, and errors

Loading is data-only. It performs no optimization, dynamic import, plugin resolution, custom-code execution, pickle load, shell command, network request, or filesystem access outside the run directory.

verify_run is also data-only. It verifies every referenced artifact's bytes, parses known environment/numerical artifacts through bounded safe readers, and reports artifact integrity, path/NPZ safety, environment compatibility, component reconstructability, and effective replayability independently. vamos results inspect performs manifest-only inspection without materializing arrays; vamos results verify performs full verification. --require-level exact fails when the effective level is lower than exact.

Verification modes are:

  • manifest: validate manifest JSON, semantics, task ID, and self-hash;
  • required (default): additionally verify artifacts required for loading;
  • all: verify every known referenced artifact.

An accessed artifact is always parsed through its bounded safe reader even in manifest mode. Unknown optional roles remain inert. An unknown role required for loading is rejected before it is opened.

Errors derive from RunArtifactError and include stable fields for operation, category, reason, expected/actual values, path/role/field, and action. Missing, modified, malformed, unsafe, oversized, unsupported-schema, collision, incomplete-run, and incomplete-metadata cases have distinct typed errors.

A directory without a supported manifest, or a manifest with another document identity/version, is rejected uniformly. The action states that this is a pre-1.0 development format and the run must be regenerated with the current VAMOS version. The reader does not inspect filenames to classify an earlier format.

10. Defensive limits

Default limits are finite: manifest 8 MiB, environment 16 MiB, one artifact or array 512 MiB, total uncompressed arrays 1 GiB, 128 descriptors/ZIP members, 64 arrays, 100 million elements, 8 dimensions, 64 KiB NPY headers, JSON depth 64, and compression ratio 1000:1. ZIP member names, overlaps, compression flags, headers, sizes, shapes, dtypes, and ratios are checked before materialization.

Trusted callers may pass an explicit LoadLimits. VAMOS never silently raises a limit following rejection.

VAMOS 1.2.0: a JSON document nested too deeply for Python's parser is rejected with the max_json_depth error, and an integer literal longer than sys.get_int_max_str_digits() (4300 digits by default) with ArtifactResourceLimitError and limit int_max_str_digits. vamos results verify exits with 3 for both, and study loading reports them as RESOURCE_LIMIT (exit 3) rather than as an infrastructure failure (exit 7).

11. Consumers

discover_runs finds manifest.json and accepts only manifests validated by the canonical reader. It leaves out replays (lineage.execution_kind is replay) unless include_replays=True, which marks them in RunInfo.replay. load_run_data and Studio load numerical values only through load_run/load_result. Aggregation reads resolved identity, seed, timestamps, outcome counters, runtime, termination, and optional metrics from the manifest. Directory names are presentation, not data recovery heuristics.

Study CSV summaries are derived reports. They may contain a run path or IDs but do not duplicate specifications or numerical arrays. A durable StudyManifest, resume/retry orchestration, and study artifact ownership are separate work.

12. Explicit non-goals

This v1 consolidation does not implement:

  • any alternate reader, writer, detector, adapter, or field spelling;
  • replay of plugins, custom Python, closures, notebook-local code, or arbitrary import paths;
  • cross-backend or best-effort replay, backend overrides, dependency installation, or environment repair;
  • a durable StudyManifest or study resume/retry system;
  • authentication/signatures (SHA-256 is integrity evidence, not trust);
  • a general CSV export API or performance optimization initiative.

13. Exact built-in replay

vamos reproduce RUN_DIR and vamos.reproduce(path) first use the same full verification service. Execution is permitted only for an effective exact run whose problem, algorithm, operators, evaluation backend, termination, and kernel use stable schema-1 built-in IDs. No plugin entry point is discovered and no manifest-provided module name is imported.

The replay plan is reconstructed solely from resolved_spec: typed algorithm configuration, operators, problem dimensions/encoding, population and reference-direction settings, archive/stopping configuration, termination budget, backend, and concrete seed. VAMOS regenerates a resolved spec from that explicit plan and requires canonical semantic equality before optimization. Current defaults never fill or replace persisted resolved values.

The stored result is the comparison target, not an initial state. Exact comparison requires F and X with identical NumPy dtype descriptor, shape, logical order, and contiguous C-order logical bytes. Every other deterministic array role present in either run is compared separately. Reports include per-array hashes, first differing logical index when safe, maximum absolute difference when meaningful, and mismatch classification. Timestamps, duration, run IDs, paths, and timing metrics are excluded.

Every executed replay publishes a new schema 1.0.0 run with a new run_id, the same content-derived task_id, immediate/root lineage, bounded depth, source-manifest hash, replay-plan hash, compatibility level, and comparison evidence. The default destination is <source-parent>/replays/<new-run-id>. Existing destinations collide and the source is never modified. Replay of a replay retains the root run ID without copying prior manifests.

If execution begins and fails, the atomic writer publishes an inspectable failed attempt containing manifest.json and environment.json, sanitized failure evidence, and replay lineage. Verification/plan refusal publishes nothing. A completed numerical mismatch is stored with mismatch evidence and is reported as failure, never exact success.

CLI exit codes are: 0 success, 2 usage, 3 integrity/path/malformed artifact, 4 unsupported or invalid schema, 5 compatibility requirement, 6 unavailable or untrusted component/replay, 7 execution or exact-comparison failure, and 8 output collision. JSON mode emits one machine-readable document. Verification is the separate results verify command; reproduction always executes.

14. Examples and acceptance

Sanitized machine-readable fixtures live in run_artifact_examples/. The normative acceptance matrix is run_artifact_acceptance_tests.md.