GitHub

Metadata Schema

Sampler metadata is the contract between a driver and benchmarking or conformance tooling. Every SampleSet returned through MOI.optimize! should carry the fields below. QUBODrivers validates this schema after it stamps framework metadata and emits a warning, not an error, for violations.

Use QUBODrivers.validate_metadata(sampleset) to check a result directly. The function returns a vector of human-readable violations; an empty vector means the metadata satisfies the current schema.

Required Fields

FieldTypeRequiredStamped byMeaning
metadata["origin"]StringyesdriverHuman-readable driver or backend origin.
metadata["algorithm"]["name"]StringyesdriverAlgorithm or sampler name used for grouping benchmark rows.
metadata["backend"]["name"]StringyesdriverBackend package, service, or solver name.
metadata["backend"]["version"]VersionNumber, String, or nothingyesdriverBackend version when known. Use nothing only when the backend cannot report one.
metadata["status"]Stringyesdriver or QUBODrivers fallbackRaw status string suitable for logs and reports.
metadata["reads"]["number_of_reads"]non-negative IntegeryesdriverReads, evaluations, or backend samples actually consumed by the sampler.
metadata["reads"]["final_number_of_reads"]non-negative IntegeryesdriverNumber of reads represented in the final emitted SampleSet.
metadata["seeds"]dictionary with string keysyesdriver and QUBODrivers seed recorderSeed values used by the sampler, backend, model generator, or optimizer. Empty is valid for deterministic solvers.
metadata["time"]["total"]non-negative RealyesQUBODriversWall-clock seconds around sample, post-sample callbacks, and attachment preparation.
metadata["time"]["effective"]non-negative RealyesdriverBackend solve or sampling seconds, excluding framework overhead when the backend exposes that split.

Drivers may include additional metadata fields for backend diagnostics. Prefer string keys and stable, machine-readable values.

Timing

metadata["time"] has two standardized entries:

  • "total" is QUBODrivers wall-clock time. It is always stamped by the framework when MOI.optimize! attaches a result.
  • "effective" is backend solve or sampling time. The driver must stamp it.

Additional timing fields are optional. Use concise names such as "build", "queue", or "decode" when their meaning is clear for the backend, or a nested dictionary when a driver needs a namespaced breakdown.

MOI.get(sampler, MOI.SolveTimeSec()) returns QUBODrivers.effective_time(sampler), not total wall-clock time. Use QUBODrivers.total_time(sampler) or QUBODrivers.total_time(sampleset) when a benchmark needs framework-inclusive wall time.

Seeds

Generated optimizers opt into the standard seed contract by declaring raw "seed" in their @setup attributes:

QUBODrivers.@setup Optimizer begin
    name = "Example Sampler"
    attributes = begin
        "seed"::Union{Integer,Nothing} = nothing
    end
end

Users can then call MOI.set(sampler, QUBODrivers.RandomSeed(), 123) or set the raw optimizer attribute "seed". When a non-nothing seed is configured, QUBODrivers records it as metadata["seeds"]["sampler"] after sampling.

Use QUBODrivers.supports_seed(sampler) to decide whether a benchmark harness can set QUBODrivers.RandomSeed() without driver-specific special cases.

Capability Traits

QUBODrivers exposes conservative trait functions for benchmark harnesses:

  • QUBODrivers.supports_seed(sampler) reports support for QUBODrivers.RandomSeed().
  • QUBODrivers.honors_final_reads(sampler) reports whether QUBODrivers.FinalNumberOfReads() controls the final emitted read count.
  • QUBODrivers.enforces_time_limit(sampler) reports whether MOI.TimeLimitSec() is enforced by the sampler contract.

The default for each trait is false. Drivers should define a method returning true only when the behavior is part of their public contract. Forwarding a time limit to a backend is not the same as guaranteeing enforcement unless the driver can rely on that backend behavior.

Validation

violations = QUBODrivers.validate_metadata(sampleset)
isempty(violations) || @warn "metadata is not benchmark-ready" violations

MOI.optimize! runs the same validation after framework metadata is stamped. Violations are warnings so existing drivers have a migration path, but conformance tests can choose to fail on any non-empty violation list.

API

QUBODrivers.RandomSeed — Type
RandomSeed()

Generic sampler optimizer attribute for a reproducibility seed.

The raw optimizer attribute key is "seed", and the default value is nothing. Generated optimizers support this typed attribute when the driver declares the raw "seed" attribute in its QUBODrivers.@setup block. When a seed is set, QUBODrivers records it under metadata["seeds"]["sampler"] in the emitted SampleSet.

source
QUBODrivers.random_seed — Function
random_seed(sampler)

Return the configured sampler seed, or nothing when the sampler does not support the standard seed attribute or no seed is configured.

source
QUBODrivers.total_time — Function
total_time(sampleset_or_sampler)

Return metadata["time"]["total"], the wall-clock seconds measured around the QUBODrivers sampling pipeline, or nothing when it is not available.

source
QUBODrivers.effective_time — Function
effective_time(sampleset_or_sampler)

Return metadata["time"]["effective"], the backend solve/sampling seconds reported by the driver, or nothing when it is not available.

source
QUBODrivers.validate_metadata — Function
validate_metadata(sampleset::SampleSet)
validate_metadata(metadata::AbstractDict)

Return a list of benchmarking metadata schema violations.

The validator is intentionally non-throwing so driver authors can use it while migrating metadata. MOI.optimize! calls it after QUBODrivers stamps framework metadata and emits a warning when the returned list is not empty.

source
QUBODrivers.enforces_time_limit — Function
enforces_time_limit(sampler_or_type)::Bool

Return whether a sampler enforces MOI.TimeLimitSec() as part of its own sampling contract. The default is conservative because many wrappers only store or forward the attribute.

source