Metadata Schema

Sampler metadata is the contract between a driver and benchmarking or conformance tooling. Every SampleSet returned through MOI.optimize! should carry the fields below. QUBODrivers validates this schema after it stamps framework metadata and emits a warning, not an error, for violations.

Use QUBODrivers.validate_metadata(sampleset) to check a result directly. The function returns a vector of human-readable violations; an empty vector means the metadata satisfies the current schema.

Required Fields

FieldTypeRequiredStamped byMeaning
metadata["origin"]StringyesdriverHuman-readable driver or backend origin.
metadata["algorithm"]["name"]StringyesdriverAlgorithm or sampler name used for grouping benchmark rows.
metadata["backend"]["name"]StringyesdriverBackend package, service, or solver name.
metadata["backend"]["version"]VersionNumber, String, or nothingyesdriverBackend version when known. Use nothing only when the backend cannot report one.
metadata["status"]Stringyesdriver or QUBODrivers fallbackRaw status string suitable for logs and reports.
metadata["reads"]["number_of_reads"]non-negative IntegeryesdriverReads, evaluations, or backend samples actually consumed by the sampler.
metadata["reads"]["final_number_of_reads"]non-negative IntegeryesdriverNumber of reads represented in the final emitted SampleSet.
metadata["seeds"]dictionary with string keysyesdriver and QUBODrivers seed recorderSeed values used by the sampler, backend, model generator, or optimizer. Empty is valid for deterministic solvers.
metadata["time"]["total"]non-negative RealyesQUBODriversWall-clock seconds around sample, post-sample callbacks, and attachment preparation.
metadata["time"]["effective"]non-negative RealyesdriverBackend solve or sampling seconds, excluding framework overhead when the backend exposes that split.

Drivers may include additional metadata fields for backend diagnostics. Prefer string keys and stable, machine-readable values.

Timing

metadata["time"] has two standardized entries:

  • "total" is QUBODrivers wall-clock time. It is always stamped by the framework when MOI.optimize! attaches a result.
  • "effective" is backend solve or sampling time. The driver must stamp it.

Additional timing fields are optional. Use concise names such as "build", "queue", or "decode" when their meaning is clear for the backend, or a nested dictionary when a driver needs a namespaced breakdown.

MOI.get(sampler, MOI.SolveTimeSec()) returns QUBODrivers.effective_time(sampler), not total wall-clock time. Use QUBODrivers.total_time(sampler) or QUBODrivers.total_time(sampleset) when a benchmark needs framework-inclusive wall time.

Seeds

Generated optimizers opt into the standard seed contract by declaring raw "seed" in their @setup attributes:

QUBODrivers.@setup Optimizer begin
    name = "Example Sampler"
    attributes = begin
        "seed"::Union{Integer,Nothing} = nothing
    end
end

Users can then call MOI.set(sampler, QUBODrivers.RandomSeed(), 123) or set the raw optimizer attribute "seed". When a non-nothing seed is configured, QUBODrivers records it as metadata["seeds"]["sampler"] after sampling.

Use QUBODrivers.supports_seed(sampler) to decide whether a benchmark harness can set QUBODrivers.RandomSeed() without driver-specific special cases.

Capability Traits

QUBODrivers exposes conservative trait functions for benchmark harnesses:

  • QUBODrivers.supports_seed(sampler) reports support for QUBODrivers.RandomSeed().
  • QUBODrivers.honors_final_reads(sampler) reports whether QUBODrivers.FinalNumberOfReads() controls the final emitted read count.
  • QUBODrivers.enforces_time_limit(sampler) reports whether MOI.TimeLimitSec() is enforced by the sampler contract.
  • QUBODrivers.supports_auto_configure(sampler) reports whether QUBODrivers.AutoConfigure() scales solver parameters automatically from problem size n (preserving explicit user attribute overrides and logging chosen values in metadata["auto_configure"]).

The default for each trait is false. Drivers should define a method returning true only when the behavior is part of their public contract. Forwarding a time limit to a backend is not the same as guaranteeing enforcement unless the driver can rely on that backend behavior.

Composite accounting

The existing schema permits additional dictionaries with string keys. A composite should describe its counters and timing scopes in a package-specific namespace, without changing the meaning of other drivers' metadata.

For example, QUBODecomposition's accounting contract uses a "decomposition" dictionary with schema version 1:

QuantityMeaning in this composite
reads.number_of_readsIndependently evaluated full candidates, including the initial state and lifted child rows; duplicate evaluations still consume work.
reads.final_number_of_readsEmitted multiplicity: one for a validated global incumbent, zero if none exists.
decomposition.candidate_evaluationsParent evaluation-budget consumption, separate from attempted/completed child-call counters.
decomposition.calls[k].reported_resultsNumber of rows reported by that child, when queried successfully; a complete scan is separately validated.
decomposition.calls[k].reported_multiplicitiesChild row multiplicities when available through public QUBOTools queries; these are diagnostics, not parent multiplicities.
decomposition.calls[k].physical_readsnothing when physical observations are unknown. Enumeration and constructed states do not establish hardware measurements.

Do not add or multiply child multiplicities to imply independently observed global samples. A composite may construct a single global state from several child solves; that state has no physical global-read count merely because each child reported reads. Keep unknown observations in optional diagnostics, rather than putting nothing in either required integer read field. This convention does not redefine the read counters of ordinary backend drivers.

For this composite, time.effective includes parent preparation, conditioning, copying/configuration, child execution, validation/reconstruction and independent original-energy evaluation. MOI.SolveTimeSec() returns that effective time. Leave time.total unset in sample so the framework fills the enclosing measurement, including post-sample callbacks. The current framework fills a missing total; it preserves a total already supplied by a driver.

The package's decomposition.phase_sec breakdown is disjoint; its sum is at most effective time, which also includes unclassified orchestration and final sample preparation. child_execution_sec aliases the execution phase and must not be added again. Parent effective time already contains child execution: neither child-reported effective times nor child wall times should be added to the parent total. Keep per-call diagnostics separate and state their scope.

A parent limit covers one invocation, including its preparation and reconstruction. A child limit covers one child call; the forwarded limit can be the minimum of its factory-configured limit, per-child cap and remaining parent time. An outer ToQUBO refinement budget covers repeated compiler/composite invocations and is another scope. Automatic refinement in released ToQUBO 0.7.1 does not establish a shared wall-clock deadline. Forwarding MOI.TimeLimitSec cannot forcibly cancel an opaque synchronous child, so this composite reports enforces_time_limit == false and records overruns.

Likewise, supports_seed == true describes the parent's public seed support; record each derived child seed and whether the child accepts it. Repeatability also depends on child behavior, ordered input, versions and effective work; wall-clock caps and arbitrary factories cannot guarantee it. This composite accepts FinalNumberOfReads but reports honors_final_reads == false. See its results contract and Public composition pattern for conservative result/status handling.

Preprocessing Metadata

When QUBODrivers.Preprocess() (raw key "preprocess") is set to an active QUBOPreprocessing.jl rule mode (:strong, :weak, :dominance, :elimination, :relations, :mixed, or :identity), QUBODrivers stamps metadata["preprocess"] and metadata["time"]["preprocess"]:

FieldTypeMeaning
metadata["preprocess"]["enabled"]Booltrue when preprocessing ran prior to sample.
metadata["preprocess"]["rules"]StringActive rule mode name (e.g., "strong" or "weak").
metadata["preprocess"]["original_variables"]non-negative IntegerVariable count before preprocessing.
metadata["preprocess"]["reduced_variables"]non-negative IntegerVariable count of the reduced model passed to sample.
metadata["preprocess"]["removed_variables"]non-negative IntegerTotal variables removed (original_variables - reduced_variables).
metadata["preprocess"]["reduction_steps"]non-negative IntegerNumber of certified reduction steps recorded in the certificate.
metadata["preprocess"]["time"]non-negative RealCombined wall-clock seconds spent in model reduction and sample reconstruction.
metadata["time"]["preprocess"]non-negative RealCombined wall-clock seconds spent in model reduction and sample reconstruction (mirrored in metadata["time"]).

Checking metadata

violations = QUBODrivers.validate_metadata(sampleset)
isempty(violations) || @warn "metadata is not benchmark-ready" violations

MOI.optimize! runs the same validation after framework metadata is stamped. Violations are warnings so existing drivers have a migration path, but conformance tests can choose to fail on any non-empty violation list.

API

QUBODrivers.RandomSeed — Type
RandomSeed()

Generic sampler optimizer attribute for a reproducibility seed.

The raw optimizer attribute key is "seed", and the default value is nothing. Generated optimizers support this typed attribute when the driver declares the raw "seed" attribute in its QUBODrivers.@setup block. When a seed is set, QUBODrivers records it under metadata["seeds"]["sampler"] in the emitted SampleSet.

source
QUBODrivers.random_seed — Function
random_seed(sampler)

Return the configured sampler seed, or nothing when the sampler does not support the standard seed attribute or no seed is configured.

source
QUBODrivers.total_time — Function
total_time(sampleset_or_sampler)

Return metadata["time"]["total"], the wall-clock seconds measured around the QUBODrivers sampling pipeline, or nothing when it is not available.

source
QUBODrivers.effective_time — Function
effective_time(sampleset_or_sampler)

Return metadata["time"]["effective"], the backend solve/sampling seconds reported by the driver, or nothing when it is not available.

source
QUBODrivers.validate_metadata — Function
validate_metadata(sampleset::SampleSet)
validate_metadata(metadata::AbstractDict)

Return a list of benchmarking metadata schema violations.

The validator is intentionally non-throwing so driver authors can use it while migrating metadata. MOI.optimize! calls it after QUBODrivers stamps framework metadata and emits a warning when the returned list is not empty.

source
QUBODrivers.enforces_time_limit — Function
enforces_time_limit(sampler_or_type)::Bool

Return whether a sampler enforces MOI.TimeLimitSec() as part of its own sampling contract. The default is conservative because many wrappers only store or forward the attribute.

source
QUBODrivers.AutoConfigure — Type
AutoConfigure()

Generic opt-in sampler optimizer attribute enabling problem-size-dependent parameter scaling (n = QUBOTools.dimension(sampler)).

The raw optimizer attribute key is "auto_configure", and the default value is false so existing solver behavior is unchanged unless explicitly enabled. Samplers that implement size-based scaling advertise support via supports_auto_configure, respect any explicitly user-set solver attributes over auto-scaled values (verifiable via has_explicit_raw_attr), clamp scaled parameters to explicit finite bounds, and log the chosen scaling details under metadata["auto_configure"].

source
QUBODrivers.auto_configure — Function
auto_configure(sampler)::Bool

Return whether automatic problem-size parameter scaling (AutoConfigure) is enabled on sampler. Samplers that do not support the attribute default to false.

source
QUBODrivers.has_explicit_raw_attr — Function
has_explicit_raw_attr(sampler, key::Union{Symbol,AbstractString})::Bool

Return true when the raw optimizer attribute key has been explicitly set by the caller on sampler (with a non-nothing value), and false when the attribute is still at its default value. Drivers implementing AutoConfigure use this check to preserve user override precedence.

source