Metadata Schema
Sampler metadata is the contract between a driver and benchmarking or conformance tooling. Every SampleSet returned through MOI.optimize! should carry the fields below. QUBODrivers validates this schema after it stamps framework metadata and emits a warning, not an error, for violations.
Use QUBODrivers.validate_metadata(sampleset) to check a result directly. The function returns a vector of human-readable violations; an empty vector means the metadata satisfies the current schema.
Required Fields
| Field | Type | Required | Stamped by | Meaning |
|---|---|---|---|---|
metadata["origin"] | String | yes | driver | Human-readable driver or backend origin. |
metadata["algorithm"]["name"] | String | yes | driver | Algorithm or sampler name used for grouping benchmark rows. |
metadata["backend"]["name"] | String | yes | driver | Backend package, service, or solver name. |
metadata["backend"]["version"] | VersionNumber, String, or nothing | yes | driver | Backend version when known. Use nothing only when the backend cannot report one. |
metadata["status"] | String | yes | driver or QUBODrivers fallback | Raw status string suitable for logs and reports. |
metadata["reads"]["number_of_reads"] | non-negative Integer | yes | driver | Reads, evaluations, or backend samples actually consumed by the sampler. |
metadata["reads"]["final_number_of_reads"] | non-negative Integer | yes | driver | Number of reads represented in the final emitted SampleSet. |
metadata["seeds"] | dictionary with string keys | yes | driver and QUBODrivers seed recorder | Seed values used by the sampler, backend, model generator, or optimizer. Empty is valid for deterministic solvers. |
metadata["time"]["total"] | non-negative Real | yes | QUBODrivers | Wall-clock seconds around sample, post-sample callbacks, and attachment preparation. |
metadata["time"]["effective"] | non-negative Real | yes | driver | Backend solve or sampling seconds, excluding framework overhead when the backend exposes that split. |
Drivers may include additional metadata fields for backend diagnostics. Prefer string keys and stable, machine-readable values.
Timing
metadata["time"] has two standardized entries:
"total"is QUBODrivers wall-clock time. It is always stamped by the framework whenMOI.optimize!attaches a result."effective"is backend solve or sampling time. The driver must stamp it.
Additional timing fields are optional. Use concise names such as "build", "queue", or "decode" when their meaning is clear for the backend, or a nested dictionary when a driver needs a namespaced breakdown.
MOI.get(sampler, MOI.SolveTimeSec()) returns QUBODrivers.effective_time(sampler), not total wall-clock time. Use QUBODrivers.total_time(sampler) or QUBODrivers.total_time(sampleset) when a benchmark needs framework-inclusive wall time.
Seeds
Generated optimizers opt into the standard seed contract by declaring raw "seed" in their @setup attributes:
QUBODrivers.@setup Optimizer begin
name = "Example Sampler"
attributes = begin
"seed"::Union{Integer,Nothing} = nothing
end
endUsers can then call MOI.set(sampler, QUBODrivers.RandomSeed(), 123) or set the raw optimizer attribute "seed". When a non-nothing seed is configured, QUBODrivers records it as metadata["seeds"]["sampler"] after sampling.
Use QUBODrivers.supports_seed(sampler) to decide whether a benchmark harness can set QUBODrivers.RandomSeed() without driver-specific special cases.
Capability Traits
QUBODrivers exposes conservative trait functions for benchmark harnesses:
QUBODrivers.supports_seed(sampler)reports support forQUBODrivers.RandomSeed().QUBODrivers.honors_final_reads(sampler)reports whetherQUBODrivers.FinalNumberOfReads()controls the final emitted read count.QUBODrivers.enforces_time_limit(sampler)reports whetherMOI.TimeLimitSec()is enforced by the sampler contract.QUBODrivers.supports_auto_configure(sampler)reports whetherQUBODrivers.AutoConfigure()scales solver parameters automatically from problem sizen(preserving explicit user attribute overrides and logging chosen values inmetadata["auto_configure"]).
The default for each trait is false. Drivers should define a method returning true only when the behavior is part of their public contract. Forwarding a time limit to a backend is not the same as guaranteeing enforcement unless the driver can rely on that backend behavior.
Composite accounting
The existing schema permits additional dictionaries with string keys. A composite should describe its counters and timing scopes in a package-specific namespace, without changing the meaning of other drivers' metadata.
For example, QUBODecomposition's accounting contract uses a "decomposition" dictionary with schema version 1:
| Quantity | Meaning in this composite |
|---|---|
reads.number_of_reads | Independently evaluated full candidates, including the initial state and lifted child rows; duplicate evaluations still consume work. |
reads.final_number_of_reads | Emitted multiplicity: one for a validated global incumbent, zero if none exists. |
decomposition.candidate_evaluations | Parent evaluation-budget consumption, separate from attempted/completed child-call counters. |
decomposition.calls[k].reported_results | Number of rows reported by that child, when queried successfully; a complete scan is separately validated. |
decomposition.calls[k].reported_multiplicities | Child row multiplicities when available through public QUBOTools queries; these are diagnostics, not parent multiplicities. |
decomposition.calls[k].physical_reads | nothing when physical observations are unknown. Enumeration and constructed states do not establish hardware measurements. |
Do not add or multiply child multiplicities to imply independently observed global samples. A composite may construct a single global state from several child solves; that state has no physical global-read count merely because each child reported reads. Keep unknown observations in optional diagnostics, rather than putting nothing in either required integer read field. This convention does not redefine the read counters of ordinary backend drivers.
For this composite, time.effective includes parent preparation, conditioning, copying/configuration, child execution, validation/reconstruction and independent original-energy evaluation. MOI.SolveTimeSec() returns that effective time. Leave time.total unset in sample so the framework fills the enclosing measurement, including post-sample callbacks. The current framework fills a missing total; it preserves a total already supplied by a driver.
The package's decomposition.phase_sec breakdown is disjoint; its sum is at most effective time, which also includes unclassified orchestration and final sample preparation. child_execution_sec aliases the execution phase and must not be added again. Parent effective time already contains child execution: neither child-reported effective times nor child wall times should be added to the parent total. Keep per-call diagnostics separate and state their scope.
A parent limit covers one invocation, including its preparation and reconstruction. A child limit covers one child call; the forwarded limit can be the minimum of its factory-configured limit, per-child cap and remaining parent time. An outer ToQUBO refinement budget covers repeated compiler/composite invocations and is another scope. Automatic refinement in released ToQUBO 0.7.1 does not establish a shared wall-clock deadline. Forwarding MOI.TimeLimitSec cannot forcibly cancel an opaque synchronous child, so this composite reports enforces_time_limit == false and records overruns.
Likewise, supports_seed == true describes the parent's public seed support; record each derived child seed and whether the child accepts it. Repeatability also depends on child behavior, ordered input, versions and effective work; wall-clock caps and arbitrary factories cannot guarantee it. This composite accepts FinalNumberOfReads but reports honors_final_reads == false. See its results contract and Public composition pattern for conservative result/status handling.
Preprocessing Metadata
When QUBODrivers.Preprocess() (raw key "preprocess") is set to an active QUBOPreprocessing.jl rule mode (:strong, :weak, :dominance, :elimination, :relations, :mixed, or :identity), QUBODrivers stamps metadata["preprocess"] and metadata["time"]["preprocess"]:
| Field | Type | Meaning |
|---|---|---|
metadata["preprocess"]["enabled"] | Bool | true when preprocessing ran prior to sample. |
metadata["preprocess"]["rules"] | String | Active rule mode name (e.g., "strong" or "weak"). |
metadata["preprocess"]["original_variables"] | non-negative Integer | Variable count before preprocessing. |
metadata["preprocess"]["reduced_variables"] | non-negative Integer | Variable count of the reduced model passed to sample. |
metadata["preprocess"]["removed_variables"] | non-negative Integer | Total variables removed (original_variables - reduced_variables). |
metadata["preprocess"]["reduction_steps"] | non-negative Integer | Number of certified reduction steps recorded in the certificate. |
metadata["preprocess"]["time"] | non-negative Real | Combined wall-clock seconds spent in model reduction and sample reconstruction. |
metadata["time"]["preprocess"] | non-negative Real | Combined wall-clock seconds spent in model reduction and sample reconstruction (mirrored in metadata["time"]). |
Checking metadata
violations = QUBODrivers.validate_metadata(sampleset)
isempty(violations) || @warn "metadata is not benchmark-ready" violationsMOI.optimize! runs the same validation after framework metadata is stamped. Violations are warnings so existing drivers have a migration path, but conformance tests can choose to fail on any non-empty violation list.
API
QUBODrivers.RandomSeed — Type
RandomSeed()Generic sampler optimizer attribute for a reproducibility seed.
The raw optimizer attribute key is "seed", and the default value is nothing. Generated optimizers support this typed attribute when the driver declares the raw "seed" attribute in its QUBODrivers.@setup block. When a seed is set, QUBODrivers records it under metadata["seeds"]["sampler"] in the emitted SampleSet.
QUBODrivers.random_seed — Function
random_seed(sampler)Return the configured sampler seed, or nothing when the sampler does not support the standard seed attribute or no seed is configured.
QUBODrivers.total_time — Function
total_time(sampleset_or_sampler)Return metadata["time"]["total"], the wall-clock seconds measured around the QUBODrivers sampling pipeline, or nothing when it is not available.
QUBODrivers.effective_time — Function
effective_time(sampleset_or_sampler)Return metadata["time"]["effective"], the backend solve/sampling seconds reported by the driver, or nothing when it is not available.
QUBODrivers.validate_metadata — Function
validate_metadata(sampleset::SampleSet)
validate_metadata(metadata::AbstractDict)Return a list of benchmarking metadata schema violations.
The validator is intentionally non-throwing so driver authors can use it while migrating metadata. MOI.optimize! calls it after QUBODrivers stamps framework metadata and emits a warning when the returned list is not empty.
QUBODrivers.supports_seed — Function
supports_seed(sampler_or_type)::BoolReturn whether a sampler advertises support for the standard RandomSeed attribute.
QUBODrivers.honors_final_reads — Function
honors_final_reads(sampler_or_type)::BoolReturn whether a sampler uses FinalNumberOfReads to control the number of samples emitted in the final SampleSet.
QUBODrivers.enforces_time_limit — Function
enforces_time_limit(sampler_or_type)::BoolReturn whether a sampler enforces MOI.TimeLimitSec() as part of its own sampling contract. The default is conservative because many wrappers only store or forward the attribute.
QUBODrivers.AutoConfigure — Type
AutoConfigure()Generic opt-in sampler optimizer attribute enabling problem-size-dependent parameter scaling (n = QUBOTools.dimension(sampler)).
The raw optimizer attribute key is "auto_configure", and the default value is false so existing solver behavior is unchanged unless explicitly enabled. Samplers that implement size-based scaling advertise support via supports_auto_configure, respect any explicitly user-set solver attributes over auto-scaled values (verifiable via has_explicit_raw_attr), clamp scaled parameters to explicit finite bounds, and log the chosen scaling details under metadata["auto_configure"].
QUBODrivers.auto_configure — Function
auto_configure(sampler)::BoolReturn whether automatic problem-size parameter scaling (AutoConfigure) is enabled on sampler. Samplers that do not support the attribute default to false.
QUBODrivers.supports_auto_configure — Function
supports_auto_configure(sampler_or_type)::BoolReturn whether a sampler implements automatic problem-size parameter scaling when AutoConfigure is set to true. Defaults to false.
QUBODrivers.has_explicit_raw_attr — Function
has_explicit_raw_attr(sampler, key::Union{Symbol,AbstractString})::BoolReturn true when the raw optimizer attribute key has been explicitly set by the caller on sampler (with a non-nothing value), and false when the attribute is still at its default value. Drivers implementing AutoConfigure use this check to preserve user override precedence.