Independent component execution

The explicit executor API requires QUBODecomposition 0.1.2 or later; see installation.

Serial execution remains the default (component_executor=nothing), including its whole-model shortcut when the input fits max_variables. Opt into the supported component job pipeline with strategy=:components and either QUBODecomposition.SequentialComponentExecutor() or QUBODecomposition.ProcessComponentExecutor(pool; max_inflight=2). Both explicit executors always plan separate components, even when the entire model fits capacity. The sequential executor is useful for comparing identical frozen jobs with process execution.

Only independent, capacity-fitting components are supported. The executor option requires the default selection=:strongest_edge, partition_plan=nothing and separator=Int[]. Other strategies/options and any oversized component produce INVALID_OPTION before child dispatch; capacity is checked even for constant models. Coupled sweeps, separator branches and frozen voting batches remain on the existing serial paths. Empty/constant models require no child calls.

Factory and worker setup

Supply a fresh empty MOI optimizer per job. Existing zero-argument factories work; a factory accepting ComponentJob can configure generic allocation and observers from its supported context. Configure solver-specific attributes in that factory; the package forwards the job seed and minimum supported time limit. Do not capture live optimizers, mutable samplers or a shared observer/ledger in transported closures. Define named factories and their dependencies on every worker first.

Use an explicitly supplied Distributed.WorkerPool of local-host processes. Initialize Julia projects, package imports, factory definitions and thread limits before solving. Workers must load the same package revision and compatible Julia/dependencies as the coordinator. Dedicate the pool to these solves; do not use the same pool/executor concurrently from several parents or unrelated tasks. The package creates no workers, persists no job/model/result cache and never terminates caller workers. Pools can be reused across solves and model changes; the caller owns addprocs, initialization and rmprocs, including cleanup after errors. A factory that reuses a still-live optimizer is rejected on its worker. The executor limits submitted, uncommitted jobs, including out-of-order results buffered for deterministic commit. It never retries a failed job.

An executable tiny offline example uses two one-thread workers and released ExactSampler children, with no native SA or service credentials:

# SPDX-License-Identifier: MPL-2.0
# Explicit component executors and absolute_deadline_sec require version >=0.1.2.
using Distributed
import MathOptInterface as MOI
import QUBODecomposition, QUBODrivers, QUBOTools

function run_process_components()
    # Workers inherit the active environment containing this package revision.
    project = dirname(Base.active_project())
    pids = addprocs(2; exeflags=`--project=$project --threads=1 --startup-file=no`,
        env=["OPENBLAS_NUM_THREADS"=>"1", "OMP_NUM_THREADS"=>"1"])
    local state, energy, status
    identities = Any[]
    try
        for pid in pids
            remotecall_wait(Core.eval, pid, Main, :(import QUBODecomposition, QUBODrivers))
            identity = remotecall_fetch(Core.eval, pid, Main, :((;
                source=realpath(pkgdir(QUBODecomposition)),
                version=pkgversion(QUBODecomposition), project=realpath(Base.active_project()),
                driver_source=realpath(pkgdir(QUBODrivers)), driver_version=pkgversion(QUBODrivers))))
            @assert identity.source == realpath(pkgdir(QUBODecomposition))
            @assert identity.version == pkgversion(QUBODecomposition)
            @assert identity.project == realpath(Base.active_project())
            @assert identity.driver_source == realpath(pkgdir(QUBODrivers))
            @assert identity.driver_version == pkgversion(QUBODrivers)
            push!(identities, identity)
        end
        model = QUBOTools.Model{Symbol,Float64,Int}([:a,:b,:isolate],
            [1,2,3], [-2.,1.,-3.], [1], [2], [-1.]; offset=4.)
        executor = QUBODecomposition.ProcessComponentExecutor(WorkerPool(pids); max_inflight=2)
        optimizer = QUBODecomposition.Optimizer(child_optimizer=QUBODrivers.ExactSampler.Optimizer,
            max_variables=3, strategy=:components, component_executor=executor, seed=17,
            absolute_deadline_sec=time_ns()/1e9 + 3600.) # Cooperative allowance, not hard cancellation.
        QUBODrivers.set_model!(optimizer, model)
        MOI.optimize!(optimizer)
        state, energy = QUBOTools.state(optimizer,1), QUBOTools.value(optimizer,1)
        @assert state == [1,0,1] && energy == -1.
        @assert QUBOTools.value(model, state) == energy
        status = MOI.get(optimizer, MOI.TerminationStatus())
        certified = pkgversion(QUBODrivers) >= v"0.6.6"
        @assert status == (certified ? MOI.OPTIMAL : MOI.LOCALLY_SOLVED)
        @assert QUBOTools.metadata(QUBOTools.solution(optimizer))["decomposition"]["separable_proof"] == certified
        # The optimizer returns workers to the caller's pool for reuse.
        MOI.optimize!(optimizer)
        @assert QUBOTools.state(optimizer,1) == state && QUBOTools.value(optimizer,1) == energy
        @assert all(pid in workers() for pid in pids) && isready(executor.pool)
        # An expired absolute deadline must stop before invoking any child factory.
        MOI.set(optimizer, MOI.RawOptimizerAttribute("child_optimizer"),
            () -> error("expired deadline dispatched a child"))
        MOI.set(optimizer, MOI.RawOptimizerAttribute("absolute_deadline_sec"), 0.)
        MOI.optimize!(optimizer)
        @assert MOI.get(optimizer, MOI.TerminationStatus()) == MOI.TIME_LIMIT
        @assert MOI.get(optimizer, MOI.ResultCount()) == 0
        data = QUBOTools.metadata(QUBOTools.solution(optimizer))["decomposition"]
        @assert data["attempted_calls"] == 0 && isempty(data["calls"])
        @assert all(pid in workers() for pid in pids) && isready(executor.pool)
    finally
        rmprocs(pids) # Caller owns workers; the optimizer never terminates them.
    end
    @assert all(!(pid in workers()) for pid in pids)
    return (; state, energy, status, worker_identities=identities,
        expired_deadline_status=MOI.TIME_LIMIT)
end

if abspath(PROGRAM_FILE) == @__FILE__
    @show run_process_components()
end
run_process_components()
(state = [1, 0, 1], energy = -1.0, status = MathOptInterface.OPTIMAL, worker_identities = Any[(source = "/home/runner/work/QUBODecomposition.jl/QUBODecomposition.jl", version = v"0.1.5", project = "/home/runner/work/QUBODecomposition.jl/QUBODecomposition.jl/docs/Project.toml", driver_source = "/home/runner/.julia/packages/QUBODrivers/Y6pt4", driver_version = v"0.6.8"), (source = "/home/runner/work/QUBODecomposition.jl/QUBODecomposition.jl", version = v"0.1.5", project = "/home/runner/work/QUBODecomposition.jl/QUBODecomposition.jl/docs/Project.toml", driver_source = "/home/runner/.julia/packages/QUBODrivers/Y6pt4", driver_version = v"0.6.8")], expired_deadline_status = MathOptInterface.TIME_LIMIT)

Supported job boundary

ComponentJob.id is (solve_token, invocation, component). A fresh opaque solve token prevents another optimizer, a prior invocation or a changed model from reusing an identity. Component numbers and original free indices are deterministic; the solve token is deliberately unique and is never a seed input. indices and labels map reduced positions to original free variables. problem owns tuples of normalized coefficients, domain, sense, signed scale and conditioned offset. reference records the frozen component assignment. seed is (parent_seed + component - 1) mod (2^31-1), or nothing. deadline is the shared absolute local-host monotonic deadline; child_time_limit_sec is the parent child limit; candidate_limit bounds result transfer by the configured candidate cap minus the initial evaluation, fixed for every job in the invocation. The coordinator still enforces the shared remaining evaluation allowance. These are generic execution concepts; reads, SA schedules and scheduled-variable work units belong in a consumer factory/executor.

execute_component(factory, job) constructs the optimizer where called, copies owned model data, sets supported limits, executes and extracts owned result rows. No private optimizer object or parent solve state is serialized. The worker uses the minimum of factory time limit, child limit and remaining parent deadline when the child supports MOI.TimeLimitSec. The result carries public termination, reported result count, tuples of rows and support/worker/timing diagnostics. Worker exceptions propagate to the coordinator as errors, without replacement.

A custom AbstractComponentExecutor implements executor_width and submit_component(executor, factory, job). Return promptly with an isready/ fetch handle producing exactly one ComponentResult or throwing an error. The handle must remain drainable on every exit. Consumers can attach per-child allocation and diagnostics using this interface without accessing optimizer internals. Preserve owned data, isolation and identity; no implicit retry. The coordinator rejects duplicate, stale, mismatched and malformed envelopes, invalid/nonfeasible rows, nonfinite reported energies and failure-class statuses. Additional diagnostic fields are retained but cannot alter acceptance/proof. This is a trusted local extension interface, not a sandbox for arbitrary code.

Acceptance, budgets and stops

All component inputs are conditioned on the same frozen reference. No nonzero edge crosses components, so changing a complement changes only a child's constant. The coordinator commits in ascending component order, merging only the component's returned variables into the current full incumbent. It evaluates every candidate in the original objective, including scale and offset; it never sums child energies with repeated conditioned constants. Within each complete scan, equal-energy candidates use lexicographic full-state order. Incumbent replacement is strictly improving, preserving its existing assignment on ties. Under nonbinding limits and deterministic factories/children, sequential and process executors have identical job inputs/seeds and final results despite completion order. Wall-clock-limited runs can complete different work depending on scheduling; identical partial results are not promised.

The initial original evaluation and every validated row share the existing candidate cap. The coordinator reserves calls before submission (factory/copy/ worker failures remain charged), and requires room for at least one evaluation per outstanding job before dispatching another. This minimum reservation is not a promise that all child rows fit. Extra rows can exhaust the shared scan cap; partial scans never commit or certify their component. Worker result transfer is bounded by the fixed invocation candidate_limit; that cap does not bound opaque child internals. A complete public OPTIMAL component proof exactly at a call/candidate cap retains OPTIMAL; heuristic completion keeps existing cap precedence.

On a shared deadline, cancellation, invalid result, failure or a normally returned INTERRUPTED status, queued dispatch stops. Readiness is inspected before refilling the window. Previously submitted jobs stay charged. A completely validated interrupted component may improve the incumbent and then ends the parent with INTERRUPTED, without a global proof. It must have enough scan/time allowance to complete its transaction. When interruption arrives out of order, dispatch stops and the coordinator processes the already-dispatched stable prefix through that component. A parent deadline takes precedence over a late child status or result. Failure stops acceptance immediately.

Every exit drains outstanding handles; late rows are recorded as drained_uncommitted and cannot mutate the incumbent. Failed/unavailable results are failed_or_unknown. The built-in executor returns each acquired worker to the pool once its call returns/errors. Opaque/noninterruptible calls can extend solve/drain wall time beyond the cooperative deadline. There is no forcible cancellation and no assumption that a reservation was unspent. Callers needing a hard wall/memory bound must enforce an OS limit on the process group. Remote hosts, worker restart and replacing failed jobs are outside this initial executor. After process loss, the caller must supply a healthy pool before another solve.

Proof and timing

This is the existing separable method with concurrent isolated children. The original objective is a constant plus independent component terms over a product of binary/spin domains, so complete validated public OPTIMAL solutions for every component assemble a global optimum. Signed scale/sense are included in each child objective and the independent full evaluation. Public LOCALLY_SOLVED (including ExactSampler on QUBODrivers 0.6.5), missing/failed components and incomplete scans cannot establish this proof. Fixed variables remain reconstructed by the existing driver MOI copy hook; optimality is over the supplied logical QUBO, with the existing ToQUBO source-feasibility qualifications.

time.effective / MOI.SolveTimeSec measure parent wall time including waiting and draining. child_execution_sec sums child execution durations and can exceed parent wall time. Failed/missing results have unavailable child durations; child_execution_timing_complete=false flags that the sum contains only available durations. In explicit executor mode, parent_processing_sec=nothing: subtracting overlapping child durations does not measure overhead. Invocation phase_sec records coordinator preparation/conditioning/validation/full-energy work; its execution/copying slots do not duplicate worker costs. Per-call executor_diagnostics separates worker copying, execution and extraction, with worker id and local-host monotonic execution start/end. The invocation execution record includes window width, maximum observed in-flight count, arrival/commit orders, wait and drain wall durations. These durations overlap and are not additive. Per-job state/status/support/seeds/maps are retained in the existing bounded call log; no benchmark-specific storage or mandatory heavy tracing is added.

The retained four-arm native SA experiment used package revision 4f39e6eba398d0fa6c9c70c0b5e30e437ceb2c29, whose tree matches the 0.1.2 preparation base 44a97d92963726a200b54f6a7c72f6c53e08fb00. Its 32 validation controls and 88 comparison slots are complete and spent; all 20 component measurement pairs matched native inputs, output multisets and final assignments. The n=256 matched four-arm analysis retained nine complete blocks because one portfolio measurement hit its deadline. These observations establish neither general speedup nor SA optimality and give no reason to change defaults. The example and tests make no performance claim; the historical campaign is not part of package installation or release verification.