Provider plugins

Public acquisition now dispatches through ProviderPlugin and a local ProviderRegistry. Explicitly register trusted installed plugins in DEFAULT_PROVIDERS; WebPolicy.provider accepts registered IDs. Manifests record version, rendering, local-browser requirements, paid status and identity support. Receipts expose provider versions. No operation installs or downloads plugins. Acquisition envelopes validate source URLs, content, HTTP outcomes and policy byte limits; unexpected exception text is withheld.

HTTP, local browser, Steel, Camoufox and Scrapling use compatibility plugins. Named identities stay on their existing authority-controlled executor path; public plugin acquisition cannot substitute for them. Explicit disabled providers fail before acquisition/cache lookup. Paid providers need the existing paid-policy grant, and unreported paid costs remain null.

Milestone P now provides registry-based candidate eligibility, configurable escalation stops and caller-controlled resource budgets. Entry-point/config registration, broader budgets, scoped backend interfaces, route memory and learned provider selection remain pending. Compatibility provider versions currently use legacy, not automatic source versioning. Plugin code is trusted local code, not a sandbox.

The focused live canary passed exact Grays/Lloyds galleries and two-page Cash Converters/ALLBIDS cases through HTTP/router arms (eight observations). It does not establish all-site reliability or release readiness.

Explicit public-provider cache keys now include the registered provider version. A replacement version cannot reuse the previous version’s projection. Disabling an explicit provider blocks both acquisition and cached-success reuse before execution. Named identity cache scopes continue to follow their existing identity authority/version checks. This increment does not yet version learned routes or replace compatibility manifests’ legacy version labels.

Image count and browser settle time are caller-controlled policy budgets with defaults of 12 images and 400 ms. They have no fixed upper ceiling. Counts must be nonnegative integers, byte budgets positive integers, and timeout a positive finite number. Raising a budget permits more work; it does not establish gallery completeness or override a provider’s capabilities.

Provider byte budgets apply independently to UTF-8 extraction content and raw response bytes before the response enters evidence storage. A small text projection cannot admit an oversized raw response.

WebPolicy.terminal_failures controls which public acquisition failures stop provider escalation. context_stop_failures controls which failures stop later requests to the same domain in that Runtime. Both are immutable tuples of failure codes; defaults stop authentication failures (and terminal not-found acquisition) while blocks, challenges and rate limits permit escalation through existing candidates. Empty tuples allow public continuation on every failure code. Named identity failures still stop their executor path and never fall back to public providers. Learned routing remains pending.

provider_candidates=("http", "camoufox", "scrapling") selects an ordered sequence of distinct registered public plugins. Caller candidates override site seeds; an explicit provider remains a single-provider override. Each candidate still undergoes enabled/paid/local-browser checks. Terminal failure policy controls continuation. Named identities retain their enrolled executor and do not use this public sequence. Without an override, registry eligibility supplies the public candidates.

A Trading Post live canary on 2026-10-02 used caller candidates HTTP, Camoufox, then Scrapling with block escalation enabled. Two fresh rounds recovered from HTTP blocks through Camoufox on both initial and continuation pages. Each continuation contained 25 new listing IDs and zero overlap, with acknowledged query scope. Scrapling was not reached. Evidence is recorded in FRANKENBENCH_PROVIDER_SEQUENCE_SNAPSHOT.json. This is two observations of one query, not longitudinal reliability, complete galleries or current-state proof.

Caller-sequence cache keys include the ordered candidates’ manifests and enabled states. Replacing a version or disabling a candidate invalidates the prior sequence’s cached projection. A disabled candidate remains a typed failed attempt and may fall through to the next permitted candidate; it does not execute. Explicit single-provider and named-identity precedence is unchanged.

render=True requires a provider manifest with rendering capability. A public sequence records VISUAL_REQUIRED for incompatible candidates and can continue to a rendering plugin. An explicit incompatible provider fails preflight without execution. Capability filtering prevents a successful HTTP fetch from satisfying a request that requires browser rendering; it does not independently validate a plugin’s rendering implementation.

Public block/challenge/rate-limit escalation is now the default, matching SPEC section 9. Set terminal_failures and context_stop_failures to include the relevant codes to restore stop-on-block behavior. Named identities continue to stop on their executor’s failure and cannot fall back to public acquisition. Eligible registered plugins supply the default public candidate sequence.

Default public candidate selection now uses registry eligibility in registration order, including installed optional compatibility workers. Disabled, identity-only, ungranted paid and disallowed local-browser providers are excluded; rendering requirements filter capabilities. Steel requires configuration. Custom registered plugins can declare an optional synchronous available(configured) probe. Core does not install plugins. Default cache scopes include eligible candidate manifests. Site preferences seed the starting provider and are pending route-memory migration; candidate order is not yet learned.

For ordinary public reads, site seeds now place their preferred provider first in the eligible registry sequence, rather than locking the operation to one provider. Failures can fall through under the caller’s terminal-failure policy. Explicit provider/candidate overrides and named identities bypass seeds. Carsales numbered navigation retains its existing single-provider workflow pending separate migration. Preferences remain temporary code seeds; persistent route memory and learned ordering are still unfinished.

The default-router canary on 2026-10-02 passed both exact eBay galleries. Gumtree failed after Scrapling readiness timed out and all eligible fallbacks were blocked; Trading Post pagination also failed. Full receipts are saved in FRANKENBENCH_DEFAULT_REGISTRY_CANARY.json. This establishes observable fallback attempts, not reliable access. The earlier public sequence successes do not override these failures. Readiness/session continuity and longitudinal verification remain necessary.

Fresh acquisition attempts now append sanitized local records to route-observations.jsonl, including provider version, outcome, failure code, latency, adapter and observation time. Cache hits add no samples. Query values are omitted; named-identity paths and identity names are omitted. Cost/token measurements remain null in this journal and independent correctness is false. This is the observation foundation for milestone M; it does not yet learn or change routing, and attempt counts are not reliability estimates.

Runtime.route_capabilities() reads the local journal and groups observations by domain, path pattern, operation, adapter/version, identity class and provider/version. It reports sample/observed counts, typed failures and median latency with its sample count. Duplicate events do not add samples; malformed records are counted and skipped. Reliability, cost and token totals remain unknown. Independent correctness counts are separate. This operator surface exposes the initial capability graph without changing routing.

For adapter operations, route memory can now prefer a free eligible provider with repeated recent successful projections in the same domain/path pattern and adapter/provider versions. Defaults require three samples within one hour; use_route_memory, route_memory_min_samples and route_memory_ttl_seconds control learning. TTL zero disables preferences. A latest failure invalidates a provider’s preference. Among qualifying free providers, measured median latency orders candidates; unmeasured paid costs are not optimized. Explicit choices and identities bypass learning. Evidence remains projection-level, not independent correctness or a reliability estimate. Cheaper-provider reprobes and full benchmark-backed learning remain pending.

A failed route resets its preference sample window: only successful projections after the latest failure count toward promotion. One recovery cannot borrow old successes to satisfy the configured threshold. Malformed request URLs still produce the runtime’s typed invalid-URL outcome when a journal exists.

navigation_max_pages now controls the numbered Carsales category-navigation budget (default 3); callers can raise it. Core validation, worker packets and receipt checks share that budget. navigation_page must be within it. Bikesales still supports its first-page category-click workflow only. Automatic item search across category pages remains pending.

Carsales exact-item reads now automatically search observed category pages from page one, following Next controls until the item appears or navigation_max_pages is reached. navigation_auto_search=False disables this; an explicit later navigation_page stays an exact-page request. navigation_settle_ms controls the pre-click settle budget (default 10000 ms). The final receipt records the actual category page and continuation statuses, validated against policy. Bikesales automatic continuation remains pending.

The previously failing default Mazda detail case passed a fresh automatic page-two canary, preserved in FRANKENBENCH_CARSALES_AUTO_NAVIGATION.json. This is one repair canary, not longitudinal reliability or current-state proof.

Bikesales numbered category navigation and automatic exact-item search now use the vehicle workflow’s policy page/settle budgets. Continuations require a same-origin search-core POST, matching rendered page marker and per-step HTTP provenance. Cross-site Carsales data cannot satisfy Bikesales extraction. A first integrated live canary observed page one but page two was blocked; AU_BIKESALES_INTEGRATED_PAGINATION.json retains both outcomes. Unit fixtures validate page-two projection, but live distinct-page coverage remains unproven. The manifest’s Bikesales pagination gap therefore remains open.

A Bikesales category/detail classification defect was corrected: category URLs no longer enter exact-detail link selection. Regression checks cover both vehicle sites’ categories and detail URLs. Three subsequent fresh pagination rounds were blocked before live continuation verification; evidence is retained in AU_BIKESALES_CLASSIFICATION_REPAIR_CANARY.json. Session continuity and intermittent public access remain unresolved.

Vehicle-worker failures now label category navigation, continuation response, continuation marker or exact-listing search when that stage is known. Core accepts only fixed diagnostic labels and withholds arbitrary worker messages. The numbered vehicle worker already loads the category and clicks Next within one browser context; latest Bikesales blocks occurred on a fresh category load. Persistent browser reuse may still warrant investigation, but the existing worker does not separate those two steps into different contexts.

A 2026-10-02 Bikesales anonymous-context diagnostic loaded its category with HTTP 200, clicked Next in the same context, observed a visible page-two marker and 22 new exact listing links, then received HTTP 403 loading the category in a fresh context. AU_BIKESALES_CONTEXT_REUSE_DIAGNOSTIC.json records the comparison. The link counts include all matching links rather than validated result-card scope, and no API body was passed through the core adapter; this is diagnostic evidence, not a pagination benchmark pass. It motivates a same-context batched pagination acquisition. Named identity authority must remain separate.

The opt-in camoufox_vehicle_pages provider captures the initial vehicle category and its next page in one anonymous Camoufox context. Use it with the versioned vehicle_pages adapter. The composite representation is excluded from ordinary read routing; explicitly select the provider. It uses the caller’s navigation and byte budgets, acquires no enrolled identity, and validates the final same-origin continuation before canonicalizing site-added navigation parameters. The adapter validates both pages through existing Carsales/Bikesales scope, DOM/API listing-ID and page-marker checks, exposing new/overlap counts. It does not claim complete catalogue traversal or longitudinal reliability.