OfflineCreator Studio
AI Router · CLI · MCPCheapest eligible quotes before you create

3–6 months · partial

Score against a corpus, not a vibe.

Route intelligence needs a fixed set of jobs. Authenticated GET /api/v1/outcomes/benchmarks returns the frozen seed corpus plus observed receipt counts. Mix shares stay the seed design. This page does not run the suite.

Purpose
Comparable scores
Scope
Three workloads
Data
Seed + overlay

Frozen seed corpus · observed receipt overlay

  • Corpus items
    120

    Frozen jobs across the three launch workloads. Seed corpus ids stay fixed.

  • Workloads
    3

    Image-to-video, product image, vertical product ad.

  • Observed receipts
    0

    No inspected receipts yet. Overlay stays empty until wait+judge writes stats.

  • Image-to-video under budget40%
  • Ecommerce product image35%
  • Short vertical product ad25%

Build a corpus the router can fail against

GET /api/v1/outcomes/benchmarks keeps the frozen 120-item seed corpus and overlays observed pass/fail from inspected receipts.

12-month plan overviewLive AI router
  1. 01

    Freeze the jobs

    Each corpus item is an intent, inputs, and the evaluation checks that define success. Changing the jobs later makes historical scores incomparable.

  2. 02

    Cover the three launch workloads

    Image-to-video under budget, ecommerce product image, and short vertical product ad. Depth on those three beats a shallow catalog of every media type.

  3. 03

    Run every new capability through the suite

    Before a route is eligible for automated selection, it has to produce measurable results on the same items the current router is scored against.

  4. 04

    Keep human review in the loop

    Community anecdotes and one-off demos do not replace the corpus. Human or agent evaluation scores are stored as signals, not as marketing copy.

  • PurposeComparable scores

    The same jobs, over time, across models, workflows, and later sellers.

  • ScopeThree workloads

    Do not recruit broad supply until these jobs have a stable baseline.

  • DataSeed + overlay

    Mix shares stay the frozen seed design. Observed receipt counts overlay when present.

This page does not charge.

Stripe Checkout, wallet env, and seller payouts stay off this page. Authenticated outcomes execute is the live charge path. Named-model MCP and /ai-router remain available.

  • TODO(generation)

    Do not call FAL or other provider credentials. Candidate scores and outputs on these pages are illustrative.

Continue the plan

Plan FAQ

Is this corpus already scoring live routes?

Authenticated callers read the frozen seed corpus plus observed receipt overlays. The mix shares are still the seed design, not a public gating suite.

Will third-party sellers see the corpus?

The plan is that new capabilities run a standard benchmark and policy check before sandbox traffic. How much of the corpus is disclosed to sellers is not decided here.

No corpus, no proof.

A router that cannot fail a known job cannot claim it beats a manual baseline.

Back to the 12-month plan