Skip to main content

​
Custom Router Training Data Format

​
Structure

{
  "problems": [
    {
      "problem_key": "string — unique ID per problem",
      "problem":     "string — raw input text the router classifies",
      "correct_models": ["model/string", "..."]
    }
  ]
}
incorrect_models is optional and ignored during training. Only correct_models is used as the multi-label target.

​
Validation (src/ml_tasks.py:138-141)

Problems are silently dropped if missing any of: problem_key, problem, correct_models. Minimum 10 valid problems required. Fewer → ValueError: No valid 'problems' found.

​
Model string format

Use OpenRouter model IDs, e.g.:
  • openai/gpt-4o
  • openai/gpt-4o-mini
  • anthropic/claude-3-haiku
  • google/gemini-2.5-flash
  • qwen/qwen3-coder
The trained router outputs these exact strings as predictions.

​
Quality tips

  • Include diverse problem types — router learns from variation
  • At least 2–3 models per problem (multi-label signal)
  • Problems where only one model ever succeeds add weak signal
  • Problems where no model succeeds are dropped (unsolvable filter)
  • 100+ problems recommended for production accuracy

​
Hosting

Data must be at a publicly accessible URL. Options:
  • Cloudflare R2 — upload with boto3, use public bucket URL
  • GitHub Gist — raw URL works
  • Any HTTP server — must return Content-Type: application/json