Maintenance and dispatch, tested before the plant
On-time orders: 50.6% → 98.9%
300 holdout scenarios, verified by simulation
Operator card · recommendation
Real vLLM (Qwen3-8B), calibrated plant
On time in holdout: 50.6% with the fixed rule, 98.9% with the best policy.
Why
Winning policy: dispatch CRITICAL_RATIO, 1 PM per shift and 5 rules. Under stress the throughput jump is 202.1 and the on-time jump is 13.4%. The gap is throughput, not on-time, so the card does not promise the same jump under that spike.
Numbers, verified by the simulator
- 98.9% On-time orders, recommended policy fixed rule 50.6%
- 50.6% On-time orders, fixed rule today same scenarios
- 819.0 Orders completed per week fixed rule 745.2
- 300 Holdout scenarios not used to train the search
Holdout objective interval: fixed rule 0.7725 [0.743, 0.802]; best policy 1.3691 [1.3537, 1.3846].
Your decision
Pending
Your turn
Try a policy
Build a maintenance and dispatch policy and the simulator scores it against the fixed rule and the LLM's best policy, on the same seeds. No LLM is called.
How we compare
Efficiency
Best objective found against simulations spent, for each search method.
| method | train | holdout | 95% CI | holdout on-time | sims | sims to 90% | source |
|---|---|---|---|---|---|---|---|
| Fixed rule | 0.7858 | 0.7725 | 0.7431–0.8017 | 50.6% | 180 | — | final |
| Optuna TPE | 1.5188 | 1.5126 | 1.5048–1.5198 | 99.9% | 69300 | 5760 | final |
| LLM without feedback | 1.2318 | 1.2259 | 1.2167–1.2349 | 91.4% | 69300 | — | real vLLM |
| Loop with feedback | 1.3755 | 1.3691 | 1.3530–1.3842 | 98.9% | 37560 | — | real vLLM |
The loop and one-shot numbers are real vLLM (Qwen3-8B) on the calibrated plant; they are not the mock. Optuna is still ahead on the composite objective; the loop beats one-shot and delivers the on-time jump. The composite objective is not bounded to 0–1 and is not compared across plants. Calibration is a shape fit on a public dataset; it is not a twin of a specific factory and it does not promise a real deployment. Under stress the gap is throughput, not on-time.