Evaluation reportGenerated by the project's CI evaluation on held-out data.About the projectCode on GitHub

Field service: evaluation

Synthetic regions of 10 technicians over four weeks; settings chosen on separate tuning regions.

91.7% vs 75.7%appointments kept: optimised plans vs the dispatcher's list (held-out)
43% lessdriving per job (10.7 km vs 18.7)
99.6% vs 72.8%first-time fix: van kit planned for the day vs a standard kit
90.4%first-time fix with the planned kit in the shifted regions, where a quarter of brands were new

Held-out regions: 10 regions

PlannerAppointments keptkm per jobOvertime h / tech-dayAvg days waitedBacklog
Dispatcher's list (nearest free technician, in booking order)75.7%18.70.090.27205
Optimised (tight windows first, cheapest insertion, moves between routes)91.7%10.70.090.10125

First-time fix: standard kit 72.8%, planned kit 99.6%.

Shifted regions (held back): busier, new brands, less history: 6 regions

PlannerAppointments keptkm per jobOvertime h / tech-dayAvg days waitedBacklog
Dispatcher's list (nearest free technician, in booking order)60.3%17.90.430.46250
Optimised (tight windows first, cheapest insertion, moves between routes)84.4%11.10.280.16196

First-time fix: standard kit 71.9%, planned kit 90.4%.

Tuning regions: 4 regions

PlannerAppointments keptkm per jobOvertime h / tech-dayAvg days waitedBacklog
Dispatcher's list (nearest free technician, in booking order)75.4%18.90.090.32109
Optimised (tight windows first, cheapest insertion, moves between routes)88.9%10.40.100.1590

First-time fix: standard kit 73.8%, planned kit 99.5%.

Gate (limits written before the held-out run)

CheckValueLimitResult
heldout_km_ratio_max0.57170.75pass
heldout_kept_gain_min0.15990.05pass
heldout_first_time_fix_gain_min0.26850.1pass
shifted_km_ratio_max0.61960.8pass
shifted_first_time_fix_gain_min0.1850.05pass