Meeting operations: evaluation
Scheduling across six time zones, and action items from transcripts. Settings tuned on separate organisations and phrasing.
75.6%fewer night and weekend meeting hours (17 vs 68 a week)
1680 of 1680requests placed in held-out organisations
0.72owner-and-task F1 on unseen phrasing (rules: 0.00)
96.4%of action items extracted or put in front of a reviewer
Scheduling
| Organisations | Policy | Out-of-hours h/week | Night h/week | Worst-off person | Focus lost | Optional can come |
|---|---|---|---|---|---|---|
| Held-out organisations | First free slot (organiser's hours) | 108.0 | 67.9 | 4.24 h | 2.37 h | 32.9% |
| Held-out organisations | First slot in everyone's hours | 103.7 | 66.1 | 4.06 h | 2.53 h | 34.5% |
| Held-out organisations | Balanced (this platform) | 93.0 | 16.6 | 3.85 h | 2.37 h | 39.6% |
| Other time zones, fuller calendars | First free slot (organiser's hours) | 122.0 | 75.9 | 4.28 h | 1.87 h | 34.9% |
| Other time zones, fuller calendars | First slot in everyone's hours | 118.7 | 74.4 | 4.28 h | 2.02 h | 35.2% |
| Other time zones, fuller calendars | Balanced (this platform) | 111.3 | 20.4 | 4.13 h | 1.73 h | 36.9% |
Action items: owner-and-task F1 (with due date)
| Phrasing | Keyword search | Hand-written rules | Learned reader | Surfaced |
|---|---|---|---|---|
| Familiar phrasing (rules built on) | 0.173 (0.173) | 1.000 (1.000) | 1.000 (1.000) | 100.0% |
| Familiar phrasing, new meetings | 0.170 (0.170) | 1.000 (1.000) | 1.000 (1.000) | 100.0% |
| New phrasing A (validation) | 0.128 (0.095) | 0.000 (0.000) | 0.752 (0.549) | 92.6% |
| New phrasing B (test) | 0.112 (0.056) | 0.000 (0.000) | 0.722 (0.335) | 96.4% |
Gate (regression limits)
| Check | Value | Limit | Result |
|---|---|---|---|
| heldout_night_vs_first_slot_max | 0.244 | 0.3 | pass |
| fresh_night_vs_first_slot_max | 0.269 | 0.33 | pass |
| heldout_out_of_hours_vs_first_slot_max | 0.861 | 0.92 | pass |
| heldout_placed_share_min | 1.0 | 0.99 | pass |
| heldout_learned_f1_min | 1.0 | 0.99 | pass |
| fresh2_learned_owner_task_f1_min | 0.722 | 0.68 | pass |
| fresh2_surfaced_min | 0.964 | 0.93 | pass |