Property management: evaluation
Maintenance requests in tenants' own words, triaged and dispatched in simulated months; and a year of rent through two ways of allocating payments.
96.8%of emergencies attended within 4 hours with triage, vs 59.2% with keyword rules and 51.4% in inbox order (fresh phrasings)
23 h vs 62 hlongest wait for an emergency: triage vs inbox order
90.4%of emergency messages caught, vs 14.8% for keyword rules, at 19.3% false alarms
0 vs 1,756late fees charged while all rent was paid: rent-first allocation vs strictly oldest first
Validation phrasings (seen while the lexicon was written)
| Messages | Emergencies caught | False alarms | Urgency right | Trade right |
|---|---|---|---|---|
| Keyword rules | 9.7% | 13.9% | 47.9% | 69.3% |
| Learned model, words only | 67.4% | 26.0% | 49.8% | 57.6% |
| Triage (hazard screen + model) | 100.0% | 15.6% | 62.7% | 88.9% |
Simulated months: 2,324 requests, 262 emergencies.
| Policy | Within 4 h | Median | 90th pct | Worst | False alarms / week | At night |
|---|---|---|---|---|---|---|
| Inbox order, no triage | 53.1% | 3.6 h | 32.9 h | 62.8 h | 0.0 | 0.0 |
| Keyword rules flag emergencies | 56.5% | 3.4 h | 32.2 h | 62.8 h | 12.8 | 2.0 |
| Triage flags emergencies | 100.0% | 2.2 h | 3.0 h | 3.2 h | 12.8 | 2.1 |
Fresh phrasings (written after the freeze): the test
| Messages | Emergencies caught | False alarms | Urgency right | Trade right |
|---|---|---|---|---|
| Keyword rules | 14.8% | 8.6% | 61.0% | 67.3% |
| Learned model, words only | 84.4% | 30.5% | 64.8% | 52.7% |
| Triage (hazard screen + model) | 90.4% | 19.3% | 71.5% | 77.2% |
Simulated months: 3,235 requests, 311 emergencies.
| Policy | Within 4 h | Median | 90th pct | Worst | False alarms / week | At night |
|---|---|---|---|---|---|---|
| Inbox order, no triage | 51.4% | 3.7 h | 28.5 h | 62.3 h | 0.0 | 0.0 |
| Keyword rules flag emergencies | 59.2% | 3.2 h | 25.1 h | 61.6 h | 8.3 | 1.3 |
| Triage flags emergencies | 96.8% | 2.2 h | 3.1 h | 23.1 h | 17.6 | 3.1 |
Emergencies triage missed (sample):
- Good morning, someone tried to force the lock and now it won't close upstairs.
- Good morning, the bathroom ceiling has come down in the kitchen since this morning.
- Hello, the plug socket in the bedroom went bang and is black in the bathroom. We have a baby at home. Cheers
- Hey, Please help ASAP! the plug socket in the bedroom went bang and is black upstairs since yesterday. Thank you!
- Hey, When you get a chance: the bedroom window frame fell out in the wind in the kitchen since yesterday. My mum is elderly and lives with us.
- Hey, the bedroom window frame fell out in the wind in the bathroom since last night. Thank you!
- Hi there, EMERGENCY: someone tried to force the lock and now it won't close in the kitchen. I work from home. Cheers
- Hi, No rush, but the bedroom window frame fell out in the wind in flat 4B. My mum is elderly and lives with us.
Rent ledger: a year, 5 portfolios
| Allocation | Late fees | Fees total | Fees while all rent was paid | Paid all rent, shown owing rent |
|---|---|---|---|---|
| rent first | 1,639 | $81,950 | 0 | 0 of 904 |
| oldest first | 3,395 | $169,750 | 1,756 | 286 of 904 |
Gate (regression limits)
Passed