Machine learning · recs.apiCI passing

Recommender System

Recommendations measured honestly: strong baselines, a temporal split, cold start and serving.

View the code Evaluation report Demo tour Run it ›_ last push 5 days ago

+29%NDCG@20 over popularity (95% CI of the per-user difference +0.034 to +0.043)

2.8xNDCG for cold-start users with the content fallback (0.134 against 0.048 for EASE alone)

+65%how much a random split flatters the model compared with the temporal split it will face

Evaluation report: NDCG for seven recommenders overall and for cold-start users, and the effect of a random versus temporal split. Screenshot from the project's demo (synthetic data).

demo Demo tour

Screenshots from the running application, with its synthetic demo data. Select one to enlarge.

Seven recommenders on the test window: NDCG overall and for cold-start users, coverage and popularity bias.

A random split flatters every model; the hyperparameters each model was given, all chosen on validation.

01 The problem

Recommender results are easy to flatter: a random train/test split lets the model learn from the future, hyperparameters get tuned on the test set, and popularity, the baseline that matters, is left out. New users with no history are where most models quietly fail.

02 How it works

Interactions up to today
EASE fitted (closed form)
Content fallback for thin history
Remove what the user has
Top-K in ~5 ms
Scored on the next 30 days

03 What I built

Popularity, item-kNN, EASE, implicit ALS and BPR, each tuned on a validation window and scored once on a later test window, with paired bootstrap intervals over users.
A cold-start hybrid that blends content (genre affinity) and popularity with the collaborative model, weighted by how much history a user has.
A serving API for one user's top-K and similar items, with catalogue coverage and popularity bias reported next to accuracy.

04 What I learned

ALS first looked worse than popularity; a wider grid on the validation window (much stronger regularisation) put it level with EASE. Weak baselines usually mean weak tuning.

Collaborative models score a third of popularity on users with three or fewer interactions; blending by history length fixes cold start without hurting anyone else.

05 Known limitations

Every repository documents what it doesn't do yet. The main ones:

Synthetic interactions and offline metrics; an online A/B test is the real judge.
Genre is the only content feature, so items released after training are still hard to recommend.

06 Stack

PythonNumPySciPy sparseFastAPI

The repository has the design document, operations runbook, full evaluation report and known limitations. All data is synthetic, generated by code in the repository.

run Run it yourself

Every project runs on a laptop: no cloud account and no paid API needed.

git clone https://github.com/codeROCK-IZZY/recommender-system.git
cd recommender-system
pip install -e ".[dev]" && pytest
recsys evaluate --gate                # writes eval/reports/report.html
recsys recommend 42
recsys serve                          # GET /users/42/recommendations