Experiments
Machine learning projects I built to learn, from predicting game winners to reading handwriting.
See the code on GitHubWhy I built it
In 2024 I spent a lot of time playing with machine learning, building small projects to learn how different models behave on real data.
What I built
Environmental sound classification: I started from agrija9’s 2018 ESC-10 project and rebuilt the evaluation so the results can be trusted. Hyperparameters are tuned with nested cross-validation, recordings can’t leak between training and test folds, and runs are seeded and checked by automated tests. On 200 clips across five sound classes, tuned logistic regression did best at 92.5%, the SVM reached 91%, and a spectrogram CNN I added reached 84–88%. Most mistakes were helicopters confused with chainsaws.
League of Legends winner predictor: decision tree and random forest models trained on over 50,000 ranked games to predict the winning team from in-game events like first blood and first tower. They reached 96–97% accuracy on games they hadn’t seen, and I wrapped them in a small Streamlit app where you enter a game’s events and it gives each team’s chance of winning.
Handwritten digit recognition: a convolutional neural network trained on the MNIST digits, using data augmentation and early stopping, that reached 99.5% test accuracy. It comes with a small desktop app: pick an image of a digit and the model tells you what it is.
Anime recommender: collaborative filtering on ratings from 73,516 users across 12,294 titles, recommending anime similar to one you already like.
Breast cancer classification: compared logistic regression, SVM, decision tree and random forest models with cross-validation. The SVM did best, at about 95%.
House price prediction: ridge regression tuned with grid search to predict house prices from housing data.
What was hard
The sound classification project was honestly really hard. I had to stop and do some quick research just to get it up to standard.
The template I started from looked fine on the surface, but once I dug in, its results couldn’t be trusted. Each script split the data randomly in its own way, some used outdated Keras code, a few had missing imports or shifted labels, and the saved feature file claimed ten sound classes when only five were actually in the repo.
Clips cut from the same recording could also land in both training and test, which quietly makes a model look better than it is.
What happened
I rebuilt it as a small, tested package. Features are re-extracted from the audio that’s actually there, every model is tuned without ever seeing its test fold, runs are seeded, and the results are saved so anyone can reproduce them.
Tuned logistic regression came out on top at 92.5%. The CNN I added didn’t beat the simpler models on this small dataset, and adding augmentation made it slightly worse (84% against 88%). Most mistakes were helicopters confused with chainsaws.
What I learned
A model’s accuracy is only as trustworthy as the way you measured it. The old numbers looked impressive, but they came from an evaluation I couldn’t defend.
With a small dataset, simpler models on good features can beat a neural network, and extra tricks like augmentation don’t automatically help. You have to test them.
| Model | Accuracy | Macro-F1 | Fold accuracy SD |
|---|---|---|---|
| Logistic regression | 92.5% | 0.925 | 4.0% |
| SVM | 91.0% | 0.910 | 4.5% |
| Extra Trees | 88.0% | 0.880 | 2.1% |
| CNN, no masking | 88.0% | 0.879 | 4.8% |
| Random forest | 86.5% | 0.864 | 2.2% |
| CNN, with masking | 84.0% | 0.838 | 3.8% |
| Always guess one class | 20.0% | 0.067 | 0.0% |
