The challenge
Spam filtering is a classic text-classification task, but a high accuracy figure says little until you know whether the test data is independent of the training data, and what the model is actually relying on. The goal was to build a complete classification pipeline and assess the result critically.
The solution
A pipeline from raw email to prediction: loading, preprocessing (lower-casing, removing punctuation and stop words), TF-IDF features with single words and word pairs, and a linear SVM. The vectoriser is fitted on the training data only, so the test data does not influence the model.
How it was built
The work builds on a provided framework with interfaces and tests; I implemented preprocessing, feature extraction, training and evaluation with scikit-learn. The dataset is a public, Enron-based collection with 14,925 emails for training and 1,655 for testing in a fixed split. In a later re-evaluation I also checked the overlap between training and test data and which signals the model can exploit.
Result
With the fixed split, 1,653 of 1,655 test emails were classified correctly. The later re-evaluation removed test emails that also appeared as exact or near-duplicates in the training data; accuracy was still about 99.8% on the remaining 1,282. The same re-evaluation showed that using only the start of each email (where technical email headers dominate) already gives about 97% accuracy. The dataset is therefore relatively easy and has strong shortcut signals.
Context
The figures come from a later, reproducible run of the pipeline, not from an originally submitted result file. The value of the project lies in a complete, checked text-classification pipeline and the robustness analysis, not in the high figure, which says little about how the model would perform on today’s spam. The solution has not been used in production.