Data & machine learning · 2 min read
Network Anomaly Detection: making a model comparison reproducible
This project classifies historical NSL-KDD connection records as normal or attack traffic. I developed the training and prediction workflow around a practical question: can someone reproduce the comparison, load the selected model, and understand what its output actually means?
- My role
- Developer · personal academic project
- Project scope
- Implemented benchmark and dashboard · private source
The main responsibilities in this workflow.
- Validate connection-record schema
- Compare model pipelines on training folds
- Select a model, then evaluate the separate test split
- Save the versioned pipeline and schema
- Load the bundle for Streamlit prediction and export
Keep model selection away from the test split
The application compares Random Forest, Decision Tree, and KNN. Each candidate uses the same stratified training folds, and the selection order considers attack F1, attack recall, and false-positive rate. Only after that selection does the workflow evaluate the chosen pipeline against the separate test split.
This separation matters because repeatedly choosing a model from its test score turns the test set into part of the development process. Keeping selection inside the training split gives the final evaluation a clearer purpose. Accuracy is still reported, alongside precision, recall, F1, ROC-AUC, PR-AUC, and a confusion matrix.
Save preprocessing with the classifier
A prediction depends on how its input was transformed. The pipelines one-hot encode categorical protocol, service, and connection-state fields. KNN also scales numerical features; the tree models keep their original numerical scale. Unseen categorical values have an explicit handling rule.
The saved artifact contains the fitted pipeline, feature schema, selected model, evaluation results, seed, and split sizes. Loading checks the bundle version and required fields. Keeping these pieces together reduces the chance of applying a trained classifier to a differently prepared table.
Treat uploaded data as a contract
The Streamlit interface accepts labeled or unlabeled records. The schema layer supports canonical headered and headerless formats and validates required numerical values before inference. Predictions, class probabilities, and exportable results are presented together.
Ground-truth labels stay separate from predictions. Attack-family descriptions come from known labels; the binary classifier does not predict individual attack families. The batch risk label is a configurable summary of the predicted attack ratio, so the interface identifies it as a heuristic.
What the project demonstrates
The implementation brings data validation, model comparison, persistence, and a usable interface into one workflow. Its synthetic-data tests cover unseen categories, pipeline preprocessing, schema failures, artifact compatibility, metrics, and a train/save/load/predict cycle.
NSL-KDD is a historical benchmark. This project does not capture live packets or block traffic, and benchmark results do not establish effectiveness on a current production network. The useful contribution is the reproducible evaluation and inference workflow; no production detection rate is claimed.
Project notes
Academic project using historical NSL-KDD records. It does not inspect live network traffic; benchmark results are not production detection guarantees.