Web servers provide websites, web applications, APIs, and other online services by processing HTTP requests received from clients. These requests generate network traffic with measurable characteristics, including flow duration, packet counts, byte counts, packet lengths, and traffic rates. Analyzing these characteristics can help security teams investigate abnormal network activity and identify potential Distributed Denial-of-Service (DDoS) attacks.
DDoS attacks attempt to disrupt service availability by generating excessive traffic from multiple sources or distributed systems. Such activity may consume network bandwidth, connection capacity, CPU resources, or other server resources, potentially affecting legitimate users. Identifying suspicious traffic is an important part of DDoS investigation, but a classification result alone does not explain why a particular traffic observation was considered malicious.
Traditional detection approaches may use fixed thresholds, predefined signatures, or known attack patterns. Machine-learning classifiers can analyze multiple traffic characteristics to distinguish normal traffic from DDoS-related activity. However, complex classification models may provide limited information about which features influenced a particular prediction.
Explainable Artificial Intelligence (XAI) addresses this limitation by providing explanations for machine-learning predictions. In this use case, HistGradientBoostingClassifier is used to classify network traffic, while SHAP (SHapley Additive exPlanations) is used to explain the features influencing the model's predictions.
An Apache HTTP Server is deployed on Ubuntu within an isolated laboratory environment. Kali Linux is used as the controlled security-testing system, and VirtualBox provides the virtualized environment for the application and testing systems.
A public DDoS traffic dataset is used as the existing data source for machine-learning development. Normal and DDoS-related traffic records are selected and processed to extract the features required for model training. The dataset is obtained from an existing public source rather than manually created for this project.
The HistGradientBoostingClassifier model is trained using the selected labelled traffic records. After model evaluation, SHAP is applied to explain the classification results by identifying features that contribute to individual predictions and by summarizing influential features across the evaluated traffic population.
During practical validation, legitimate and controlled test traffic associated with the Apache web server is captured and analyzed. The trained classifier evaluates compatible traffic features, and SHAP explanations are generated to support the investigation of selected predictions. The resulting explanations are reviewed against the known laboratory test scenario to assess their usefulness and limitations.
The primary objective is to build a practical DDoS investigation workflow that combines traffic classification with interpretable model explanations. The project focuses on explaining detection decisions rather than automatically blocking traffic.
Public DDoS Traffic Dataset → Data Preprocessing → Traffic Feature Extraction → HistGradientBoosting Training → Model Evaluation → Apache HTTP Server on Ubuntu → Network Traffic Capture → Feature Extraction → Traffic Classification → SHAP Explanation → Influential Feature Identification → Investigation Report → Controlled Laboratory Validation
Service: Apache HTTP Server on Ubuntu
Apache HTTP Server is the target web-server service in this use case. It receives and processes HTTP requests from clients and provides access to web content or application functionality.
The service is deployed within an isolated laboratory environment to support legitimate traffic generation, controlled security testing, traffic capture, and investigation of model predictions.
A web server can experience availability problems when excessive network traffic consumes its available resources. DDoS-related traffic may exhibit characteristics that differ from legitimate usage, but distinguishing malicious activity from normal traffic can be difficult when traffic patterns overlap.
Machine-learning classifiers can help identify traffic patterns associated with known DDoS activity. However, investigators also need to understand which features influenced a prediction to review suspicious traffic, examine possible false positives, and assess whether the model's decision is supported by the observed traffic characteristics.
Without an explanation mechanism, a classifier's output may provide a label without sufficient information for investigating the reasons behind that prediction.
The proposed solution uses HistGradientBoostingClassifier to classify normal and DDoS-related traffic using features extracted from an existing public traffic dataset. SHAP is then applied to explain the model's predictions and identify the traffic features that influence individual classification results.
Specific Attack: Controlled Distributed Denial-of-Service Traffic Test Against an Apache HTTP Server
The controlled attack scenario evaluates whether the trained HistGradientBoostingClassifier can identify traffic associated with a controlled DDoS test and whether SHAP can provide useful explanations for the resulting predictions.
The test is performed only against the Apache HTTP Server deployed within the isolated laboratory. The objective is to generate controlled traffic for security testing, capture its characteristics, and investigate how the model classifies the resulting observations.
The model is trained using an existing public DDoS traffic dataset. The controlled laboratory traffic is subsequently analyzed using the trained model and a compatible feature-extraction workflow. SHAP explanations are generated for selected predictions to identify features contributing to the model's output.
The experiment does not assume that every controlled test request will be classified correctly. Detection results and explanations must be evaluated against the known testing scenario and the limitations of the available traffic features.
Explainable AI-Based DDoS Traffic Classification and Investigation
The primary security concept is Explainable Artificial Intelligence combined with machine-learning-based network traffic classification.
The first objective is to establish a labelled traffic representation using an existing public DDoS traffic dataset. Normal and DDoS-related records are processed into numerical features suitable for HistGradientBoostingClassifier.
The second objective is to train and evaluate the classification model to determine how effectively it distinguishes normal traffic from DDoS-related traffic. The model is evaluated using held-out records and appropriate classification metrics.
The third objective is to explain the model's predictions using SHAP. Global explanations are used to examine features that influence predictions across a population of traffic records. Local explanations are used to investigate individual observations classified as normal or DdoS-related.
The resulting explanations help an investigator understand the model's reasoning in terms of its input features. They do not independently prove that an attack occurred or establish causation. The explanations must be interpreted alongside traffic evidence, ground-truth labels, and the limitations of the model.
The security concept defines the overall investigation strategy.
The following are the specific defensive mechanisms implemented in this use case.
HistGradientBoostingClassifier learns patterns from labelled traffic features and classifies traffic observations as normal or DdoS-related.
The same feature definitions and compatible preprocessing procedures are applied to training data and traffic evaluated during testing.
Global SHAP analysis summarizes how traffic features influence the model's predictions across the evaluated dataset.
Local SHAP analysis explains how individual input features contribute to a selected traffic observation's prediction.
Model predictions and SHAP explanations are reviewed alongside captured traffic, available labels, and the known controlled testing scenario.
The model is evaluated using appropriate classification metrics, including precision, recall, F1-score, and false-positive rate.
Apache HTTP Server provides the web-server service used to evaluate network traffic classification and explainable investigation.
Ubuntu provides the environment for hosting Apache HTTP Server and executing the traffic-analysis workflow.
Kali Linux provides the controlled testing environment used to generate authorized test traffic and inspect the target service.
VirtualBox provides the isolated virtual environment used to host Ubuntu and Kali Linux.
Python provides the primary environment for data processing, model training, prediction generation, explanation analysis, and evaluation.
Scikit-learn provides the HistGradientBoostingClassifier implementation used for traffic classification.
SHAP provides feature-contribution explanations for the trained classification model.
Pandas is used to load, filter, clean, and organize the existing public traffic records.
NumPy supports numerical transformations required for feature preparation and model processing.
Wireshark is used to inspect captured network communication during controlled laboratory testing.
tcpdump is used to capture packets from the authorized laboratory network interface when packet-level evidence is required.
Nmap is used to identify exposed services on the designated laboratory server.
An existing public DDoS traffic dataset provides labelled network traffic records for model development and evaluation.
Matplotlib is used to visualize classification performance and SHAP analysis results.
An isolated cybersecurity laboratory is established using VirtualBox, Ubuntu Linux, and Kali Linux, providing a controlled environment for investigating DDoS-related traffic against an Apache HTTP Server.
The Apache HTTP Server is deployed and validated within the laboratory, establishing a functioning web service for legitimate HTTP requests and controlled security testing.
An existing public DDoS traffic dataset is processed to select labelled normal and DDoS-related records, providing the data required for supervised classification and explainability research without manually creating the training dataset.
The selected traffic records are preprocessed and converted into numerical features compatible with HistGradientBoostingClassifier, establishing a consistent representation for model training and evaluation.
A HistGradientBoostingClassifier model is trained and evaluated using held-out data. Classification metrics, including precision, recall, F1-score, and false-positive rate, are recorded to assess its ability to distinguish normal and DDoS-related traffic.
SHAP is integrated with the trained classification model using a compatible explanation method, enabling the analysis of feature contributions associated with the model's predictions.
Global SHAP explanations are generated to identify influential traffic features across the evaluated observations, providing an overview of the characteristics that affect the model's classification behavior.
Local SHAP explanations are generated for selected traffic observations, helping investigators examine why individual records received particular predictions and compare the model's reasoning across normal and DDoS-related traffic.
Captured laboratory traffic is processed and evaluated using the established classification and explanation workflow. The results are correlated with the known test conditions and available traffic evidence to examine false positives, missed detections, and explanation limitations.
The complete workflow of public traffic-data processing, HistGradientBoosting model training, classification evaluation, SHAP-based global and local explanations, controlled Apache server testing, traffic evidence correlation, and final security investigation is demonstrated within the authorized laboratory environment.
Research Breakthrough Possible @S-Logix