Location Research Breakthrough Possible @S-Logix pro@slogix.in

XGBoost-Driven Detection and Prevention of Malicious PDF Upload Attacks Against Nextcloud Files Through Static Document Feature Analysis

Description

File-sharing applications allow users to upload, store, access, and share documents through web-based interfaces. Nextcloud Files provides file-management and sharing functionality that can be deployed within a controlled environment for security testing.

PDF documents are widely used for sharing reports, forms, invoices, and other information. However, a PDF file can contain potentially dangerous structures or embedded content that may be associated with malicious activity. A document's PDF extension or its ability to open successfully does not establish that the document is safe.

In a file-sharing environment, a malicious document may be uploaded and stored before its risk is identified. If the application accepts the file without an additional security-inspection layer, users may subsequently download or distribute the document.

In this use case, Nextcloud Files is deployed in an isolated Ubuntu and Kali Linux laboratory environment. A public PDF-security dataset is used as the existing data source for machine-learning training and evaluation. The dataset is inspected to identify its available benign and malicious document labels and static document features.

XGBoost is used to train a classification model using the selected labelled PDF features. The model learns patterns associated with the classes represented in the training data and predicts whether a submitted document resembles the benign or malicious samples.

A controlled malicious-PDF upload scenario is then performed against the local Nextcloud instance. The uploaded PDF is inspected using the selected static-feature extraction process. The resulting feature vector is provided to the trained XGBoost model, and its prediction is used by a separate validation component to determine whether the upload should be flagged or blocked.

The detection component is implemented separately from Nextcloud's standard file-upload functionality. The project must therefore include an integration mechanism that checks the classification result before the document is accepted for ordinary use. A model prediction alone does not automatically block a file unless this enforcement mechanism is implemented and tested.

The complete workflow is:
Public PDF-Security Dataset
→
Dataset Inspection
→
Label Identification
→
Static Feature Preparation
→
XGBoost Model Training
→
Model Evaluation
→
Nextcloud Files Deployment
→
Controlled Malicious PDF Preparation
→
PDF Upload Request
→
Static Feature Extraction
→
XGBoost Classification
→
Upload Security Decision
→
Suspicious PDF Flagging or Blocking
→
Detection Validation

Existing Security Problem

Application : Nextcloud Files

Nextcloud Files is the target file-sharing application in this use case. It provides file-upload, storage, retrieval, and sharing functionality for users within the laboratory environment.

The application is deployed within an isolated laboratory so that legitimate PDF uploads and controlled malicious-document testing can be performed without exposing real users or production systems.

Existing Problem :

File-sharing applications must accept documents in supported formats while preventing potentially harmful files from being distributed through the application.

A PDF file may appear to be an ordinary document based on its filename and extension while containing structures that require additional security inspection. Extension-based checks alone are therefore insufficient to establish that a document is safe.

If uploaded documents are not examined by an effective security-validation mechanism, potentially malicious files may be stored and later accessed by other users.

The existing problem is:

Nextcloud Files

↓

User Selects a PDF Document

↓

PDF Upload Request

↓

File-Type and Upload Validation

↓

Document Stored in the Application

↓

Potentially Malicious PDF Remains Undetected

↓

Document May Be Downloaded or Shared

↓

Potential Exposure to Malicious Document Content

↓

Requirement for Additional Document Security Analysis

The proposed solution adds a machine-learning-based document-classification component to the upload workflow. Static features extracted from a PDF are analyzed by the trained XGBoost model. The resulting classification is passed to an upload-validation component, which applies the configured security policy.

Attack

Specific Attack : Controlled Malicious PDF Upload Attack Against Nextcloud Files

The controlled attack scenario evaluates whether the proposed detection mechanism can identify a PDF document associated with the malicious class when it is submitted to the local Nextcloud Files application.

A public PDF-security dataset provides the labelled records used for model development. A suitable PDF sample is selected from the available dataset or its associated downloadable files for controlled laboratory validation.

The selected sample is uploaded only to the isolated Nextcloud instance. Its features are extracted using the same compatible feature-processing procedure used by the trained model.

The XGBoost model classifies the extracted features. The classification result is then passed to the upload-validation component, which applies the configured action, such as flagging or blocking the document.

Attack Behavior :
Controlled Malicious PDF Sample Selected
→
PDF Sample Verified for Laboratory Use
→
PDF Submitted to Nextcloud Files
→
Upload Request Received
→
Upload-Validation Component Invoked
→
PDF Static Features Extracted
→
Feature Vector Prepared
→
XGBoost Model Performs Classification
→
Classification Result Evaluated
→
Upload Policy Applied
→
Suspicious PDF Flagged or Blocked
→
Detection Result Compared with Known Label

Security Concept

XGBoost-Based Static PDF Threat Classification and Upload-Time Security Validation

The primary security concept combines static document-feature analysis with machine-learning classification to identify PDF files that resemble known malicious samples before they become available through Nextcloud Files.

Static PDF Feature Analysis : The selected PDF-security dataset is inspected to identify the available document features and security labels. The feature set may include structural properties, metadata, object counts, embedded-content indicators, or other attributes provided by the dataset.

XGBoost-Based Classification : XGBoost learns patterns from the selected labelled document features. The trained model predicts a class for a new PDF based on its extracted features.

The model is evaluated using data that was not used during training. Precision, recall, F1-score, and the confusion matrix are used to examine the classification results, particularly the model's ability to identify malicious samples without incorrectly flagging too many benign documents.

Upload-Time Security Enforcement : A separate integration component connects Nextcloud's upload workflow with the PDF inspection and classification process.When a PDF is uploaded, the component extracts the required features and submits them to the trained model. The classification result is then evaluated against the configured security policy.A suspicious result can trigger a block or quarantine action if that enforcement behavior has been implemented and verified. The system must not assume that XGBoost itself can block an upload; the enforcement component is responsible for applying the decision.

The assessment follows:
PDF Upload Request
→
PDF Identification and Validation
→
Static Feature Extraction
→
Feature Preprocessing
→
XGBoost Classification
→
Benign or Malicious Prediction
→
Upload Security Policy Evaluation
→
Allow, Flag, or Block Decision
→
Security Result Recording
→
Controlled Detection Validation

Defensive Mechanism

The security concept defines the overall security strategy.

The following are the specific defensive mechanisms implemented.

Static PDF Feature Inspection :

The uploaded PDF is inspected to extract the document features required by the trained model.

Purpose: Identify document characteristics that can be evaluated for potential malicious behavior

XGBoost-Based Threat Classification

The extracted document features are provided to the trained XGBoost model for classification.

Purpose
  • The extracted document features are provided to the trained XGBoost model for classification.

Consistent Feature Extraction

The same compatible feature representation is used for the training dataset and newly uploaded PDF documents.

Purpose
  • Ensure that the model receives the expected feature types and ordering during inference.

Upload-Time Security Validation

The classification result is evaluated before the uploaded PDF is accepted for ordinary use.

Purpose
  • The classification result is evaluated before the uploaded PDF is accepted for ordinary use.

Suspicious Document Flagging

PDF documents classified as suspicious are recorded for security review.

Purpose
  • Make potentially malicious documents visible for further investigation

Upload Blocking or Quarantine

The upload integration component blocks or quarantines a document when the configured security policy requires that action.

Purpose
  • The upload integration component blocks or quarantines a document when the configured security policy requires that action.

Security Event Recording

The upload result, model prediction, and relevant document information are recorded.

Purpose
  • The upload result, model prediction, and relevant document information are recorded.

Detection Result Validation

Predictions and upload decisions are compared with the known labels of the controlled test documents.

Purpose
  • Measure detection performance and verify whether the configured upload policy is applied correctly.

Security Tools

Primary File-Sharing Application: Nextcloud Files

Nextcloud Files provides the target application for PDF uploads, storage, and controlled file-sharing security testing.

Purpose
  • Receive legitimate PDF uploads.
  • Receive controlled PDF test samples.
  • Provide the upload workflow for security inspection.
  • Validate the behavior of the upload-enforcement integration.

Operating System: Ubuntu Linux

Ubuntu hosts the Nextcloud deployment and the document-analysis components.

Purpose
  • Run the local file-sharing environment.
  • Execute Python-based feature extraction and classification.
  • Host the upload-validation integration.
  • Support controlled security testing.

Security Testing Platform: Kali Linux

Kali Linux is used as the controlled client environment for submitting PDF files to the laboratory application.

Purpose
  • Generate legitimate upload requests.
  • Submit controlled malicious PDF samples.
  • Observe upload responses.
  • Preserve evidence from the laboratory tests.

Virtualization Platform: VirtualBox

VirtualBox provides the isolated virtual-machine environment for the application and testing systems.

Purpose
  • Host Ubuntu and Kali Linux virtual machines.
  • Separate the application and testing environments.
  • Support repeatable experiments.
  • Reduce unintended interaction with external systems.

Programming Environment: Python

Python implements dataset processing, feature extraction, model training, classification, and upload-policy integration.

Purpose
  • Process PDF-security data.
  • Implement the static-feature extraction workflow.
  • Connect XGBoost inference with the upload-validation component.
  • Record classification and enforcement results.

Machine-Learning Library: XGBoost

XGBoost provides the classification algorithm used to analyze the prepared PDF features.

Purpose
  • Train the PDF classification model.
  • Generate predictions for previously unseen feature vectors.
  • Support evaluation of benign and malicious document classification.

Data Processing Tool: Pandas

Pandas organizes the dataset records, document metadata, and classification results.

Purpose
  • Load and inspect the public PDF-security dataset
  • Prepare labelled feature records.
  • Organize training and evaluation data.
  • Analyze classification results.

Numerical Processing Tool: NumPy

NumPy supports numerical operations required for feature preparation.

Purpose
  • Prepare numerical feature arrays.
  • Support feature transformations.
  • Maintain compatible model-input representations.

PDF Inspection and Feature Extraction: PDF Parsing Library

A suitable PDF parsing library, selected according to the actual dataset and required features, is used to inspect document structure and extract compatible static attributes.

Purpose
  • Read supported PDF structures.
  • Extract the features required by the model.
  • Process controlled PDF samples consistently.
  • Provide feature data to the XGBoost classifier.

Machine-Learning Evaluation Tool: Scikit-learn

Scikit-learn provides dataset-splitting and classification-evaluation utilities.

Purpose
  • Prepare suitable training and evaluation partitions.
  • Calculate precision, recall, F1-score, and confusion matrix.
  • Evaluate model predictions against known labels.
  • Support analysis of false-positive and false-negative results.

HTTP Testing Tool: cURL

cURL is used to generate controlled HTTP requests against the authorized Nextcloud environment where appropriate.

Purpose
  • Submit repeatable upload requests.
  • Observe HTTP response codes.
  • Support controlled upload testing.
  • Provide request-level evidence.

Web Security Testing Tool: OWASP ZAP

OWASP ZAP can be used to inspect HTTP traffic exchanged with the laboratory Nextcloud application.

Purpose
  • Inspect upload requests and responses.
  • Preserve relevant HTTP traffic evidence.
  • Support controlled web-application testing.
  • Assist in verifying upload-enforcement behavior.

Event Storage: SQLite

SQLite stores the relevant upload records, model predictions, and security decisions.

Purpose
  • Store document-analysis results.
  • Preserve upload-validation outcomes.
  • Maintain controlled test records.
  • Support later review and evaluation.

Dashboard Framework: Streamlit

Streamlit provides an interface for reviewing PDF classification and upload-validation results.

Purpose
  • Display document-analysis results.
  • Present model predictions and security decisions.
  • Support review of flagged documents.
  • Provide a clear view of detection evidence.

Process

STEP 01

Prepare the Isolated Laboratory Environment

  • Create the laboratory environment using VirtualBox.
  • Configure Ubuntu Linux as the application and machine-learning environment
  • Configure Kali Linux as the controlled security-testing environment.
  • Establish the intended network connectivity between the virtual machines.
  • Verify that Kali Linux can reach the local Nextcloud instance.
  • Confirm that testing is restricted to the authorized laboratory environment.
Tools: VirtualBox + Ubuntu Linux + Kali Linux
STEP 02

Deploy Nextcloud Files

  • Deploy Nextcloud within the Ubuntu laboratory environment.
  • Configure the required application services and storage
  • Start the Nextcloud instance and verify that its interface is accessible.
  • Create a dedicated laboratory user account.
  • Configure a test directory for PDF uploads.
  • Verify that a legitimate PDF can be uploaded and retrieved successfully.
  • Record the initial application and upload behavior.
Tools: Nextcloud Files + Ubuntu Linux
STEP 03

Establish the Normal Upload Baseline

  • Use the designated laboratory account to upload legitimate PDF documents.
  • Observe the upload response and resulting file availability.
  • Verify that the documents appear in the intended Nextcloud directory.
  • Download a test document to confirm normal retrieval behavior.
  • Record the results of the legitimate upload tests.
  • Preserve the benign samples for later comparison.
Tools: Nextcloud Files + cURL
STEP 04

Identify the Laboratory Service Exposure

  • Identify the authorized Ubuntu laboratory target from Kali Linux.
  • Perform controlled service discovery against the laboratory environment.
  • Identify the Nextcloud web service and its listening port
  • Verify that the application is reachable from the intended testing network.
  • Record the observed service exposure.
  • Confirm that no unrelated systems are included in the testing scope.
Tools: Kali Linux + Nmap
STEP 05

Prepare the Existing Public PDF-Security Dataset

  • Obtain the public PDF-security dataset selected for the experiment.
  • Inspect the dataset documentation and available files.
  • Determine whether the dataset provides original PDF documents, extracted feature vectors, or both.
  • Identify the available benign and malicious labels.
  • Inspect the feature definitions and determine how the labels correspond to the samples.
  • Verify that the available data supports the intended classification task.
  • Record any limitations affecting the extraction of compatible features from uploaded PDFs.
Tools: Python + Pandas + NumPy
STEP 06

Select Suitable PDF Training Records

  • Review the available dataset records.
  • Identify samples with usable security labels.
  • Select the benign and malicious records required for model development.
  • Exclude corrupted, incomplete, or unsuitable records where necessary.
  • Inspect the distribution of the available classes.
  • Preserve the original labels and the associated feature definitions.
  • Document the selection criteria used to prepare the training data.
Tools: Python + Pandas
STEP 07

Preprocess the PDF Feature Data

  • Load the selected dataset records into Python.
  • Inspect missing, inconsistent, or invalid feature values.
  • Apply appropriate preprocessing to the selected features.
  • Convert compatible categorical values into numerical representations where required.
  • Ensure that the target security label is not included as an input feature.
  • Preserve the feature order and preprocessing configuration.
  • Record the transformations required for later model inference.
Tools: Python + Pandas + NumPy
STEP 08

Extract or Prepare Static PDF Features

  • Identify the static features used by the selected public dataset.
  • If original PDF files are available, inspect them using a suitable PDF parsing library and extract the required attributes.
  • If the dataset contains only pre-extracted features, prepare the feature matrix directly and verify whether the original PDF files are available for upload validation.
  • Extract compatible attributes from the controlled upload samples.
  • Convert the extracted values into the same representation used during training.
  • Verify that the feature names, ordering, and data types match the model requirements.
  • Do not claim end-to-end upload detection if the dataset features cannot be reproduced from the actual uploaded PDF files.
Tools: Python + PDF Parsing Library + Pandas + NumPy
STEP 09

Prepare Training and Evaluation Data

  • Separate the prepared PDF feature records into training, validation, and test partitions as appropriate.
  • Preserve the associated security labels.
  • Prevent duplicate documents or equivalent feature records from appearing across partitions where this would cause data leakage.
  • Review the distribution of benign and malicious samples.
  • Choose a suitable evaluation strategy for the available class distribution.
  • Record the number of records in each partition and the method used to create the split.
  • Prepare the resulting feature matrices for XGBoost training.
Tools: Python + Pandas + Scikit-learn + NumPy
STEP 10

Train the XGBoost Classification Model

  • Configure the XGBoost classifier for the selected PDF security labels
  • Provide the prepared training feature matrix and corresponding labels
  • Train the model using the selected records.
  • Monitor the training process and review the model configuration.
  • Evaluate validation performance where applicable.
  • Preserve the trained model and its associated feature-processing configuration.
  • Record the model settings so that the experiment can be reproduced.
Tools: Python + XGBoost + Scikit-learn
STEP 11

Evaluate the Trained Classification Model

  • Load the trained XGBoost model.
  • Provide the test feature records that were not used during training.
  • Generate predictions for the test records.
  • Compare the predictions with their known security labels.
  • Calculate precision, recall, F1-score, and the confusion matrix.
  • Review the model's ability to identify malicious samples.
  • Examine false-positive and false-negative cases.
  • Record the actual evaluation results without assuming a particular detection rate.
Tools: Python + XGBoost + Scikit-learn + Pandas
STEP 12

Prepare the Controlled Malicious PDF Test Sample

  • Select a suitable malicious PDF sample from the available dataset or its associated downloadable files.
  • Verify that the sample can be used safely in the isolated laboratory.
  • Do not execute embedded content or open the sample in an unrestricted environment.
  • Confirm that its known label and source are recorded.
  • Prepare a benign PDF sample for comparison.
  • Ensure that the selected samples can be processed by the available static-feature extraction workflow.
Tools: Ubuntu Linux + Python + PDF Parsing Library
STEP 13

Submit the Controlled PDF Upload

  • Use the designated laboratory account to submit the selected PDF sample to the local Nextcloud instance.
  • Capture the upload request and the application response.
  • Record whether the upload is accepted by the unmodified application workflow.
  • Preserve the original test file and its known label.
  • Repeat the upload process with a benign sample for comparison.
  • Ensure that all testing remains within the isolated laboratory.
Tools: Kali Linux + Nextcloud Files + cURL + OWASP ZAP
STEP 14

Extract Features from the Uploaded PDF

  • Pass the uploaded PDF to the configured document-inspection component.
  • Verify that the file can be processed without executing its contents.
  • Extract the static features required by the trained XGBoost model.
  • Apply the same preprocessing transformations used during model training.
  • Verify the feature names, ordering, and data types.
  • Confirm that the resulting feature vector is compatible with the trained model.
  • Preserve the extracted features for the classification stage.
Tools: Python + PDF Parsing Library + Pandas + NumPy
STEP 15

Perform XGBoost-Based PDF Classification

  • Load the trained XGBoost model.
  • Provide the extracted and preprocessed PDF feature vector.
  • Generate the classification prediction for the uploaded document.
  • Record the predicted class and model probability or confidence information where supported.
  • Associate the prediction with the original PDF sample and its known label.
  • Preserve the classification result as security evidence.
Tools: Python + XGBoost + NumPy
STEP 16

Apply the Upload Security Decision

  • Pass the model prediction to the upload-validation component.
  • Evaluate the prediction against the configured security policy.
  • Allow documents that meet the defined acceptance conditions.
  • Flag or block documents classified as suspicious according to the implemented policy.
  • Where quarantine is implemented, store the document in a restricted location rather than making it normally accessible.
  • Record the action applied to the upload.
  • Verify that the decision is enforced by the integration component rather than merely displayed as a model prediction.
Tools: Python + Nextcloud Files + SQLite
STEP 17

Store and Display the Detection Evidence

  • Organize the original upload record, extracted PDF features, model prediction, and resulting security decision.
  • Store the relevant records in SQLite.
  • Associate each result with its controlled test scenario and known label.
  • Create Streamlit views for displaying PDF classification and upload-validation results.
  • Present the predicted class and the action applied to the document.
  • Verify that the displayed results match the original test evidence.
Tools: Python + SQLite + Streamlit + Pandas
STEP 18

Perform Final Detection Validation and Security Review

  • Repeat the workflow using benign and controlled malicious PDF samples.
  • Review the model's evaluation results on the held-out test data.
  • Verify that uploaded PDFs are processed using the expected static-feature pipeline.
  • Compare the model predictions with the known document labels.
  • Confirm that the configured upload policy is correctly enforced.
  • Review false-positive and false-negative results.
  • Document the limitations of the dataset, feature extractor, classification model, and upload integration.
  • Finalize the assessment of XGBoost-based malicious PDF detection for Nextcloud Files.
Tools: Nextcloud Files + Python + XGBoost + Scikit-learn + SQLite + Streamlit + Kali Linux

Outcome

Nextcloud Files is deployed within an isolated Ubuntu and Kali Linux laboratory environment to provide the target application for controlled PDF-upload security testing.

An existing public PDF-security dataset is inspected to identify its available security labels and static document features suitable for machine-learning development.

The selected PDF feature records are preprocessed and prepared for XGBoost training, with the original labels preserved for evaluation.

The XGBoost model is trained using the selected labelled PDF features and evaluated using records that were not used during training.

Classification performance is measured using precision, recall, F1-score, and confusion matrix, providing evidence of the model's ability to distinguish the available benign and malicious document classes.

Controlled PDF uploads are performed against the laboratory Nextcloud instance, generating upload requests and application responses for security analysis.

Static features are extracted from compatible uploaded PDF samples and analyzed by the trained XGBoost model to generate document-classification predictions.

The classification predictions are compared with the known test labels, allowing the experiment to document correct classifications, false positives, and false negatives.

A separate upload-validation component applies the configured security policy by allowing, flagging, or blocking uploaded documents according to the implemented and verified enforcement logic.

The complete workflow demonstrates the integration of public PDF-security data, static document-feature analysis, XGBoost classification, Nextcloud upload validation, controlled malicious-PDF testing, and evidence-based detection evaluation within an isolated laboratory environment.