Journal of Engineering Research and Reports

In the evolving landscape of online transactions, fraud is increasingly becoming sophisticated; fraud detection is a significant problem. This study investigates the use of machine learning methods to improve online transaction fraud detection. In this research, a comprehensive machine learning-based fraud detection system using the PaySim dataset is presented, which is a simulated dataset replicating real-world mobile money transfers. It is methodologically based on thorough exploratory data analysis, MinMax normalization, and class balancing using SMOTE to deal with severe data imbalance and splitting into training (80) and testing (20) sets. XGBoost and LightGBM are two advanced gradient boosting models that are assessed using standard measures such as ROC-AUC, accuracy, precision, recall, F1-score, and confusion matrices. The results of the experiment indicate that XGBoost performs better than all the models with 99.91% and 99.98% accuracy, and LightGBM is 96.72% with good overall performance. The proposed framework is also superior to the existing methods, such as FinSafeNet, Logistic Regression, and GAN-VAE, as it can be further verified through the comparative analysis. The results indicate that gradient-boosting models are suitable for high-volume financial transaction settings and provide a reliable baseline for applied fraud detection research.