Implementasi Sistem Deteksi Malware Berbasis Random Forest pada File Portable Executable Windows

Abstract

This research addresses the limitations of signature based malware detection against new variants in Windows Portable Executable files, along with the absence of an automated system with real time monitoring that requires no user intervention. Limited and homogeneous benign samples also introduce bias and a high false positive rate. To address this, the research develops a Random Forest based malware detection system integrated with a web dashboard and event driven file monitoring. The contribution includes designing an automated system, extracting static features from PE headers without executing files, and presenting confidence scores and feature importance on a web interface. The method involves feature extraction with the pefile library, directory monitoring with the Watchdog library, and training a Random Forest model on the PE-Malware-Dataset containing malware samples from five families along with benign samples from the Windows system. The research began modeling with an exploration of multi class classification, but accuracy remained low because header features did not sufficiently distinguish between families, so the final approach was set as binary classification, malware versus benign. Real time testing results confirmed that the end to end pipeline runs automatically. Initial evaluation on a small test set showed the model captured all malware samples but produced false positives on benign files, resulting in high recall paired with low precision and a moderate F score. Most false positives came from third party administrative utilities with a Portable Executable structure similar to malware. In conclusion, the system functions as a proof of concept for real time detection based on Random Forest and a web dashboard, but real world performance remains limited by benign data bias, so diversifying benign samples is needed to reduce false positives.