Content uploaded by Ali Feizollah
Author content
All content in this area was uploaded by Ali Feizollah on Nov 17, 2014
Content may be subject to copyright.
A preview of the PDF is not available
Evaluation of machine learning classifiers for mobile malware detection
Abstract and Figures
Mobile devices have become a significant part of people's lives, leading to an increasing number of users involved with such technology. The rising number of users invites hackers to generate malicious applications. Besides, the security of sensitive data available on mobile devices is taken lightly. Relying on currently developed approaches is not sufficient, given that intelligent malware keeps modifying rapidly and as a result becomes more difficult to detect. In this paper, we propose an alternative solution to evaluating malware detection using the anomaly-based approach with machine learning classifiers. Among the various network traffic features, the four categories selected are basic information, content based, time based and connection based. The evaluation utilizes two datasets: public (i.e. MalGenome) and private (i.e. self-collected). Based on the evaluation results, both the Bayes network and random forest classifiers produced more accurate readings, with a 99.97 % true-positive rate (TPR) as opposed to the multi-layer perceptron with only 93.03 % on the MalGenome dataset. However, this experiment revealed that the k-nearest neighbor classifier efficiently detected the latest Android malware with an 84.57 % true-positive rate higher than other classifiers.
Figures - uploaded by Ali Feizollah
Author content
All figure content in this area was uploaded by Ali Feizollah
Content may be subject to copyright.
... ROC is a graph showing the degree of a true prediction in contrast to a false prediction. The ROC graph describes the relative trade-off between the true positive rate (TPR, plotted in the Y axis) and the false positive rate (FPR, plotted in the X axis) (Narudin et al. 2014). In our study, we selected the reference fields to calculate the TPR and FPR, and changed the threshold value from 0.1 to 1.0 to generate the ROC space. ...
... In our study, we selected the reference fields to calculate the TPR and FPR, and changed the threshold value from 0.1 to 1.0 to generate the ROC space. In the ROC space, each threshold value creates a diverse point, and the optimal cutoff point is where TPR is high and FPR is low (Narudin et al. 2014). In Fig. 5, the optimal cutoff point and the different threshold values for different study areas is illustrated in ROC curve. ...
Delineation of agricultural fields is desirable for operational monitoring of agricultural production and is essential to support food security. Due to large within-class variance of pixel values and small inter-class difference , automated field delineation remains to be a challenging task. In this study, a strategy is proposed to effectively address this issue. Firstly, a framework was developed using the Canny operator connected with the Watershed segmentation algorithm (CW) to quickly label the training dataset, which minimizes the workload of dataset generation in comparison with the commonly used manual vectorization. Secondly, a CW-trained deep semantic segmentation network, recurrent residual U-Net, was selected to mine the low level and deep semantic features. Finally, a boundary connecting method (to integrate fragmented boundaries) was used to generate the agricultural field boundary. The proposed methods are tested over smallholder agricultural landscape in Hei-longjiang province, China, using Sentinel-2 imagery. Compared with the U-Net (overall accuracy (OA) 82.18%), the residual U-Net (ResU-Net, with OA 85.78%), traditional object-based image analysis (OBIA, with OA about 82%), and the existing 10-m resolution global land cover map (FROM-GLC10), the proposed method shows an improved performance (OA 89.28%, and Kappa 0.85). The successful application of the proposed method suggests that the recurrent residual U-Net model has great universality in agricultural field boundary extraction, and the automated technique has the potential of being applied to other regions.
... In the specific development process, how to integrate the corresponding technology and teaching, promote the comprehensive development of students, solve the key and difficult problems in the process of various teaching development, motivate the universalization and fairness of education, and improve the effective and steady improvement of quality of education. By constructing the personalized push of corresponding learning resources, the personalized and self-organized development of online learning resources can be realized, the self-cognition of students can be maximized, and the development trend and future requirements can be kept up [21][22][23]. ...
... Moreover, we conclude that the optimum results in terms of all evaluation metrics used in these experiments were achieved by KNN in Dataset 3IGFS using10-fold crossvalidation. We determined KNN to be the most suitable ML classifier in terms of classifying smartphone applications' network traffic based on different levels of behaviour and interaction.Narudin et al.[150] discussed, the time taken by classifiers to build a model is very crucial and affects the resource consumption of a wireless device. Thus, considering the processing time of classifiers to build a model is very important. ...
Energy is a vital resource in wireless computing systems. Despite the increasing popularity of Wireless Local Area Networks (WLANs), one of the most important outstanding issues remains the power consumption caused by Wireless Network Interface Controller (WNIC). To save this energy and reduce the overall power consumption of wireless devices, a number of power saving approaches have been devised including Static Power Save Mode (SPSM), Adaptive PSM (APSM), and Smart Adaptive PSM (SAPSM). However, the existing literature has highlighted several issues and limitations in regards to their power consumption and performance degradation, warranting the need for further enhancements. This thesis proposes a novel Context-Aware Listen Interval (CALI), in which the wireless network interface, with the aid of a Machine Learning (ML) classification model, sleeps and awakes based on the level of network activity of each application. We focused on the network activity of a single smartphone application while ignoring the network activity of applications running simultaneously. We introduced a context-aware network traffic classification approach based on ML classifiers to classify the network traffic of wireless devices in WLANs. Smartphone applications’ network traffic reflecting a diverse array of network behaviour and interactions were used as contextual inputs for training ML classifiers of output traffic, constructing an ML classification model. A real-world dataset is constructed, based on nine smartphone applications’ network traffic, this is used firstly to evaluate the performance of five ML classifiers using cross-validation, followed by conducting extensive experimentation to assess the generalisation capacity of the selected classifiers on unseen testing data. The experimental results further validated the practical application of the selected ML classifiers and indicated that ML classifiers can be usefully employed for classifying the network traffic of smartphone applications based on different levels of behaviour and interaction. Furthermore, to optimise the sleep and awake cycles of the WNIC in accordance with the smartphone applications’ network activity. Four CALI power saving modes were developed based on the classified output traffic. Hence, the ML classification model classifies the new unseen samples into one of the classes, and the WNIC will be adjusted to operate into one of CALI power saving modes. In addition, the performance of CALI’s power saving modes were evaluated by comparing the levels of energy consumption with existing benchmark power saving approaches using three varied sets of energy parameters. The experimental results show that CALI consumes up to 75% less power when compared to the currently deployed power saving mechanism on the latest generation of smartphones, and up to 14% less energy when compared to SAPSM power saving approach, which also employs an ML classifier.
... With billions of devices constantly communicating over networks, the threat landscape has become increasingly complex and sophisticated. Traditional security methods struggle to keep pace with the dynamic nature of IoT ecosystems, leaving them vulnerable to cyber-attacks, data breaches, and privacy violations [14].This challenge has sparked a paradigm shift in cybersecurity strategies, leading to the integration of Machine Learning (ML) techniques to safeguard IoT environments [15]. Machine Learning, a subset of artificial intelligence, equips systems with the ability to learn from data and make intelligent decisions without explicit programming. ...
The Internet of Things (IoT) has revolutionized the way we interact with the physical world, embedding everyday objects with sensors and connectivity to enhance efficiency and convenience. However, the rapid proliferation of IoT devices has raised significant concerns regarding security and privacy. Traditional security mechanisms often fall short in addressing the dynamic and diverse nature of IoT ecosystems. This paper explores the paradigm shift towards securing IoT through the integration of Machine Learning (ML) techniques. This research delves into the innovative fusion of IoT and ML, presenting a comprehensive analysis of how machine learning algorithms can fortify the security infrastructure of IoT networks. By leveraging ML algorithms, IoT systems can detect and respond to evolving cyber threats in real-time. This proactive approach enhances anomaly detection, intrusion prevention, and incident response capabilities, mitigating potential risks before they escalate. The study discusses various ML models such as deep learning, clustering, and reinforcement learning, elucidating their applications in IoT security. Deep learning algorithms, particularly Convolutional Neural Networks (CNNs) and Recurrent Neural Networks (RNNs), are examined for their prowess in analyzing intricate patterns within large datasets, ensuring the integrity of IoT data transmission. Additionally, clustering algorithms are explored for their efficiency in grouping similar IoT devices, enabling the implementation of tailored security protocols for specific device clusters. Reinforcement learning techniques are also investigated to optimize security strategies dynamically, adapting to evolving threats in real-time. Furthermore, the paper sheds light on the challenges and opportunities in the integration of ML with IoT security. Ethical considerations, data privacy, and the energy efficiency of ML algorithms are discussed in the context of resource-constrained IoT devices. The research also explores the potential of federated learning, enabling collaborative ML model training across distributed IoT networks without compromising data privacy.
Більшість пристроїв-датчиків в системах інтернету речей базуються на енергоефективних мікроконтролерах, обчислювальні ресурси яких обмежені, як і обсяги наявної пам’яті. Підвищення захищеності використання таких пристроїв за допомогою нейромереж є важливою та актуальною проблемою. В статті описана можливість використання штучних нейронних мереж у малих мікроконтролерах з обмеженими ресурсами. Мета даної роботи полягає у перевірці можливості обчислення нейромереж на базі цілочисельної арифметики задля зменшення часу обчислення нейромережі та виключення операцій по нормалізації даних, а також оцінка доцільності використання таких нейромереж у сфері безпеки інтернету речей у порівнянні з традиційними методами, такими як чорні та білі списки. Отримані наступні результати: при переході на цілочисельну арифметику, у порівнянні з плаваючою точкою, точність обчислень результату знаходиться у межах допустимої похибки навчання нейромережі, тобто не змінилася. Час виконання зменшився на 30-96%, в залежності від архітектури мікроконтролеру. Розмір програми знизився на 22-48% також в залежності від архітектури мікроконтролеру. Висновки. Було доказано можливість та доцільність використання нейромереж, оптимізованих для мікроконтролерів з обмеженими ресурсами. Це підвищить захищеність систем інтернету речей особливо перед загрозами автентифікації пристроїв та виявленню вторгнень. Визначено перспективи подальших досліджень.
The challenge of developing an Android malware detection framework that can identify malware in real-world apps is difficult for academicians and researchers. The vulnerability lies in the permission model of Android. Therefore, it has attracted the attention of various researchers to develop an Android malware detection model using permission or a set of permissions. Academicians and researchers have used all extracted features in previous studies, resulting in overburdening while creating malware detection models. But, the effectiveness of the machine learning model depends on the relevant features, which help in reducing the value of misclassification errors and have excellent discriminative power. A feature selection framework is proposed in this research paper that helps in selecting the relevant features. In the first stage of the proposed framework, t-test, and univariate logistic regression are implemented on our collected feature data set to classify their capacity for detecting malware. Multivariate linear regression stepwise forward selection and correlation analysis are implemented in the second stage to evaluate the correctness of the features selected in the first stage. Furthermore, the resulting features are used as input in the development of malware detection models using three ensemble methods and a neural network with six different machine-learning algorithms. The developed models’ performance is compared using two performance parameters: F-measure and Accuracy. The experiment is performed by using half a million different Android apps. The empirical findings reveal that malware detection model developed using features selected by implementing proposed feature selection framework achieved higher detection rate as compared to the model developed using all extracted features data set. Further, when compared to previously developed frameworks or methodologies, the experimental results indicates that model developed in this study achieved an accuracy of 98.8%.
In today world the mobile malware shows the significant threat to the security and privacy of the society using smartphones. These malware aims to access the sensitive data and harm the devices of users. This paper conducts a comprehensive comparison between the various machine learning and traditional methods for mobile malware detection based on the research papers published by the authors. Signature-based detection depends upon the predefined and common patterns, while the anomaly based techniques analyse the deviation from the regular normal behaviour. This study discusses the strengths and limitations of different approaches and highlights the need for adopting the malware detection methods to fight the growing threats. It also examines the role of machine learning algorithms, like Decision Trees, Random Forests, Convolutional Neural Networks, Support Vector Machines, and Naïve Bayes, for better malware detection. Latest findings and research highlights the importance of the continuing innovation to fight the emerging threat to the user privacy, data and security due to malwares. Keywords: Mobile Malware, Artificial Intelligence, Virus, Signature-based Detection, Machine Learning
Identifying malware is a critical task to ensure computer system’s security, and machine learning algorithms have shown magnificent performance in this area due to their ability of learning patterns from large datasets. However, the quality and relevance of features used in the model can negatively affect the performance of machine learning algorithms. In this paper, we traverse the effectiveness of six different machine learning algorithms, namely, logistic regression, decision tree, Naïve Bayes, SVM, random forest, and k-nearest neighbor, for identifying malware along with feature selection including dragonfly optimization (DFO) and particle swarm optimization (PSO). We evaluate these algorithms on a memory-based balanced dataset containing a mix of benign and malicious files and then examine their performance in terms of accuracy, precision, recall, and F1-score. Our experimental results show that the dragonfly feature selection technique achieves the highest overall performance. The K-nearest neighbor algorithm outperforms the other algorithms by achieving accuracy of 99.97%. Our findings suggest that incorporating feature selection techniques can improve the performance of machine learning algorithms for malware detection, and the dragonfly technique provides the best results.
With the growth of networked computers and associated applications, intrusion detection has become essential to keeping networks secure. A number of intrusion detection methods have been developed for protecting computers and networks using conventional statistical methods as well as data mining methods. Data mining methods for misuse and anomaly-based intrusion detection, usually encompass supervised, unsupervised and outlier methods. It is necessary that the capabilities of intrusion detection methods be updated with the creation of new attacks. This paper proposes a multi-level hybrid intrusion detection method that uses a combination of supervised, unsupervised and outlier-based methods for improving the efficiency of detection of new and old attacks. The method is evaluated with a captured real-time flow and packet dataset called the Tezpur University intrusion detection system (TUIDS) dataset, a distributed denial of service dataset, and the benchmark intrusion dataset called the knowledge discovery and data mining Cup 1999 dataset and the new version of KDD (NSL-KDD) dataset. Experimental results are compared with existing multi-level intrusion detection methods and other classifiers. The performance of our method is very good.
Analyzed Bayesian classifier with string, n-gram and API as features, we found that it is very difficult to improve Bayesian classifier detection accuracy because selected features are not completely independent. In order to solve this problem, we propose a new improved choose features method which are most representative properties, and show that our method achieve high detection rates, even on completely new, previously unseen malicious executables. (C) 2011 Published by Elsevier Ltd. Selection and/or peer-review under responsibility of [CEIS 2011]
Malicious applications pose a threat to the security of the Android platform. The growing amount and diversity of these applications render conventional defenses largely ineffective and thus Android smartphones often remain un-protected from novel malware. In this paper, we propose DREBIN, a lightweight method for detection of Android malware that enables identifying malicious applications di-rectly on the smartphone. As the limited resources impede monitoring applications at run-time, DREBIN performs a broad static analysis, gathering as many features of an ap-plication as possible. These features are embedded in a joint vector space, such that typical patterns indicative for malware can be automatically identified and used for ex-plaining the decisions of our method. In an evaluation with 123,453 applications and 5,560 malware samples DREBIN outperforms several related approaches and detects 94% of the malware with few false alarms, where the explana-tions provided for each detection reveal relevant properties of the detected malware. On five popular smartphones, the method requires 10 seconds for an analysis on average, ren-dering it suitable for checking downloaded applications di-rectly on the device.
Mobile malware has been growing in scale and complexity as smartphone usage continues to rise. Android has surpassed other mobile platforms as the most popular whilst also witnessing a dramatic increase in malware targeting the platform. A worrying trend that is emerging is the increasing sophistication of Android malware to evade detection by traditional signature-based scanners. As such, Android app marketplaces remain at risk of hosting malicious apps that could evade detection before being downloaded by unsuspecting users. Hence, in this paper we present an effective approach to alleviate this problem based on Bayesian classification models obtained from static code analysis. The models are built from a collection of code and app characteristics that provide indicators of potential malicious activities. The models are evaluated with real malware samples in the wild and results of experiments are presented to demonstrate the effectiveness of the proposed approach.
The widespread adoption and contextually sensitive nature of smartphone devices has increased concerns over smartphone malware. Machine learning classifiers are a current method for detecting malicious applications on smartphone systems. This paper presents the evaluation of a number of existing classifiers, using a dataset containing thousands of real (i.e. not synthetic) applications. We also present our STREAM framework, which was developed to enable rapid large-scale validation of mobile malware machine learning classifiers.
In this paper, we present a smart phone dual defense protection framework that allows Official and Alternative Android Markets to detect malicious applications among those new applications that are submitted for public release. Our framework consists of servers running on clouds where developers who wish to release their new applications can upload their software for verification purpose. The verification server first uses system call statistics to identify potential malicious applications. After verification, if the software is clean, the application will then be released to the relevant markets. To mitigate against false negative cases, users who run new applications can invoke our network traffic monitoring (NTM)tool which triggers network traffic capture upon detecting some suspicious behaviors e.g. detecting sensitive data being sent to output stream of an open socket. The network traffic will be analyzed to see if it matches network characteristics observed from malware applications. If suspicious network traffic is observed, the relevant Android markets will be notified tore move the application from the repository. We trained our system call and network traffic classifiers using 32 families of known Android malware families and some typical normal applications. Later, we evaluated our framework using other malware and normal applications that used in the training set. Our experimental results using 120 test applications (which consist of 50 malware and 70 normal applications) indicate that we can achieve a 94.2% and 99.2% accuracy with J.48 and Random forest classifier respectively using our framework.
It is a straightforward idea to detect a harmful mobile application based on the permissions it requests. This study attempts to explore the possibility of detecting malicious applications in Android operating system based on permissions. Compare against previous researches, we collect a relative large number of benign and malicious applications (124,769 and 480, respectively) and conduct experiments based on the collected samples. In addition to the requested and the required permissions, we also extract several easy-to-retrieve features from application packages to help the detection of malicious applications. Four commonly used machine learning algorithms including AdaBoost, Naïve Bayes, Decision Tree (C4.5), and Support Vector Machine are used to evaluate the performance. Experimental results show that a permission-based detector can detect more than 81% of malicious samples. However, due to its precision, we conclude that a permission-based mechanism can be used as a quick filter to identify malicious applications. It still requires a second pass to make complete analysis to a reported malicious application.
Recent smartphone platforms based on new operating systems, such as iOS, Android, or Windows Phone, have been a huge success in recent years and open up many new opportunities. Unfortunately, 2011 also showed us that the new technologies and the privacy-related data on smartphones are also increasingly interesting for attackers. Especially, the Android platform has been the favorite target for malware, mainly because of the openness of the platform, the ability to install applications from other sources than the Android Market, and the significant gains in market share. Although the processes of detecting and analyzing malware are well known from the PC world, where the arms race between attackers and defenders has continued for the past 15 years, they cannot be directly applied to smartphone platforms because of differences in the hardware and software architectures. In this paper, we first give an overview of the current malware situation on smartphone platforms with a special focus on Android and explain relevant malware detection and analysis methods. It turns out that most of the current malware relies on the installation by the user, who represents the last line of defense in malware detection. With these conclusions, we then present a new malware detection method that focuses on the information that the user is able to see prior to the installation of an application—the metadata within the platform's software market. Depending on the platform, this includes the application's description, its permissions, the ratings, or information about the developer. To analyze these data, we use sophisticated knowledge discovery processes and lean statistical methods. By presenting a wide range of examples based on real application metadata extracted from the Android Market, we show the possibilities of the new method. With the possibilities, we argue that it should be an essential part of a complete malware analysis/detection chain that includes other well-known methods such as network traffic analysis, or static, or dynamic code inspection. Copyright © 2013 John Wiley & Sons, Ltd.
As mobile devices have supported various services and contents, much personal information such as private SMS messages, bank account information, etc. is scattered in mobile devices. Thus, attackers extend the attack range not only to the existing environment of PC and Internet, but also to the mobile device. Previous studies evaluated the malware detection performance of machine learning classifiers through collecting and analyzing event, system call, and log information generated in Android mobile devices. However, monitoring of unnecessary features without understanding Android architecture and malware characteristics generates resource consumption overhead of Android devices and low ratio of malware detection. In this paper, we propose new feature sets which solve the problem of previous studies in mobile malware detection and analyze the malware detection performance of machine learning classifiers.