Hybrid Botnet Detection Based on Host and Network Analysis

Botnet is one of the most dangerous cyber-security issues. The botnet infects unprotected machines and keeps track of the communication with the command and control server to send and receive malicious commands. The attacker uses botnet to initiate dangerous attacks such as DDoS, ﬁshing, data stealing, and spamming. The size of the botnet is usually very large, and millions of infected hosts may belong to it. In this paper, we addressed the problem of botnet detection based on network’s ﬂows records and activities in the host. Thus, we propose a general technique capable of detecting new botnets in early phase. Our technique is implemented in both sides: host side and network side. The botnet communication traﬃc we are interested in includes HTTP, P2P, IRC, and DNS using IP ﬂuxing. HANABot algorithm is proposed to preprocess and extract features to distinguish the botnet behavior from the legitimate behavior. We evaluate our solution using a collection of real datasets (malicious and legitimate). Our experiment shows a high level of accuracy and a low false positive rate. Furthermore, a comparison between some existing approaches was given, focusing on speciﬁc features and performance. The proposed technique outperforms some of the presented approaches in terms of accurately detecting botnet ﬂow records within Netﬂow traces.


Introduction
Most of the people depend on the Internet for their day-today tasks such as business, education, and entertainment [1]. Botnet is one of the most popular security threats [2]. e term "botnet" is formed from two words: robot and network. Robot means an automatic program that is run and performs the intended tasks without user interaction. e bot can be good or bad. e bot in the botnet is a bad (malicious) software that is run on the victim's machine without his knowledge. e attacker who owns the bot takes control of the machine and formed, with other bots, a network of infected machines. e attacker is known as botnet master. e botnet master uses special mechanism to communicate with these bots and exchange commands through the botnet. is mechanism called Command and Control server (C&C).
Bots act in a coordinated manner and follow the botnet masters instructions, given the power of botnets to execute various malicious activities and massive attacks against e-commerce website, governments' websites, etc. [3]. e bot has the capabilities to lunch dangerous attacks such as Distributed Denial of Service (DDoS), fishing, data stealing, click fraud, and spamming. A survey from Kaspersky shows that DDoS botnet attacked 79 countries online resources in the first quarter of 2018 [4]. is number of attacks is increasing each year. e goal of botnet activities is typically to gain a financial benefit for different purposes. One of the botnet activities is to steal sensitive information from the compromised machines and send this information back to the C&C server. en, this sensitive information is sold by the botnet master. An example of sensitive information is personal bank details or any other information that an unauthorized person should not have the right to access to it.
Organizations firewalls are designed to allow legitimate traffic such as DNS, HTTP, and P2P. e botnet master takes this as an advantage to pass the organization firewall and download the bot into the users' computers. e huge amount of packets and data exchanged in the network prevent network administrator to detect this intrusion since he cannot monitor all these information. ere are many types of botnets that are identified by the previous researchers [5][6][7]. e first developed botnets are Internet Relay Chat (IRC) botnets, followed by Hypertext Transfer Protocol (HTTP) botnets, and finally Peer to Peer (P2P) botnets. IRC bot is the easiest type of botnets to detect because it uses centralized architecture which means all bots are supervised from a center point. e existence of a central point makes it visible and therefore easy to be detected and blocked [8]. However, P2P bot is the most difficult one to detect because it uses distributed architecture which means the botnet master transfers command to an infected bot peer who transfers it to other peers. In this case, a single failure in distributed botnet does not create significant disruption [7]. e first action that must be taken to secure a network is to detect machines infected by bots. However, the botnet is employing and developing hiding techniques along with the development of the technology. ese hiding techniques can be done by various ways such as trying to imitate the legitimate network traffic and using process injection. Fortunately, botnet follows a unique communication pattern that is different from nonmalicious (legitimate) traffic in different criteria such as packet size, flow size, flow and duration time.
Botnet detection may be implemented at the network level or at the host level. Botnet detection at the network level plays a critical role in security by monitoring the network traffic and providing warning to the network administrator when any unusual event is detected. On the other hand, the detection at the host level plays a crucial role in the detection of malware infection by monitoring files system modification, registry modification, and network traffic on the host.
Given the discussion above, our research question is as follows: Can botnets be detected effectively by combining both network traffic flow data analyzer and host processes data analyzer?
In this research, we propose a general hybrid technique capable of detecting botnets in early phase. Early phase means that we try to detect malicious behavior when the bot tries to propagate the bot malicious code to infect other machines or from the first packets exchanged between the bot and the C&C server. Our technique is deployed at the host level and the network level. e botnet communication traffic that we are interested in includes HTTP, P2P, IRC, and DNS using IP fluxing. Our technique consists of three components: network analyzer, host analyzer, and detection report: (i) e host analyzer observes the process operation in file system and registry.
(ii) e network analyzer observes the network traffic activity of the host process. ese activities include botnet propagation operation and botnet communication with command and control server. (iii) e detection report component generates a report with infected machines' IP addresses.
To achieve our goal, we develop an algorithm called HANABot (Host And Network Analyzer for Botnet detection) to preprocess the data to detect bots based on set of features. ese features have been combined to generate a classification technique capable of differentiating botnet and legitimate flow records with a high degree of accuracy. Although some of these features were previously studied, they are reused and ordered in a different way to improve the accuracy in differentiating legitimate traffic from botnet traffic. In fact, these features are fundamental to achieve high botnet detection accuracy. Moreover, new features are proposed to enhance the botnet detection especially in early stage.
Our proposed approach has the following three contributions: (1) A general botnet detection technique capable of detecting the three types of botnet (IRC, HTTP, and P2P). For the network traffic analysis, our solution monitors two activities: botnet propagation techniques and network data flow. Botnet propagation detector aims to detect newly infected hosts before this bot starts the communication with the C&C server.
(2) Our technique, for the host process analysis, monitors file system creation and registry modification. (3) An effective algorithm, HANABot for Host and Network Analyzer for Botnet detection, is developed.
is paper is organized as follows. Section 2 gives an overview of some research works proposed for botnet detection. Section 3 describes our methodology to reach our objectives and the laboratory experimental setting. Section 4 introduces the proposed technique, HANABot, and its components. Section 5 presents HANABot performance and a comparison with some previous solutions. Finally, we conclude our paper in Section 6 and recommend some future works.

Literature Review
One of the hot research topics is the botnet detection. Many researches in the literature focus on botnet detection techniques. Many of these techniques are focused only on specific botnet type. Botnet behavior-based detection approaches can be classified into three classes: host-based detection, network-based detection, and hybrid detection. Here, we briefly describe some of the previous researches that propose different solutions to detect botnet.

Botnet Detection at the Host Side.
Using antivirus software and a firewall to protect the host are not enough to prevent its infection by botnet malware. Moreover, even if we can stop the C&C server, the infected host (with bot) can be launched again into future attack. In this case, we need a host-based detection to eliminate the bot program from the host. ere are various researches in host-based detection techniques [9][10][11]. Huang [10] proposes an effective bot host detection solution based on network failure tracking in a host during short period. e solution consists of two phases: (1) training phase and (2) detection phase. e first phase is used to extract features from failure flows. e second phase analyses the data based on knowledge obtained during training phase using C4.5 algorithm. In [9], Etemad and Vahdani deal with the detection of C&C centralized botnet by using host analysis. eir solution is based on incoming and outgoing host real traffic analyses. It consists of two basic components: is approach is based on the traffic analyzer in the host and not on process detection. It does not process encrypted packets (command) from the botnet master to bots. Moreover, it is limited to centralized botnet and does not deal with P2P botnet.
In [12], Zeng et al. proposed a per-process level containment technique for each host in the monitored network. e per-process containment consists of two components: behavior analysis component and containment model. e behavior analysis component consists of various monitor and suspicion-process level generators. e process is monitored in the runtime behavior at the operating system level in registry, file system, and network stack. en based on the process activities analysis, they apply the SVM algorithm to each running process to assign a suspicious level for that process. In the containment, there is a mapping function optimizer to transfer the suspicious score to a threshold for each process. eir approach has many advantages: (1) ey propose a technique that integrated both behavior analysis and containment for early and an automatic defense against malicious network worms. (2) ey generate a suspicious level for each process using a machine learning classification technique and develop an algorithm to map each suspicious level score to a threshold for rate-limiting procedure.
(3) ey perform in-depth analysis and experiment using traces of real world worm binaries and normal programs.

Network-Based Botnet Detection.
Earlier, botnet detection techniques are based on payload inspection analysis techniques which check the TCP and UDP packets contents for malware signature. Payload analysis techniques are resource consuming that require processing large amount of packet data and it is a slow process. Moreover, new botnet generation employs encryption algorithms and other methods to hide their communication traffic and to crush packet payload inspection analysis techniques. Flow-based attributes extracted from the network traces were similar to NetFlow features such as bytes-per-packet, bytes-per-flow, and bytes-per-second. In the last few years, botnet detection techniques using flow analysis has emerged.
ere are many network traffic flow detection techniques that have been proposed in the last years. In this section, the discussion of some of those techniques and their limitations will be stated. Anomaly detection can be through miningbased detection techniques which are used to extract unexpected network traffic patterns. Hence, it can detect abnormal traffic even if the packets are encrypted. Many techniques used flow analysis [5,7,[13][14][15].
In [5], general botnet detection that is able to detect different types of botnet is proposed. is approach analyzes the network traffic flow during constant time intervals. en, statistical correlation in the network traffic flow is performed for building effective classification system. is approach can detect botnet regardless of the topology or the protocol used. Moreover, it is capable of detecting unknown botnet.
In [7], Liao and Chang propose a P2P botnet detection approach based on analyzing and monitoring network traffic using data mining scheme. ey evaluate their solution using three data mining popular algorithms: J48, naïve Bayes, and Bayesian networks. e accuracy rate of these algorithms is 98%, 89%, and 87%, respectively.
Zhao et al. in [13] propose a new solution for P2P to detect botnets by analyzing network traffic. eir idea consists in selecting 12 features from the network flow to be analyzed and hence extract flow behavior for predefined time window. ey use machine learning algorithm to isolate the botnet traffic from the legitimate traffic. ey select decision tree using the reduced error pruning algorithm. en, they use correlation attributes evaluator to choose the key discriminating attribute for botnet detection. eir approach can detect single bot activity offline or in live traffic. Moreover, it is able to detect unknown bots and detect bots in early stage through its activity in C&C phase.
Hung and Sun in [14] propose a botnet detection system based on machine learning. e solution is based on network traffic. First, a set of flow-based features is selected and extracted from the network traffic. en, to make sure the model performs well, some noise is added to payload, interarrival time, and features. is solution achieves high accuracy that reaches 99.7% for some botnet types.
Compared with an existing botnet detection system, the proposed solution shows that it can resist more noise.
Alauthaman et al. [15], propose a P2P Botnet detection scheme based on decision tree and adaptive multilayer neural networks.
is framework passively monitors the network traffic to detect the botnet communication with the C&C server and between bots. First, network traffic reduction is done to manage the enormous amount of network traffic. en, 29 features are proposed for botnet detection. Features with small influence on classification model are eliminated by using the feature reduction approach: the classification and regression tree (CART). e target of this feature reduction is to keep only worthy features to obtain better rates of neural network learning and classification accuracy. e evaluation of this approach shows that it gives very good accuracy with 99.2% and outperforms some existing solutions.
DNS detection techniques are based on specific DNS features analysis produced by a botnet. However, these DNS techniques are not useful to differentiate C&C server traffic and new botnet types. Some solutions focus on DNS traffic to detect botnet which relays on DNS to discover the C&C server.
Zhao et al. [16] propose an IDns system to detect malicious domain names in the C&C server for APT (advanced persistent threat) attack.
e proposed system is deployed at the network side which can reduce the network traffic volume that has to be registered then analyzed. eir approach consists of four components:

Hybrid-Based Detection.
Hybrid-based detection is a correlation between network traffic analysis and host traffic analysis.
Zeng et al. [17] propose a botnet detection technique that incorporates both host level detection and network level detection. eir solution is the first that combines the host and network level detections and correlating the alerts which could increase the detection accuracy. ey propose to use 9 features for host analysis: 6 features for file and registry operations and 3 features for network traffic in the host. For network analysis, 17 features are extracted from netflow data. ey evaluate the combined host and network detection technique and show that their solution is effective against IRC, P2P, and HTTP botnets.
In [18], a botnet detection technique that incorporates both host level detection and network level detection is developed.
e proposed solution, called EFFORT, is implemented as a multimodule approach to correlate information from different host and network level aspects to achieve an efficient and effective detection. e solution is implemented and evaluated in real-word machines. Results show that EFFORT effectively detects until 15 real-word bots with low false positive rate.
In [19], Abdullah et al. deal with the P2P botnet architecture. eir solution depends on the combination between the host-and network-based analyses. In the host level, the analysis will be on the file system: registry and log file.
is analysis looks for abnormal behavior and characteristics in every single activity that happened in the host. In the network level, the analysis is on the full packet payload. is analysis can differentiate both the C&C servers and the infected hosts with bots in the network.
is solution can be also considered as prevention mechanism. In fact, it is effective in detecting bot in early stage from the host analysis. However, it takes time to complete the analysis due to the network violations in P2P bots. For accurate detection rate, it combines the host analysis result with network analysis result.

Discussion.
We have presented a brief state of the art related to botnet detection, significantly contributing to localize and detect bots in the network.
We distinguished between (1) botnet detection at the host side, (2) botnet detection at the network side, and (3) hybrid botnet detection. It is clear that the highest benefits will be obtained by a solution that (i) Combines the host and network analysis which means hybrid botnet detection: to be efficient, some solutions integrate more than one strategy. Using hybrid botnet detection, many parameters are combined and correlated to localize and detect abnormal behavior. In this case, if the bot traffic is not detected by the network analyzer, the host analyzer can detect it using other rules and taking into account other features and the reverse is true (if the bot behavior is not detected by the host analyzer, the network analyzer can detect it using other rules and taking into account other features). (ii) Minimizes the complexity: the target of a botnet detector is to detect botnet traffic as early as possible while reducing the complexity and the processing time of the analyzer. For example, the network analyzer must analyze and classify all the ingoing and outgoing traffic. Consequently, an efficient botnet detector should filter the traffic before starting the analysis. (iii) Easily adapted to detect new botnets and scalable: a botnet detector may have a good detection rate for known botnets. However, the solution should also detect unknown botnet with a very high accuracy. In all cases, the target of all these solutions is to detect as soon as possible the botnet and stop it. e ideal solution is to obtain 100% of detection rate with 0% of false positive rate. To be adopted, such solutions should be effective with low complexity and capable to detect new or unknown botnets.

Methodology
In this section, we present the research methodology to be able to perform an investigation as well as analyzing the bot in the infected host.

Laboratory Environment.
e laboratory environment in this research consists of physical machines and virtual machines (VMs). One of the physical machines is used to collect malwares, and the other physical machines are used to analyze malwares. Using the VM software, in this research, affords an efficient and safe environment to analyze the botnets and makes a flexible way to deploy a laboratory to analyze botnets.
is laboratory consists of several computers. If this laboratory depends only on physical computers to analysis botnets, the cost of the research will be very high. However, the advantages of using the VM are first to reduce the cost and second to be able to restore the computer to its original state if the VM is infected. is will save the time in repeating the experiment multiple times which help us to provide results with high accuracy and in safe way.
On the other hand, this research is dealing with a malicious activity that can threat the security of the researcher's network. erefore, we need first to consider the security of the researcher's network. e solution is to use a demilitarized zone (DMZ) on the physical machine that collects malwares.
is DMZ provides a honeypot in an isolated environment to capture all the malicious traffic in experimental machine.

Laboratory Components.
is subsection presents the laboratory components used for the experiment of this research. ese components are shown in Figure 1.
It consists of a physical computer (the host) with Linux as an operating system. On this host, we install a VM with Windows 7 as an operating system. e VM will be the target of the attack in this research, and the Linux operating system is the controller on which honeypot is installed. For a dynamic analysis, different tools will be used to carry out the experiment: (i) Honeypot: this tool is used to collect malware samples from the Internet. In fact, the collection of the malwares using this tool is the safest way. e physical computer is connected to the Internet directly to expose the computer vulnerabilities and hence attracts attackers. e honeypot chosen for this research is the low-interaction honeypot Dionaea [20]. (ii) Controller: the controller is the Linux-based operating system. It will be used to monitor the network activity and to control the windows virtual machine activities. It will construct a database to collect malware signatures. (iii) Virtual target: it is a windows 7 VM. e malware analysis tools will be installed on it. is will allow us to examine the bot behavior. (iv) Dynamic analysis: this is a VM with Windows 7 as an operating system; we will have an ability to perform a dynamic analysis. A dynamic analysis will be performed in this research separately.

Dataset Collection.
We have run the honeypot for one month and collected six types of malware binaries. Unfortunately, only two of the collected binaries belong to botnet. Hence, we download the rest of wanted malware binaries from VirusTotal. en, we do the experiment to build our dataset by capturing the activities of normal and malicious processes. e monitored applications are eMule, Bittorrent, web services, Rbot, Zues, Neris, and Storm. e legitimate application traces were collected from clean (malware-free) VM PCs in regular use. e dataset collected is presented in Table 1.
To construct our proposed technique training dataset, we have applied the publicly available botnet dataset from multiple sources. We collect three different botnet datasets and one benign dataset from Dalhousie University [21], two botnet datasets and one normal dataset from CTU University [22], one botnet dataset from the Sourcefire Vulnerability Research Team (VRT) [23], and one dataset from ISCX which is a mixture of several accessible publicly available botnet and normal datasets [24]. Table 2 shows a detailed description of the collected datasets.
To evaluate our botnet detection technique, we have used the collected dataset from the experiment.

Research Design.
e research design is divided in two phases. e first phase consists of the collection of malware signatures by using honeypot. ese malware binaries will be used to infect the VMware host in order to monitor their activities.
e target of monitoring the infected host, using sniffer tool (wireshark) and malware analysis tools (process monitor and process explorer), is to be able to extract the communication pattern characteristics between the C&C server and this host. After that, we use these tools to analyze the collected malware behavior and gather behavior information artifacts that we need. e second phase of this research consists of the development of an algorithm (using python) to extract needed Journal of Computer Networks and Communications artifacts. en, we perform preprocessing steps to collect features which conduct to the final decision. is final decision will be produced using machine learning algorithm using Knime.

Extracted Proprieties for Features Building.
e bots need the Internet to perform various actions such as communication with C&C server and the propagation of the botnet by infecting other computers. As a result, monitoring the network traffic and processes activities in hosts are an essential aspect to detect the botnet. e artifacts needed to our features for host processes analysis are explained in Section 4.2.
In the following, we review the features and attributes that are used for classifying P2P, IRC, HTTP, and IP fluxing activities from bots activities in network monitoring. In the traffic classification field, finding a combination of features and attributes that distinguishes each category with a high accuracy and successfully determines the label of each unknown item to a right class is the challenge. e characteristics needed for our features vector analysis can be extracted directly from the network flow record. For each flow record, we first determine the traffic type then compute the features needed based on the traffic type. Table 3 shows the artifacts needed to build up features for machine learning tools. e four first artifacts are used to determine packets belonging to the same flow. e artifact numbers five and six are used to compute the failed connection ratio feature presented in Section 4.1.1. e last five artifacts are used as follows.
3.5.1. Packet Size. P2P application and Internet services that need a large amount of bandwidth will typically have large packet sizes. is is because these packets are linked with a large payloads content and contain a lot of data and information. ese large packet sizes are generated by network bandwidth consuming services such as uploading or downloading files, video streaming, and P2P file sharing. e (TCP or UDP) packets, Maximum Transmission Unit (MTU), size running on the Ethernet is approximately 1400 bytes and for P2P application is between 1000 and 1400 as shown in Figure 2.
However, the botnet uses small packet size to minimize their observable impact on the network traffic and does not consume network bandwidth. ere are two reasons for that: first, to maintain the connection undetectable and hidden in the network; second, to maximize the reliability of the connection between the bots and command and control server. e packet size used in the botnet communication is   Journal of Computer Networks and Communications less than 400 bytes as shown in Figure 2. We consider the packet size as small packet size if its size is less than 400 bytes as a measurement for botnet detection. For HTTP traffic, 3 ranges can be identified for the request and response size which are 0-500, 501-1000, and 1001-1500 bytes. In HTTP traffic, we distinguish two cases: (i) e abnormal behavior in which HTTP request packet size is in range1 (0-500) and the response packet size is in the same range (range1). (ii) e normal behavior in which HTTP request packet size is in range1 (0-500) and the response packet size is in the different range (range2 or range3).
Example of HTTP request and response ranges is presented in Table 4.

Equal small Packet size.
As mentioned in subsection 3.5.1, bots use small packet size. Furthermore, during the life-cycle of the botnet, the communication activities between the C&C server and bots follow constant pattern (since they are hard coded in the bot program). In fact, the packets exchanged between a C&C server and a bot have the same content.
is means the same packet size which is generally small as explained above. us, the bot follows uniformity in their communication traffic behavior such as packet size, whereas legitimate users use distinct packet sizes which are generally large packet sizes. e packet size and the number of equal small packet size are used to make decision on the traffic if it is normal or abnormal behavior. erefore, for each flow the first incoming packet size (IPS) if it is small packet or the first outgoing packet size (OPS) if it is small packet is taken as reference. ese values are used at the end of the flow to conclude. en, (1) e number of equal incoming small packet (EIS) is computed. To achieve that, the number of all small incoming packet that their size is equal to IPS is counted. en, the total number of packets in the flow called total incoming packet (TIP) is counted. At the end of the flow, the following equation is computed: where σ is a positive parameter used since all packets are not equal to the first packet (IPS) in botnet flow. From our experiment, we notice that there are some distinct values. erefore, we choose σ � 0.75. (2) e same step is used in the case of outgoing traffic. e number of equal outgoing small packet (EOS) is computed. Hence, the number of small outgoing packets (NOPs) where the size of this packet is equal to OPS is counted. e total outgoing packets is also counted. en, the following equation is computed: where σ � 0.75.

HANABot Technique Description and Analysis
Our solution, HANABot (Host And Network Analysis for Botnet detection), works at the network level and host level. In the network level, it can monitor the traffic of all devices connected to this network and analyze it to observe specific connection patterns. In the host level, our solution can monitor the host processes operations and network traffic to detect the bot process. e monitored host process operations involve registry and file system. en, specific features are varied to decide if a flow including a host's IP address or a process in a host is suspected of bot activity or not. Figure 3 shows our technique architecture in which we distinguish three components: network analyzer component, host analyzer component, and detection report. Total number of connections 6 Number of failed connections 7 First packet length 8 Packet size 9 Total number of packets 10 Number of input small packets 11 Number of output small packets   In this research, we try to detect the botnet as efficiently as possible. In Section 4.1, we detailed each component of our architecture.

Network Analyzer.
e network traffic of the botnet is classified into two main categories: (i) Propagation and infection of other computers in the network either through scanning for other vulnerabilities in the network or through social engineering. Scanning techniques may be a port scanning or failed connection. Social engineering techniques may be an e-mail spam. (ii) Botnet master communicates with the botnet members through the C&C server in the same botnet with a specific communication pattern. e communication traffic of HTTP, IRC, IP fluxing, and P2P is discussed.
As a starting step for the classification technique, we need to distinguish between P2P activities and nonP2P activities. e packets are parsed from the Wireshark packet analyzer (PCAP file) to our HANABot algorithm. e process for distinguishing the network traffic is based on the source port number, destination port number, and the protocol used. In non-P2P traffic, we identify the following network traffic: (i) HTTP using TCP protocol and port number 80 (ii) IRC using TCP protocol and port number 194 (iii) IP fluxing by DNS using UDP protocol and port number 53 If the port source number and port destination number is bigger than 1024, then the traffic is considered P2P traffic.
In this stage, we filter out all irrelevant network traffic flows. Notice that this filtering step is just an optional procedure and not critical to our network analyzer. e purpose of this step is to reduce the computational cost by reducing the total number of flows. en, the network traffic is split into different categories (HTTP traffic, IRC traffic, etc). In each category, all packets that belong to the same flows are extracted. Next, our HANABot algorithm extracts all the artifacts presented in Table 3 from these flows. After that, the HANABot algorithm preprocesses these extracted artifacts to obtain the relevant features of each flow. e preprocessing is applied on the network traffic to collect valuable information that based on the decision is made at the end. In the following, we present HANABot preprocessing steps. Figure 4 shows the network analyzer components. In the following, we present each component in more details.

Botnet Propagation Detector.
is component is common for P2P and non-P2P botnet. In fact, in all case, the botnet uses propagation techniques to spread its malicious code and infect new machines. e bots will be looking for known vulnerabilities in new victim machine as part of the propagation process. Bots look for a new target randomly or a specific machine and scan it. e propagation techniques that we interested in are as follows: (i) In traditional non-P2P traffic, such as HTTP, DNS, and SMTP, the server works well and generally all connections are successful. erefore, the failed connection is low in client/server networks. However, in P2P networks, such as BitTorrent, the failed connection is high. e reason for that is that a peer maintains the connection with other peers and tries to connect to the peers that have been disconnected in P2P networks. However, IRC botnet and HTTP botnet have high failed connection ratio since the bot is trying to find the C&C server IP address. Regarding P2P botnets, it has low failed connection ratio because the bot communicate with known peers via hard coded IP addresses.   Table 5. We have a failed connection if the outgoing connection is not successful. e failed connection ratio is computed using the following equation: where FCR means failed connection ratio, FCN means failed connections number, and ACN means all connections number. (ii) Port scanning can be detected by monitoring the failed connection from the same source IP to different hosts in the network. If the packet is non-P2P packet (if the source port or the destination port is <1024), many failed connections to different hosts in sliding time window is considered as abnormal behavior. Based on Table 5, we keep track of the number of failed connections issued from the same sender IP address. Every time a failed connection is tracked, a variable flag is_failed is set to 1. en after 2 seconds, we sum up the is_failed value for each sender in the monitored network. (iii) E-mail spam is a method for sending unwanted e-mail messages frequently with commercial or marketing content in large quantities to a random set of recipients. Delivering e-mail spam procedure is same as delivering legitimate e-mail, utilizing Simple Mail Transfer Protocol (SMTP) on port 25 [25]. e botnet master uses botnet for diffusing spam e-mails at a large scale. e senders of e-mails may belong to different botnet networks and also they may serve different botnet masters for different purposes and this spam emails can have different contents. We keep track of the number of SMTP packets issued from the sender IP address. Every time a SMTP packet is tracked, a variable flag is_SMTP is set to 1. en after 120 seconds passes, we sum up the is_SMTP value for each sender in the monitored network.

Features for botnet propagation detection:
(1) Failed connection ratio (FCR) (2) Number of failed connection from same IP address during an interval of time (2 seconds in our case) (3) Number of emails (port 25) from same IP address during an interval of time (120 seconds in our case)

P2P Communication Traffic Detector.
In [26], we present our work to detect P2P botnet. Here, we mention briefly the presented approach. If both the source port number and the destination port number of the packet is greater than 1024, then the packet is considered as P2P packet. In P2P application, each node can act as server to download data or as client to upload data. In order to  (ii) If the size of UDP communication request packet is more than 120 bytes and the response packet size exceeds 400 bytes, then the traffic is considered as suspicious bot. In fact, if the peer needs to retrieve other peers IP addresses who has the wanted file using DHT, it should send a request UDP packet size as mentioned above and receives packet with a size less than 400 bytes as in case of BitTorrent. However, the response, in case of botnet, contains the malicious code which is generally larger than 400 bytes.
TCP communication establishment: a normal three handshake TCP communication establishment is performed. erefore, the detection for TCP communication is performed in the data transmission stage.
(2) Data transmission stage: After the establishment of the connection, the data transmission starts by uploading or downloading data through the P2P network. During this transmission, peers use large packet sizes to reduce the overhead in the network. ese packet sizes are generally higher than 1000 bytes. However, bots use small packet size to minimize the impact of its traffic on the network as explained above in this paper. Equations (1) and (2) computed in the preprocessing step are used to make a decision concerning the traffic type: (i) Based on Equation (1), if NIP > � EIS, then the number of incoming equal small packet size is high (abnormal behavior).
(ii) Based on equation (2), if NOP > � EOS, then the number of outgoing equal small packet size is high (abnormal behavior). Table 6. e flow is terminated in TCP connection by TCP finish flag or TCP reset flag. ere are other cases where the flow can be terminated in a nonstandard way such as failure in the physical link. In this case and in UDP transmission protocol, we check the time of the last packet arrived if it exceeds 120 seconds (this time is the default value used by open source linux kernel's connection tracking module), we consider this flow as terminated.

An example of Equation (1) computation is presented in
Features for P2P communication traffic detection: (i) Packet request size for UDP protocol (ii) Packet communication request size for UDP protocol (iii) Packet response size for UDP protocol (iv) NIP, EIS, NOP and EOS for TCP protocol (for each flow)

Non-P2P Communication Traffic Detector. Non-P2P
application is any application that runs on well-known port number (if the source port number or the destination port number is less than 1024). In non-P2P detector, we monitor HTTP, IRC, and IP fluxing.
(1) HTTP Detector. HTTP traffic uses the TCP transmission protocol. e standard HTTP traffic contains request and response messages.
(i) e request can be one of three types: GET, POST, and HEAD. (ii) Each response has a status codes. is status code can be one of five classes. e interesting response status code for us is 200 from the second class (2xx) which means that the HTTP request successful. e actual response code status number will depend on the HTTP request method type used. In a HTTP GET request, the HTTP response status code 200 will contain an information entity corresponding to the requested web resource. In a HTTP POSTrequest, the HTTP response status code 200 will contain an information entity containing the result of the activity. We analyze three types of parameters which are the HTTP request size, HTTP response size, and payload packet size. In normal HTTP traffic, to download data, legitimate users will send small request packet size and receive large response packet size with large incoming packet size. To upload data, legitimate users will send large request packet size with large outgoing packet size and receive small response packet size. However, C&C server communication with bots using HTTP as shown in Figure 6 can be categorized in three cases.
(i) e C&C server sends to the bot instructions to do it or malicious code to download them. In this case, the incoming packets size will be small and the number of equal small incoming packets size should be large. (ii) e bot sends the compromised host machine's information to the C&C server. In this case, the outgoing packets size will be small and the number of small outgoing packets that have the same size should be large. (iii) e bot maintains the connection with C&C for updates. In this case, the incoming and the outgoing packet size will be small. Moreover, the request size and response size will be in same range (both are in small range).
e preprocessing of number of small packet is done by equations (1) and (2). e flow is terminated by TCP finish flag or TCP reset flag. ere are other cases where the flow can be terminated in a nonstandard way such as failure in the physical link. In this case, we consider this flow is terminated if the last packet arrived 120 seconds ago.
Features of the HTTP detector are as follows: For each flow, (i) HTTP request size (POST or GET messages) (ii) HTTP response size (status code 200) (iii) NIP, EIS, NOP, and EOS (2) IP Fluxing. DNS protocol is used to resolve the server name to the IP address. When the host performs DNS query for a specific domain, it will retrieve DNS response records. e response record can be one of the different types such as A record for IPv4. e mapped IP address in the response record can contain one or many IP addresses if the domain name was located. is response record optionally has a time to live (TTL) period. is time period is predefined by the authority name server for the zone. Caching resolver performs lookup and stores the results for a TTL period of time.
is period informs how long (in seconds) a server and application should cache this address. Most of the legitimate services use long caching times such as 86400 seconds or even more.
is long caching relieves the loads on the authority servers and provides response faster [28][29][30]. e main challenge for the botnet master is to hold the C&C server undetected to longer time as many security administrators try to block down and track the IP addresses of these servers [30]. Moreover, to better manage large distributed infrastructure, botnet masters have implemented fast flux techniques using DNS traffic for four reasons: (i) ey intend to minimize cache time by setting a low TTL period for C&C domain names. is is because long caching delays the propagation of new IP of the C&C servers. (ii) ey obtain the flexibility to change the IP addresses of the malicious servers that they manage. (iii) e C&C server becomes more difficult to identify and take down. (iv) To hide their critical servers behind proxy services.
DNS, for load balancing, uses a round robin technique in two steps: (i) e domain name is mapped to multiple IP addresses. (ii) e DNS server rounds through these IP addresses and returns each time a different IP address mapping [31].
Whenever a computer is infected with a bot, it joins to the botnet as an asset to the collection [29]. Malicious domains IP addresses are gathered from bots that exist in different autonomous systems and countries. Botnet masters do not aim a specific country or a specific IP range [32].
We can detect IP fluxing in DNS by keeping track of two artifacts: IP address assigned to domain and domain cache TTL. Every time the host sends a query for specific domain, we keep track of the last IP assigned (if there), the new one and the domain cache TTL. In fact, if newest IP address is not in the same class subnet of the previous IP and cache TTL period is less than 1000, then this domain is suspicious. Indeed, normal domain assigns multiple IP addresses in the same class subnet and long domain cache TTL period.
Features for IP fluxing detector are as follows: (i) IP address assigned to domain (ii) Next IP address assigned to domain (iii) TTL value (3) IRC Detector. We follow the same procedure as discussed in P2P data transmission. We keep track of the number of small packet size and the ratio of the equal small packet size. Table 7 shows our features for host process analysis.  Host analyzer, as shown in Figure 7, consists of three components: process behavior analysis, process correlation, and process behavior accumulation. In the host analyzer, we monitor three activities: registry keys, file system, and process active time, for a period of time. e monitoring time is randomly chosen from 10 min to 30 min. In Section 4.2.1, we discuss these components in more details.

Process Behavior Analysis.
We develop an algorithm to monitor the registry keys, file system, and active process time. e first action to reduce the propagation speed of botnet is to identify processes that perform malicious activities in a host computer. e botnets share certain behavior patterns, such as creating autorun registry key and creation of EXE in system directory, that are different from normal application. However, a single activity, such as creation of an autokey in the registry or creation of EXE in system directory, alone could appear harmless, while the combination of multiple activities may expose a malicious intent.
In our analysis, we extract the common botnet behavior. We focus on software process run-time behavior vector including registry modification, file system creation, and network traffic. e host process network traffic analyzing and detection is performed and followed the procedure explained in Section 4. In this section, we explain the features that are used by our technique to analyze the monitored registry, file system, and active time of processes in the host.
(1) Registry. Bots, in the registry, deal with two keys entries operations: (i) Bots will modify the value of critical registry key and the creation of autorun key. e means of critical registry key value is the value of any software meant to do process monitoring to detect any malicious action. An example of process monitoring software is Windows Task Manager, Sysinternals tools, and antivirus. e bot changes the value of the software registry to evade detection such as Taskmgr disable, overriding antivirus, windows firewall disable, etc. An example of value of the registry key that had been modified by a bot is HKEY_LOCAL_MACHINE\SOFTWAR-E\Microsoft\Windows\CurrentVersion\Policies\System. (ii) Bots will create an autorun key in the registry to automatically run themselves when the operating system starts up. e value of the autorun key will set to the name of the executable file of the bot. An example of bot creation of an autorun key is Maripose [33].
We use the HANABot algorithm to periodically check the registry location mentioned above. en, HANABot extracts the value of the registry key and finds the process ID who own this value.
(2) File System. When a victim computer is infected with a bot, a bot always downloads its malicious binary from the Internet to the file system directory. is is because it could be launched again when the operating system starts up.
Bayer et al. [34] indicate that malware files creation belongs to one of the two following categories: (i) e first one contains executable files (EXE). Most of the bots choose system directory location such as C: \WINDOWS or one of its subfolders as a typical location to drop the binary. is is because normal users rarely check the system directory and the bot

Malicious command
Maintenance and update Connection C&C server Bot   binary is less noticeable among the system files which could be a thousands of files. New bots start to target user folder which is C:\USERS\username\-Documents and Settings. (ii) e second one contains nonexecutable files which are dynamic link library (DLL), temporary data files, and batch scripts. Most of these files are placed either in the windows directory or in the user folder. Internet traffic creates significant amount of temporary Internet files. ese files are used to download content from the Internet.
Based on the monitored location, we use the HANABot algorithm to monitor the creation of DLL and EXE files access in windows directory. en, HANABot extracts the process ID owner of this file.
(3) Active Time of the Bots. Bots need to be always online to maintain the connection with the botnet master. If the bots are disconnected, then the botnet life cycle will be affected. As a result, the active time of the bots (operating in the background and waiting for updates) should be similar to the underlying compromised operating system. In contrast, the active time of any legitimate process is determined by the user, which is likely to be transient from one user to another.
e HANABot algorithm is used to monitor the active time of the process. It takes the creation time of the process. If its time is closer to OS starts up time, then it checks periodically to see if this process is terminated or not.

Process Correlation.
To perform malicious activities, bots can act as follows: (i) e bot behaves on its own process to conduct its malicious activities. (ii) e bot divides its activities through multiple processes, so the processes collaboratively perform its assigned malicious activities. To mitigate against such sophisticated bots whose one process malicious activity is not considered as a bot, we take into account process correlation during the monitoring. We keep track of the interprocess relations and aggregate the behavior vector from the processes correlation. e obtained behavior vectors are equal for all the correlated processes like a parent process and its children processes. We can easily exclude some normal process with correlation such as windows services by maintaining a white list. erefore, it is not hard to differentiate a set of processes behaving malicious altogether. is means that these processes have exactly the same behavior vectors.

Process Behavior Accumulation.
e bot, during the monitoring time window, may perform either multiple malicious activities or single malicious activity. To deal with this type of bot, we collect the value of each behavior feature vector in host monitor as shown in Table 8. e behavior vector features value summation (accumulation) are those rarely seen from normal software, such as creating EXE file in system directory or creating an AutoRun key in the registry, etc. e accumulation procedure is straightforward. An example of the accumulation procedure is shown in Table 8.
Using this procedure, the process final score will not decrease if the process does one malicious activity in single time window. Every window time, the process conducts malicious activity and its score will increase. e same concept is applied to a set of processes when each one shows malicious activity in various time windows.
After the bot process is identified in the monitored host, the file associated with this process will be scanned by the administrator. is file can provide valuable information about the C&C server IP address and how the communication is established between the bot (the compromised machine) and the C&C server. Moreover, the administrator can know how the botnet is propagated and infect other machines. is knowledge permits to the administrator to, in one hand, block all the incoming and outgoing traffic with the C&C server, and, in another hand, improve the overall detection mechanism by exploiting these information to build new rules.
Applying only host analyzer technique may not be effective because host resident bots may compromise the detection technique.

Classification Algorithms.
As malicious bot software spreads, the demand for an automated alternative is increased because manually generated rules become impractical. Machine learning algorithms allows the detection systems to fit fast changing malware. Moreover, it allows an automated method for learning specific patterns and properties that may be presented by botnet.
Our target is to effectively distinguish between legitimate traffic and botnet traffic. To achieve that, we consider the most memorable classification algorithm, namely, decision tree ensemble classifier and Naive Bayesian statistical classifier.
Naive Bayesian classifier (NB) represents a supervised learning method and a statistical technique for classification. It assumes an implicit probabilistic model. It allows to capture and analyze the relationship between each feature and the corresponding class to derive a conditional probability that links feature values and its class. e decision tree classifier (DT) utilizes a decision tree as a predictive model which maps an item observation to conclusions about the item's target class. It is one of the predictive modelling approaches used in data mining and machine learning. e classification tree is a tree model where the target variable can take a finite set of values. In this tree structure, leaves represent class labels and each branch represents the feature so the conjunctions of these features lead to those class labels.

Performance Metrics.
To avoid data bias, multiple metrics are measured. Accuracy, precision, recall, F-measure, and false positive rate are the five metrics used to evaluate the performance of our solution. Accuracy depends on the sample size. However, precision and recall measures are independent of the sample size.
Precision rate indicates the credibility of stated detection result, whereas Recall rate indicates the portion of hosts belonging to a specific class that can be identified. e metric F-measure is a coordinated mean of Precision and Recall. e last metric which is false positive rate indicates an error in classifying an item in botnet class whereas the right class is legitimate class. Table 9 shows the performance results for the five presented metrics. As shown in this table, the decision tree classifier achieves high performances and outperforms the naive Bayesian classifier in host level and network level detection. Its accuracy reaches 99.6% in case of hybrid detection with 0.01 as false positive rate. at is why we choose the decision tree classifier for our solution.

Results and Discussion.
Our technique consists of two main components: host analyzer and network analyzer. e purpose of our technique is to detect unknown botnet and infected hosts with bots. To build the ground truth, we collect the data set from the publicly available botnet datasets. To evaluate the classifier detection technique accuracy rate, we use the 10-fold cross validation method to divide our dataset into 10 random subsets. 9 subsets are used for the training, and one subset is used for the evaluation.
is same process is repeated until each subset of the 10 subsets have been used as the testing set exactly once. e evaluation must show that it can detect unknown botnet that are not in the training dataset. For this purpose, we use the data set collected from our experiment as explained in Section 3. e detection accuracy rate of our network analyzer is 99.6%. We report the following: (i) Neris bot, which belongs to IRC botnet, generates high failed ratio and performs scanning technique. Our technique is able to detect the IRC bot in the propagation phase. (ii) e Storm bot behavior is similar to bitTorrent in term of number of failed connections. Moreover, they both implement DHT to obtain the peer list. erefore, Storm bot can escape to the detector during the propagation stage. However, since in bitTorrent, the DTH UDP packet size is bigger than 67 bytes while the bot uses less than this size, the P2P detector will detect this bot. e detection technique achieves very high accuracy.
(iii) From our experiment, the failed connection ratio result for some P2P application is lower than the threshold fixed for botnet detection; eMule is an example of this case. In fact, eMule generates very small failure connection rate. is is due to the use of a "bad peer source list," which will be added to the application. is list is updated with the peers IP addresses when the eMule application fails to connect with these peers IP addresses. Consider this peer source list of IP addresses as dead and block for predefined time (15 to 45 minutes). Even if eMule application behaves similar to bot in terms of failed connection rate, eMule is not detected by our technique since we check also equal small packet size feature which is not satisfied in this case. Table 10 shows a comparison of the two classifier performances at host level. e first classifier uses all selected features. e performance of this classifier generates high false positive rates and low accuracy rate. is is due to two features that are common between normal P2P application and botnet. ese features are as follows: create and set the value of AutoRun key in the registry and Active time of the bot process. ese features are not effective to differentiate between bots and legitimate P2P application detection and increase the false positive.
is is due to the nature of P2P application, and it creates an AutoRun key to launch when the OS starts up and stay working online in the background (the active time of this process is equal to the active time of the OS). Botnet acts in the same way. erefore, we evaluate the second classifier by eliminating these two features from the behavior vector. e second classier decreases the false positive rate and increases the accuracy rate.  Accumulated feature score  win0  EXE file creation  1  1  win1  AutoRun creation  1  2  win2 DLL file creation 0 2  In our experiment, when host process is infected with Rbot, the Rbot process injects the svchost process (windows service). e svchost performs DNS query to obtain the C&C server IP address. en, the Rbot process establishes TCP connection to this address. Rbot injects the svchost process with malicious code. e host analysis does not cover any process injection methods. erefore, the Rbot process will evade our host analysis analyzer. However, if we use both levels host analyzer and network analyzer, this bot will be detected by the network analyzer. In fact, the detection accuracy rate is increased, if we combine the network and the host analyzer. In this case, all the flows are classified correctly in the right class (botnet or legitimate).

Comparison.
We compare our technique with the research works proposed in [5,13,15]. e performance comparison of our solution to some published approaches is shown in Table 11. e existing solution performances are computed by each developer. Zhao et al. [13] used two P2P botnets, namely, Storm and Waledac, and two HTTP botnets, namely, BlackEnergy and Weasel, to confirm their model efficiency. Kirubavathi and Anitha [5] used three botnets, namely, Zeus, Spyeye, and BlackEnergy botnets to evaluate their solution. Alauthamanet al. [15] used two P2P botnets, namely, Storm bot and Waledac bot to validate their model.
From the performance comparison table, we can remark that we achieve very good accuracy rate with the lowest false positive rate using HANABot with the network analyzer only. However, our proposed solution, which is hybrid solution based on network and host analysis, outperforms existing solutions in terms of accuracy rate and false positive rate. e works in [9,12] utilize the effectiveness of host level analysis for more accurate results. ese approaches, as our approach, monitored registry and file system in each host. However, we differentiate in the feature selection of the registry such as AutoRun key creation which increases the botnet detection.

Conclusion
In this research, we proposed a general technique that is capable of detecting a new botnet implemented on three levels: the host level, the network level, or a combination of both. e botnet communication traffic we are interested in includes HTTP, P2P, IRC, and DNS using IP fluxing. Our proposed technique consists of three components: host analyzer, network analyzer, and detection report: (i) e network analyzer monitors two activities: botnet propagation and botnet communication with the C&C server. e network analyzer consists of three detectors: the propagation detector, the P2P detector, and the non-P2P detector. (ii) e host analyzer monitors three activities: the registry keys, the file system, and the duration of the active process over a period of time. e host analyzer consists of three components: process behavior analysis, process correlation, and process behavior accumulation. (iii) e detection report is responsible for producing the final score result based on the host analysis or network analysis and provides a report of infected machines.
We developed the HANABot algorithm to preprocess host traffic data, host processes operations, and network traffic activity to detect the bots based on specific rules. All botnet flow records were successfully extracted by exploiting specific connection patterns and setting up feature vectors that is unique for botnet network traffic and its process operations. A comparison between an existing approach was given, focusing on specific features and performance.
In our future work, we will apply our solution for realtime classification. In fact, the proposed technique implements the classification algorithm in an offline mode, where the Internet network traffic traces are stored in a PCAP file before processing them. e technique can be enhanced to obtain real time classification instantaneously with a high level of accuracy. is can be achieved by making relatively short the export time period of the Netflow traces collector and by supplying the exported Netflow traces to the classification technique instantly, as they arrive. e technique can then differentiate botnet activities and take action on this basis. e processing speed at which real time classification takes place should vary, depending on various factors such as the export time interval and the size of the exported Netflow traces. Moreover, this work can be extended, in the host level, to support the detection of process injection used by botnet to exert full control over the process [35].

Data Availability
e dataset used to support the findings of this study are available from the corresponding author upon request.

Conflicts of Interest
e authors declare that they have no conflicts of interest.