MTEDS: Multivariant Time Series-Based Encoder-Decoder System for Anomaly Detection

Intrusion detection systems examine the computer or network for potential security vulnerabilities. Time series data is real-valued. The nature of the data influences the type of anomaly detection. As a result, network anomalies are operations that deviate from the norm. These anomalies can cause a wide range of device malfunctions, overloads, and network intrusions. As a result of this, the network's normal operation and services will be disrupted. The paper proposes a new multi-variant time series-based encoder-decoder system for dealing with anomalies in time series data with multiple variables. As a result, to update network weights via backpropagation, a radical loss function is defined. Anomaly scores are used to evaluate performance. The anomaly score, according to the findings, is more stable and traceable, with fewer false positives and negatives. The proposed system's efficiency is compared to three existing approaches: Multiscaling Convolutional Recurrent Encoder-Decoder, Autoregressive Moving Average, and Long Short Term Medium-Encoder-Decoder. The results show that the proposed technique has the highest precision of 1 for a noise level of 0.2. Thus, it demonstrates greater precision for noise factors of 0.25, 0.3, 0.35, and 0.4, and its effectiveness.


Introduction
Anomaly detection is a widely researched topic in machine learning and is crucial in many areas such as fraud detection, cyber security, and complex system health monitoring. As a result, it continues to be an active and challenging research area. e industrial multivariate time series data is the focus of this paper. Industrial systems, such as power plants, wind turbines, and engines, generate large amounts of time series data during their normal operations thanks to an increasing number of sensors and cost-effective data transmission and storage solutions. It is critical to monitor these systems for abnormal behavior, which, if not detected early, can have serious reliability consequences. Such relationships will not adhere to the learned representation in an abnormal situation, resulting in deviations. Today, new business opportunities are established through the use of the Internet and corporate networks. Numerous businesses and governments have established complex networks [1]. ese networks are equipped with techniques for dispersed data storage, encryption, decryption, and authentication. Corporate networks are more accessible and utilize virtual private networks to enable employees to connect [2], thereby increasing the vulnerability of today's network to attackers. Network-based attacks are increasing in frequency and severity, resulting in increased financial losses. Firewalls have ceased to provide appropriate protection [3]. According to a recent survey, forty new vulnerabilities are uncovered each month.
is unsecured environment has necessitated the development of intrusion detection and prevention systems.
Intrusion detection systems do an analysis of the computer or network for potential security vulnerabilities. It identifies actions that jeopardize the integrity, reliability, and confidentiality of the system [2]. Time-series data are real-valued. Time series data are frequently used in financial, medicinal, and meteorological applications [4]. Routers and switches are used to facilitate network communication. Each of these entities contributes to the network's behavior. Due to the dynamic nature of the Internet protocol, it is difficult to comprehend the system. To comprehend network behavior, it is vital to understand how network probes are used [5]. is is a characteristic of uncontrolled networks. With an understanding of the network's layout, it is simple to identify network irregularities and performance bottlenecks. e type of anomaly detection varies according to the nature of the data [5]. us, network anomalies are operations that depart from the usual. ese abnormalities can result in a variety of device malfunctions, overloads, and network incursions, among other things. is will cause a disruption in the network's normal operation and services. e normal behavior of a network is determined by a variety of factors, including the volume of traffic, the type of application, and the type of network data [5].
Today's commercially available network management systems monitor network activity in real-time and alert users to irregularities. e network manager monitors the environment and generates alarms when an unusual one occurs. Anomalies in networks are defined by changes in the measured data. Anomalies in networks are classified into two categories [5]. e first category results in network failures and performance problems such as file server errors and congestion. is abnormality will affect all users who share the network's available capacity. e second type results in security difficulties such as denial of service assaults, intrusions, and so forth. e correct type of network information is critical for any anomaly detection system. e more precise the traffic behavior, the more accurate the detection of anomalies. e remainder of the article's section is organized as follows: Section 2 discusses the various studies on time series-based anomaly detection that have been addressed. e proposed research methodology is illustrated in Section 3, followed by the results and discussion in Section 4, and finally the conclusion in Section 5.

Related Works
Anomaly detection in time series data has attracted the interest of a growing number of researchers in recent years. Numerous accessible techniques have difficulty in resolving these issues. Wang et al. in [6] created an online self-learning anomaly detection method to detect the abnormality automatically. Time series data are generated as a result of a succession of events, equipment sensors, or other sources. Anomalies can be structural, anomalous, or point in nature. In this case, the data points diverge. Structural does not follow the expected pattern, and anomalous time series contain deviations. In real-world applications, the difficulty is the occurrence of new anomalies with unknown behavior and duration. e study proposes techniques for detecting anomalies in univariate and multivariate time series. e algorithm is capable of detecting time series, locations, and sensors associated with them. Two algorithms were developed: SLADE TS and MTS. Initially, the algorithm splits the time series automatically into variable-length windows. Objects' behavioral patterns are not identified in all dimensions. However, accuracy and the determination of precise measurements and locations continue to be a challenge.
Memarzadeh et al. in [7] concentrated on identifying the National Airspace System's vulnerabilities. It is difficult to detect an anomaly in a high-dimensional heterogeneous time series. To capture the complicated patterns, the authors trained a convolutional variational auto-encoder. e model is composed of two components: an encoder that converts the original data space to a lowdimensional latent space, and a decoder that reconstructs the original data. e model optimizes mapping and quantifies the reconstruction error. is method is used to detect anomalies. Although the model performs well with high-dimensional data, dealing with heterogeneous multivariate time series data remains a barrier to vulnerability discovery. In [8] the authors surveyed the most advanced anomaly detection algorithms in the aviation domains. Anomaly detection is critical in a variety of industries. Due to the complexity of aviation's traffic circumstances, current anomaly detection approaches are insufficient. ey evaluated current improvements in deep learning algorithms for scaling high-dimensional data in light of the availability of a vast amount of sensor data.
us, it is clear that detecting anomalies in real-world applications continue to be a challenge.

Computational Intelligence and Neuroscience
Anomaly detection is the process of identifying anomalous behavior within a specified period. Li et al. in [9] proposed a method for detecting anomalies in multivariate time series using clustering. Due to the temporal link between points and individual time series, multivariate time series detection remains difficult.
e authors classified aberrant signals into two categories based on their amplitude and shape. e fixed-length sliding window is initially constructed from multivariate subsequences. Subsequently, expanded Fuzzy-C-Means clustering is used.
us, the anomaly score is identified, which quantifies the deviation from normal data. To deal with the initial feature space, the clustering procedure employs the Euclidean distance. Correlation of time series encapsulates shape information and identifies time series-based dissimilarity. Based on simulated datasets, the technique was tested. A confidence index was used to determine the cluster numbers and the interval length. As a result, the proposed approach recognized shape irregularities. While the approach performs admirably, it is limited in that it does not reveal the dimension of multivariate time series. Additionally, heavy computation generates overhead and is a time-consuming operation.
Khan et al. in [10] concentrated on anomaly detection in the aerospace domain, with a particular emphasis on machine learning techniques. Unsupervised anomaly detection is critical for detecting unknown cases in multivariate time series. e authors explored a variety of issues in their attempt to make a credible prognosis of the scenario. ese include data availability, unsupervised learning for processing vast amounts of heterogeneous data, measurement error, and the complexity of learning real-time patterns. Despite the algorithms' strong results, determining the appropriate threshold level for isolating the anomaly remains a challenge. In [11] Han proposed an unsupervised method for detecting medical anomalies through the use of generative adversarial networks. e innovative method consists of two steps and is capable of detecting a variety of ailments. is phase covers the rebuilding of unhealthy and aberrant scans. e diagnosis phase then establishes the ground truth. e receiver operating characteristic (ROC) and area under the curve (AUC) were calculated. e study made use of the OASIS 3 clinical dementia rating dataset. us, the disease was diagnosed using many MRI sequence images. In [12] Fan and others examined the autoencoder's performance in detecting anomalies and constructing energy data. Autoencoders are a kind of neural network that is capable of doing unsupervised learning. ere is no label variable in the learning process. e encoder takes identical inputs and converts them to high-level characteristics. e auto encoders deal with a variety of issues and are composed of fully connected layers. e autoencoder ensemble approach analyses energy data for abnormalities. e system was able to detect anomalies related to abnormal occurrences, operational errors, and inefficiency in the control strategy, among other things. However, developing unsupervised anomaly detection autoencoders remains a difficulty. e authors [13] concentrated on developing a secure and trustworthy system through the use of an anomaly detection technique. ey deployed a DeepLog, a deep neural network with Long Short Term Memory that was used to represent the system log. DeepLog automatically discovers log patterns based on the log data. e approach makes use of the massive amount of system logs. e reason for this is that log data is unstructured and varies by system. It's difficult to diagnose unstructured log files. e recurrent neural network loops back from the previous state to the present input and maintains a record of previous predictions. e long-term repercussions are recalled. In a streaming approach, DeepLog trains on small data sets and recognizes the standard log sequence. However, the scientists noted that employing a recurrent neural network to diagnose a system remained challenging.
Al Osman and other experts in [14] place a premium on stress management through the use of ubiquitous biofeedback serious games. e device continuously monitors the subject's mental stress by monitoring their heart rate, respiration rate, and degree of activity. As with a biological sensory paradigm, the approach operates as a loop between the body and the brain. It takes physiological data throughout game sessions. Mental stress is monitored using electrodermal sensors. e two-way communication between the game and the player is established. As the player becomes immersed in the game, biological signals are captured and the time required to assess relaxation is recorded. e u-biofeedback system achieves continuous monitoring ubiquity. e algorithm has two modes of operation: limited and indefinite. e limited play can establish their monitoring period for a short time, whereas with indefinite, monitoring continues for the duration of the game. As a result, the technique aids individuals in determining their stress levels. With the proliferation of interconnected devices and sensors in the industrial system, it is critical to monitor for threats. is is especially crucial in power grids, water treatment facilities, and communication networks because as complexity develops, it becomes more difficult to spot anomalies. In [15], they introduced a novel graph deviation network to learn the sensor relationship graph. Four components comprise the approach: sensor embedding, graph structure learning, attention-based forecasting, and deviation score. e trials used datasets from water treatment plants to demonstrate the accuracy of the anomaly detection algorithm. While the model comprehends the anomaly, the authors' issue about improving anomaly detection with hyperparameters remains unresolved. e authors in [16] introduced a novel GAN model for anomaly detection and a time series imaging localization framework. e sequence of distance images from one to the next is memorized. e encoder and decoder convert multivariate time series to two-dimensional pictures. e encoder's pointwise convolution ensures that temporal information is encoded. e experiment is conducted using the smart power plant dataset. As a result, the generator generates a sequence of distance images. However, detecting anomalies in image localization continues to be a challenge.
Zhang et al. [17] noted that multivariate time series data are becoming more prevalent in the actual world. For instance, power plants and wearable technology. When dealing with Computational Intelligence and Neuroscience multivariate time series, the anomaly is identified, and the problem is capturing the temporal dependency inherent in each time series. Additionally, the system should be noiseresistant. Although the number of unsupervised anomaly detection algorithms continues to grow. e authors proposed the use of a Multiscale Convolutional Recurrent Encoder-Decoder (MSCRED) for anomaly detection in multivariate time series data. To begin, the technique generates a signature matrix that is used to characterize multi-level system states. Following that, the time-series correlation is encoded using the signature matrices in a convolutional encoder. Precision, recall, and F1 scores were determined during the examination.
us, when applied to synthetic datasets, the model outperformed state-of-the-art approaches. In [18] the authors suggested a one-step-ahead load forecasting algorithm for estimating electricity requirements. e framework incorporates both the linear autoregressive and artificial neural network techniques. e Bayesian information criterion is used to counteract human-induced random events. e experimental results showed that the study is effective at selecting features from nonstationary time series data. Utilizing sophisticated approaches for anomaly prediction in the future remains a challenge.
e Temporal Hierarchical One-Class (THOC) network was proposed in [19]. Initially, it used a recurrent neural network to extract multi-scale temporal properties from time series. e approach performed hierarchical clustering using the intermediate layer characteristics.
is is because the upper layers include the characteristics with the lowest resolution. Multiple hyperspheres define the normal behaviors for each resolution. e intricate properties of realworld systems are captured through the use of multiscale support vector data descriptions. e one-class objectives are stated, and the end-to-end training of multiscale features and hypersphere centers is permitted. e deviation score is used to detect anomalies, and the approach outperforms contemporary state-of-the-art approaches. In [20] developed an improved deep model for detecting time series anomalies. e method was applied to a collection of hyperspheres with a hierarchical structure. To have access to all of time's events. e representation of features was enhanced to account for orthogonality and temporal self-supervision loss. Each component's effectiveness was evaluated.
Li et al. in [21] offered two solutions to the challenge of segmenting moving objects. e authors began with a transposed convolutional neural network and then added a Features Pooling Module. e first stage involved mapping the features to a pixel-level foreground probability map using the multiscale feature encoder and decoder. en the binary segmentation labels were created. e proposed approaches are straightforward and operate with networks with numerous inputs. e review was conducted using publicly available datasets from CDnet2014, SBI2015, and UCSD. In [22] discussed the difficulties associated with anomaly detection in the Internet of ings. e Internet of ings facilitates data communication without the direct involvement of human agents. e gadgets interact with the environment via sensors, actuators, computers, and smart objects.
e authors have highlighted several significant obstacles, including (i) Real-time processing, i.e., anomaly detection, which requires real-time decision-making. When the detector takes an extended period, the system fails. (ii) Incremental technique, i.e., maintaining the complete dataset for analysis is challenging, and using an incremental approach reduces memory needs. (iii) Online adaptive learning necessitates system reconfiguration.
Multivariate data on the environment and other sensor information must be addressed [23]. us, the author's point on historical anomaly detection for univariate exists. Yet multivariate data stream-based anomaly detection continues to be a difficult problem, which motivated this research. Although intrusion detection systems have evolved, nextgeneration anomaly detection systems face numerous obstacles. ese include legacy intrusion detection systems that are incapable of adapting to emerging network paradigms such as wireless and mobile networks. ey have not been scaled to meet the requirements of high-speed networks. Noise, constantly shifting traffic patterns, and massive networks go unaddressed [2]. False alarms must be suppressed. Despite the multiple methodologies used, there are currently no universally accepted criteria for evaluating intrusion detection systems. As a result, evaluating an intrusion detection system fairly using systematic criteria continues to be a challenge. Additionally, in today's actual network environment, having the necessary data for IDS evaluation is crucial [24]. Apart from this, another significant obstacle is the proliferation of attacks in various forms. e majority of intrusion detection systems are ineffective at detecting intrusions on time. us, the most difficult problem is multivariant time-series detection. While considerable work has been done in this area, additional research is necessary to discover realistically effective solutions [25].

Materials and Methods
Today's real-world systems generate an increasing amount of data. Identifying anomalous states and isolating the underlying reasons is challenging. e purpose of this study is to detect anomalies in multivariate time series data using a synthetic dataset of real power plants. In modern industrial facilities, complex systems are used. Numerous sensors are used to monitor the activity of these systems. e sensors generate multivariate time series data, which must be analyzed for anomalies [26]. In today's industrial world, multivariate time series data are generated by monitoring the behavior of complex systems, for instance, temperature and pressure sensor readings in a power plant. ese systems must be repaired immediately to address the difficulties mentioned. Managing this procedure and undertaking exact detection would come at a significant financial and human cost [23]. In the long run, small perturbations in a system cannot be viewed as system failures. As a result, several different anomaly scores must be offered to aid system operations. is enables operators to take remedial action and rectify any difficulties that may arise. e discovered anomaly score shall indicate the system failure based on the sensor data. Accurate and prompt judgments are required to avert catastrophic losses in the case of a power plant. Additionally, it is required to locate and repair the sensors 4 Computational Intelligence and Neuroscience responsible for the anomaly. Anomaly detection at various severity levels will finally reveal the robustness of the systems [27]. Systems that self-detect abnormalities are constructed using supervised algorithms. Although unsupervised algorithms exist, they face significant difficulties. In the event of an abnormality, multivariate time series data is critical for determining the aberrant condition and establishing the core causes [28,29]. Constructing time series-based systems is challenging because it requires the intercorrelation of numerous time-series data. us, to achieve the abnormal status of identifying and diagnosing anomalies based on time series, this research study proposes a Multi-variant Time series-based Encoder-Decoder System (MTEDS), in which the system utilizes time-series inter-correlation with the help of the state vector and signature matrices, which aids in analyzing the anomaly score based on the client matrix series variation (residual matrices) in our dataset. Following an adequate number of training sessions, the network parameters are used to infer the reconstructed signature matrices for the validation and test data. en, using latent vector characteristics, the anticipated number of anomalies may be checked against the enormous variety of Anomalies identified using this reconstruction [6]. e work introduces a novel MTEDS for handling anomalies in time series data with multiple variables. It begins by constructing multi-scale signature matrices to detect anomalies in the system's condition in a variety of time series. is value represents the degree to which the system is anomalous. Following that, the variational autoencoder is used to decode the time series data's correlation patterns. A convolutional long-short-term memory is used to capture the temporal patterns. us, the correlation between sensors and temporal information is sent to the Variational auto decoder for the reconstruction of the signature and residual matrices. e signature matrices are labeled S normal and S malicious , indicating which actions are considered normal and deleterious. us, the MTEDS was designed for anomaly detection and problem diagnosis. To evaluate the proposed MaVES system's performance, it is compared to three existing approaches: Multiscaling Convolutional Recurrent Encoder-Decoder (MSCRED), Autoregressive Moving Average (ARMA), and Long Short Term Medium-Encoder-Decoder (LSTM-ED). e proposed MTEDS framework is represented in Figure 1. Initially, the system generates multi-scale signature matrices. e input is fed into the encoding vector, and each dimension represents a learned attribute. For each dimension, the suggested encoder employs a single value. Similarly, the decoder recreates the input's original value. e variation autoencoder depicts the observation in latent space, with the properties of the observation being used to calculate the probability distribution. Normally, the latent state for the input vector creation is chosen at random during the decoding phase. It is the feasible range of values as output feed into the decoder in the proposed model. Each time series' temporal dependency is correlated, and the system is noise-resistant. e various components of the proposed MTEDS framework are herewith described below.

Signature Matrices.
e system status is characterized using different time series. Here for a multivariate time series of m − w to m, we construct n × n signature matrices S m . For a two-give time series as mentioned in e correlation S m ij € S m is calculated using Here the rescaling factor is k and k � w. e signature matrix is assumed to be robust to noise and it does not capture the shape similarities. e segment interval is assumed as 10 and the signature matrix for s � 3 is constructed for lengths w � 10, 30, and 60.

Variational Auto Encoder.
e variational autoencoder is a latent model that uses the regularized version of the autoencoder. e z, hidden codes enforce to draw samples using regular sampling. Here φ(p) is the deterministic function, and the recognition model t(z/p). e model employs a Gaussian disturbance over z with a neural network condition on p. e calculation is expressed in [21].
is decodes every point in the latent space and has a reasonable probability.

Convolutional LSTM.
e convolutional LSTM adapts hidden states across various time series. e feature maps p m,l from the lth layer of the convolutional neural network and H m− 1,l the hidden state belongs to R n1×n1×d1 . erefore, the calculations are expressed in where α i the previous weight is expressed in where vec() denotes the vector, k the rescaling factor. us the spatial patterns of the signature matrix are mapped with the temporal information at the convolutional layer.
Computational Intelligence and Neuroscience

Loss Function.
e loss function defines the reconstruction errors over the signature matrix and is expressed in where c € R n×n . e Adam optimization minimizes the loss function by employing stochastic gradient descent. e number of training epochs is sufficiently obtained the signature matrix is validated using the learned neural network. Based on the obtained residual matrix the anomaly is detected. Further, the performance of the proposed technique was evaluated using metrics such as anomaly score, noise factor, attention weight, anomaly count, precision, and recall.

Results and Discussion
is section describes the experimental setup, datasets, and various evaluation metrics.

Datasets.
e synthetic dataset of a power plant was used. For the dataset the time series was formulated as follows:

Experimental Setup.
e experiment was conducted on a PC with a 3.6 GHz CPU, 12 GB of RAM, and Windows 10 OS. All codes are implemented in python language respectively.

Performance Evaluation Criteria and Parameters.
e experiments were conducted to evaluate the various performance metrics such as anomaly score, noise factor, attention weight, anomaly count, precision, and recall. Finally, validate the proposed system against three existing approaches: Multiscaling Convolutional Recurrent Encoder-Decoder (MSCRED), Autoregressive Moving Average (ARMA), and Long Short Term Medium-Encoder-Decoder. Figures 2 and 3 depict the anomaly score results of the considered synthetic dataset. Multiple levels of system status in various periods must be characterized. Following that, given the signature matrices, a convolutional encoder is used to encode the intertime-series correlations based on the Client metric and changes, resulting in an alert and an increase in the anomaly score. e anomaly scores are used to assess the proposed MTEDS performance. e results in Figures 2 and 3 show that the suggested model is effective in terms of inter-sensor correlation and temporal patterns. e results reveal that the anomaly score is more consistent and detectable, with fewer false positives and negatives. e anomaly periods are shown in blue in Figure 4, and the cutting threshold is shown in white. e results demonstrate the efficacy of the proposed model. e matrix generated is shown in Figure 5 and Table 1. e matrices obtained from the residual and signature time series are validated based on the large variety of anomalies detected. Based on this, the reconstruction takes place, which is a latent vector where the predicted amount of the variation series is increased, and the reconstruction is enabled with the next set of matrices. e network weights are updated using backpropagation after defining a differentiable loss function. Further variational autoencoders reduce the reconstruction error between the network's input and output by minimizing the generative model parameters m simultaneously. With the use of residual matrices, the evaluation of the vector time series sequence is subdivided into several parts with the help of the residual matrices by this the differentiation of how the anomaly variates are determined. e distribution of attention weights over the previous timestamps is shown in Table 2 and Figure 6. Both the normal and abnormal attention weights are displayed separately. Even minor changes in the system status cause the attention to become more sensitive, which is useful for anomaly detection. e fourth and fifth-time steps, for instance, have an abnormal status of 0.22 and 0.23, respectively, whereas the first and second-time steps appear to be normal, and therefore the weights are assigned less than the normal segments.   Step of Conv Normal Abnormal      Computational Intelligence and Neuroscience Table 3 and Figure 7 illustrate the performance results of the proposed MTEDS in terms of noise factors, precision, and is anomaly. e noise factor found appears to be relatively low, ranging between 0.2 and 0.45. e high precision value of 1 can be found in the majority of timestamps. As a result, the proposed system's efficiency is demonstrated. Further, the efficiency of the proposed MTEDS is compared with the three existing approaches: Multiscaling Convolutional Recurrent Encoder-Decoder (MSCRED), Autoregressive Moving Average (ARMA), and Long Short Term Medium-Encoder-Decoder. Figure 8 shows the recall measure in comparison to the two previous systems, LSTM-ED and MSCRED. When     Table 4 and Figure 9. e proposed approach had a precision of 1 for a noise level of 0.2. e precision value decreases as the noise level rises. However, for noise factors of 0.25, 0.3, 0.35, and 0.4, the proposed system had greater precision of 1, 1, 0.98, and 0.95, respectively. When compared to the proposed, the MSCRED has a precision of 1, 0.90, and 0.378. e least efficient systems were LSTM-ED and ARMA, demonstrating the efficacy of the proposed system.

Conclusion
To detect anomalies in multivariate time series data, the proposed research study MTEDS was implemented. It creates multi-scale signature matrices to describe many levels of system status in various time steps. As a result, a radical loss function is defined to update network weights via backpropagation. Multiple levels of system status were assessed over time. e suggested MTEDS performance is evaluated using anomaly scores. According to the findings, the anomaly score is more stable and traceable, with fewer false positives and negatives. e proposed technique provides the highest precision of 1 for a noise level of 0.2, according to the results. As the noise level increases, the precision value lowers.
e proposed system, however, showed greater precision for noise factors of 0.25, 0.3, 0.35, and 0.4, proving its effectiveness. Although the proposed system performed better in terms of anomaly detection, the notion of anomaly would change as the dataset changes. It could be extended in the future to improve the method by applying it to different real-time datasets. Scaling and deploying the algorithm in complex operations might also be addressed while diagnosing the core cause of the problem. Other variables that could be examined for performance evaluation include latency and power analysis.

Data Availability
e data used to support the findings of this study are available on request.

Conflicts of Interest
e authors declare that they have no conflicts of interest. Computational Intelligence and Neuroscience 9