Resilient State Estimation for Control Systems Using Multiple Observers and Median Operation

This paper addresses the problem of state estimation for linear dynamic systems that is resilient againstmalicious attacks on sensors. By “resiliency” we mean the capability of correctly estimating the state despite external attacks. We propose a state estimation with a bank of observers combined through median operations and show that the proposed method is resilient in the sense that estimated states asymptotically converge to the true state despite attacks on sensors. In addition, the effect of sensor noise and process disturbance is also considered. For bounded sensor noise and process disturbance, the proposed method eliminates the effect of attack and achieves state estimation error within a bound proportional to those of sensor noise and disturbance. While existing methods are computationally heavy because online solution of nonconvex optimization is needed, the proposed approach is computationally efficient by using median operation in the place of the optimization. It should be pointed out that the proposed method requires the system states being observable with every sensor, which is not a necessary condition for the existing methods. From resilient system design point of view, however, this fact may not be critical because sensors can be chosen for resiliency in the design stage. The gained computational efficiency helps real-time implementation in practice.


Introduction
Feedback control systems resilient against malicious attacks have received increasing attention in recent years [1][2][3][4].This is because, combined with advances in computing and communications, feedback control systems now operate in a more connected manner with remotely located sensors, actuators, and other subsystems, which increase vulnerability of the systems compared to isolated ones in the past.The same trend is clearly seen in networked control systems [5,6] and Cyber-Physical Systems [7,8].In particular, for applications to critical infrastructures of our society [9], such as power grid [10], public transportation, and nuclear facility, the consequence of malfunction due to attacks may be disastrous.Malicious attacks on control systems of trams, power grids, water distribution systems, and sewage plant have occurred in reality as reported in [11][12][13].More potential attacks have been illustrated (see [1][2][3][4] and references therein for details).
In this paper, we develop a state estimation method for feedback control systems that is resilient against malicious attacks on sensors.Resilient state estimation is a method that can correctly estimate the true state of the system despite attacks on sensors.Such a method is sometimes referred to as secure state estimation.
The scenario considered here is the situation in which malicious attacks corrupt sensor outputs with the aim to degrade the control performance or fail the control systems.Such attacks, referred to as integrity attack [1], include the case where the sensors are physically destroyed and yielding false values or the case where the communication channels between sensors and controllers are compromised so that measurement values are intentionally altered.Altered sensor values can be arbitrary and no assumptions are made on their values or statistical properties.
Feedback systems under consideration are those with multiple sensors.First we consider the case where multiple sensors measure the same physical quantity redundantly and then we consider the case of multiple sensors measuring different physical quantities.The rationale is that the systems with multiple sensors can retain its functionality with a properly designed state estimation mechanism, despite the fact that some sensors, not all, are compromised.We assume that compromising more sensors at once requires more effort and resource for the adversaries.
Our approach is based on Luenberger state observers.Specifically, for redundant sensors that measure the same physical quantity, sensor outputs are combined through a median operation, which then feed to a state observer to estimate the state.For multiple sensors that measure different physical quantities, multiple observers are constructed first, and states estimates are combined through element-wise median operations.Analyses are provided for conditions under which resilient state estimation is guaranteed.Additionally, experimental results on a magnetic levitation system are also given to illustrate the efficacy of the proposed approach.
State observers have been used previously to detect faults in the systems [14].Most existing work designs an observer based scheme to generate residual signals that are used to detect faults.However, combining multiple state estimates using median operation in order to ensure resiliency has not been exploited to date.
Median operation has been used previously to ensure system tolerance to faults.For example, [15] designs a Guidance Navigation and Control (GNC) system where outputs from encoders, decoders, and data process units are combined through median operation to detect faults in the Data Processing Unit (DPU).Tripple Modular Redundancy (TMR) used in airline industry [16] executes voting based on AND-OR operation at logic level, which could be interpreted at selecting the median of the values from three computing units.However, it has not been used in the context of resilient state estimation where integrity attacks on the sensors are of the main concerns.
It should be acknowledged that seminal work of resilient state estimation is [17].Formulated in discrete time linear systems setting, the method in [17] accumulates sensor outputs for multiple sampling periods, and process state estimation using techniques developed in compressed sensing literature [18,19].This work has been extended to systems with uncertainty, noise, and disturbance [20].In [17], conditions for the correct estimation are given and an  0 optimization problem is formulated.Since solving  0 optimization online is computationally heavy (NP-hard), a relaxation condition on system parameters is given under which the solution of  0 optimization is identical to a relaxed  1 optimization.However, the relaxation condition narrows the class of the systems to which the method is applicable.
In an attempt to reduce computational effort, [21] approaches the problem of resilient state estimation using multiple observers.Contrast to the setting of [17,20], [21] formulates the problem in continuous time linear dynamic systems setting and combines the estimates from multiple observers using the technique from compressed sensing.This method reduces  0 optimization search space to a finite set leading to substantial reduction of computational effort from NP-hard to polynomial time.In addition, it is applicable to a large class of systems, compared to  1 optimization method in [17], whose states are observable from the sensors.
Adaptive parameter estimation methods with various nonlinear elements [22,23] may be used to solve resilient state estimation problem.When combining multiple observer outputs, especially, when each observes different number of states, adaptive fuzzy technique [24][25][26][27][28][29] can be utilized.These venues, however, have not yet been actively pursued.
The approach of current paper follows the setting of [21] and achieves computational complexity in the order of O() with  being the number of states and  being the number of sensors, under the assumption that the system states are observable from each sensor.
It should be pointed out that the proposed method requires the system states being observable with every sensor, which is not a necessary condition for the existing methods.From resilient system design point of view, however, this fact may not be critical because sensors can be chosen for resiliency in the design stage.On the other hand, the gained computational efficiency helps real-time implementation in practice.
The contributions of this paper are to propose multiple observers combined by median operation as a means to solve resilient state estimation problem and achive higher computational efficiency compared to existing methods for a class of systems.
The outline of this paper is as follows.The problem formulation is given in Section 2. Section 3 presents the main designs and analyses, and Section 4 provides experimental results.Comparison to existing methods is given in Section 4 as well in terms of applicability and computational effort.The conclusions are formulated in Section 5.

Problem Formulation
Consider a linear time invariant system given by where  ∈   is the plant state,  ∈   is control,  ∈   is the plant output,  ∈   is the measurement for feedback control,  ∈   is process disturbance,  ∈   is sensor noise, and  ∈   is a vector that represents the altered output value by external malicious attack.The matrices , , and  are in appropriate dimensions.Let the matrix  be written by where each   for  = 1, 2, . . .,  is a row vector that corresponds to the th output   of the output vector .The th sensor being under attack is described by th element of the vector (), denoted by   (), being nonzero, and the value of   () represents the amount of measurement altered by the external attack.
In order to denote the set of sensors under attack, we introduce the following notation.The support of the vector () is defined as and the cardinality of the set supp(()) is denoted by |supp(())|.The elements in the set supp(()) are the indices of the attacked sensors.We now introduce assumptions for the system of (1).
Assumption 1 states that strictly less than half of all the sensors in the system may be under integrity attack.This is a standard assumption for resilient state estimation [17,21] and in fact a necessary and sufficient condition for resilient state estimation problem to be solvable.The rationale is that the adversaries who attack the sensors have limited resource only enough to compromise a subset of the sensors.
This assumption ensures that a bank of  observers can be constructed.This assumption can be viewed as restrictive.However, from system design point of view, one can select sensors that satisfy Assumption 2. Assumption 3 states that the process disturbance and measurement noise are bounded.
We now formulate the following design problems.
It should be pointed out that unknown input observers (see, e.g., [30]), which address the problem of estimating states correctly despite unknown disturbances, may appear similar to Problems 4 and 5.However, the framework deals with unknown input entering the state dynamics instead of output equation, which differentiates Problems 4 and 5 from the problem of unknown input observers.
Another aspect that differentiates Problems 4 and 5 from existing work is that we seek a method of asymptotic estimation formulated in continuous dynamics, while [17,20] seek instantaneous estimation formulated in discrete dynamics.
The above formulated problems aim to achieve asymptotic state estimation and do not consider the effect of process disturbance and measurement noise.In practice, modeling errors, external process disturbance, and measurement noise exist.Hence, we formulate the following analysis problems.Problem 6.Let Assumptions 1, 2, and 3 hold.Analyze the effect of disturbance and measurement noise on the system of (1) and the state estimator of Problem 4.
Problem 7. Let Assumptions 1, 2, and 3 hold.Analyze the effect of disturbance and measurement noise on the system of (1) and the state estimator of Problem 5.
Solutions to Problems 4-7 are given in Section 3. We now examine the property of median operation in the context of the system of (1).Suppose there are  measurements denoted by   with  = 1, 2, . . ., , each measuring the same value denoted by  0 .Let   =  0 +   and   ̸ = 0 for  ∈ .We denote the cardinality of  by ; that is,  = ||.Then, it is straightforward to notice that, as long as the number of measurements  is greater than twice the number of elements in , or equivalent to say 2 < , the median value is equal to  0 ; that is,

Resilient State Estimation
Notice that the fact above holds regardless of the values of   () as long as 2 <  at any given time.Note also that (4) holds even if the elements of  change in time.As an illustration, an example is given.
For the case when measurement noise exists, we have the following property for the median.Let , , , and  be -dimensional vectors.The vector  is of the form  =  0 [1 1 ⋅ ⋅ ⋅ 1]  with  0 ∈ , the vector  represents noise, and as in Assumption 3, each element of the vector  is bounded by a constant  max ; that is, |  | ≤  max , the vector () satisfies |supp(())| =  with 2 < , and let the vector  be given Mathematical Problems in Engineering by  =  +  + .Then, we have the following for the sample median operation: med ( 1 , . . .,   ) −  0      ≤  max .
In words, this means when all the measurement is subject to bounded noise, sample median is also subject to noise, with the same bound as that for each element of the vector representing measurement noise.The derivation of ( 5) is in the appendix.

Design of
where the gain matrix  is chosen such that  −  0 is Hurwitz.Then it can be shown that, for the system of (1) with () = 0 and () = 0, the state observer (6) satisfies x() → () as  → ∞.In words, asymptotic state estimation is obtained by using (6).Specifically, since all the sensors measure the same output, we can denote this output by  0 where  0 =  0 .As explained earlier, under Assumption 1, med( 1 , . . .,   ) =  0 is obtained.Also, due to Assumption 2, the matrix  can always be chosen to render  −  0 Hurwitz.This ensures the state estimate x asymptotically converges to .
Therefore, the state observer of ( 6) is a solution to Problem 4. It ensures asymptotic state estimation despite external attack () as long as the number of attacked sensors is less than half of all the sensors (Assumption 1).We emphasize that this solution is computationally very efficient as the computational complexity of median operation of  variables is given by O().
Next we consider the case where not all  sensors measure the same physical quantities.As given in Assumption 2, the system states are observable from each sensor.For each sensor output   , one can design a Luenberger type observer that estimates the state  asymptotically.The state estimate from th sensor is denoted by   ∈   with a superscript .Then, Assumption 2 allows design of the observer, ż  =   +  +   (  −     ) ,  = 1, 2, . . ., , (7) where   can be selected such that ( −     ) is Hurwitz.By combining  state estimates  1 ,  2 , . . .,   through median operation, we can obtain a state estimate where For the method in ( 7)-( 9) to work, an additional assumption is needed.
Assumption 9.The set supp(()) does not change over time.
The additional assumption is needed to avoid the case that attacks excite the transients response of each observer in (7) in a manner that prevents x() from converging to ().With Assumption 9, it can be shown that, for the system of (1) with () = 0 and () = 0, the state estimation method given by ( 7)-( 9) achieves x() → () as  → ∞.This is possible because, under Assumption 1, more than half of  observers yield correct state estimates.Combining them through median would remove the effect of nonzero attack vector () and ensure asymptotic state estimate.Detailed derivation is given in the appendix.Therefore, the state estimation method of ( 7)-( 9) for the system (1) provides a solution to Problem 5. Note that the additional computational effort for resiliency in this case is O(), which is more scalable than NP-hard [17,20], or polynomial time of [21].

Effect of Measurement Noise and Process Disturbance.
Now we analyse the proposed state estimation method when measurement noise and process disturbances exist.In the presence of measurement noise and disturbance, asymptotic state estimation is generally not possible even without external attack.Hence, we focus on finding a bound on the estimation error.From a practical point of view, we deal with measurement noise and process disturbance that are bounded.Hence, Assumption 3 applies throughout this subsection.
First we consider the system of ( 1) with multiple sensors that measure the same physical quantity; that is,   =  0 for  = 1, 2, . . ., .Then, it can be shown that there exist some positive constants  and  such that the state estimation given by (6) yields Note that inequality (10) implies that the estimation error is bounded when bounded noise as well as process disturbance is present.Note moreover that the first term in (10) diminishes as time goes and the bound on the remaining term in the error is proportional to the bounds of the measurement noise  max and  max .This solves Problem 6 given in Section 2.
The derivation of ( 10) is given in the appendix.
For the case with the sensors measuring different physical quantities, resilient state estimation is achieved by the method given in (7)- (9).When measurement noise and process disturbance exist, the method of ( 7)-( 9) does not achieve asymptotic estimation.It turns out, however, that (10) holds for this case as well although the derivation now is more involved using  observers and element-wise median operation.Hence (10) solves Problem 7. The detailed derivation for this case is given in the appendix.
We would like to emphasize that the bound on estimation error does not depend on attack vector ().Attack can be arbitrarily large, but the effect is eliminated by resilient state estimator construction, and the bound on error only depends on the initial error, the bound  max of process disturbance, and the bound  max for the measurement noise.

Experiment
4.1.Modeling.The proposed methods of resilient state estimation are experimentally validated using a magnetic leviation control system.Figure 1 shows the magnetic levitation system developed by Quanser for control education purpose.It consists of electromagnet, infrared ray position sensor, a steel ball, voltage amplifier, ADC converter, and data acquisition system connected to a PC using USB cable.A control algorithm is implemented using real-time workshop in Matlab/Simulink.
The system model is given by [31] where  1 is the position of the ball,  2 is the velocity of the ball,  is the gravitational constant,  is the current applied to the electromagnet,   is the electromagnet force constant, and   is the metal ball mass.Values for parameters   and   are specified in [31].By linearizing the dynamics of ( 11) at the equilibrium point of  eq = [0.0060]  and  eq = 1, the following linear model is obtained: where Δ is used to indicate deviations from the equilibrium state  eq and input  eq .Quanser magnetic levitation system has only one sensor that measures the position of the steel ball.In order to apply the proposed state estimation method, we virtually create in Matlab an additional position sensor and a velocity sensor.Then, the system output equation including attack can be written as where It can be easily verified that the system of ( 12) and ( 13) satisfies Assumption 2.

Attack Scenario and State Estimation
Results.We construct resilient state estimator given in ( 7)-( 9).In the case of the magnetic levitation plant, the method yields three Luenberger observers as we have three sensors.Each observer dynamics is given by where  = 1, 2, 3 is the index for the th sensor,   is the th observer state, and   is the output of the th sensor.
Finally, the state estimate around the equilibrium is obtained by x = Δx +  eq .We consider the scenario where the velocity sensor, which provides the third measurement, is compromised by adversaries.The attack on the sensor,  3 (), consists of constant, ramp, sinusoid, and square waves as shown in Figure 2.
The estimated state x and true state  are shown in Figure 3.As expected, x is practically identical with  despite the attack on the velocity sensor.Slight mismatches between the two are due to modeling uncertainty which act as if they were disturbance.
For further investigation, Figure 4 shows state estimates  1 ,  2 , and  3 from the three observers.The effect of attack is clearly present in  3 .As shown by the analysis in Section 3,  the element-wise median operation removes the effect of  3 on x.
As illustrated by the above experiments, the proposed state estimation method is resilient against external attacks on the measurement.

Comparison with Existing Methods.
Here we consider the method of [17] with the magnetic levitation system.It is difficult to apply the method of [17] on the magnetic levitation system for two reasons.First, the exact optimization using  0 norm is computationally expensive (NP-hard) and no efficient method is known for  0 optimization.Second, the relaxation condition in [17] for enabling  1 convex optimization is not satisfied for the magnetic levitation system.Hence we do not implement and compare the method of [17] in the context of experiment with magnetic levitation.
The proposed method and that of [21] are compared in the following manner.From the above experiment, data from the sensors are stored.Then, two state estimation algorithms coded in Matlab m-file are executed on the stored sensor data, respectively.In this way, the execution times for the two algorithms alone (separated from the computation needed for control and communications) can be measured and compared.
We compared the two for the cases of 3, 5, 7, and 9 sensors.The cases of 5, 7, and 9 sensors use duplicated data from the first sensor for the sake of simplicity.The sensor data is collected over 58001 samples, and the time for 58001 executions of each algorithm is measured to obtain average value.Each Matlab code is executed on a computer with Intel i7-4790 CPU, 3.60 GHz clock speed, 32 GB RAM, and 64-bit Windows operating system.Both algorithms correctly estimate the true states despite attacks, although no plots are shown as our main interest here is the computational efficiency.Average execution time for the two algorithms is listed in Table 1.
Clearly, the proposed method is superior to the method in [21] in terms of computational effort, showing smaller computation time by orders of magnitude.We point out that method of [21] is superior to the proposed method in terms of applicability: the condition of systems states being observable from every sensor is not necessary for [21].

Conclusion
This paper addresses the problem of resilient state estimation against malicious attacks on the sensors.We propose a state estimation with a bank of observers combined through median operations.Then, we show that this method is resilient in the sense that state estimation converges to the true state despite existence of attacks on sensors.For practical considerations, the effect of sensor noise and process disturbance on the proposed state estimation is analyzed.
We point out that the proposed method requires the system states being observable with every sensor, which is not required for the existing methods.This may not be a critical limitation because sensors can be chosen in the system design stage in applications where resiliency is of importance.
We emphasize that the proposed method is computationally efficient compared to existing methods in the literature, yielding the complexity of O() with  being the number of system states and  being the number of sensors.The gained computational efficiency helps real-time implementation for feedback systems in practice.Due to the simplicity of the state estimator structure and computational advantage over the existing method, the proposed method will benefit the design of resilient control systems.
Developing resilient state estimation methods using adaptive parameter estimation techniques is a future work.

Figure 2 :
Figure 2: Attack signal on the third sensor.

Figure 3 :
Figure 3: The estimated state and the true state.

Figure 4 :
Figure 4: The states estimates from each Luenberger observer.
Resilient State Observer.Now we propose a solution to Problem 4. Since all  sensors are measuring the same physical quantity, that is,   =  0 , for 1 ≤  ≤ , we construct a Luenberger state observer in the following manner: The observer gain matrix   is selected such that ( −     ) is Hurwitz for all  = 1, 2, 3. Specifically, the gains are  1 = [200 13271]  ,  2 = [210 14299]  , and  3 = [4.0584200]  , respectively.Then, Δx is computed by

Table 1 :
[21]utation time comparison between the proposed method and the method in[21].