Postprocessing of Accidental Scenarios by Semi-Supervised Self-Organizing Maps

. Integrated Deterministic and Probabilistic Safety Analysis (IDPSA) of dynamic systems calls for the development of efficient methods for accidental scenarios generation. The necessary consideration of failure events timing and sequencing along the scenarios requires the number of scenarios to be generated to increase with respect to conventional PSA. Consequently, their postprocessing for retrieving safety relevant information regarding the system behavior is challenged because of the large amount of generated scenarios that makes the computational cost for scenario postprocessing enormous and the retrieved information difficult to interpret. In the context of IDPSA, the interpretation consists in the classification of the generated scenarios as safe, failed, Near Misses (NMs), and Prime Implicants (PIs). To address this issue, in this paper we propose the use of an ensemble of Semi-Supervised Self-Organizing Maps (SSSOMs) whose outcomes are combined by a locally weighted aggregation according to two strategies: a locally weighted aggregation and a decision tree based aggregation. In the former, we resort to the Local Fusion (LF) principle for accounting the classification reliability of the different SSSOM classifiers, whereas in the latter we build a classification scheme to select the appropriate classifier (or ensemble of classifiers), for the type of scenario to be classified. The two strategies are applied for the postprocessing of the accidental scenarios of a dynamic U-Tube Steam Generator (UTSG).


Introduction
The number of dynamic scenarios considered in an Integrated Deterministic and Probabilistic Safety Analysis (IDPSA) of dynamic systems increases with the number of failure events that can occur and the consideration of their timing and sequencing.This can make the computational cost for scenario postprocessing enormous and the retrieved information difficult to interpret [1][2][3][4].The main goal of postprocessing is the classification of the dynamic scenarios generated as safe, failed, Near Misses (NM), and Prime Implicants (PIs) clusters.Safe scenarios are those that, even if several components failures are included, keep the system working in safe conditions.Failed scenarios, instead, result from a combination of failure events that lead the system into a failed condition.Among failed scenarios, PIs are those scenarios containing events representing the minimal combinations of component failure necessary for system failure [5] (i.e., the dynamic systems equivalent of Minimal Cut Sets (MCSs)).Among safe scenarios, NMs are dangerous sequences of events that lead the system to a quasi-fault state [6].
Many methods have been proposed in literature for the classification task.A first step could be distinguishing failed scenarios from safe scenarios, for example, by a fuzzy--means (FCM) classifier [6], a Mean-Shift Methodology (MSM) [7], or a decision tree [8].Methods have been proposed for the identification also of PIs and Near Misses.For example, PIs identification has been performed with a differential evolution-based method [9] or a visual interactive method [10], where the number of components whose behavior is specified in the accident sequence is selected as most important feature for the PIs identification: the accident sequences associated with the lowest literal cost are selected and stored as PIs (most reduced sequences, i.e., with least number of events, that cannot be covered by any other implicant).Regarding the identification of the Near Misses 2 Science and Technology of Nuclear Installations sequences, an unsupervised clustering problem based on an optimized wrapper algorithm and the -means clustering algorithm has been proposed (MacQueen, 1967) [4].A comprehensive method for accidental scenarios classification can be provided by Self-Organizing Maps (SOMs) [11], which have been widely used in various engineering and physical applications, including fault detection and diagnosis in complex systems [12,13].SOMs capture nonlinear relationships of high-dimensional data and visualize them on a lowdimensional interface, normally a 2D structure of, so-called, neurons.In this structure, data are assigned to the most similar neuron called Best Matching Unit (BMU) (usually by measuring the smallest Euclidean distance), so that the available data are divided into regions with common characteristics (i.e., data with high similarity to the same BMU are mapped close to each other).Three kinds of SOM exist: the Unsupervised SOM (USOM), the Semi-Supervised SOM (SSSOM), and the Supervised SOM (SSOM).We have shown in [14] a SSSOM performs best in identifying safe, failed, NMs, and PIs groups of scenarios.In particular, assigning the set of discrete variables (i.e., the failure sequences) to a BMU, a SSSOM (implemented with a Manhattan distance as similarity measure) is particularly suitable to properly treat the MVL approximation needed for the representation of the dynamic scenarios (the usual binary variables representation used in Boolean Logic, in which the modeling is limited only to the occurrence or not of certain events [2-4, 6, 9, 10, 15], is not sufficient).In this work, it will be shown that the SSSOM performance in classifying different groups of scenarios depends on the feature of the SSSOM that is used as discriminating characteristics for choosing the BMU (e.g., assigning the data to the cluster with the geometric barycenter more similar to the input data or to the cluster with the maximum (minimum) neuron (i.e., with the maximum (minimum) weights) more similar to the input data).The results confirm that depending on this, some classifiers overperform the stand-alone SSSOM for some classes and vice versa.This suggests adopting an ensemble approach for an improved classification of accidental scenarios.
The main objective of this work is to propose a postprocessing tool for dynamic accidental scenarios, which exploits an ensemble of classifiers.In fact, by doing so, it is possible to leverage the classifiers complementary characteristics and to boost overall classification accuracy (in terms of the multiobjective precision sensitivity and specificity) [16].In general, strategies for boosting diversity include (i) using different types of classifiers (this is the technique we adopt for our application); (ii) training individual classifiers with different data sets; (iii) using different subsets of features.Various methodologies exist for aggregation of the outcomes of individual classifiers: majority vote [17], Borda count [18], threshold voting [Ho 1994], weighted average [19], fuzzy integral [20], fuzzy templates [21], and Dempster-Shafer theory [22].Furthermore, methods have been developed to dynamically select a classifier from the set of available ones, based on local information [23]: different classifiers perform best in different regions and this aggregation can lead to improving classification results; in a supervised setting, the individual classifier performance can be calibrated based on historical data with known target values; each individual classifier performance value reflects the degree to which we want each classifier to contribute in the ensemble aggregation: the best performing classifier for a given scenario type should contribute most [16].On these premises, we propose two alternative strategies based on locally weighted aggregation of SSSOMs outcomes: a locally weighted ensemble and a decision tree based on an ensemble.For both strategies, we resort to the Local Fusion (LF) principle [19] for building the ensemble outcome, based on each classifier local performance, measured by the classification accuracy on scenarios in the neighborhood of (i.e., similar to) the test scenario considered.
In the locally weighted ensemble strategy, we ensemble the classification outcomes of the SSSOMs whose assignments to a BMU are given with respect to the different features characterizing the SSSOM (e.g., the Mean Quantization Error (MQE) based SSSOM, the barycenter based SSSOM, the minimum neuron based SSSOM, the maximum neuron based SSSOM, and the stand-alone SSSOM).
Differently, the decision tree based classification scheme chooses one single classifier (or ensemble of classifiers), based on the local performance of the test scenario.In this way, input scenarios with similar characteristics are treated by the same classifier (or ensemble of classifiers) for which the branch of the tree is most effective.
The feasibility of combining local information for postprocessing IDPSA scenarios for their classification into safe, failed, NMs, and PIs, is demonstrated with respect to a dynamic U-Tube Steam Generator (UTSG) of a NPP [24].For IDPSA scenarios generation, a dynamic simulation model has been implemented in SIMULINK and a multivalued logic (MVL) scheme [4] has been adopted for describing the different component operational states in the scenarios.
The paper is organized as follows.In Section 2, the UTSG and its SIMULINK model are presented.In Section 3, the SSSOMs are presented and different features are considered as discriminating characteristics for the classification; also the LF process for ensembling is outlined.In Section 4, the locally weighted ensemble of SSSOMs and the decision tree based ensemble of SSSOMs are presented, and the results on the case study considered are reported.In Section 5, some conclusions and final remarks are given.

Case Study
We consider the dynamic scenarios of a UTSG used in nuclear power plants for scenarios generation.A SIMULINK model has been used to describe the UTSG response at different power levels  0 [4].The component failures considered for UTSG are as follows (Figure 1): the steam valve failure, the safety relief valve failure, the interruption of the communication between the sensor that monitors the water level (governed by the balance between the incoming and exiting feed water) and the Proportional Integrative Derivative (PID) controller, and the PID failure.A mission time ( miss ) of 4000 (s) has been considered for allowing complete development also of slow dynamic accident scenarios occurring at early/medium times.The component failures are considered occurring at any continuous time instant, with any order in the sequence and magnitude.Assumptions on the failure occurrence process have been made in order to (i) favor the occurrence of multiple failures in the scenarios, (ii) capture the dynamic influence of all factors of interest, and (iii) treat a comprehensive (but still manageable) problem for which scenarios postprocessing is required for a robust risk quantification.
For the tractability of the problem, we resort to a multivalued logic (MVL) computational framework in which the components can fail at discrete times and magnitudes [9].The discretization consists in the following: All possible combinations of multiple component failures, each represented by a MVL vector of time, magnitude, and order of occurrence, lead to a total of  = 100509 accidental MVL scenarios to be treated for the quantification of the risk related to the UTSG operation.

The Ensemble
The design of a successful ensemble consists of two important parts [25,26]: (1) the design of the individual classifiers (Section 3.1); (2) the design of the aggregation mechanism (Section 3.2) [27].

Design of Classifiers.
For postprocessing the  = 100509 multivalued dynamic scenarios of the UTSG, we resort to a Semi-Supervised Self-Organizing Map (SSSOM) based on the Manhattan distance (shown in Figure 2(c)).This SSSOM has been shown in [14] to be efficient for grouping the scenarios in four distinct regions of the map and retrieving safety relevant information, and it is hereafter shown to be capable of further improvement when trained to classify new data based on different features of this same SSSOM to be used as BMUs and, then, their outcomes are ensembled into the final classification: we shall see that certain classifiers overperform the others for certain classes and vice versa.Specifically, we build  = 5 classifiers the stand-alone SSSOM, the MQE based SSSOM, the barycenter based SSSOM, the minimum neuron based SSSOM, and the maximum neuron based SSSOM and show how, for different classes, none of these is the best and all would mutually benefit from each other, namely.

The Stand-Alone
where ‖ ⋅ ‖ is the absolute value of the difference between the two vectors along the -dimension [14].By doing so, the MVL formalism is accommodated within the similarity assessment between data vector and neurons.The map of

The MQE Based SSSOM.
Let us consider a generic scenario   and a generic neuron of the map   .As stated in the Appendix (A.5), a commonly used quality measures that can be used to determine the performance of the map is the MQE and it can be defined as in the following equation.
where   are the weights associated with the BMU neuron   .
Basically, the lower the MQE of the BMU is, the more the scenario features vector is similar to its weight vector and, thus, the more the knowledge is learnt by the SSSOM.Computing the MQE for each input data and grouping them classwise, we can obtain the empirical probability density functions (PDF) referring to the distribution of the MQE for each class (Figures 2(b), 2(a), 2(e), and 2(d)).Equation (3) shows an example of computation of the MQE for a generic class : where   is the number of scenarios belonging to the class ,    is a generic scenario belonging to the class , and   is the weight vector of the BMU neuron in the map to which    is assigned.The classification of a new input to a  class with the MQE based SSSOM proceeds as follows: its MQE  is calculated as in (2) and, then, it is assigned to the class with the larger PDF value for the calculated MQE.The rationale is that, for a particular value of MQE, the larger the PDF, the more probable the value: if for a class , the PDF associated with a MQE value is larger than for the other classes, it is more probable that the scenario belongs to that class.For example, if the MQE of an input is equal to 1.5, we assign it to the safe class because the PDF of the safe class (Figure 2 2(c) is exploited as an alternative classifier by using the geometric barycenter of each cluster as a reference for the choice of the BMU (circles in Figure 2).When a new   is fed to this SSSOM, we select the closest of the four barycenter neurons as the most similar neuron, where similarity is quantified based on the Manhattan distance: where   bar is anyone of the four barycenters of the four classes.The rationale is that the geometric barycenter is most representative of the characteristics of the class.

The Minimum Neuron Based SSSOM.
Considering again the SSSOM trained as in the Appendix and shown in Figure 2(c) for each cluster  we locate on the map the neuron with the minimum weight   min (represented in the map with a triangle in the map of Figure 2) and for the classification we assign the new vector   to the cluster with the minimum neuron most similar to the considered input, based on the Manhattan distance: The rationale is that if the vector of the features of a scenario is similar to that of neuron with the minimum weight of a specific cluster, it will be assigned to this cluster because it is very different to the neurons with minimum weight vectors of the other classes.

The Maximum Neuron
Based SSSOM.The maximum neuron based SSSOM is complementary to the previous one in that it is based on the neuron with the maximum weights for each cluster   max represented by a rectangle in the map of Figure 2(c).
3.1.6.Classification Performance.The four classifiers of Sections 3.1.2-3.1.5are compared to the stand-alone SSSOM of [14], on the UTSG scenario postprocessing task.The performances of the classifiers are quantified by the calculation of [28]: (i) Precision: the larger, the better the capability of the th classifier to not include samples of other classes in the considered th class: where    is the total number of scenarios assigned to the th class and   is the number of scenarios belonging to class  and correctly assigned to class .(ii) Sensitivity: the larger, the better the capability of the th classifier to correctly recognize samples belonging to the th class: where   is the total number of scenarios belonging to the th class.
(iii) Specificity: the larger, the better the capability of each th class of the th classifier to reject the samples of all the others: where    is the total number of samples assigned to the th class: In Tables 1-5, the performances for the MQE based, the barycenter based, the minimum neuron based, the maximum neuron based, and the stand-alone SSSOMs, for each class, are reported.
For failed and PIs scenarios, the parameters of the standalone SSSOM (precision of 0.83 and 0.016, sensitivity of 0.773 and 0.911, and specificity of 0.911 and 0.949, respectively) are larger than for all the other classifiers.For example, for the minimum neuron based SSSOM, the precision for failed scenarios is equal to 0.5418, which is much lower than the precision obtained with the stand-alone SSSOM.It is worth noticing that this is always true for all the parameters values when dealing with failed and PIs scenarios.On the contrary, looking at the NMs and safe scenarios, we see that the other classifiers overcome the stand-alone SSSOM performances.For example, the specificity in classifying safe scenarios is higher for the maximum neuron based SSSOM than for the stand-alone SSSOM (0.955 versus 0.927), and the precision in classifying NMs is higher for the MQE based SSSOM than for the stand-alone SSSOM (0.0406 versus 0.034), and also the specificity in classifying NMs for both the MQE based SSSSOM (0.9616) and the minimum neuron based SSSOM (0.9902) is higher than the stand-alone SSSOM (0.911).
In Figure 3 Figure 3(a) confirms that the stand-alone SSSOM, on average, overperforms the other classifiers, except for safe and NMs classes: for these scenarios in Figures 3(b) and 3(d), respectively, a Pareto front can be identified and highlighted with a solid line for the suboptimal solutions of classifiers that do not dominate all the others with respect to all the three performance objectives.For example, we can see that the precision in classifying the NMs is higher for the MQE based SSSOM (0.0406) than for the stand-alone SSSOM (0.034) and so is the specificity in classifying NMs (0.9616 versus 0.911), but the sensitivity for the same class is higher for the standalone SSSOM (0.957 versus 0.491).
These results suggest the possibility of developing a general method for aggregating the multiple classifiers outputs considered as an ensemble, whose aggregation mechanism (as proposed in the following sections) would consider the local performances of the different classifiers in dealing with the different types of scenarios and automatically selecting the classifier to be used.
As a last remark, it is worth clarifying that the proposed ensemble method is designed such that it could be applied to other cases than the UTSG here presented.Therefore, even if, under this circumstances, one might argue it would be more convenient to rely on the stand-alone SSSOM only when dealing with failed and PIs scenarios and on the ensemble for the other classes of scenarios because (i) it is always true that the stand-alone SSSOM overcomes the other classifiers when dealing with failed and PIs scenarios and (ii) it is not when dealing with NMs and safe scenarios, not to limit the generality of the proposed ensemble method, we proceed by devising an automatic aggregation mechanism, rather than case based.

Design of the Aggregation Mechanism: The Locally Weighted Fusion. Let 𝑤 𝑘
be the weight that classifier  carries in assigning scenario   to a class of a dataset of  scenarios to be classified: where the Mean Error (ME) me    is the error that classifier  makes in classifying the scenario   , defined as and    is the error that the classifier  commits in classifying the th scenario whose real class is   (with  = 1, . . ., ).In this work, the error    is computed in two different ways: being ŷ  = 1, . . .,  the class the th classifier assigns to   ; the first way for computing    (1) is given in the following equation: where the error is null if the estimated class ŷ  is the same as the real class   (where ŷ  = 1 and   = 1 means that the estimated and real class of the scenario are safe, respectively, ŷ  =   = 2 means failed, ŷ  =   = 3 means NMs, and ŷ  =   = 4 means PIs), whereas in the second way    (2) is calculated as the Manhattan distance between the real and the predicted class by the following equation: Usually the error is computed by relying on a subset of , called neighbor set of scenarios to   and defined as in (10): where In this way, the th classifier performance is expected to be similar to the one that would be obtained with a new (unknown) scenario.A weight     can, thus, be associated with each of the individual  classifiers of the ensemble depending on its performance, as it will be shown in the next section.

The Proposed Ensemble Strategies
In the following, we describe the details of the implemented ensemble strategies, namely, the locally weighted ensemble of SSSOMs and the decision tree based on an ensemble of SSSOMs.These approaches rely on the five classifiers introduced in Sections 3.1.1-3.1.5(the stand-alone SSSOM, the MQE based SSSOM, the barycenter based SSSOM, the minimum neuron based SSSOM, and the maximum neuron based SSSOM, respectively), whose classification outcomes are combined into two different ways, as we shall see in what follows.

Locally Weighted Ensemble of SSSOMs.
For the locally weighted ensemble of SSSOMs, we directly apply the algorithm of the neighborhood based approach, as described in Section 3.2 to the  = 100509 dynamic scenarios.For each scenario to be classified we retrieve the 100 neighbors based on the considerations made before: relying on the neighborhood of each scenario we compute the classification errors (with both ( 12) and ( 13)) and, through the errors, also the weights associated.The classification outcomes of the five different trained SSSOMs are ensembled and the assignment to a class is given accounting for the different performances of these classifiers when assigning the weight (the larger the number of neighbors of the input scenario correctly classified, the lower the error, and the larger the weight and the reliability for the th classifier).For the computation of the weight associated with each classifier  for each scenario, thus, we resort ( 10) and (11) where, for the th generic scenario and the th classifier, we calculate where   is one of the  = 100509 dynamic scenarios,     is the weight associated with this scenario, and me    is the ME associated with this scenario and computed as in (13): where    = 100 and    is the classification error.Once the weights are computed for all the  = 5 classifier, the input data   is assigned to the class with the larger weight  = arg((max  ( ))), because this is the most reliable classifier for the th vector.

Training of the Locally Weighted Ensemble of SSSOMs.
Table 6 shows the classification results for all scenarios and for those of NMs and PIs classes.The rows of the table report the results obtained when ( 12) and ( 13) are used for computing the classification error.We can see that, in both cases, the total number of correctly assigned scenarios (irrespectively of being safe, failed, NMs, and PIs) exploiting the locally weighted ensemble of SSSOMs increases with respect to the stand-alone SSSOM (whose results are reported in Table 7): this latter, in fact, scores a total amount of 78288 rightly assigned scenarios [14], while with the locally weighted ensemble of SSSOMs, we achieve 84141 overall correct assignments when the error is given by ( 12) and 81512 overall correct assignments when we resort to the Manhattan distance of (13) for computing the error.
If we, instead, focus on NMs and PIs classes, both ensembles are penalized with respect to NMs and PIs classification (second and third column): the stand-alone SSSOM correctly assigns 318 out of 332 NMs and 82 out of 90 PIs (as reported in Table 7).It is worth pointing out that, even if the ensembles do not correctly classify all NMs and PIs scenarios, we can consider these results satisfactory for the operational risk quantification which the classification is aiming at contributing to (i.e., the consequences of the scenario occurring and to its probability of occurrence): as already said, PIs normally are made of many component failures but because of this also have low probability of occurrence and, thus, the risk that is not accounted for due to the misclassification of PI is very low, whereas for the NMs those scenarios that are not correctly classified are classified as either safe (with no extra risk quantification being both safe and NMs leading to safe states) or failed scenarios (with a conservative overestimation of the system operational risk).Table 8 reports the same results in terms of percentage of correct assignment.
Looking at the two locally weighted ensembles, we can say that the one based on the Manhattan distance is more effective in the assignment of NMs and PIs than the other: we see in fact that the percentage of correctly assigned NMs is 31.33%when ( 12) is used and 92.77% when (13) is used, whereas for PIs the percentage increases from 73.33% to 85.56% when the Manhattan distance is used.Even if (12) is used, the percentage of correct assignment is larger than when the Manhattan distance is used (83.71%versus 81.1%), since NMs and PIs are the most safety relevant classes and, thus, are those we have to guarantee to be better classified during the postprocessing of dynamic scenarios.
Furthermore, Tables 9 and 10 list the precision, sensitivity, and specificity values for the two ensembles for all the four Locally weighted ensemble by using ( 13) 1599 8 11 classes.The best performances are obtained by using ( 13): the precision is larger using ( 13) than ( 12) for all the four classes and what we gain in terms of sensitivity in classifying NMs and PIs scenarios and specificity in classifying safe and failed scenarios justifies a negligible loss in terms of sensitivity in classifying safe and failed scenarios and specificity in classifying NMs and PIs scenarios.In fact, (i) the specificity for NMs and PIs decreases (from 0.9704 to 0.9327 and from 0.9653 to 0.9615, respectively); (ii) the sensitivity for safe and failed scenarios decreases (from 0.8453 to 0.8266 and from 0.8277 to 0.782, respectively).
Using (13), we gain a consistent benefit; namely, (i) the sensitivity for NMs and PIs increases (from 0.3133 to 0.9277 and from 0.7333 to 0.8556, respectively); (ii) the specificity for safe and failed scenarios increases (from 0.9124 to 0.9216 and from 0.8958 to 0.9142, respectively).
In conclusion, it is possible to assert that the approach based on (13) leads to superior results of classification.

Test of the Locally Weighted Ensemble of SSSOMs.
We test the locally weighted ensemble of SSSOMs approach with a set of scenarios in which the time is not discretized anymore, but it is continuous.A new set of input data  test of 2000 scenarios have been generated, in which components can fail randomly between 0 and the mission time of 4000 (s).
Then, the trained classifiers are used to classify  test .In Table 11, the results of the test conducted on the locally weighted ensemble of SSSOMs are reported.Within the set of the 2000 input data, there are 8 NMs and 11 PIs.We can see in Table 11 that the test classification results confirm the considerations made for the training.The ensemble based on the Manhattan distance is more efficient in the assignment of NMs and PIs than the other, even if the total correct assignment is larger for (12) than for (13): using (9) all the NMs and PIs scenarios are correctly classified, whereas only 2 NMs and 9 PIs are assigned to the right class, if (12) is used.

Decision Tree Based on an Ensemble of SSSOMs.
The decision tree based classification scheme chooses one classifier (or ensemble of classifiers), depending on its local performance of the test scenario considered (see Figure 4).In this way, a test scenario with similar characteristics is treated  with the same classifier (or ensemble of classifiers) for which the branch is most effective.A decision tree [29] is defined as a classification procedure that recursively partitions the scenarios into smaller subdivisions on the basis of a set of rules defined at each branch.The tree is composed of a root node (formed from all the input scenarios), a set of internal nodes (splits), and a set of terminal nodes (leaves), in which the scenarios are divided per groups with common characteristics.In this framework, the scenarios are classified by sequentially subdividing them according to the decision framework defined by the tree, and a class label is assigned to each scenario according to the leaf node into which the scenario falls.
In the following, we describe the decision rules used for the construction of the tree structure: (i) If the stand-alone SSSOM classifies the scenario as failed or PI, then we accept such classification.The rationale is that as shown before in Figure 3 and Tables 1-5, the stand-alone SSSOM is the best classifier of failed and PI scenarios.
If the stand-alone SSSOM classifies the scenario as safe or NM, we consider multiple classifiers, as the three objectives (precision, sensitivity, and specificity) show a Pareto front where two (or three) classifiers can provide equally plausible classification results (see Tables 1-5).In this case, we aggregate the classification outcomes of the classifiers on the Pareto front, weighting them proportionally to the inverse distance between the point in the space of the three objectives for the specific classifier and the optimum, represented by the point [1, 1, 1] in the same space: the lower the distance, the larger the associated weight.
If the stand-alone SSSOM classifies the scenario as safe or NM (i) the scenario is assigned to the class given by the standalone SSSOM with a distance < 0.1; (ii) if the stand-alone distance is larger than 0.1, we resort to the locally weighted ensemble of SSSOMs (Section 4.1).It is worth mentioning that the threshold distance is chosen equal to 0.1 (i.e., a reasonably low error), because in such way we would rely on the stand-alone SSSOM (for safe or NMs) only if the assignment can be done with large confidence (otherwise, we resort to the locally weighted ensemble of SSSOM).

Training of Decision Tree
Based on an Ensemble of SSSOMs.Table 12 shows the classification results for all scenarios and for those of NMs and PIs classes.As for the locally weighted ensemble of SSSOMs, we focus, in particular, on these two classes, because these are the two more relevant for quantifying the operational risk of the system.Both the decision trees based on ( 12) and ( 13) overperform the classification of the stand-alone SSSOM (whose results are reported in Table 7): the approach that uses (12) scores 82520 correctly assigned scenarios, whereas the approach based on (13) scores 80192 correct assignments.As for the locally weighted ensemble, both classifications are penalized with respect to NMs if compared with the stand-alone SSSOM, because the first approach based on (12) correctly classifies 104 NMs, whereas the approach that uses (13) correctly classifies 306 NMs.The number of correctly classified PIs corresponds to the number of the stand-alone SSSOM classified rightly, because in the classification algorithm of the decision tree, the classification of the PIs leans on the stand-alone SSSOM only.Table 13 reports the classification result of the decision tree in terms of percentages.
Looking at the two decision trees, the one based on the Manhattan distance is more effective in the classification of NMs than the other: we see in fact that the percentage of correctly assigned NMs is 31.33%when ( 12) is used and 92.17% when (13) is used.The PIs have the same percentage of correctly classified scenarios because they are assigned with the same classifier (the stand-alone SSSOM).Using (12), the percentage of correct assignment is larger than when the Manhattan distance is used (82.1% versus 79.79%), but we know that NMs and PIs are the most safety relevant classes and, thus, are those we have to guarantee to be better classify during the postprocessing of dynamic scenarios.
Tables 14 and 15 list the precision, sensitivity, and specificity values for the two decision trees for all the four classes.The best performances are obtained by using (13): the precision is larger using (13) than (12) for all the four classes and, as for the locally weighted ensembles, what we lose in terms of sensitivity in classifying safe and failed scenarios and specificity in classifying NMs is justified by the growth in terms of sensitivity in classifying NMs and specificity in classifying safe and failed scenarios.In fact, (i) the specificity for NMs decreases (from 0.9707 to 0.934); (ii) the sensitivity for safe and failed scenarios decreases (from 0.8216 to 0.8058 and from 0.8275 to 0.7824, respectively).
Using ( 13), (i) sensitivity for the NMs increases (from 0.3133 to 0.9217), (ii) specificity for the safe and failed scenarios increases (from 0.9197 to 0.9251 and from 0.891 to 0.9083, respectively).
Also in this case we can say that the Manhattan approach for the computation of the classification error (as in ( 13)) is preferable.

Test of Decision Tree Based on an Ensemble of SSSOMs.
We test the decision tree based on an ensemble of SSSOMs approach with the same set of scenarios in which the time is continuous, as in Section 4.1.2.Table 16 shows the classification results.In this case, the two approaches (using ( 12) and ( 13)) lead to the same results of classification of NMs (as regards PIs, it is obvious that the classification leads to the same number of correctly classified scenarios, because only the stand-alone SSSOM is used for the class assignment): this can be due to the small number of scenarios used for the test phase, if compared with the total number of scenarios ( = 100509).Similarly to other cases, the total number of correctly classified vectors is greater using (12) that scores 1544 correctly classified scenarios, than using (13) which obtains 1537 right classifications.But since the NMs and the PIs are the two classes for which we need to guarantee a good classification for the quantification of the operational risk of the system and considering the results of the training phase, we can say that the approach that uses (13) is better than the approach based on (12).
Both the locally weighted ensemble of SSSOMs and the decision tree based on an ensemble of SSSOMs show better performances when the classification error is computed with (13), rather than with (12).This is due to the fact that (12) smooths down the large errors of the classifier (e.g., using (12) an error computed for a safe scenario (labeled with 1) misclassified as NMs (labeled with 2) is equal to an error computed for a safe scenario (labeled with 1) assigned to PIs class (labeled with 4)).
In conclusion, we can assert that, between the decision tree based on an ensemble of SSSOMs and the locally weighted ensemble of SSSOMs (both based on (13)), the best compromise choice falls on the former approach: indeed, it can guarantee a very large overall correct assignment rate (80192 out of 100509), a large number of NMs correctly classified (306 out of 332 real NMs), and the largest number of PIs assigned to the right class (82 out of 90 real PIs), whereas the weighted ensemble of SSSOMs correctly classifies only 77 real PIs.

Conclusions
The postprocessing of IDPSA accidental scenarios of a dynamic system is a fundamental task for retrieving safety relevant information for the system operation and maintenance.In practice the task can be challenged by the combinatorial explosion of the scenarios generated due to the dynamic dependence of components failure events and the consideration of timing and magnitudes of failure events in the accidental scenarios generation.
In this paper, for UTSG scenario generation, a SIMULINK dynamic simulation model has been used, within a MVL scheme that describes the different component operational states, and has presented two alternatives strategies of scenario classification (i.e., the locally weighed ensemble of SSSOMs and the decision tree based on an ensemble of SSSOMs) with the twofold purpose of (1) leveraging the classifiers complementary characteristics and (2) boosting overall classification accuracy.In general terms, it has been shown (in Section 3.1.6)that ensemble approach can benefit from each independent classifier by capitalizing (within either a locally weighted strategy or a decision tree based strategy) their complementary characteristics.The methodology highlights the need of taking into account different classifiers to recover information that would have been lost if neglected.Examples have been provided with respect to the capability of the ensemble of SSSOM to improve the classification of NMs and PIs, especially, accepting that safe scenarios that do not cause a negative contribution to system operational risk quantification might be misclassified.

Figure 1 :
Figure 1: Sketch of the failures that can be injected into the system.

Figure 2 :
Figure 2: The stand-alone SSSOM (c): different shades of color indicate different classes, circles are the geometric barycenters of the classes, triangles are the minimum neurons of the classes, and rectangles are the maximum neurons of the classes.(a) PDF of the MQE for failed scenarios; (b) PDF of the MQE for NMs scenarios; (d) PDF of the MQE for PIs scenarios; (e) PDF of the MQE for safe scenarios.
, a 3D representation of the performance parameters values of Tables 1-5 is given for each implemented SSSOM and each scenario class: stars indicate the MQE based SSSOM values, circles the barycenter based SSSOM values, squares the minimum neuron based SSSOM values, diamonds the maximum neuron based SSSOM, and crosses the stand-alone SSSOM values.

Figure 3 :
Figure 3: 3D representation of the performance parameters.Red refers to PIs, black to NMs, blue to failed, and green to safe scenarios.
measure between the generic scenario in input   and the weight   of the  neurons of the map, with the Manhattan distance: SSSOM.A SSSOM of  = 3025 neurons  = [ 1 ,  2 , . . .,   ], each of which is assigned a weight vector   = [ 1 ,  2 , . . .,   ], is trained on the  = 100509 UTSG dynamic scenarios  belonging to a  = 12dimensional space, say  = [ 1 ,  2 , . . .,   ], where the th sample is   = [ 1 ,  2 , . . .,   ].For the sake of completeness, the training algorithm is presented in the Appendix.In particular, this SSSOM is constructed by replacing the Euclidean distance as similarity

Table 2 :
Barycenter based SSSOM performances: precision, sensitivity, and specificity for each class.

Table 3 :
Minimum neuron based SSSOM performances: precision, sensitivity, and specificity for each class.

Table 4 :
Maximum neuron based SSSOM performances: precision, sensitivity, and specificity for each class.
2, , . . .,  , ⟩ is a set of -dimensional scenarios, (  ) is the neighborhood of   that is in this work defined as a set of    = 100 scenarios (i.e., a subset of  scenarios) whose Manhattan distance for the instance to be classified is lower than 10 (i.e., being  = 12, a threshold value equal to 10 means   and its neighbors have not to differ too much), and, thus,  = 1, . . ., 100.

Table 16 :
Decision tree classification results.