A Quantitative Assessment Approach to COTS Component Security

The vulnerability of software components hinders the development of component technology. An effective assessment approach to component security level can promote the development of component technology. Thus, the current paper proposes a quantitative assessment approach to COTS (commercial-off-the-shelf) component security. The steps of interface fault injection and the assessment framework are given based on the internal factors of the tested component. The quantitative assessment algorithm and formula of component security level are also presented. The experiment results show that the approach not only can detect component security vulnerabilities effectively but also quantitatively assess the component security level. The score of component security can be accurately calculated, which represents the security level of the tested component.


Introduction
In the area of software engineering, component-based software engineering (CBSE) has currently become a research point.The various development technologies of components further improve the development efficiency and performance of components, but the problems on the reliability and security of components have not yet been resolved.Current component testing approaches have focused on component functionality testing, which, to some extent, guarantees the correctness and integrity of component functionality.However, component security testing is rarely researched as a special subject.Component security testing mainly involved in testing the vulnerabilities of component which can be exploited to crack software component.There are some general component vulnerabilities such as memory leakage, stack overflow, access beyond boundary, and access protected page.In addition, the special difficulties for the security testing of COTS component are as follows.Firstly, the source codes are unavailable.Secondly, The general security pattern is difficult to attain by analyzing the general classification model of software vulnerability.Finally, The COTS component is very independent, which results in some problems such as gaining the effective component information and running the independent component.On the other hand, the assessment of component security has not been solved.Research on the quantitative assessment of component security level is rather rare.In the field of software testing, quantitatively assessing the security level of the component system not only guarantees and enhances the system's reliability and security but also conducts a quantitative assessment level to software developers and facilitates users in selecting software products based on its security to reduce the risk of attack.
Nevertheless, component security vulnerabilities are usually studied, but the quantitative assessment of component security is not common.Most scholars only research on the theoretical description of security assessment, but practical implementation methods are not presented.A basic assessment approach was given in the literature [1].Based on the knowledge of information security, the main approach is dividing the component security vulnerability into several ranks [2].The security level of components is assessed using the software testing technology.The relevant experiments were conducted but were not studied in depth.
Enough internal information should be collected before assessing the security level for the software component.But, it is usually very difficult to obtain the internal information for COTS (commercial-off-the-shelf) components, because its source codes are unavailable.In order to obtain more information about COTS component, it is necessary that some data are inputted the tested component to drive the component running.When the tested component is running, some internal information including some security information can be collected by monitoring the running state of the tested component.The data-driven method can obtain internal security information for COTS component.In the current research, we propose a feasible approach to assess the component security level quantitatively through using data-driven method and learning from relevant statistical and probabilistic methods.The exceptional component information monitored by the dynamic monitoring system can be obtained by the testing system of component vulnerability [3].Relevant security factors are assigned to appropriate security exceptions according to the classification of the vulnerability in the Common Vulnerabilities and Exposures (CVE) vulnerability library.Each of the exceptional vulnerability factor values can be obtained from the triggering causes of software security vulnerabilities, which is a harmful degree combined with the characteristics of the component security vulnerabilities.Finally, six commercial components, in which security vulnerabilities exist, are collected for experimental analysis in the CVE Web site and are assessed by the proposed approach.The number of each interface and methods used for each interface can be obtained by dynamically executing the tested components.The executed probabilities of each interface and methods used for each interface can be determined by statistical method.At the same time, fault injection operators and the logical prediction rules of exceptional database (PRED) are proposed according to the component security exceptions commonly used.This approach not only can assess the component security level but also effectively assess the security level of each interface, thus providing better assessment capabilities.

Related Work
At present, commercial-off-the-shelf (COTS) components, such as military, mechanization, financial components, and so on, are used by more and more commercial software including some crucial security software.COTS security takes an important part in component systems because the security of the whole software system is determined by it.Therefore, assessing the security level of the component is used to test the security of the entire software system, which has become an essential part in component testing.To the best of our knowledge, research on assessing the software component security mainly focuses on component security requirements and assessment approaches, vulnerability assessment, and quantitative risk assessment.
Han and Zheng [4] and Jabeen and Jaffar-Ur Rehman [5] studied the security requirements and security assessment methods of components but did not give the specific relevant assessment methods.Md.Khan et al. examined the security testing description and component assessment [6][7][8].Models of the user data security, component security framework, and security assessment model of software were proposed.Khan and Han developed an assessment scheme for the security properties of software components.The assessment scheme provided a numeric score that indicates the relative strength of the security properties of the component [8].However, the approach consists of two prerequisites: (1) a security requirement specification of the candidate component and (2) a component-specific security rating.The approach requires the source codes of candidate component to be available.The security property descriptions of the components based on the security assessment standards of information technology were proposed by Khan et al. [9].The main attributes of resource allocation, user data protection, and communications were included.The model was based on the active interface, which could provide the basis for reasoning and assessing a component's security to meet certain security requirements of a particular application.Common criteria (CC) provide a schema for evaluating information system and enumerate the specific security requirements for such systems [10].The functional requirements in CC describe the security behavior or functions expected of an information system to counter threats, but not all the requirements in CC may be applicable directly to software components because of the distributed nature of software components and their compositional complexity.
Alhazmi and Malaiya proposed a time-based model for the total vulnerabilities and an alternative model which is analogous to the software reliability growth models [11].However, this model cannot quantitatively assess the security level of software.Zhang et al. presented a vulnerability risk assessment method based on the attack tree model and Bayesian network model [12].The analysis method can assess the risk probability of the software system but not its security level.The security risk assessment was introduced to the classical MDA (Model-Driven Architecture) framework [13].The extension offers early assessment and early validation of security requirements.However, the improved framework still has not provided the quantitative mechanism of the software security.A systematic methodology was proposed for some software vulnerability assessment and security function verification [14].A scalable and adaptable automatic test system was implemented to test hundreds of software releases, but the security assessment of software releases was not implemented.The state of the architecturebased approach to reliability assessment of component-based software was presented [15].The common requirements of the architecture-based models were identified, and the classification was proposed.However, the approach only gave a framework for the reliability assessment of componentbased software.
Idongesit proposed a quantitative risk assessment model based on annualized loss expectancy and the exposure factor, which is the percentage of asset loss due to the potential threat [3].The model can assess the potential threats to a development effort and rank these threats based on the risk metric calculation.The Common Vulnerability Scoring System (CVSS) [16] provides an open framework for communicating the characteristics and effects of IT vulnerabilities.Its quantitative model ensures repeatable accurate measurement while enabling users to see the underlying vulnerability characteristics used to generate the scores.However, CVSS measures software vulnerability based on the external factors of software system.Occasionally, the result cannot precisely represent the software vulnerability.
In summary, we propose a new quantitative assessment of component security (QACS) approach to represent more accurately the security level of the COTS component based on the internal factors of software component.The internal factors of software component are some important information inside the component instead of the outside enviornment.There are some main internal factors such as the component objects, component interfaces, component methods, and component attributes.These internal factors reflect more accurate security features for the tested component.

A Quantitative Assessment Framework for Component Security
With the further development of component-based software engineering, COTS components have become frequently used in software systems.The source codes of the COTS component are unavailable and highly independent.The traditional testing approach of interface mutation is suitable for system integration testing.That is, mutation operators are only effective at the interface between the module and the subsystem.Therefore, the traditional method is unsuitable for testing the COTS component system.Interface mutation testing, which is suitable for the COTS component, can inject faults at the component interface level by extending the mutation operators in the program mutation and increasing the mutation operators at the component interface level.The defects of components can be identified through observation and analysis of the testing result of the system.Figure 1 shows the quantitative assessment framework of the component security level.
The interface fault injection operators of the COTS component include irregular input parameter operators, mutation operators based on program language, mutation operators based on interface specifications [17], and mutation operators based on interface definition language (IDL) [18].The fault injection operators, which are commonly used, are as follows: boundary value parameter, parameter beyond the length, parameter attribute replaced, parameter swap, parameter change, parameter assignment, parameter set null, and so on.
The test cases of fault injection can be generated through the selection of a suitable fault injection operator based on parameter types and number of component methods in the interface information.The test scripts can be generated according to the test cases.The component can be run with the test scripts.The running state can be monitored through the dynamic monitoring mechanism.Component vulnerability of the interface methods can be determined by the running state of the component, security prediction mechanism, prediction rules, and vulnerability factor in various methods.The vulnerable score can be obtained under Whether the generated test cases of fault injection are representative is the key to designing the fault injection operator, that is, whether the generated test cases of fault injection can trigger the component's security vulnerability as greatly as possible.Two errors, namely, environmental errors and interface parameter errors, can trigger component security vulnerabilities in a component system.The first error, which is discussed in the previous paper [1], does not repeat it again.The focus of the current paper is the interface parameter fault injection.The errors in the interface parameter include three types: IDL description errors, boundary value errors, and irregular input errors.In detail, they are the method errors defined by the IDL description, all boundary value, and irregular input value errors, respectively.Based on the cause of triggering software security vulnerabilities in practical experience, 18 fault injection operators of component vulnerability are specifically designed by comprehensively analyzing the interface mutation operators and the operators of parameter fault injection.These operators are PAS In addition, the reverse distribution function of the parameter input domain is generated using the input operator PRI of the parameters of reverse probability distribution.The parameters of reverse probability distribution are then generated.The experimental result shows that component security exception can be effectively triggered through the irregular value of input domain.The input distribution function, which is usually difficult to obtain, is a probability density function.The one dimensional and multidimensional functions are the two commonly used kinds.Q is assumed the input distribution function, of which the value is in the interval [0, 1], and the value 0 indicates the probability of selection input field is 0. The algorithm of reverse probability distribution   is given in the literature [19].
A number of models have been designed to facilitate the automatic management of the fault injection operator, such as the automatic learning model of the fault injection operator, adaptive model, topic model, intelligent vulnerable trees, among others.They can automatically increase and manage the operator libraries of fault injection.

Exceptional Predication Rules and Vulnerability Factors
Definition 3. The component exceptional predication rule is a set of all the possible exceptions that may arise under the exceptional circumstances.Definition 4. The vulnerability factor of the component represents the degree of the component security vulnerabilities resulting from exceptions.The maximum value is 1, and the minimum value is 0. Table 1 shows the component exceptional predication rules and relative vulnerability factors used to assess the component security level.Each of the factor value of exceptional security vulnerability, which can objectively describe the component security level, is obtained from the triggering causes of software security vulnerabilities, which is a harmful degree combined with the characteristics of component security vulnerabilities based on experience software engineering.

Component Security Assessment Algorithm
Suppose  is a component, and  is a set of interface in .Let   represent the execution probability of interface , and   is a method properties set of interface .Meanwhile,   is the executable probability in method  of interface .Suppose || = , |  | = . is the input parameter of the method attribute in . is vulnerable error test cases set and  = { 1 ,  2 ,  3 , . . .,   }.Δ is all the input set of component .  is the reverse probability distribution of Δ. PRED is the logical prediction rules of exceptional database.VC is the vulnerability factor of exceptional function.Before the vulnerability assessment method of component is given, the following definitions should be introduced first.Definition 5. Let  represent an  ×  probability matrix that can be described as  = (  ) × , row  represents the interface number of component , and column  represents the method number of interface .The element   states the executable probability for function  at interface  of component , where   =   ×   .Definition 6. Interface method coverage (MC), where MC = (the number of methods executed at least once/the total number methods ) × 100%, is used to measure the sufficient degree of the interface method during component testing.

Definition 7. Interface coverage (IC)
, where IC = (the number of interfaces executed at least once/the total number interface ) × 100%, is used to measure the sufficient degree of the interface test during component testing.Definition 8. Component test adequacy CA can be measured by a two-tuple (i.e., MC and IC).Generally, the value is determined by the total score of MC and IC, that is, CA = [∑ =1,  (MC  )]/+IC.A greater value of CA indicates a more comprehensive test; the relationship between MC and IC is parallel.
Suppose Sum  represents the number of test methods, cnt represents the number of vulnerability methods in component , and VC  represents the vulnerability factors of exceptional method.The value of vulnerability factor VC  is assigned to 0 in case the interface method has no security exceptions.The execution probability of the vulnerability method  is labeled   .The component vulnerability assessment formula is given as follows: The value 1 − Vul  , the security level of the component , can be derived from the Formula (1).The vulnerability assessment formula of the component has the following properties.
Proof.For any vulnerability method, 1 ≥   ≥ 0 and 1 ≥ VC  ≥ 0 can be determined according to the definition of VC.Therefore, Property 9 shows that the value of any component's vulnerability index is between 0 and 1, with a larger value indicating a higher vulnerability index.Proof.For any vulnerability method, 1 ≥   ≥ 0 and 1 ≥ VC  ≥ 0 can be determined according to the definition of VC.Therefore, max(  ×VC  ) ≥ 0. In addition, 1 ≥ 1−  ×VC  ≥ 0 Property 10 shows that the vulnerability index of any one component is not less than the peak value of the methods' vulnerability indexes of the component.All situations of the vulnerability methods of component are synthetically considered by the index of the vulnerability of component.
Based on the component vulnerable assessment method based on the analysis of the interface, the algorithm of the component's security assessment level is given as follows.
The input probability matrix  in Algorithm 1 is not only derived from the component requirement specifications or equal probability but is also statistically calculated after the component is practically executed.The vulnerability factor VC  is assigned to 0 given that the component has no security exceptional interface method.We usually set the value of  and  from 0 to 1 according to the complexity of tested component and the testing requirement.If the number of interfaces and methods of tested component is no more than 200, we usually let the value of  and  to be 1 to ensure a complete testing of COTS.In addition, the test cases of interface parameters are generated using the TGSM [20,21] algorithm.First, the test cases of the interface parameter faultinjection, expressed in matrix form, are generated in terms of fault injection operator.The K-factor coverage solution matrix is generated by the TGSM algorithm, which is based on the parameter set of the matrix form.The fault injection test cases are composed of row data of the solution matrix.Test cases generated by the TGSM algorithm not only greatly reduce the scale of the test cases but also have certain coverage.The algorithm can trigger most of the security exceptions with fewer test cases.
Cnt is the number of vulnerability methods of the component  in the algorithm results.Vul  is the vulnerability ratio of the method  at the interface  of the component .Vul  represents the vulnerability level of the component . 1 − Vul  , the security level of the component , represents the security of the method  in the interface  of component .

Experiment Analysis
To validate the effectiveness of the proposed approach, the approach was implemented in our component security testing system (CSTS) platform [22].The component security testing and assessing system based on interface fault injection (CSTASboIFT) was implemented using C # language in the Microsoft.NET platform.It is used as a subsystem of the CSTS platform.Figure 2 shows the function flow chart of CSTASboIFT.
In the experiment, six commercial components (  experiments were conducted for these components.A component composed of six interfaces and each interface consisting of eight methods were assumed so that the component could be regarded as a 6 × 8 matrix.The first group of experiments tested the condition of   assigned to a fixed matrix and VC  divided into interval descending.As VC  changed, the 1 − Vul  was affected.That is, only the level of vulnerability factors affected the component security level.The second group of experiments tested the condition of VC  assigned to a fixed matrix and   divided into interval descending.As   changed, 1 − Vul  was affected.That is, only the executed probability affected the component security level.The last group of experiments considered the effect on 1 − Vul  , as both vulnerability factor and executed probability simultaneously changed.

Experiment One.
The vulnerability factor value is divided into intervals based on the knowledge of security vulnerability provided by the CVE Web site.  represents the executable probability of the method  at the interface , VC  represents the vulnerability factors of the method  at the interface , and Vul  represents the vulnerability level of component . 1 − Vul  is the security level of component .
The first experiment assumes that   is a 6 × 8 fixed matrix, which is made up of random values.For every interval,   is the same matrix.VC  is divided into 10 intervals:    the average result.Under the condition that   is regarded as a 6 × 8 matrix at every interval of VC  , VC  affects the value 1 − Vul  .
When Vul  is equal to   × VC  , as   is a fixed value, the values of Vul  and Vul  are proportional to the value of VC  .The value of 1 − Vul  is reversely proportional to VC  .A larger VC  value indicates a lower security level.As the vulnerability number increases, the security level decreases slowly.The results are shown in Figure 3.The testing results show that the proposed assessment Formula (1) is effective.That is, the component is more insecure.As the number of cnt increases, the decline rate of the security level is initially fast and then becomes slow.The results, as presented in Figure 4, show that the proposed assessment Formula (1) is effective.show that the proposed assessment Formula (1) is suitable for assessing the COTS component.

Experiment Four.
The components were assessed in Table 3 based on Algorithm 1.The value of IC and MC is set to 1.   can be determined by executing the tested component 100 times or more.
During the system testing process, the test cases of fault injection can be implemented by testing the component information and fault injection operators.The testing state can be recorded through the dynamic monitoring mechanism after running the tested component.The vulnerability type of the component method and vulnerability factor can be derived from comparing the testing state with the component information record in PRED.The execution probability   of each method in the component can be determined by running the tested component 100 times, respectively.Then, the component security level can be determined using Formula (1).The testing results are shown in Table 3.Because different components have different method parameters which have different parameter types such as integer, string, and boolean, we select suitable fault injection factors to test the component security based on the above presented 18 fault injection operators of component in Table 3.
A higher level of security indicates better relative security of the component.The component is secure if and only if the component security level is 1.Otherwise, it has security vulnerability.In contrast to the test results in Table 3, the level of security is closer to the vulnerability information published in the CVE Web site.The values of execution probability and vulnerability factors have been proved to affect the component security.The results agree with the previous experimental results, further confirming the feasibility and effectiveness of this approach.4 shows the comparison with other assessment approaches.The related assessment approaches have various merits as shown in the analysis of Table 4.Our QACS approach mainly focuses on quantitatively assessing the security level of the COTS component.The source codes of the COTS component are  generally unknown.The internal factors of the component are considered in QACS.The QACS approach is implemented in the CSTASboIFT that is integrated into our CSTS [22] system developed to test the security of the COTS component.Aside from the above-mentioned approaches, CVSS is a widely used quantitative assessment approach for software component.Khan and Han developed an assessment scheme for the security properties of software components.The assessment scheme provided a numeric score that indicates the relative strength of the security properties of the component [8].Results of the comparison with the CVSS approach and Khaled approach are shown in Table 5.In addition, the security level of six tested components is also listed according to the reported empirical data of security Web sites in Table 5.The initial CVSS Base Score is calculated by the CVSS formula [16].

Experimental Comparison. Table
Analyzing Table 5, the QACS approach is found to be close to the empirical data reported on some security Web sites regarding the security of the tested COTS components.The security score of the QACS approach is more accurate than that of others because the internal factors of the tested component are considered.The assessment process of the QACS approach is objective, without human intervention.Nevertheless, the CVSS approach is mainly focused on assessing the vulnerability of the general software component.If an assessor is not familiar with the assessed software, the vulnerability score becomes inaccurate because of the assessor's subjectivity in the assessment process.On the other hand, the Khaled approach is an approximate assessment approach based on the assumed component environment.In addition, presently there are no more effective assessment approaches for the security level of software component according to our studies.

Conclusion
In component-based software development and maintenance activities, the assessment of the component reliability and security level is essential in the third-party component.
Research on an effective quantitative assessment method for component security level is valuable in the CBSE.The

Figure 2 :
Figure 2: The function flow chart of CSTASboIFT.

Figure 3 :
Figure 3: The comparison for different VC  and cnt.

10 Mathematical
Three.Both parameters   and VC  are assumed to influence the value of 1 − Vul  in Experiment three.The two parameters are randomly divided into

Figure 4 :
Figure 4: The comparison for different   and cnt.

Figure 5 :
Figure 5: The comparison for different   , VC  , and cnt.

Table 1 :
Exceptional predication rules and vulnerability factors.

Table 2 )
, in which security vulnerabilities exist, were collected for experimental analysis in the CVE Web site.Several group Input: Interface information XML file; Fault injection operator; Prediction rules PRED; Probability matrix ; MC's threshold and IC's threshold value  and . is the number of the testing methods 10 Generate method parameter values set according to fault injection operators; 11 Call testing cases generating algorithm TGSM(); //call the generation algorithm of the minimum  factors combined cover test case based on solution matrix 12 Running  and   ; 13 If (the output after running  and   satisfies PRED) VC  ×   ; // the vulnerability level of the method  in the interface  Algorithm 1: QACS (quantitative assessment of component security) algorithm.

Table 2 :
The six typical tested components.
Two. VC  is assumed a 6 × 8 fixed matrix made up of random values in the second experiment.Then,   is divided into 10 intervals: [0.00, 0.09], [0.10, 0.19], [0.20, 0.29], [0.30, 0.39], [0.40, 0.49], [0.50, 0.59], [0.60, 0.69], [0.70, 0.79], [0.80, 0.89], and [0.90, 1.00].The tests were done about ten times with different random values of   and VC  , respectively, in the experiment, and then the value 1−Vul  was got by calculating the average result.Under the condition that VC  is regarded as a 6 × 8 matrix at every interval of   ,   affects the value of 1 − Vul  .When VuL  is equal to   × VC  , as the value of VC  is fixed, the values of Vul  and VuL  are proportional to the value of   .The value of 1 − Vul  is reversely proportional to VC  .A larger   value indicates a lower security level.