The Assessment and Foundation of Bell-Shaped Testability Growth Effort Functions Dependent System Testability Growth Models Based on NHPP

This paper investigates a type of STGM (system testability growth model) based on the nonhomogeneous Poisson process which incorporates TGEF (testability growth effort function). First, we analyze the process of TGT (testability growth test) for equipment, which shows that the TGT can be divided into two committed steps: make the unit under test be in broken condition to identify TDL (testability design limitation) and remove the TDL. We consider that the amount of TGF (testability growth effort) spent on identifying TDL is a crucial issue which decides the shape of testability growth curve and that the TGF increases firstly and then decreases at different rates in the whole life cycle. Furthermore, we incorporate five TGEFs: an Exponential curve, a Rayleigh curve, a logistic curve, a delayed S-shape curve or an inflected S-shaped curve which are collectively referred to as Bell-shaped TGEFs into STGM. Results from applications to a real data set of a stable tracking platform are analyzed and evaluated in testability prediction capability and show that the Bell-shaped function can be expressed as a TGF curve and that the logistic TGEF dependent STGM gives better predictions based on the real data set.


Introduction
Generally, system testability is defined as the probability of fault detection or isolation for a specific period of time in a specified environment, which is quantified by various testability indexes, such as FDR, FIR, and FAR [1][2][3].Over the last several decades, many testability test technologies, especially testability demonstration test for equipment, have been researched [4][5][6][7][8][9][10][11][12][13][14][15].However, some research work indicates that the result of testability demonstration test departures from the actual value of testability greatly and makes the result of testability demonstration test unauthentic [4].The root cause for this is that the vacancy of TGT in tracking and measuring the growth of testability as equipment is being developed.
In general, any fault diagnosis system similar to reliability for large-scale and complicated equipment like a missile may be premature when the fault diagnosis system has been developed, in which many TDLs hide, such as nonexpectant failure, test vacancy, ambiguity group, fuzzy point, improper resistance tolerance, or threshold.And then, the designers analyze and judge the root cause of TDL and further track to the design of equipment testability, like UUT, redesign of test equipment or interface equipment, circuit of BIT, fault diagnosis software, and test program of ATE.Thus, almost all the fault diagnosis systems for large-scale and complicated equipment need a certain extent of time period to develop TGT to identify and remove the TDLs to attain the expected value under contract.
The TGT is an important and expensive part of equipment testability development.A TDL in UUT leads to an output that differs from specifications and requirements.The aim of TGT is to identify and remove the TDL and further to increase the fault detection/isolation probability that a designed testability will work as intended in the hands of the designers.The testability growth test phase aims at identifying and removing the TDLs.Typically, TGT is more constructive and has high confidence level for system testability rather than the testability demonstration test at the acceptance stage only.
A common approach for measuring system testability is by using an analytic model whose parameters are generally estimated from available data on fault detection/isolation.In this paper, the analytic model is referred to by us as STGM.
A STGM provides a mathematical relationship between the number of TDLs removed and the test time during the whole life cycle and generally is used as a tool to estimate and predict the progress of system testability.At the same time, the STGM can be used for planning and controlling all test resources during development and can assure us about the testability of equipment.There is an extensive body of literature emphasizing the importance of TGT, but no depth of research is done on STGM [3,8].
In the context of TGT, the basic objective is to identify the TDLs and remove them one by one.Therefore, the process of TGT contains not only a TDL identification process but also a TDL removal process and that the key issues are the TGE which are consumed to identify the TDL and the effectiveness of TDL removal, respectively.In general, the TGE can be represented as man-hour, TGT cost, the times of fault injection, and so forth.The functions that describe how a TGE is distributed over the TGT phase are referred to by us as TGEF.The shape of the observed testability growth curve depends strongly on the time distribution of the TGE.Due to the randomness of TGT and consider the designer's capability and familiarity to UUT synchronously, the other key issue (i.e., the effectiveness of TDL removal) can be referred to by us as a constant TDL identification rate and a constant TDL removal rate, respectively.
There is an extensive body of literature  on software reliability growth model describing the relationship between the test time and the amount of test-effort expended during that time, in which the test effort was often described by the traditional Exponential [19,21], Rayleigh [16,19,21], logistic [29,30,37,38], delayed-shaped [19,39] or inflected S-shaped [18,39] curves.Thus, to address the issue of TGEF, this paper will use the above five test effort curves which are collectively referred to as Bell-shaped TGEFs to describe the relationship between the test time and the amount of TGE expended during that time by analyzing the consumption rule of TGE.Sometimes the TGE can be represented as the number of faults injected or occurred naturally instead of man-hours or TGT cost.Similarly, to address the issue of effectiveness of TDL removal, we assume that the TDL identification rate and the TDL removal rate are constant.
In this paper, the process of TGT is decomposed into fault injection or occurrence, TDL identification, and TDL removal processes which are all based on NHPP.We show how to integrate time dependent TGEF, constant TDL identification rate, and constant TDL removal rate into STGM to find the framework of STGM.We further pay main attention on the consumption pattern of TGE based on fault injection collection of fault occurring naturally and then discuss how to integrate five Bell-shaped TGEFs into STGM.A method to estimate the model parameter is provided.Experimental results from a stable tracking platform are analyzed and the five STGMs based on the above five Bell-shaped TGEFs are compared with each other to show which STGM can give better prediction.
The remainder of the paper is organized as follows.In Section 2, we found a framework of STGM considering TGEF.The fault occurrence, TDL identification and TDL removal process based on NHPP are also described and analyzed in this section.In Section 3, we pay main attention to analyzing the consumption rule of TGEF which is represented by fault injection or collection of fault occurred naturally at design & development stage and trial & in-service stage, respectively.At the same time, we show how to incorporate the five Bell-shaped TGEFs into STGM.Parameters of the proposed STGMs as estimated by the method of LSE are discussed in Section 4. Finally, Section 5 discusses the goodness of the above five STGMs on prediction with the application of these models to a real data set.

A General Framework of STGM Based on NHPP
In general, the precondition for testability growth test is that the UUT is in failed condition.That is to say, there is one or more faults occurred in the UUT.Under the precondition, TGT is the process of running a testability diagnostic program and capturing the fault detection and isolation data to identify the TDLs and remove them.In other words, the TGT aims at identifying the TDLs and removing them one by one under the precondition of fault occurred.Therefore, the process of removing TDLs can roughly be divided into three steps: fault occurrence, TDL identification, and TDL removal.At the same time, each of the above three steps can be described effectively as a counting process based on NHPP.
In practice, the most important factor which affects the STGM's evaluation and predication accuracy is TGE.TGE is the cumulative testability growth effort consumed in TGT which can be measured by the number of fault injection, test cost, staff, and so on.The consumed TGE indicates how TDLs are identified effectively in the UUT and can be modeled by different curves considering the consumption pattern in TGT.In this section, a framework for foundation of STGM with TGEF is proposed.
2.1.Fault Occurrence, TDL Identification, and Removal.TDL identification may be realized under the condition that the UUT must be broken.Once the UUT is in broken condition which usually can be realized by fault injection or collection information of fault occurred naturally, the design of testability begins to run to identify the TDL and analyze the root cause of the TDL.When the specific root cause is recognized, the designers can remove the TDL accordingly.
We can use the number of residual TDLs as a measure of system testability.The great the number of TDLs, the lower of testability level would be.So, the process of TGT is the process to identify and remove the TDLs existing in the UUT.In other words, the process of TGT is to identify and remove the TDL one by one.So, the process of TGT can be described The valid detection and The valid detection and The remaining invalid (d) The process of removing TDL exist in the UUT (e) The process of fault detection and isolation after TGT The event of fault occurred or injected The event of detecting and isolating a fault successfully The event of identifying a TDL The event of removing a TDL successfully The event of removing a TDL unsuccessfully The event of detecting and isolating a fault unsuccessfully  by Poisson process considering the randomness of fault injection, fault detection and isolation, TDL identification, and removal etc.Here, we give an apt shown in Figure 1 to describe the process of TGT.The fives processes given by Figure 1 about TGT and fault detection and isolation are (a) the process of fault occurred naturally or injected of the UUT, (c) the process of identifying that TDL exists in the UUT, (d) the process of removing TDL exist in the UUT, (b) and (e) depict the fault detection and isolation before and after the TGT, respectively.
In (a), { 1 ,  2 , . . .,   , . . .,   } is the failure mode set which is obtained by FMEA and  is the total number of failures in the UUT.With the progress of TGT, the time-dependent failure phenomena which can be realized by fault injection or collection of fault information occurred naturally at random time serial { 1 ,  2 , . . .,   , . . .,   }.
In (b), When some failures occurred at random time serial { 1 ,  2 , . . .,   , . . .,   }, the system testability is activated to detect and isolate the failures accordingly.If a failure cannot be detected and isolated successfully, a TDL is identified.Practically, some failures may be easy to detect and isolate, and some others are not.For example,  1 and  4 cannot be detected and isolated correctly, so two TDLs are identified and are noted as TDL 1 and TDL 2 , respectively.
With the progress of TDL identification, increasing TDLs are identified gradually; thus we have the process (c) which describes the counting process of TDL identified.
In (d), the designers analyze the root cause of the TDL and try their best to modify the design of testability and remove the TDL accordingly.But, not all the TDLs identified can be removed successfully because of designer's familiarity to UUT.For example, the TDL  presented in process (c) still cannot be removed successfully after an infinite amount of test time and test effort consumed.All the TDLs which can be identified but cannot be removed successfully form the process (d).In TGT, we should pay more attention to the expected average number of TDLs identified and removed successfully.
By comparison with process (b), process (e) gives the growth effect of fault detection and isolation after TGT apparently.

NHPP in TGT.
There is an extensive body of literature [12,32] on NHPP used for description of counting process effectively, such as the probabilistic failure process in software and equipment.In this paper, the NHPP is introduced to describe the TDL detection and removal process.() is a nonnegative integer and a time-dependent nondecreasing function, which describes the cumulative number of TDLs identified and removed up to time .If  < , () − () is the counts of the TDL identified and removed in the interval (, ).
The following characters of () can be derived from the assumption that "the probability that two or more faults occurred synchronously is wee and can be neglected in testability engineering" [3].
(2) The process has independent increment.
Thus, the process of {(),  ≥ 0} follows a Poisson process with a parameter (), where () is the TDL intensity of   () (i.e., number of TDLs identified and removed correctly per unit time).The counting process with a constant () has a smooth increment, which is named homogeneous Poisson process.Otherwise, the counting process with a timedependent () has a fluctuant increment, which is named nonhomogeneous Poisson process.HPP can be considered as a special case of NHPP, and NHPP is an extend case of HPP.
Make   () = [()]; then   () is the expected mean number of TDLs identified and removed successfully in time (0, ], so   () can be expressed as So To identify the inherent TDLs in the UUT, the process is expressed by To remove the TDLs identified by the tester, the To evaluate the value of system testability after TGT, noted as T e b 2 by process is expressed To estimate the value of system testability before TGT, noted as T(t 0 ) To any  ≥ 0,  ≥ 0, ( + ) − () follows the Poisson distribution with parameters   ( + ) −   (), so we have We can use either the number of TDLs identified and removed or the number of remaining TDLs as a measure of testability quality.Here, we use the number of TDLs identified and removed successfully (i.e.,   ()) as a measure of system testability quality.Parallel to the analysis of Figure 1,   () is critical for both the mean and variance of testability estimation and prediction.Hence, calculation of   () is our main focus in the following parts of this paper.

A General Framework of STGM Considering TGEF.
A STGM provides a mathematical relationship between the number of TDLs identified and removed successfully (  ()) from the UUT and the test time.STGM can be used as a tool to estimate and predict the progress of system testability.A TDL inherent in the UUT leads to an output that the fault occurred or injected cannot be detected and isolated correctly.The TGT aims at identifying these TDLs and removing them.At any time during the TGT phase, the basic process of TGT includes four steps which are shown in Figure 2.
In Figure 2,   is the target testability level which has been fixed at the beginning of TGT.But it is frequently realized that this target may not be achievable for a number of reasons, like inadequacy of TGE or inefficiency of the test team.So, TGT is a repetitive work of "test-identification-correctiontest" generally until the conditional expression ( 0 ) <   is satisfied.( 0 ) is the estimation value of testability at time  0 .
Firstly, we need to estimate ( 0 ) at time  0 .From the above analysis, the main cause leading to ( 0 ) <   is the TDL existing in the UUT.In this paper, we assume that the removal of a TDL from the UUT includes two phases.In the first phase the TDL identification team, primarily consisting of test personnel, identifies a TDL by making the UUT be in failed condition.Concretely, the identification team can either collect the fault occurred naturally or injected to verify the level of testability.During the TGT phase, much testability growth effort is consumed.The consumed TGE indicates how the TDLs are identified effectively in the UUT and can be modeled by different curves.Actually, the system testability is highly related to the amount of testability growth effort expenditures spent on identifying TDLs.In this step, the work in the process of TDL identification is calculated by (), such as man-hour, TGT cost, and the times of fault injection.() signifies the cumulative TGE consumed in identifying the TDLs up to the given time .Then another team, primarily consisting of designers, analyzes the reason for the TDL and modifies the design of testability to remove the TDL by redesigning of UUT or ATE and the corresponding interface, optimizing the circuit of BIT, debugging the diagnostic software and rewriting the diagnostic program of ATE, etc.All the work about removing the TDLs will be done in the third step of Figure 2 by the designers of system testability, which is described by a constant  2 .Finally, all the teams evaluate the value of system testability after TGT noted as () and give a test conclusion that if () ≥   , the TGT can be stopped, else the TGT will be continued until the condition expression () ≥   is satisfied.Apparently, testability can be enhanced if TGT is done for a prolonged period, that is, The STGM developed below is based on the following assumptions.
(1) The TDL identification and its removal processes all follow NHPP.
(2) The failure of fault detection/isolation at random times is caused by the TDLs which are inherent in the UUT.
(3) All TDLs are mutually independent and have the same contribution to the failure of fault detection/isolation occurred in the system.
(4) There is a one-to-one correspondence between a failure and the corresponding test.
(5) Two phases can be observed within the TGT process: TDL identification and TDL removal.There is no time lag between the TDL identification and its removal.Whenever a TDL is identified, the goal of TDL removal is to analyze the root cause of the TDL and to remove it.At the same time, no new TDLs are introduced.Let   () be the MVF of the expected number of TDL identified and let   () be the MVF of the expected number of TDLs identified and removed in time (0, ].Then, according to the above assumptions, we describe the STGM based on TGEF as follows: Note that here we assume  1 ̸ =  2 .Solving (4) under the boundary condition   (0) =   (0) = 0 we have Due to the space limitations, here we only propose the testability growth model of FDR in general.FDR is defined as the capability to detect fault occurred in the UUT, and FDR is also used to describe the average detection probability to the failure mode set which is gained from FMEA.The mathematical model of FDR can be formulated as [1][2][3] where  is the total number of failures.  is the number of failures which can be detected accurately by the system testability design.According to the above assumption (4), we can have that the number of failures which can be detected accurately by the system testability design is Substituting it into (8), we obtain Substitute   () (in (6)) into (10); the () can be rewritten as Depending on how elaborate a model one wishes to obtain is, one can use () to yield more complex or less complex analytic solutions for ().Different () reflect various assumption patterns of TGE in TGT.

Mathematical Problems in Engineering
Equation (11) is the modeling framework which considers TGEF into the NHPP STGM.At the same time, (11) can provide designers or testers with an estimation of the time needed to reach a given level of FDR.And that, it may also be used to determine the appropriate ending time for TGT, and can provide useful information for making decision on turn phase.

STGM with Bell-Shaped TGEF
Actually, the system testability is highly related to the amount of testability growth effort expenditures spent on identifying the TDLs.In this section we develop some STGMs in which five Bell-shaped TGEFs: an Exponential TGEF, a Rayleigh TGEF, a logistic TGEF, a delayed S-shaped TGEF, and an inflected S-shaped TGEF are taken into account.The forms of TGEF are all discussed in the literature about software reliability growth model .Here, we choose appropriate forms of TGEF in view of the TGT's specialty.

Bell-Shaped TGEF.
In TGT, one of the key factors is the TGE which can be represented as the number of manhours, the times of fault injection, and the cost of TGT, and so forth.The functions that describe how an effort is distributed over the testability growth phase are referred to by us as TGEF.TGEF describes the relationship between the test time and the amount of TGE expended in the test time.Actually, fault injection is one of the best effective measures to make the UUT be in broken condition and to identify the inherent TDLs in the UUT further.There is an extensive body of literature on fault injection used for identifying TDL in testability test [1][2][3][41][42][43].Therefore, we will use the times of fault injection as a measurement of TGE, which has an advantage that it is very intuitive and that testability growth effort can be quantified exactly at the same time.
In general, the whole life-cycle of equipment can be divided into three stages: demonstration stage, design & development stage, and trial & in-service stage.But the empirical analysis shows that the effective and credible stages to conduct TGT are design & development stage and trial & in-service stage considering the cost effectiveness in which fault injection and collection information for naturally occurred faults are done, respectively [3,6].At the same time, TGE are calculated by times of fault injection and the number of naturally occurred faults at design & development stage and trial & in-service stage, respectively.
Actually, the more faults we inject or collect, the more confidence we obtain in the estimation of system testability.Unfortunately, TGT with redundant fault injections or a large number of failure data may lead to excessive cost and too much time consumption.It is impracticable with the constraint of test cycle and total cost.
To give an accurate and reasonable description of TGE, we need to know not only how many faults need to inject at design & development stage but also how many failure data need to collect at trial & in-service stage.That is to say, we need to analyze the TGE's consumption pattern in TGT by combining Figure 3 which gives an abridged general view of testability growth process in the whole life-cycle.be identified as long as we inject less fault.In order to identify a certain number of TDLs at the design & development stage, the number of faults need to inject grow up with the level of FDR, which can be formulated as [1][2][3] For example, with reference to (12), the number of faults which need to be injected with the change of FDR level and number of TDLs, respectively, are shown in Figure 4.As seen from Figure 4, we find that at the same number of TDLs, the FDR level needs to achieve increase as time goes on, and the number of faults needs to inject increases accordingly.At the same level of FDR, the greater number of TDLs needs to identify, the greater number of faults needs to inject.Consequently, TGEF is a time-dependent increasing function at design & development stage.
Collection of failure which occurred naturally is also an effective measure to identify TDL When the UUT is put into the trail & in-service stage.Actually, the number of faults at trail & in-service stage is "small sample" under several constraints such as high reliability requirements or specified test cycle.Considering the growth of reliability, the expected number of faults at regular intervals will decrease gradually.That is to say, at trail & in-service stage, TGEF is a timedependent decreasing function which has the similar variation tendency shown in Figure 5.
In conclusion, at the whole TGT phase, TGEF increases firstly and then decreases at different rate.Based on the increase-decrease characteristic, we will use a kind of Bellshaped function which increases firstly and then decreases to fit the practical growth rate of TGEF.Bell-shaped function, just as its name implies, draws a time-dependent curve like a bell which has a similar shape to the curve shown in Figure 6.
It should be noted that Figure 6 is just a schematic used to depict the Bell-shaped function; the concrete functional expression depends on the actual TGE data set.Because actual testability growth effort data represent various expenditure patterns, sometimes the testability growth effort  expenditures are difficult to describe only by a Bell-shaped curve.In this paper, the Exponential TGEF, the Rayleigh TGEF, the logistic TGEF, the delayed S-shaped TGEF, and the inflected S-shaped TGEF have been used to explain the testability growth effort, which can be derived from the assumption that "the efforts that govern the pace of identifying the TDL for the UUT increase firstly and then decrease gradually." 3.1.1.The Exponential TGEF.Yamada et al. [21] found that the TEF in software reliability growth could be described by a Weibull-type distribution with two cases: the Exponential curve and the Rayleigh curve.
The Exponential TGEF over time period (0, ] can be expressed as The current TGE expenditure rate at time  is where   () = ∫  0   ()d.The Exponential curve is used for process that declines monotonically to an asymptote.
Substitute the  *  () in ( 13) into (7); we obtain 3.1.2.The Rayleigh TGEF [16,19,21,37].Another alternative of the Weibull-type distribution is the Rayleigh curve.It has been empirically observed that software development projects follow a life-cycle pattern described by the Rayleigh curve.
The Rayleigh TGEF over time period (0, ] can be expressed as The current TGE expenditure rate at time  is where   () = ∫  0   ()d.Therefore,   () is a smooth Bell-shaped curve and reaches its maximum value at time The Rayleigh curve first increases to a peak, and then decreases at a decelerating rate.
Substitute the  *  () in ( 16) into (7), we obtain 3.1.3.The Logistic TGEF.Logistic TEF was originally proposed by Parr [16] about software reliability growth model.It exhibits similar behavior to the Rayleigh curve, except during the early part of the project.Huang et al. [28,30,37] proposed that a logistic testing effort function can be used instead of the Weibull-type curve to describe the test effort patterns during the software development process.In some two dozen projects studied in the Yourdon 1978-1980 project survey, the Logistic TEF appeared to be fairly accurate in describing the expended test efforts [44].
The logistic TGEF over time period (0, ] can be expressed as The current TGE expenditure rate at time  is where   () = ∫  0   ()d.Therefore,   () is a smooth Bell-shaped function, and reaches its maximum value at time Substitute the  *  () in ( 20) into ( 7), we obtain Ohba [17,18] found that the test effort function in software reliability growth could be described by S-shaped distribution with two cases: the delayed S-shaped curve and the inflected S-shaped curve.[17,40].The delayed Sshaped TGEF over time period (0, ] can be expressed as

The Delayed S-Shaped TGEF
The current TGE expenditure rate at time  is where  DS () = ∫  0  DS ()d.Therefore,  DS () is a smooth Bell-shaped function and reaches its maximum value at time Substitute the  * DS () in ( 24) into (7); we obtain   [18,40].An inflected Sshaped TGEF over time period (0, ] can be expressed as

The Inflected S-Shaped TGEF
The current TGE expenditure rate at time  is where  IS () = ∫  0  IS ()d.Therefore,  IS () is a smooth Bell-shaped function and reaches its maximum value at time Substitute the  * IS () in ( 28) into (7); we obtain With reference to the above five Bell-shaped TGEFs, the instantaneous TGEs which are shown in Figures 7, 8, 9, 10, and 11 decrease ultimately during the testability growth lifecycle because the cumulative TGEs approach a finite limit.As shown from ( 15), ( 19), ( 23), (27), and (31), we find that not all the inherent TDLs in the UUT can be fully removed, even after a long TGT period because the total amount of TGE to be consumed during the TGT phase is limited to  max .This assumption is reasonable because no system testability design company will spend infinite resources on TGT considering the restricted cost & development cycle.

Bell-Shaped TGEF Dependent STGM.
In this section, we will use the above five TGEFs in the foundation of STGM.One of these can be substituted into (10) to have five kinds of STGM which are formulated as follows.13) into (11); the EX-STGM can be formulated as

Estimation of STGM Parameters
Fitting a proposed model to actual TGT data involves estimating the model parameters from the real TGT data set.Two popular estimation techniques are MLE and LSE [45].The MLE estimates parameters by solving a set of simultaneous equations.However, the equation set may be very complex and usually must be solved numerically.The LSE minimizes the sum of squares of the deviations between what we actually observe and what we expect.In order to avoid the solution of complex simultaneous equations of MLE, in this section, we employ the method of LSE to estimate the parameters of the above five TGEFs.Using the estimated TGEFs, the other parameters ,  1 ,  2 in ( 5), ( 6) can also be estimated by LSE.Due to the limitations of paper size, reference to the above five TGEFs, only the parameters  1 ,  1 of the Exponential TGEF in (13) are estimated by the method of LSE.At the same time, the parameters ,  1 given in (5) and  2 given in (6) can all be estimated by LSE.For the method of LSE, the evaluation formula   ( 1 ,  1 ),    (,  1 ) and    (  ,  2 ) are as follows: Differentiating   ,    ,    with respect to ( 1 ,  1 ), (,  1 ), and (  ,  2 ), respectively, setting the partial derivatives to zero, and rearranging these terms, we can solve this type of nonlinear least square problems.For a simple illustration, we only consider the generalized Exponential TGEF.
Take the partial derivatives of   with respect to  1 ,  1 ; we get Thus, the least squares estimator  1 is given by solving the above equation, yielding Next, we have The parameter  1 can also be obtained by substituting the least squares estimator  1 into the above equation.

Data Description.
To validate the proposed STGMs with the above five Bell-shaped TGEFs, TGT on a stable tracking platform have been performed.The TGT data set employed (listed in Table 1) was from the testability laboratory of National University of Defense Technology for a stable tracking platform which can isolate the movement of moving vehicle, such as car, ship, and aircraft.FMEA of the stable tracking platform had been done and had gained that the stable tracking platform consisted of approximately 350 functional circuit level failures.Over the course of 12 weeks at the design & development stage, 72 TDLs were identified by injecting 203 functional circuits level failures.Failures were injected by 1553B fault injection equipment, ARINC 429 fault injection equipment, RS232/422 fault injection equipment, CAN bus fault injection equipment, and the like.Further, testability designers analyzed the root cause of the TDLs and had tried their best to modify the design of testability and removed 46 TDLs successfully.On the other hand, over the course of 12 weeks at the trial & in-service stage, 110 functional circuit level failures had occurred naturally.14 TDLs were identified, in which 7 TDLs were removed successfully.

Criteria for Model Comparison.
A STGM can generally be analyzed according to its estimation capability, fitness capability, and predictive capability.That is to say, a STGM can be analyzed according to its ability to reproduce the identified TDL and behavior of the UUT and to predict the future behavior of the UUT from the observed TDL data.In this paper, the STGMs are compared with each other based on the following three criteria.[39].For practical purposes, we will use AE to calculate the accuracy of estimation.AE is defined as

The Accuracy of Estimation Criterion
is the actual cumulative number of identified TDL after the TGT, and  is the estimated number of initial TDLs.  is obtained from system testability TDL tracking after TGT.

The Goodness-of-Fit Criteria.
To quantitatively compare long-term predictions, we use MSE because it provides a well-understood measure of the difference between actual and predicted values.The MSE is defined as [30,37,45] A smaller MSE indicates a smaller fitting error and better performance.

The Predictive Validity Criterion.
The capability of the model to predict TDL identification and removal behavior from present & past TDL behavior is called predictive validity.This approach which was proposed by Musa et al. [45]

STGM Performance Analysis.
In this section, we present our evaluation of the performance of the proposed STGMs when applied to DS listed in Table 1.
All the parameters of the Exponential TGEF, the Rayleigh TGEF, the logistic TGEF, the delayed S-shaped TGEF, and the inflected S-shaped TGEF are also estimated by LSE.Firstly, the two unknown parameters  1 ,  1 of the Exponential TGEF are solved by LSE, giving the estimated values  1 = 2592.7 (fault times) and  1 = 0.0060/week.Correspondingly, the estimated parameters of the Rayleigh TGEF are  2 = 329.57(fault times), and  2 = 0.0123/week.In similarly, the estimated parameters of the logistic TGEF, the delayed Sshaped TGEF, the inflected S-shaped TGEF are  3 = 311.18(fault times),  = 39.36, 3 = 0.3524/week,  4 = 406.06(fault times),  4 = 0.1313/week,  5 = 314.44(fault times),  = 27.09, and  5 = 0.3232/week.In order to clearly show the comparative effectiveness of the fitness for the observed TGE data, the comparisons between the observed five current TGE data and the estimated five current TGE data are illustrated graphically in Figures 12 and 13, respectively.Similarly, the comparisons between the observed five cumulative TGE data and the estimated five cumulative TGE data are illustrated graphically in Figures 14 and 15, respectively.
In order to check the performance of the five above TGEFs and make comparisons with each other, here we select some comparison criterions for the estimation of TGEFs [30,37,45] The PE, Bias, and Variation for the above five TGEFs are listed in Table 2. From Table 2, we see that the inflected Sshaped TGEF has lower values of PE, Bias, and Variation than   the other four TGEFs.On average, the inflected S-shaped TGEF yields a better fit for this data set, which can also be drown from Figures 12 to 15 approximately.Table 3 lists the estimated values of parameters of different STGMs.We also give the values of AE, RE, and MSE in Table 3.It is observed that the STGM with inflected S-shaped TGEF (i.e., IS-STGM) has the smallest value of MSE when compared with other STGMs.
From Table 3 we see that the LO-STGM has lower values of AE and RE than the other four Bell-shaped TGEF dependent STGMs and that the RA-STGM has lower value of MSE than the other four Bell-shaped TGEF dependent STGMs.On average, the logistic TGEF dependent STGM yields a better fit for this data set.Substitute the estimated value of ,  1 ,  2 listed in Table 3 into (32), (33), (34), (35), (36), we can find five TGEF dependent STGMs of FDR.Figures 21,22,23,24, and 25 depict the growth curve of FDR at the whole TGT stage.
Finally, the performance of STGM strongly depends on the kind of data set.If a system testability designer plans to employ STGM for estimation of testability growth of UUT during system development processes, the testability designers need to select several representative models and apply them at the same time.Although models sometimes give good results, there is no single model that can be trusted to give accurate results in all circumstances.From our results, we can draw the following conclusions.
(1) The Bell-shaped TGEF may be a good approach to providing a more accurate description of resource consumption during TGT phase.Particularly, the inflected Sshaped TGEF has the smallest value of Bias, Variation, and PE compared with the other four TGEFs when applied to DS for a stable tracking platform. (

Figure 1 :
Figure 1: An apt sketch map of TGT.

and b 1 Figure 2 :
Figure 2: The basic process of TGT.

( 6 )
The mean number of TDL identified in the time interval [,  + Δ] by the current TGE is proportional to the mean number of TDLs unidentified in the UUT.The mean number of TDL identified and removed in the time interval [,  + Δ] by the current testability growth effort is proportional to the mean number of TDLs uncorrected in the UUT.The proportions are constants and expressed by  1 and  2 , respectively.(7)The consumption of TGE is modeled by ().

Figure 6 :
Figure 6: A schematic of Bell-shaped current TGE consumption.

Figure 7 :
Figure 7: An Exponential curve with fixed parameters.

Figure 8 :
Figure 8: A Rayleigh curve with fixed parameters.

Figure 9 :
Figure 9: A logistic curve with fixed parameters.

Figure 10 :Figure 11 :
Figure 10: A delayed S-shaped curve with fixed parameters.

Figure 12 :
Figure 12: Observed and three estimated current TGEFs for DS.

Figure 13 :
Figure 13: Observed and two estimated current TGEFs for DS.

Table 1 :
Data set for a stable tracking platform.
Assuming that we have identified and removed  TDLs by the end of TGT time   , we employ the TDL data up to time   (  ≤   ) to estimate the parameters of   ().Substituting the estimates of these parameters in the MVF yields the estimate of the number of TDLs   (  ) by time   .The estimate is compared with the actual number .The procedure is repeated for various values of   .We can check the predictive validity by plotting the relative error for different values of   .Numbers closer to zero imply more accurate prediction.Positive values of error indicate overestimation and negative values indicate underestimation.

Table 2 :
Comparison results for different TGEFs applied to DS.

Table 3 :
Estimated parameters values and model comparisons for DS.The symbol "-" indicates that the value is too large to characterize the MSE.
) By incorporating the Bell-shaped TGEF into the structure of STGM, the STGMs with different TGEFs are very powerful and flexible for various test environments.The RA-STGM has the smallest value of MSE compared with the other four STGMs when applied to DS for a stable tracking platform.Similarly, the LO-STGM has the smallest value of AE and RE compared with the other four STGMs.STGM: A STGM with an Exponential TGEF RA-STGM: A STGM with a Rayleigh TGEF LO-STGM: A STGM with a logistic TGEF DS-STGM: A STGM with a delayed S-shaped TGEF IS-STGM: A STGM with an inflected S-shaped TGEF LSE: Co n s ta n tT D Lr e m o v alr a t e ,wh i c hi s the TDL removal intensity for   () (): Cumulative TGE consumption up to time , which can be measured by the number of fault injected or occurred naturally, test cost, man-hour, and so on (): Current TGE consumption at time  (): Time-dependent FDR function in the whole life cycle  * (): () − (0)  max (): Total amount of TGE eventually consumed   (): The Exponential TGEF   () −   (0)  DS (): The delayed S-shaped TGEF  4 : T otalTGEeventuallyconsumedof a delayed S-shaped TGEF  4 : TGE consumption rate of the delayed S-shaped TGEF  * DS ():  DS () −  DS (0)  IS (): The inflected S-shaped TGEF DS ():  IS () −  IS (0)   (  ): Expected number of TDLs by time   estimated by a model   : ActualnumberofTDLsbytime  .