Test Sequence Reduction of Wireless Protocol Conformance Testing to Internet of Things

1 School of Computer Engineering and Science, Shanghai University, Shanghai 200444, China 2Shanghai Key Laboratory of Computer Software Testing and Evaluating, Shanghai 201114, China 3Department of Information Science and Technology, Taishan University, Taian 271000, China 4Computing Center, Shanghai University, Shanghai 200444, China 5Shanghai Key Laboratory of Intelligent Manufacturing and Robotics, Shanghai 200072, China


Introduction
Wireless communication is essential and critical to Internet of Things (IoT) [1,2].The reliability of wireless communication transmission largely depends on whether the wireless communication protocol is implemented as specified.Conformance testing [3,4] is widely used to check whether an implementation conforms to its specification in areas such as traditional communication protocols and reactive systems; i.e., there must be the same behavior in the implementation for any I/O behavior observed in the specification.In FSMbased conformance testing, a protocol is called a specification and is expressed as a Finite State Machine (FSM), while an implementation under test is considered to be a "black box", the I/O behavior of which can only be observed.A test sequence is an I/O sequence such that whether an implementation conforms to the specification may be concluded by delivering the input sequence of the test sequence to the implementation and comparing the resulting output sequence with the output sequence in the test sequence.
Test techniques based on state identification [5][6][7][8][9][10] are well-known in FSM-based conformance testing.Test techniques based on state identification are supposed to identify every state and verify every transition of the specification in the implementation.A state is said to be identified when a state identifier is delivered to the state.A transition is said to be verified when the end state of the transition is identified.A state identifier of a state is a nonempty set of input sequences for the state such that the set of corresponding output sequences can characterize the state.Unique input/output (UIO) sequence is a popular state identifier such that UIObased techniques are commonly used for test sequence generation.
Since wireless communication protocols can also be modeled by FSMs, FSM-based conformance testing is also applicable to wireless protocol conformance testing in 2 Security and Communication Networks  IoT.Zigbee is a typical wireless communication protocol and is divided into four layers: physical layer (PHY), media access control lay (MAC), network layer (NWK), and application layer (APL).The connection and release process of nodes in the MAC layer is modeled by the FSM in Figure 1 and UIO sequences of every state are listed in Table 1.On this basis, test sequences based on FSMs can be constructed using UIO sequences as state identifiers.There is a test sequence  1 ( 1 / 1 ) 4 ( 2  3 /  1  2 ) 3 ( 5 ) 2 ( 6 / 1 ) 1 ( 1 / 1 ) 4 ( 2  4 / 1  3 ) 1 ( 1 / 1 ) 4 ( 7 ) 1 ( 1 /  1 ) 4 ( 2  3 / 1  2 ) 3 ( 7 ) 1 ( 1 / 1 ) 4 of the FSM in Figure 1 based on UIO sequences in Table 1.When the input sequence of this test sequence is applied to an implementation, every state of the FSM in Figure 1 can be identified and every transition of the FSM in Figure 1 can be verified in the implementation.
Meanwhile, an output sequence can be obtained.It can be concluded whether the connection and release process of nodes is executed as the specification specifies by comparing the obtained output sequence with the expected one in the test sequence.
Test sequence reduction has long been an active research topic in FSM-based conformance testing.One approach is to convert the problem into the Rural Chinese Postman Problem from which a test sequence is extracted.For the purpose of transition verification every transition of an FSM is followed by an appended UIO sequence in the test sequence.Bo Yang et al. reduced test sequences by overlapping and multiple UIO sequences [11].Benefiting from overlapping, more than one transition may be verified by a single appended UIO sequence.Hierons improved the method of UIO sequences through the use of an invertibility criterion, thereby achieving more overlapping [12], i.e., verifying even more transitions of an FSM with a single appended UIO sequence.However, multiple UIO sequences which are conducive to test sequence reduction are not considered by Hierons.
In order to reduce the cost of testing while assuring the effectiveness, this paper presents an improved method to generate reduced test sequences for wireless protocol conformance testing of IoT.Transitions in a test sequence are of three kinds: a copy of transitions in an FSM, UIO sequences which have been appended to the test sequence for the purpose of transition verification for the FSM, and the transitions which have been appended to the test sequence.In the improved method, invertibility is taken into account to leverage as much overlapping as possible such that all the transitions of an FSM can be verified with as few appended UIO sequences as possible.Rural symmetric augmentation is the main measure for test sequence generation.The replicated transitions during the rural symmetric augmentation are exactly the transitions for concatenation of test subsequences in the test sequence.More options of rural symmetric augmentation can be supplied by multiple UIO sequences than those supplied by single UIO sequences; i.e., a sensible choice of UIO sequences can lead to a minimum number of replicated transitions during a rural symmetric augmentation, and a minimum number of transitions for concatenation are achieved in the test sequence.In this way, further reduced test sequences are obtained by the proposed method.
The rest of the paper is organized as follows.In Section 2, the basic concepts and assumptions used in this paper are presented.The improved method is described with simple examples in Section 3, and the soundness and effectiveness of the method are discussed.Experimental evaluation is set out in Section 4.Then, the related work about the testing of IoT is reviewed in Section 5 and the paper is concluded briefly in Section 6.

Preliminaries
In this section, we introduce the definitions related to FSMs and graphs, together with the assumptions necessary for FSM-based conformance testing.

2.1.
Definitions.An FSM is formally defined as a 6-tuple  = (, , ,  0 , , ) where (i)  and  are finite and nonempty sets of input symbols and output symbols, respectively; (ii)  is a finite and nonempty set of states; (iii)  0 ∈  represents the initial state of ; Security and Communication Networks 3
According to this definition, when an input symbol  is delivered to the current state ,  moves to the state   = (,) with an output produced by (, ).The transition function and output function can be extended to finite input sequences, i.e., for an input symbol , an input sequence  ∈  * (where  * is the set of finite sequences of input symbols), and a state , (, ) = ((, ), ), and (, ) = (, )((, ), ) where concatenation is denoted by juxtaposition.
A transition  is defined by a tuple (,   , /) where  is the start state,  is an input of ,  = (, ) is the associated output, and   = (, ) is the end state.A transition (,   , /) is invertible if it is the unique transition ending at   with input  and output .
A UIO sequence of a state is an input/output sequence such that the input/output behavior exhibited by the state is unique, i.e., given a UIO sequence  of a state , and the input sequence of  is denoted by   ; for any state   ̸ = , it is always true that (,   ) ̸ = (  ,   ).A UIO sequence is irreducible if it is not a UIO sequence anymore when the end / symbol of the sequence is deleted.UIO sequences in this paper will be supposed irreducible unless otherwise specified.There may be more than one UIO sequence for a state and the UIO sequences may be of different length.
An FSM  can be perceived as a labeled, directed graph .A state of  is represented as a node of , and there is an edge labeled with / from node  to   in  if and only if (, ) =   and (, ) =  in , where  and   are the start and end node of the edge.The numbers of incoming and outgoing edges of a node are called the in-degree and out-degree of the node, respectively.If the in-degree equals out-degree at each node then the directed graph is symmetric.Suppose that  is an asymmetric directed graph; if  * is a symmetric directed graph generated from  by making copies of the edges then  * is a symmetric augmentation of .When the total cost of copies of edges is minimized,  * is said to be a minimal symmetric augmentation of .Suppose that  is the set of edges of  and   ⊆ , if  * is a symmetric directed graph generated from  by making copies of the edges such that each edge in   is included in  * at least once and the total cost of copies of edges is minimized then  * is called a rural forms a path of  where  1 and   are the start and end node of the path, respectively.A path that starts and ends at the same node forms a tour, furthermore, if the tour traverses every edge of  exactly once then it is an Euler tour.A rural postman tour is a tour that traverses a given set of edges at least once.The Rural Chinese Postman Problem is to find a minimum-length rural postman tour for a given set of edges.FSMs and their directed graph representations are used interchangeably throughout this paper.

Assumptions.
Given a specification FSM  with  states, the fault domain Ψ() of  is the set of all possible implementations of  over the input alphabet  of .Ψ  () refers to the implementations with up to  states in Ψ().This paper only focuses on implementations in Ψ  ().
An FSM  is strongly connected if for any two distinct states  and   there is an input sequence  that takes  from  to   ; i.e., ∀,   ∈ ,  ̸ =   , ∃ ∈  * ⋅ ((, ) =   ).Two states  and   are equivalent if for any input sequence there are always the same output sequences from  and   .An FSM  without equivalent states is said to be minimal or reduced; otherwise  is reducible by joining equivalent states.
An FSM  is deterministic if at any state for any input there is at most one transition leading to the next state.Otherwise,  is nondeterministic.
Only strongly connected, minimal and deterministic FSMs are considered in this paper.In addition, it is assumed that UIO sequences for each state of an FSM are available and are derived from successor trees in advance.It is noted that only state transition functions in forms of  ×  →  are considered; i.e., state transitions without any input in IoT are out of the scope of this paper.

Multiple UIO Sequences.
In the improved method, for any maximum-length IDOS ( 1 ,  2 ,  1 / 1 )( 2 ,  3 ,  2 / 2 ) . . .(  ,  +1 ,   /  ), the associated test subsequence is expressed as where  is a UIO sequence of  +1 .A minimum-length rural postman tour covering all the test subsequences is subsequently constructed by a rural symmetric augmentation and a test sequence is obtained from the tour.Accordingly, transitions for concatenation of test subsequences in the test sequence are derived from the transition replications during the rural symmetric augmentation.It is noted that different choice of UIO sequences may result in different rural symmetric augmentations; i.e., a minimum number of transition replications can be achieved by a sensible choice of UIO sequences during the rural symmetric augmentation.The minimum number of transition replications during the rural symmetric augmentation indicates the minimum number of transitions for concatenation of test subsequences in the test sequence, leading to a reduced test sequence.In other words, the result of using single UIO sequences can only in best-case scenarios obtain the same length of minimum-length rural postman tours as that of using multiple UIO sequences.

Design of the Method.
It is known from Theorem 3 that noninvertible transitions restrict the generation of IDOSs.
From this point of view, FSMs can be partitioned into two subsets.One is FSMs with only invertible transitions and the other is FSMs with noninvertible transitions.For FSMs with only invertible transitions, if the FSMs are symmetric test sequences can be obtained directly.Otherwise, the FSMs should be augmented firstly.So FSMs with only invertible transitions can also be partitioned into two subsets.One is symmetric FSMs with only invertible transitions and the other is asymmetric FSMs with only invertible transitions.Generally, FSMs are classified into three categories: symmetric FSMs with only invertible transitions, asymmetric FSMs with only invertible transitions and FSMs with noninvertible transitions.The improved method is described in two steps for each type of FSMs and the detail varies for different types.
Step 1. Construct the set of maximum-length IDOSs.
Step 2. With the consideration of multiple UIO sequences, generate a minimum-length rural postman tour covering the set of maximum-length IDOSs and extract a test sequence from the tour.

Symmetric FSMs with Only Invertible Transitions
Step 1 (maximum-length IDOSs generation).There is an Euler tour in a directed graph if and only if the directed graph is strongly connected and symmetric.Under the assumption that all the FSMs are strongly connected, there must be Euler tours for symmetric FSMs with only invertible transitions.An Euler tour starting from and ending at the initial state is definitely an IDOS since there are only invertible transitions; furthermore, it is a maximum-length IDOS from the initial state since there is no other one containing more transitions.In other words, an Euler tour of a symmetric FSM with only invertible transitions is the only sequence in the set of maximum-length IDOSs.
Note that an Euler tour starting from and ending at the initial state may not be unique; i.e., there may be nonunique sets of maximum-length IDOSs.Nonetheless, all the sets of maximum-length IDOSs hold the following properties.
(i) There is only one maximum-length IDOS covering all the transitions of the FSM in every set of maximum-length IDOSs.
(ii) The start state of the maximum-length IDOS is the initial state of the FSM in every set of maximum-length IDOSs.
(iii) The end state of the maximum-length IDOS is the initial state of the FSM in every set of maximum-length IDOSs.

Asymmetric FSMs with Only Invertible Transitions
Step 1 (maximum-length IDOSs generation).An FSM is asymmetric which refers to the fact that there are nodes whose out-degree does not equal in-degree.It is known that ∑ ∈   () = ∑ ∈   () = || where   () and   () denote the out-degree and in-degree of a node, respectively [13].The notation || refers to the number of edges in a directed graph.It is concluded from where  − and  + are the sets of nodes whose out-degree outnumbers in-degree and in-degree outnumbers out-degree, respectively.Correspondingly,  > () and  > (  ) represent the amount of out-degree over in-degree for a node in  − and the amount of in-degree over out-degree for a node in  + , respectively.The relation ∑ ∈   () = ∑ ∈   () = || implies that if a path passes through as many edges as possible exactly once then the path consumes as many out-degree and indegree as possible.Every path through a node takes up one incoming edge as well as one outgoing edge, so if the outdegree outnumbers in-degree at a node, its outgoing edges cannot be traversed completely by paths passing through the node; i.e., some of its outgoing edges must instead be traversed by paths starting from the node.Thus, it is advisable to start the paths from nodes whose out-degree outnumbers in-degree if all the edges should be traversed exactly once with as few paths as possible.Moreover, it can be proved by contradiction that the paths from nodes whose out-degree outnumbers in-degree must end at nodes whose in-degree outnumbers out-degree otherwise the paths will continue to extend.
According to the above analysis, maximum-length IDOSs generation for asymmetric FSMs with only invertible transitions is performed as follows: start from a node  ∈  − and go down along an outgoing edge until a node   ∈  + is reached and all the outgoing edges of s  have been traversed.Thus a maximum-length IDOS from  to   is obtained.Add the maximum-length IDOS to the set of maximum-length IDOSs and delete the corresponding edges in the directed graph.Repeat the above process until all the nodes of the directed graph are isolated which implies that the set of maximumlength IDOSs is obtained.
Similarly, there may be nonunique sets of maximumlength IDOSs for asymmetric FSMs with only invertible transitions and all the sets of maximum-length IDOSs hold the following properties: (i) For any set of maximum-length IDOSs,  − is the set of start states for maximum-length IDOSs.(ii) For any set of maximum-length IDOSs,  + is the set of end states for maximum-length IDOSs.
(iii) For any node  ∈  − , the number of maximum-length IDOSs starting from  is  > () in every set of maximumlength IDOSs.
(iv) For any node   ∈  + , the number of maximumlength IDOSs ending at   is  > (  ) in every set of maximum-length IDOSs.
Step 2 (test sequence generation).With the set of maximumlength IDOSs, the same method as that of Bo Yang et al. is used to construct test sequences and the detail of the method is described in Algorithm 1.The core of the method is the rural symmetric augmentation over an FSM augmented by maximum-length IDOSs and the multiple UIO sequences for the end states of maximum-length IDOSs.The core of the symmetric augmentation is the states involved.According to the above properties, for any set of maximum-length IDOSs, there is no difference about the states involved in the symmetric augmentation such that a random set of maximum-length IDOSs will do for asymmetric FSMs with only invertible transitions when there is more than one set of maximum-length IDOSs.

FSMs with Noninvertible Transitions
Step 1 (maximum-length IDOSs generation).To address an FSM with noninvertible transitions in a similar way to the Given a union of maximum-length IDOSs of the remainder excluding noninvertible transitions as well as all the noninvertible transitions, there may be nonunique sets of maximum-length IDOSs for the FSM with noninvertible transitions because of different concatenation.The properties of nonunique sets of maximum-length IDOSs for FSMs with noninvertible transitions are described as follows: (i) In any set of maximum-length IDOSs, for any state  which is the start state of a maximum-length IDOS, if there are  maximum-length IDOSs starting from  then there must be  maximum-length IDOSs starting from  in every other set of maximum-length IDOSs.
(ii) In any set of maximum-length IDOSs, for any state  which is the end state of a maximum-length IDOS, if there are  maximum-length IDOSs ending at  then there must be  maximum-length IDOSs ending at  in every other set of maximum-length IDOSs.
Step 2 (test sequence generation).For FSMs with noninvertible transitions, test sequence generation is in the same way as asymmetric FSMs with only invertible transitions, i.e., by means of rural symmetric augmentation over an FSM augmented by maximum-length IDOSs and the multiple UIO sequences for the end states of maximum-length IDOSs.Table 4: UIO sequence of  3 .
Proof.The soundness of  is first proved; i.e.,  is a test sequence of .Then the effectiveness of  is assessed in terms of test generation and test execution cost, respectively.From a general standpoint, the notion of cost in the context of testing is complex and can be related to many factors.In our context, the effort required for generating test sequences is measured in terms of test sequence computational complexity.Test sequence execution cost is measured by the length of test sequences.Although these are clearly approximation methods, for practical reasons such methods have been commonly used in a number of testing studies [14][15][16].
(1) Soundness Analysis (i) Check whether every transition defined in  is verified in the implementation All the maximum-length IDOSs in  are included in  since  is a sequence over .Algorithm 1 indicates that every maximum-length IDOS in  is followed by a UIO sequence that verifies the last transition of the sequence.According to Definition 2, i.e., for any maximum-length IDOS ( 1 ,  2 ,  1 / 1 )( 2 ,  3 ,  2 / 2 ) . . .(  ,  +1 ,   /  ), if (  ,  +1 ,   /  ) is verified then every transition (  ,  +1 ,   /  ) (1 ⩽  ⩽  − 1) is verified, it is inferred that transitions of all the maximum-length IDOSs in  are verified in .According to Definition 4, i.e., every transition in  is included in one and only one IDOS of , it is concluded that every transition defined in  is verified in the implementation by .
(ii) Check whether every state in  is defined in the implementation  is supposed to be strongly connected such that for any state  of  there is at least one transition ending at .It is proved that every transition defined in  is verified in  by identifying the end state of the transition; i.e., every state of  is checked in the implementation by .
(2) Effectiveness Analysis (i) Analysis of Computational Complexity.The test sequence  has the same computational complexity as those of Aho et al. and Hierons since all the three test sequence generation methods are based on the max flow/min cost problem and the networks used in every method are of the same order.
(ii) Analysis of Length Reduction.As mentioned above, transitions in a test sequence are of three kinds and the cost of a test sequence comes from the UIO sequences which have been appended for the purpose of transition verification and the transitions which have been appended for the purpose of concatenation of test subsequences to generate a minimumlength rural postman tour.
For asymmetric FSMs with only invertible transitions, all the transitions are divided into a least number of maximumlength IDOSs; i.e., all the transitions are verified by a least number of appended UIO sequences such that the cost of  from the appended UIO sequences is minimal.For any maximum-length IDOS ( 1 ,  2 ,  1 / 1 )( 2 ,  3 ,  2 / 2 ). ..(  ,  +1 ,   /  ), there is a test subsequence ( 1 ,  2 ,  1 / 1 )( 2 ,  3 ,  2 / 2 ). ..(  ,  +1 ,   /  )⋅ where  is a UIO sequence of  +1 . is obtained from a minimum-length rural postman tour covering all the test subsequences.The minimum-length rural postman tour is generated by the rural symmetric augmentation and thus the cost of  for concatenation comes from the replicated transitions during the rural symmetric augmentation.Benefiting from multiple UIO sequences, a minimum number of transition replications is reached by a sensible choice of UIO sequences during the rural symmetric augmentation; i.e., the cost of  from the transitions for concatenation is minimal.
For FSMs with noninvertible transitions, transitions excluding noninvertible ones are divided into a least number of IDOSs.According to Theorem 3, the acquired IDOSs concatenate with noninvertible transitions as much as possible such that a set of maximum-length IDOSs with a least number of maximum-length IDOSs is obtained; i.e., all the transitions of an FSM with noninvertible transitions are verified by a least number of appended UIO sequences such that the cost of  from the appended UIO sequences is minimal.Similarly, a minimum number of transition replications during the rural symmetric augmentation is reached by a sensible choice of UIO sequences; i.e., the cost of  from the transitions for concatenation is minimal.
In short, for all types of FSMs, the execution cost is reduced effectively without increasing the generation cost such that  is a reduced test sequence of .

Case Study
We experiment with the aforementioned  1 ,  2 , and  3 which are random generation of different types of FSMs. 4 and  5 are also randomly generated FSMs; moreover, they are example FSMs used by Bo Yang et al. and Hierons, respectively.While the experimental results shed some light on how the improved method behaves with randomly generated FSMs, they bring no insight on the test sequence reduction of FSMs that are produced by software designers.For this reason, experiments on FSMs modeling INRES protocol [17], GUI for password modification in a property management system [18], page function of Gmail system [19] and the connection and release process of MAC layer in Zigbee protocol are carried out, and the FSM modeling page function of Gmail system is adjusted to satisfy the strong connectivity and UIO availability assumptions.
The FSMs used in the experiment are admittedly small.However, it is important to note that FSMs are mostly used to model the behavior of complex classes or class clusters, particularly complex control classes in protocols or reactive systems.They are rarely used to model entire systems which will result in large and unmanageable models for software engineers and testers.Completeness degree is a general factor to reveal the complexity of both large and small FSMs such that test sequence reduction of large FSMs can to some extent be learned through relatively small FSMs with the same distribution of completeness degree.Given an FSM with  inputs,  transitions and  states, completeness degree of  is denoted by /( × ); moreover, the higher the completeness degree is, the more complex the FSM is.In this section, completeness degrees of FSMs range from 0.24 to 1.
The associated data of the experiment is illustrated in   [29].Locating sequences are used to make sure that every element of a characterizing set is applied to the same state.Jourdan et al. generated shorter test sequences by means of reducing the number of locating sequences [30].Baumgartner et al. proposed a mixed integer nonlinear programming (MINLP) model to formalize how the total cost of testing depends on the sequence and the parameters of the elementary test steps [31].To provide an efficient formalization of the scheduling problem and avoid difficulties due to the evaluation of an objective function during the relaxation of the integer variables, the MINLP was formulated as a process network synthesis problem.Hierons et al. affirmed the importance of invertibility in test sequence reduction and considered three optimisation problems associated with invertible sequences [32].Petrenko et al. addressed the problem of extending the checking experiment theory to cover a class of FSMs with symbolic extensions [33].They also reported the results that further lift the theory of checking experiments for Mealy machines with symbolic inputs and symbolic outputs.Hierons et al. described an efficient parallel algorithm that uses manycore GPUs for automatically deriving UIOs from Finite State Machines [34].The proposed algorithm uses the global scope of the GPU's global memory through coalesced memory access and minimizes the transfer between CPU and GPU memory.Song et al. introduced a practical conformance testing tool that generates high-coverage test input packets using a conformance test suite and symbolic execution.This approach can be viewed as the combination of conformance testing and symbolic execution [35].Bokil et al. presented an automated black box test suite generation technique for reactive systems [36].The technique is based on dynamic mining of specifications in form of an FSM from initial runs.The set of test cases thus produced contain several redundant test cases, many of which are eliminated by a simple greedy test suite reduction algorithm to give the final test suite.

Conclusions
Taking Zigbee protocol as an example, this paper introduces how FSM-based conformance testing works in wireless protocol conformance testing of IoT.An improved method in which both overlapping by invertibility and multiple UIO sequences are considered is proposed to achieve test sequence reduction for wireless protocol conformance testing of IoT.Based on invertibility, transitions of all types of FSMs are verified with as few appended UIO sequences as possible.Multiple UIO sequences contribute to generate a shorter test sequence by means of reducing transitions for concatenation of test subsequences in the test sequence.Moreover, test sequences can be further reduced by removing the transition sequence which follows the UIO sequence of the maximumlength IDOS that is verified last in the tour.Theory and experiment indicate that the execution cost is reduced effectively by the improved method under the premise of not increasing the generation cost.The improved method is also applicable to traditional protocol conformance testing as well as reactive systems.Numerous experimental data and practical examples about wireless protocols of IoT will be gathered in the future work to analyze the effectiveness of the method.

Figure 1 :
Figure 1: FSM corresponding to the connection and release process of Zigbee.

Table 3 :
UIO sequence of  2 .firsttwo types of FSMs, noninvertible transitions are removed from the FSM and saved to a set.The remainder excluding noninvertible transitions is either an FSM or more than one connected component with only invertible transitions.And thus maximum-length IDOSs of the remainder excluding noninvertible transitions are generated in the same way as symmetric or asymmetric FSMs with only invertible transitions.Naturally, it comes to the same conclusion as the first two types of FSMs that a random set of maximum- length IDOSs will do when there are nonunique sets of maximum-length IDOSs.Next, a union of maximum-length IDOSs of the remainder excluding noninvertible transitions and all the noninvertible transitions is derived; moreover, whenever the start state of a maximum-length IDOS is the end state of a noninvertible transition, the maximum-length IDOS is concatenated with the noninvertible transition.If a maximum-length IDOS is an Euler tour then a node which is the end state of a noninvertible transition is preferred to be the start state of the tour.The set after all possible concatenations is a set of maximum-length IDOSs of the FSM with noninvertible transitions.

Table 5 ,
||, || and || denote the number of inputs, states and transitions of every FSM.Completeness degree of every FSM is calculated and listed. + ,  V , and  V+ are test sequences resulting from Bo Yang et al., Hierons, and the improved method, respectively.| + |, | V |, and | V+ | represent the length of the corresponding test sequences.For symmetric FSMs with only invertible transitions,  V and  V+ are both Euler tours starting from the initial state followed by a minimum-length UIO sequence of the initial state.Thus, | V | and | V+ | are always the same for symmetric FSMs with only invertible transitions.When the Euler tour conforms to the definition of fully overlapping transition sequences (FOTSs) [11], | + | is the same as | V | and | V+ |; otherwise | + | tends to be longer because of more appended UIO sequences as well as the possible extra transitions for concatenation.

Table 5 :
Experimental objects and associated data.

Table 6 :
Single sample K-S check.

Table 8 :
[28]istical results from ( Â12 ).John Esquiagola et al. used the current version of their IoT platform to perform performance testing[25].Daniel Kuemper et al. described how concepts for semantically described web services can be transferred into the IoT domain[26].Philipp Rosenkranz et al. propose a testing framework which supports continuous integration techniques and allows for the integration of project contributors to volunteer hardware and software resources to the test system[27].A connective and semantic similarity clustering algorithm (CSSCA) and a hierarchical combinatorial test model based on FSM are proposed by Kai Cui et al.[28].Test sequence generation and reduction has long been an active research topic.Porto et al. used identification sets which are subsets of a characterizing set to identify states and obtained reduced test sequences