Network robots provide services to users in smart spaces while being connected to ubiquitous instruments through wireless networks in ubiquitous environments. For more effective behavior planning of network robots, it is necessary to reduce the state space by recognizing a smart space as a set of spaces. This paper proposes a space classification algorithm based on automatic graph generation and naive Bayes classification. The proposed algorithm first filters spaces in order of priority using automatically generated graphs, thereby minimizing the number of tasks that need to be predefined by a human. The filtered spaces then induce the final space classification result using naive Bayes space classification. The results of experiments conducted using virtual agents in virtual environments indicate that the performance of the proposed algorithm is better than that of conventional naive Bayes space classification.
The states of ubiquitous instruments, the daily supplies connected through the Internet of Things (IoT), are integrally managed in ubiquitous environments. In such environments, network robots can be operated in smart spaces via wireless networks. A network robot can effectively interact with ubiquitous instruments based on IoT and provide services required by senior citizens or patients in a smart space.
However, a space classification approach in which a smart space is reduced and identified as a space unit is required to execute behavior planning in real time for more practical services by network robots. Graph-based space classification and naive Bayes space classification are among a variety of space classification approaches that utilize objects identified in the environment [
In graph-based space classification, the graph is predefined by a human being. Thus, it includes the subjective thoughts of the human being predefining the graph and so the classification results vary. Naive Bayes classification needs to calculate and compare the probabilities of all spaces that can be classified. The problem with this calculation is that the number of operations is proportional to the number of spaces that can be classified. This paper proposes an approach that classifies the space efficiently by solving the problem above.
This paper makes three primary contributions. The first contribution involves classification of the space by subdividing it. Whereas conventional space classification approaches divide a space into rooms and corridors [
This paper proposes an approach that automatically generates a graph using the appearance ratio of objects without predefinition by a human being in the graph-based space classification. In the proposed approach, the space with the highest possibility is extracted by filtering the spaces that can be classified using the automatically generated graph. The spaces extracted by filtering are then classified using naive Bayes space classification and the final result identified. By utilizing our approach, the quantity of data inquiry is decreased to 44.75% in average, compared to existing naive Bayes space classification based approach. As our approach can perform space classification using much less operations than conventional approach, it is suitable to be performed in real time.
The remainder of this paper is organized as follows: Section
This section discusses previous research conducted on space classification and primarily analyzes the advantages and disadvantages of various space classification methods applicable to virtual agents. Space classification research can be classified into five main approaches: graph-based space classification, space classification using the naive Bayes classifier, predicate logic-based space classification, robot action pattern and image pattern-based space classification, and place recognition-based space classification.
Graph-based space classification classifies spaces using a graph. Its prerequisite is the classification of a room and a corridor [
Naive Bayes classifier- (NBC-) based space classification utilizes probability [
Predicate logic-based space classification predicatively predefines the objects in a space to classify the space [
Space classification using robot action patterns uses the action patterns of a robot in a specific space for space classification [
Space classification using image patterns uses the directional histogram, configured using the images acquired by executing the rotational pattern of a robot, identified in a space [
In place recognition-based space classification, a human being directly generates a topological map describing the environment using a map editor and designates the relationship between an image taken by Omnicam and the space. On the basis of the map and relationship established by the human being, the present space is classified by comparing the image taken moving around in the environment and the database [
In recent space classification research, a variety of approaches depending on environment have been proposed: graph-based space classification using a graph defined by a human being, NBC-based space classification collecting and classifying the appearance ratio of objects, predicate logic-based space classification describing objects and their relationship, space classification using robot action pattern enabling space classification without vision sensor, space classification using image patterns without the object recognition process, and space recognition identifying the present location by creating a database with the images taken in the space and matching them with the images taken in real time. While a space can be classified using these conventional approaches, most of them depend substantially on information predefined by a human being.
Consequently, this paper focuses on NBC-based space classification, which has fewer tasks to predefine, and graph-based space classification, which has the lowest number of calculation requirements. The approach proposed in this paper is differentiated from previous approaches in two aspects.
First, the proposed approach reduces the number of tasks that need to be predefined by a human being, as compared to conventional approaches. In most conventional approaches, a significant number of tasks are predefined by a human being. The approach proposed in this paper uses graph-based space classification to automatically create a graph without any tasks being predefined.
Second, the proposed approach improves the performance of NBC-based space classification. NBC-based space classification requires a significant amount of time for space classification in order to calculate the probabilities of all spaces. The proposed approach conducts classification more quickly than NBC-based space classification by calculating only the probability of specific spaces.
The overall structure of the approach proposed in this paper comprises an automatic probability based graph generation module and a priority space classification module that uses the normalized probability and the generated graph.
The automatic probability based graph generation module generates the graph by collecting the object probability (appearance ratio) identified in a space. The first knowledge base of a virtual agent has no data so that it can execute the task to classify a space. Space classification using the naive Bayes classifier requires a process in which the number of appearances of an object in a space is identified and saved in the knowledge base. The approach proposed in this paper automatically creates a graph using the appearance ratio of objects acquired by searching a space based on probability. The graph describes the relationship between an object and a space and is connected to the space with the highest appearance ratio of an object. The graph automatically generated using the appearance ratio varies according to the environment in which the appearance ratio is acquired. The graph varies according to the environment to be investigated. Thus, the closer the environment is to the real world, the closer the relationship between an object and a space acquired on the graph is to the real world.
In automatic probability based graph generation, a process that normalizes the probability and creates a graph is executed. The process reduces the quantity of data required for classification. When the probability of an object in a space is acquired, the sum of the probability of the object identified in each space is irregular. The sum of the probability is normalized as one and the ratio to total probability can be estimated only with the probability of one space. Since only one standard value is requested and compared without requesting and comparing the probability of all spaces, the quantity of data required for classification is reduced.
Priority space classification uses the graph created by the automatic probability based graph generation module and the standard value normalizing the probability. The space, which is highly possible to convert as the result of space classification, is filtered before space classification by the automatically generated graph. In accordance with the filtering result, the space classification is executed from the space with a higher possibility. A space is classified using the standard value according to the priority of the space acquired by filtering. The standard value of the space with higher priority of an identified object is requested and compared with the total standard value. If the value is greater than the total standard value, the relevant space is returned as the result. However, if it is less than the total standard value, the standard value of the next priority object is requested and the process above is continued.
Automatic probability based graph generation generates a graph by collecting the probability of an object being identified in a space (appearance ratio). To create the graph, the probability of the object being identified in each space (appearance ratio) is collected in various environments using the data collection module. The collected probability is then saved in the knowledge base and the generation module normalizes the probability of the saved knowledge base and generates a graph. The result from the automatic probability based graph generation module is saved as the automatically generated graph and the standard value normalizing probability in the knowledge base.
Section
The automatic probability based graph generation system comprises a data collection module, a generation module, and a knowledge base, as shown in Figure
Structure of the automatic probability based graph generation system.
The data collection module uses the structure of the space classified using naive Bayes classifier as is. This module recognizes the objects existing in the environment and updates the probability of objects identified in the space to the knowledge base.
The generation module normalizes the object probability data of the knowledge base collected using the data collection module and generates a graph.
The data collection module comprises three modules that save the object identification probability.
The first module is the object identification module. This module obtains the image data in the environment recognized by the virtual agent and recognizes the objects using an image processing technique that incorporates the scale invariant feature transform (SIFT) algorithm to check what kinds of objects exist in the image data. The focus of this paper is on the space classification algorithm; hence, the image processing technique is beyond its scope. In this paper, an object is identified when it is included in the sight of a virtual agent. If an object is included in the sight of a virtual agent, the object type is recognized and the results are delivered to the space classification module.
The second module is the NBC-based space classification module. This module classifies the space containing the virtual agent by calculating the probability of an object using the object recognition module. It receives the list of identified objects from the object recognition module, classifies the space, and delivers the classification results. The data collection module includes the naive Bayes classifier to identify any change in classification accuracy in accordance with the update probability of an object in the knowledge base using the probability update module:
Equation (
The third module is the probability update module. This module receives the results from the space classification module, ascertains whether the results are matched to a real space, and updates the probability of identified objects. A virtual agent at first has no data in the knowledge base so that the space classification results are not proper. The classifier can properly complete the classification by saving the appearance ratio of objects in a space using the update module and repeating the processes above in a variety of environments.
The generation module operates after the data collection module. It comprises two modules: the probability normalization module and the graph generation module.
The probability normalization module normalizes the sum of the probabilities of objects in each space to 1.0. As shown in (
Equation (
As shown in Table
Probability before normalization.
Bedroom | Living room | Kitchen | Bathroom | Total | |
---|---|---|---|---|---|
Bed | 0.9 | 0.1 | 0.0 | 0.0 | 1.0 |
Dressing table | 0.7 | 0.0 | 0.0 | 0.0 | 0.7 |
Sofa | 0.2 | 0.8 | 0.0 | 0.0 | 1.0 |
Table | 0.3 | 0.3 | 0.5 | 0.0 | 1.1 |
Desk | 0.3 | 0.3 | 0.0 | 0.0 | 0.6 |
Bathtub | 0.0 | 0.0 | 0.0 | 1.0 | 1.0 |
Probability normalization aims at space classification only by identifying the probability in one space without the probability in other spaces, as normalizing the sum of irregular probability identifying the objects is 1.0. For example, when the dressing table in Table
Probability after normalization.
Bedroom | Living room | Kitchen | Bathroom | Total | |
---|---|---|---|---|---|
Bed | 0.9 | 0.1 | 0.0 | 0.0 | 1.0 |
Dressing table | 1.0 | 0.0 | 0.0 | 0.0 | 1.0 |
Sofa | 0.2 | 0.8 | 0.0 | 0.0 | 1.0 |
Table | 0.27 | 0.27 | 0.46 | 0.0 | 1.0 |
Desk | 0.5 | 0.5 | 0.0 | 0.0 | 1.0 |
Bathtub | 0.0 | 0.0 | 0.0 | 1.0 | 1.0 |
The graph generation module generates a graph using the standard value. Unlike conventional graph-based space classification approaches, which require a human being to predefine the graph, the approach in this paper automatically generates the graph using the normalized probability value with which an object is identified. The graph is generated by connecting the space with the highest value (Max) among the standard values of objects to an object:
The graph is generated using (
Figure
The graph generation process.
Graph-based priority space classification requires advance automatic probability based graph generation. A graph is automatically generated using probability that needs to be normalized. Graph-based priority space classification classifies the space using a graph and a standard probability. It classifies the space on the assumption that there is no feedback from the environment. It also executes filtering using a graph and determines the priority of the space with higher possibility to be returned as the classification result. Further, following graph filtering, it classifies the space using the naive Bayes classifier in accordance with the order of priority.
This section proposes graph-based priority space classification, which reduces the time for classification compared to existing NBC-based approaches that use generated graphs. Section
The graph-based priority space classification system comprises an object identification module, a graph filtering module, and the priority naive Bayes classification module, as shown in Figure
Structure of the graph-based priority space classification system.
The object identification module identifies the object in the sight of the virtual agent. The graph filtering module receives data describing the identified object from the object identification module, searches for the graph, and filters it. In accordance with the filtering results, the spaces with higher possibility are arranged in order and the results returned. The priority naive Bayes classification module receives the graph filtering result, calculates the probability from the space with higher possibility to be returned in accordance with the classification results, and delivers the space classification results.
The graph filtering module receives the data associated with an identified object from the object identification module and filters the space in the order of higher priority to be returned as the final classification result. The filtering result is delivered to the priority naive Bayes classification module. Graph filtering searches for the identified object in a graph and determines the order of calculation depending on which space is connected. Algorithm
For Except ( End For
Example of graph filtering.
If a bed and a desk object are identified, both objects are searched in the graph to find the space connected with each object. In accordance with the search results, the bed is connected to a bedroom and the desk to a living room. After completing the search, the connected spaces are counted and filtered. In accordance with the filtering, the space is filtered in the order of bedroom, living room, kitchen, and bathroom.
Priority naive Bayes classification classifies a space using the graph filtering result and the standard value. The standard value compares the probability of a specific space and total standard value without the specific space (1 − standard value). When the space with the highest probability is returned in accordance with the graph filtering and space classification, only the standard value of one space is searched and the space can be classified. Algorithm
For If Return End If End For
Equation (
Equation (
Figure
Space classification by priority naive Bayes classifier.
This section compares the experimental results obtained using the graph-based priority space classification proposed in this paper with those obtained using the conventional naive Bayes space classification. Experiments were executed in a 3D virtual environment generated based on the interior design book. The experiments analyzed the performance of the existing classification approach and proposed a classification approach on the basis of two standards: quantity of data inquiry from knowledge base for implementation of classification algorithm and accuracy on whether the classification result was identical to the interior design book.
Section
The aim of the experiments was to verify that graph-based priority space classification has less data inquiry than the conventional naive Bayes space classification with the same classification accuracy.
Two hundred virtual environments were produced in an invisible file to identify the probability with which an object is identified in a space. In the experiments, 10 kinds of 3D virtual environments were created, listed in Table
The experimental 3D virtual environments.
Virtual environment number | Configuration image |
---|---|
1 |
|
|
|
2 |
|
|
|
3 |
|
|
|
4 |
|
|
|
5 |
|
|
|
6 |
|
|
|
7 |
|
|
|
8 |
|
|
|
9 |
|
|
|
10 |
|
The experiments assumed 10, 50, 100, 150, and 200 virtual spaces to collect the probability with which an object is identified in a space and generate a graph. Ten kinds of 3D virtual environments were used to identify changes in the accuracy and the quantity of data inquiry carried out by the conventional naive Bayes classification and the graph-based priority space classification. Subsequently, the probability of object identification was collected from 200 spaces and a graph generated. Finally, the conventional naive Bayes classification and the graph-based priority space classification were compared in each 3D virtual environment.
The experiments were implemented in the 3D virtual environment using the Gamebryo engine. The Gamebryo-based 3D virtual environment was configured as shown in Figure
Gamebryo-based 3D virtual environment.
In the experimental environment, a virtual agent moved to each space with the objects in its sight displayed on the bottom left. A door was used to determine whether the virtual agent had moved to another space. When the virtual agent went to another space through a door, the data on the object identified in the previous space was initialized. The virtual agent identified the objects when they were included in the area of frustum indicating the sight of the character. (Frustum is inserted to describe the sight and was not seen during the experiment.) When objects in a space were completely identified, the space was either classified using the existing naive Bayes space classification or graph-based priority space classification. The classification results are displayed in the top left of the screen.
The experimental results for the existing naive Bayes space classification and the graph-based priority space classification are divided into two kinds depending on the experimental approach. The results obtained for the experiment in which 10, 50, 100, 150, and 200 virtual spaces were made to generate a space and the performance in 10 kinds of 3D virtual environments compared are presented in Figures
Change in the quantity of data inquiry depending on the number of virtual spaces.
Change in accuracy in the virtual space.
Figure
Figure
Next, the performances in 200 virtual spaces and 10 kinds of 3D virtual environments were compared. The graph-based priority space classification proposed in this paper reduced the quantity of data inquiry with the same accuracy as the conventional naive Bayes space classification. In accordance with the experimental results, space classification was executed a total of 40 times because there were 10 kinds of 3D virtual environments, with four spaces for each environment.
Figure
Quantity of data inquiry.
Figure
Reduction ratio in quantity of data inquiry.
Figure
Accuracy of space classification.
This paper proposed a space classification approach that uses standard probability and automatically generated graphs for virtual agents. The proposed approach is designed to satisfy real-time virtual environments; consequently, the proposed space classification generates a graph without predefinition. The performance of the graph-based priority space classification was found to be better than that of conventional naive Bayes space classification.
The results of the experiments conducted indicate that the proposed approach reduces the quantity of data inquiry, the graph filtering results improved as the number of virtual spaces increased, compared to conventional naive Bayes space classification. The proposed approach reduced the quantity of data inquiry by 44.75% on average compared to the conventional approach in 10 kinds of 3D virtual environments. Further, even though the quantity of data inquiry was reduced, the classification accuracy remained the same as that of conventional naive Bayes space classification. The approach proposed in this paper can be applied to behavior planning for network robots in ubiquitous environments.
Further study that considers the relationship among objects for establishing more intelligent action plans on the basis of the space classification approach is needed. Furthermore, it is necessary to study how to apply the proposed algorithm in real environments. To expand the approach proposed in this work for facilitating it in more large and complex smart spaces, a spatial feature should be considered additionally, as well as an object feature. Therefore, research of space classification algorithm based on machine learning (e.g., support vector machine algorithm) should be preceded, in order to apply learning and recognition of multidimensional feature vectors.
The authors declare that there is no conflict of interests regarding the publication of this paper.
This research was supported by the MSIP (Ministry of Science, ICT and Future Planning), Korea, under the ITRC (Information Technology Research Center) support program (NIPA-2014-H0301-14-1021) supervised by the NIPA (National IT Industry Promotion Agency).