Ubiquitous Computing is moving the interaction away from the human-computer paradigm and towards the creation of smart environments that users and things, from the IoT perspective, interact with. User modeling and adaptation is consistently present having the human user as a constant but pervasive interaction introduces the need for context incorporation towards context-aware smart environments. The current article discusses both aspects of the user modeling and adaptation as well as context awareness and incorporation into the smart home domain. Users are modeled as fuzzy personas and these models are semantically related. Context information is collected via sensors and corresponds to various aspects of the pervasive interaction such as temperature and humidity, but also smart city sensors and services. This context information enhances the smart home environment via the incorporation of user defined home rules. Semantic Web technologies support the knowledge representation of this ecosystem while the overall architecture has been experimentally verified using input from the SmartSantander smart city and applying it to the SandS smart home within FIRE and FIWARE frameworks.
Although in their initial definition and development stages pervasive computing practices did not necessarily rely on the use of the Internet, current trends show the emergence of many convergence points with the Internet of Things (IoT) paradigm, where objects are identified as Internet resources and can be accessed and utilized as such. In the same time, the Human-Computer Interaction (HCI) paradigm in the domain of domotics has widened its scope considerably, placing the human inhabitant in a pervasive environment and in a continuous interaction with smart objects and appliances. Smart homes that additionally adhere to the IoT approach consider that this data continuously produced by appliances, sensors, and humans can be processed and assessed collaboratively, remotely, and even socially. In the present paper, we try to build a new knowledge representation framework where we first place the human user in the center of this interaction. We then propose to break down the multitude of possible user behaviors to a few prototypical user models and then to resynthesize them using fuzzy reasoning. Then, we discuss the ubiquity of context information in relation to the user and the difficulty of proposing a universal formalization framework for the open world. We show that, by restricting user-related context to the smart home environment, we can reliably define simple rule structures that correlate specific sensor input data and user actions that can be used to trigger arbitrary smart home events. This rationale is then evolved to a higher level semantic representation of the domotic ecosystem in which complex home rules can be defined using Semantic Web technologies.
It is thus observed that a smart home using pervasive and semantic technologies in which the human user is in the center of the interaction has to be adaptive (its behavior can change in response to a person’s actions and environment) and personalized (its behavior can be tailored to the user’s needs and expressed using more advanced and complex home rules). In the case of smart homes, the user’s acceptance has become one of the key factors to determine the success of the system. If the home system aims to be universally usable, it will have to accommodate a diverse set of users [
To be more precise, a user model [
Recently, the emergence of ubiquitous or pervasive computing technologies that offer “anytime, anywhere, anyone” computing by decoupling users from devices has introduced the challenge of context-aware user modeling. So far, most of the context-aware systems focus on the external context known as physical context which refers to context data collected by physical sensors. Thus, they involve context data of physical environment, distance, temperature, sound, air pressure, lighting levels, and so forth. The external context is important and very useful for context-aware systems, as context-aware systems provide recommended services. However, from a broader scope, context may be considered as information used to characterize the situation of an entity [
The semantic formalization idea is to provide a functional ontological and reasoning platform that offers unified data access, processing, and services on top of the existing IoT-A ubiquitous services and to integrate heterogeneous home sensors and actuators in a uniform way. From an application perspective, a set of basic services encapsulates sensor and actuators network infrastructures hiding the underlying layers with the network communication details and heterogeneous sensor hardware and lower-level protocols. A heterogeneous networking environment indeed calls for means to hide the complexity from the end-user as well as applications by providing intelligent and adaptable connectivity services, thus providing an efficient application development framework. Thus, to face the coexistence of many heterogeneous sets of things and home appliances, a common trend in IoT applications is the adoption of an abstraction layer capable of harmonizing the access to the different devices with a common language and procedure [
The scope of most of the applications or services with respect to smart homes so far has focused on the concept of small regions like laboratory, school, hospital, smart room, and so forth. Furthermore, algorithmic and strategic models for gaining the revenue by using context-aware systems are very few. Additionally, technologies related to context-aware systems are merely standardized. The architecture, the context modeling method, the algorithm, and the network implementation as well as the devices of users in each project are different. Moreover, middleware, applications, and services make use of different level of contexts and adapt the way they behave based on the current context. Therefore, according to the level and type of contexts along with the goal of context-aware systems, the context modeling process, the inference algorithm, and interaction method of personas (humans known as personas for computational representation purposes) are changed. Although the interaction between personas and cooperation between components of the same architecture have been investigated, standard interaction, cooperation, and operation in the different context-aware systems have not been studied. Thus, the novelty of our proposed approach is to provide a common context-aware architecture system in which the user (“eahouker” in SandS) is able to control his household appliances in a collective way via the SNS (Social Network Service) and in an intelligent way via the adaptive social network intelligence. As our system is human-centered, the UM (user modeling) is related to the user’s activity inside the ESN (Eahoukers Social Network), while the context-aware environment refers to the contextual information that characterizes the situation and conditions of the system’s entities.
Finally, the modeling of the contextual information is completed through the capture of the semantics of the relationships between the user and the various entities of the ecosystem (other users, appliances, and recipes) to further improve the overall user experience. The semantic description framework of our proposed approach is based on a number of home rules that are defined for a specific household and eahouker. Since the SandS architecture consists of two layers, high and low, respectively, we have on the one hand recipes for common household tasks produced and exchanged in the SandS Social Network that are described in near-natural language. Additionally, on the other hand, we have every user’s context which consists of the actual appliances that the user has in house with their particular characteristics (type, model, brand, etc.). Finally, to ensure the executability and compatibility of a recipe and to deal as well with any uncertainty and vagueness in modeling the contextual information, a number of some axioms, to enforce constraints to all objects (things in IoT paradigm) of the ecosystem, have been introduced in the proposed Web Ontology Language (OWL) that was adopted. To conclude, the experimental results for the above framework are presented which have been conducted inside the “Social & Smart” (SandS) [
As correctly stated in [
Reviewing how “user models” term has been approached, within the HCI literature, it is indicated that users are part of an enlarged communication group in which users change through time and according to the environmental conditions and the experience they gain. Thus, in the end, there are three types of users: “novel,” “intermediate,” and “expert” [
Within the area of multimedia content, the work presented in [
Based on ontology approaches to characterize users capabilities within adaptive environments, in 2007, the GUMO ontology has been proposed [
Typically, a user model represents a collection of personal data associated with a specific user of a system. Following a similar definition, a user model [
As one may expect, there are also different design patterns for user models, though often a mixture of them is used [
A persona is an archetypal user that is derived from specific profile data to create a representative user containing general characteristics about the users and user groups and is used as a powerful complement to other usability methods, whereas it is more tangible, less ambiguous, easier to envision, and easier to empathize with. The use of personas is an increasingly popular way to customize, incorporate, and share the research about users [
Personas development supports the design process by identifying and prioritizing the roles and user characteristics of a system’s key audience. In the general case, personas development is initiated by introducing assumptions about user profiles, based on data from initial research steps conducted. Through interviews and observation, researchers expand and validate the profiles by identifying goals, motivations, contextual influences, and typical user stories for each profile. Having such a fictional person (persona) representing a profile grounds the design effort in so-called “real users.” For each persona, the user modeling description typically includes key attributes and user characteristics, such as name, age, and information that distinguishes each persona from others.
The herein proposed approach for modeling user information following a personas-based inspiration is discussed within this subsection. More specifically, according to the notation followed within our system, the so-called “eahouker profile” (
In a more formal manner, the profile of an eahouker
As a last point to consider and in order to further illustrate the herein proposed approach, we provide an example of a typical eahouker persona: the Papadopoulos family composed of four family members, namely, the parents, John and Maria, and their children, Nikos and Ioanna. Their household is located in Athens, Greece, and it contains five smart household appliances: A Samsung 55′′ TV set, model UN55F6300 An AEG washing machine, model AEG L60260 A Nescafe coffee machine, model KP1006 An LG refrigerator, model LFX31995ST A GE bread maker, model GE106732
Potential users are of course
Let us consider a set of eahoukers
In addition, the use of ontologies for capturing knowledge from a domain of interest has grown significantly lately; thus, we also consider a domain ontology
Efficient user model representation formalism using ontologies [
As far as a relevant mathematical notation is concerned, given a universe
Apart from the above described set of concepts, we need to introduce and illustrate a set depicting potential relations between the aforementioned concepts. Thus, we introduce
In order to define, extract, and use a set of concepts, we rely on the semantics of their fuzzy semantic relations. As discussed in Section
Based on the relations
Last but not least, thing to consider in our approach is the actual selection of meaningful relations to consider for the production of combined relation
Semantic relations used for generation of combined relation
Name | Inverse | Symbol | Meaning | Example | |
---|---|---|---|---|---|
|
| ||||
Belongs | Owns |
|
|
House | Device |
Manufactured by | Constructs |
|
|
Siemens | Fridge |
Friend | NotRelated |
|
|
George | Bruno |
Execute | ExecutedBy |
|
|
Recipe | User |
Triggers | TriggeredBy |
|
|
Recipe | Rule |
It is worth noticing that all relations depicted within Table
Relation
As observed in Figure
Concepts and relations example.
So far and in compliance with the notion introduced in [
In order for us to provide a measure for the evaluation of similarity between two eahoukers’ profiles, we first need to establish an evaluation of similarity for each profile component. In the following, we define a set of functions
Two eahoukers are considered identical if their gender, city, role in the house, and marital status are the same. This property is expressed through functions Two eahoukers are considered identical if their difference of age is less than 5 years. Indeed, their behavior and habits inside the house can be considered the same even if they have a slight difference of age. For example, two people, one at the age of 30 and one at the age of 32, probably would have the same behaviors, according to their age. On the other hand, a person at the age of 30 would have quite different behaviors from a person at the age of 50 or 60. This property is expressed by the function Finally, two eahoukers are considered identical if they have more or less the same number of children. For example, a parent with 3 children would have similar behaviors and demands to a parent of 4 children. This property is expressed by the function
Having introduced the functions for the evaluation of profile similarity, we can define a function that uses these evaluations to provide the level of similarity of two eahoukers. Let
Filling a home with sensors and controlling devices by a computer are nowadays not only possible, but also common. Sensors are available off the shelf which localize movement in the home, provide readings for light and temperature levels, and monitor usage of doors, phones, and appliances. Small inexpensive sensors are attached to objects not only to register their presence but also to record histories of recent social interactions [
As social interaction is an aspect of our daily life; social signals have long been recognized as important for establishing relationships, but only with the introduction of sensed environments where researchers have become able to monitor these signals. Hence, it is possible to look at socialization within the smart home and cities (such as entertaining guests, interacting with residents, or making phone calls) and examine the correlation between the socialization parameters and productivity, behavioral patterns, or even health. These results will help researchers not just to understand social interactions but also to design products and behavioral interventions that will promote more social interactions.
Proliferation of sensors in the home results in large amounts of raw data that must be analyzed to extract relevant information. Most smart home data from environmental sensors can be processed with a small computer. Once data is gathered from wearable sensors and smartphones (largely accelerometers and gyroscopes, sometimes adding camera, microphone, and physiological data), the amount of data may get too large to handle on a single computer, and cloud computing might be more appropriate. Cloud computing is also useful if data are collected for an entire community of smart homes to analyze community-wide trends and behaviors.
Collecting and handling with concurrently enormous ubiquitous data, information, and knowledge that have different formats within the SmartSantander [
In everyday social contextual situations, humans are able to, in real time, perceive, combine, process, respond to, and evaluate a multitude of information including semantics meaning of the content of an interaction, nonverbal information such as facial and body gestures, subtle vocal cues, and context, that is, events happening in the environment. Multimodal cues unfold, sometimes asynchronously, and continuously express the interlocutors’ underlying affective and cognitive states, which evolve through time and are often influenced by environmental and social contextual parameters that entail ambiguities. These ambiguities with respect to contextual aspect range from the multimodal nature of emotional expressions in different situational interactional patterns [
Understanding that the human behavior in terms of decision-making process is inherently a multidisciplinary problem involving different research fields, such as psychology, linguistics, computer vision, and machine learning, there is no doubt that the progress in machine understanding of human interactive behavior and personality is contingent on the progress in the research in each of those fields.
Attempting to provide a formal definition for context-aware applications and Human-Computer Interaction (HCI) systems, a starting point would be to investigate how the term context has been defined. The word “context” has a multitude of meanings even within the field of Computer Science (CS). To illustrate this, we group the different definitions of the term context in the area of artificial intelligence, natural language processing, image analysis, and mobile computing, where every discipline has its very own understanding of what context is.
According to the first work which introduced the term context awareness in CS [
Based on context’s broader approach [
With an entity’s location, we can determine what other objects or people are near the entity and what activity is occurring near the entity. From these examples, it should be evident that the primary pieces of context for one entity can be used as indices to find secondary context (e.g., geolocalization) for that same entity as well as primary context for other related entities (e.g., proximity to other homes). This context model was later enlarged [
Identity specifies personal user information like gender, age, children, social and marital status, and so forth. Time, in addition to its intuitive meaning, can utilize overlay models to depict events like working hours, holidays, days of week, and so on. Location refers either to geographical location or to symbolic location (e.g., at home, in the shop, or at work). Activity relates to what is occurring in the situation. It concerns both the activity of the entity itself and the activities in the surroundings of the entity.
For real-world context-aware HCI computing frameworks, context is defined as any information that can be used to characterize the situation that is relevant to the interaction between the users and the system [
All these context-aware systems that model the relevant context parameters of the environment depend on the application domain and hence face difficulties in modeling context in an independent way and also lack of models to be compared. Setting aside the fact that sometimes the domains such as context-aware computing, pervasive environments, and Ubiquitous Computing entail similarities with respect to the necessity of managing context knowledge, the concrete applications and approaches domains are different. In the area of pervasive computing, the work of [
Unfortunately, such ambiguities with respect to the human behavior data understanding are usually context independent due to the fact that the human behavioral signals are easily misinterpreted if the information about the situation in which the shown behavioral cues have been displayed is not taken into account. Thus, to date, the proposed methodology has approached one or more of the above presented contextual aspects either separately or in groups of two or three using the information extracted from multimodal input streams [
An issue related to the use of data collected continuously [
Recently, many face analysis research works have gradually shifted to facial images captured in the wild with the introduction of Labelled Faces in the Wild (LFW) [
Aligned with the aforementioned trend of collecting contextual data in nonstandard situations (in the wild), there also has been much work in creating large-scale semantic ontologies and datasets. Typically, such vocabularies are defined according to utility for retrieval, coverage, diversity, availability, and reusability. Moreover, semantic concepts such as objects, locations, and activities in visual data can be easily automatically detected [
Nevertheless, not all of them are full of rich meta information such as the entities involved, the situational context, the demographic aspects, their social status, their cultural background, and their dialect and, thus, it is not certain whether tasks such as these can be used to make reliable generalizations about natural conversation [
However, the main criticism of that type of data is that they do not address all aspects of social interactions. Consequently, the existing resources should be revisited and repurposed every time new research questions arise. The above presented reasons justify the quality of data that we have so far, where the context is relatively stable (meetings, radio programs, laboratory sessions, etc.) and the variability related to such a factor is limited. Thus, there is a need for having mechanisms to collect feedback from users in the wild (such as software systems upon smartphones that ran continuously in the background to monitor user’s mood and emotional states), to further establish large-scale spontaneous affect databases efficiently with very low cost [
Mobile devices can collect a large amount of contextual information (geographic position, proximity to other people, audio environment, etc.) for extended periods of time. Big Data analytics can make sense of that data and provide information about context and its effect on behavior. Thus, it is possible to overcome limitations such as the collection of affect-related data in a large population as well as having involved participants in the experiment for too long. With the advent of powerful smart devices with built-in microphones [
Due to the huge growth of collecting wearable data in the wild and access to more contextual information, respectively, affect analysis has recently started to move into the realm of Big Data. For example, in terms of physiological data, having enough participants being able to own and wear sensors at all times and being willing to allow contextual data to be collected from their phones, it might allow a large collection of physiological signals with high-confidence affect labels. Data could then be labelled with both self-report and contextual information such as time of day, weather, activity, and who the subject was with so as to make an assessment of affective state. Consequently, with sufficiently ground truth datasets, it will likely be able to develop better contextually aware algorithms for individuals and like groups even if the sensor data are noisier. These algorithms will enable HCI in a private, personal, and continuous way and allow our sensors to both know us better and be able to communicate more effectively on our behalf with the world around us. Taking into account the fact that personalization is desirable, that is, the system adapts itself to the user by regarding this behavior, emotions, and intentions, specifically this leads to technologies with companion-like characteristics [
Another important issue is the interplay among the personality, the situational context, and the contextualized behavior. The problem of context has been controversial in the HCI community [
So far, the issue is still open for technologies dealing with social and psychological phenomena like personality [
Particularly, data from smart wearable devices can indicate personality traits using machine learning approaches to extract useful features, providing fruitful pathways to study relationships between users and personalities, by building social networks with the rich contextual information available in applications usage, call, and SMS logs. “Designing” smart homes in terms of enhancing the comfort is also challenging for mobile emotion detection. The friendly design of an intelligent ecosystem responsive to our needs that can make users feel more comfortable for affective feedback collection and may change user’s social behavior is very promising to boost the affect detection performance and explore the possibility of further HCI techniques.
Moreover, it is necessary to discover new emotional features, which may exist in application logs, smart device usage patterns, locations, order histories, and so forth. There is a great need to thoroughly monitor and investigate the new personality and behavioral features. In other words, establishing new HCI databases in terms of new social features could be a very significant research topic and could bring “ambient intelligence” in the home closer to reality.
Gradually, the new multidisciplinary area that lies at the crossroads between Human-Computer Interaction (HCI), social sciences, linguistics, psychology, and context awareness is distinguishing itself as a separate field. It is thus possible to better recognize, interpret, and process “recipes,” to incorporate contextual information, and finally to understand the related ethical issues about the creation of homes that can enhance shelter. For applications in fields such as real-time HCI and big social data analysis [
Semantic context concept-based approaches [
Context-level analysis also ensures that all gathered rules are relevant for the specific user. In the era of social context (where intelligent systems have access to a great deal of personal identities and social dependencies), such rules will be tailored to each user’s preferences and intent. Irrelevant opinions will be accordingly filtered with respect to their source (e.g., a relevant circle of friends or users with similar interests) and intent.
Context data in a smart pervasive environment such as a smart home can come from various sources as follows: In-place sensors such as temperature, humidity, luminosity, noise, or human presence sensors located in the various rooms or outside, in the vicinity of the house Power and water consumption meters of the house Smart city sensors providing additional information such as pollution levels, temperature, and total electrical power consumption of the city, optionally with geospatial information
Users sometimes need their appliances to perform a specific action in their house taking into account the context information. For example, they may not want to wash clothes when it is raining or the temperature in the city is quite low. For this reason, there are defined actions for the smart home system. These actions are called home rules. These home rules are handling whether the appliances should be switched on or off.
In a more high-level approach, the structure of the home rules can be customized as “if it is valid, do/do not do that.” Figure
Home rules structure.
The “if it is valid, do/do not do that” structure consists of three parts: “If it is valid,” a trigger that consists of the following:
An input type and the value of the input that is defined by pervasive and context information such as the ones described in Section An operator A reference value, which is input by the user (e.g., 20 degrees Celsius) “Do/do not,” what to do when the rule is triggered, where any smart home system action/reaction can be inserted “That,” which consists of an optional parameter (e.g., lower the house blinds by using that percentage)
Moreover, more complex rules such as the temperature in specific interval of values are expressed with multiple rules that are logically joined together.
In this section, semantic technologies are used in order to represent the knowledge of an ecosystem. In general terms, an ecosystem with respect to the Internet of Things (IoT) which is often considered as the next step in Ubiquitous Computing [
In this ecosystem, we can define a number of rules, which we will call home rules, for example, defining under which conditions house appliances should be switched on or off. Another more concrete example would be “do not operate the air-condition when the outside temperature is high.”
The OWL 2 Web Ontology Language (OWL 2) [
Figure The Appliances which contain all the different types of the ecosystem’s appliances, such as (a) the refrigerator, (b) the washing machine, (c) the air-condition, and (d) the television The Location, which contains both the house and the city The Sensor, which is a class that contains the individuals of all the existing sensors The Person, which contains all the individuals The Gender, the HouseRole, and the SocialStatus which for the different types of gender, house roles, and social status implement the user model
An example of the ontology properties, the hierarchical structure, and the individuals used for our experiments.
Object properties
Data properties
Ontology hierarchy
Ontology individuals
The ontology also comprises a series of properties. These properties are both object properties and data properties. Object properties provide ways to relate two objects (also called predicates). Object properties relate two objects (classes), of which one is the domain and the other is the range. The object properties of the ontology of this ecosystem are mainly used to relate the sensors with a specific location and the inhabitants of the house and the appliances. Some of the ontology’s object properties are described below: hasGender, which relates a Person class with a Gender class according to Section hasSensor, which relates a Sensor class with a specific location hasHouseRole, which relates a Person class with a house role isLocatedIn, which relates a house with a city livesIn, which relates a person with a house builtIn, which relates a house with a city
On the other hand, data properties are similar to object properties with the sole difference that their domains are typed words. In our ontology, they relate the actual sensor values with a sensor, power on or off status of the appliances, and the user properties with numerical features. Some of them are described below: hasNoise, which relates a sensor with the actual captured noise value, for example, 40 dB hasTemperature, which relates a sensor with the actual captured temperature value, for example, 25°C isOn, which has a true value if the appliance is turned on and is false otherwise numberOfChildren, which relates a person with the number of his/her children, which must be a nonnegative integer
The object’s and the data’s properties of the ontology appear in Figure
The ecosystem in all contains a large number of appliances, sensors, and people. Every single appliance, sensor, and person is represented in the ontology as an individual of the Appliances, Sensor, or Person class, respectively. Figure
In the current section, we provide a novel semantic representation of the home rules of the ecosystem. These home rules are expressed using the Semantic Web Rule Language (SWRL) [
For this reason, a data restriction has to be created in the Appliances class. A data property called “restriction” is created. Its domain is an appliance and its range is boolean, but it is also restricted to create an appliance with the restriction property. Then, every home rule is transformed to a SWRL, and if the left side of the rule is satisfied, it leads to the creation of the “restriction” property for an appliance. This makes our ontolgy inconsistent; in other words, the appliance is restricted to start working. So every time a database record changes or a new one is added, the ontology individuals are populated with the new values querying the database. Then, using the Pellet reasoner, the system checks for possible existence of any inconsistency. Finally, the inconsistency is being handled by forcing the appliance to switch off or switch on. Using the Semantic Web technologies, the restriction is added to every appliance in order not to create any restriction data property for any individual of the class after the reasoning. In this subsection, some indicative home rules transformed to SWRLs are presented. Do not operate any washing machine when the external temperature is greater than 26°C:
The washing machine must not be operating if a person is in the house and there exists too much noise:
If the local time is between 10 p.m. and 8 a.m., the television must not be switched on:
As it is clear, the built-ins for SWRLs, such as “equal,” “lessThan,” “greaterThan,” “lessThanOrEqual,” and “lessThanOrEqual,” are used for comparisons. By using these built-ins, it is possible to create home rules in which a value comparison of environmental values is needed such as the temperature, the humidity, and the noise level, or more elaborated boolean values such as the human presence detection in a house. Additionally, rules can be used in conjunction between each other in order to express more elaborated rules, such as the third home rule.
In this section, we present the rudiments of what constitutes SandS, our smart home environment, which we define as a city in which information and communication technologies are merged with traditional infrastructures, coordinated, and integrated using the IoT technologies. These technologies establish the functions of the city and also provide ways in which citizen groups can interact in augmenting their understanding of the city and also provide essential engagement in the design and planning process. We first sketch our vision defining three goals which concern us: feeding the home rules with the signals provided by the smart city system, to represent a simple interoperability test; introducing limitations on the use of the appliances related to environment conditions, like the power or water consumption reckoned by the city environment sensors, the short-term weather forecasting, and so forth, which represents a logical test on the DI scheduler and consistency checker; and managing alarm messages sent by the municipality. We begin by presenting how our data have been collected within a social network in order to create and exchange content in the form of so-called recipes and to develop collective intelligence which adapts its operation through appropriate feedback provided by the user. Additionally, we approach SandS from the user’s perspective and illustrate how users and their relationships can be modeled through a number of fuzzy stereotypical profiles (user-centered experimental validation). Furthermore, the context modeling in our smart home paradigm is examined through appropriate representation of context cues in the overall interaction (pervasive experimental validation).
In this subsection, we present our approach towards the vision of smart home that supports inhabitants’ high-level goals, emphasizing collecting our data in the wild in terms of having been captured in real-world and unconstrained conditions. Thus, our smart home technologies deal with interference with IoT technologies and react to nonstandard situations. More precisely, data was collected by the SandS consortium and partners during a small-scale mockup according to the “in-house” and “out-house” sensors such as mobility sensors, traffic and parking sensors, environmental sensors, and park and garden irrigation sensors, respectively. Finally, this context data information collected through the sensors is sent periodically to the ecosystem. These values are stored in a specific table of a database overwriting the previous record that was stored.
Regarding the experimental dataset to validate the formation of personas, data was collected by the SandS consortium and partners during a small-scale mockup. SandS also opened up its user base towards the FIRE and related communities such as the Open Living Labs. The dissemination call for user participation pointed to a user registration form, illustrated in Figure
SandS user registration form.
This registration form comprised several user-related fields: first name, last name, date of birth, senior/junior, gender, single/married, and city.
In large-scale tests of the unified user in a smart home in a smart city, SandS will use context sensor data gathered at SmartSantander. SmartSantander [ Mobility sensors: they are placed on buses, taxis, and police cars. They are in charge of measuring main parameters associated with the vehicle (GPS position, altitude, speed, course, and odometer) Traffic and parking sensors: they are buried under the asphalt. They are accountable for sensing the corresponding traffic parameters (traffic volumes, road occupancy, vehicle speed, queue length, and free parking availability) Environmental sensors: the task is to collect data concerning temperature, noise, light, humidity, wind speed, and detection of specific gases like CO, PM10, O3, and NO2 Park and garden irrigation sensors: in order to control and make the irrigation in certain parks and gardens more efficient, these sensors register information about wind’s speed, quantity of rain, soil temperature, soil humidity, atmospheric pressure, solar radiation, air humidity, and temperature, as well as water consumption
SmartSantander sensors locations.
At the moment, the data collected by these sensors are stored in the USN/IDAS SmartSantander cloud storage platform. This platform stores in its databases all the observations and measurements gathered by the sensors. It contains live and historical data. These databases are migrating on the Fi-lab platform as an instance of the FIWARE [
In very minimal terms, our experiments will manage the integration of the two systems only in one direction: by exploiting SmartSantander data in favor of SandS with special regard to the empowerment of the home rules used by the domestic infrastructure (DI), which is the core of the proposed system and handles the home rules and the appliances, manages the users, and updates the database with any new value gathered from a sensor. Hence, the contact between the two systems will happen via the home rules which may be fed by the smart city sensor data either in their current version or in an enlarged one to be capable of profiting from the data. Available sensor data, related to the SandS domain, include the following: temperature, noise, light, humidity, and quantity of rain. Other data, for instance, those concerning traffic, could be considered in a more long-term planning and scheduling approach.
Finally, our goal would be to stress the following case studies: Feeding the home rules with the signals provided by the smart city system. It represents a simple interoperability test Introducing limitations on the use of the appliances related to environment conditions, such as the power or water consumption reckoned by the city environment sensors and the short-term weather forecasting. It represents a logical test on the DI scheduler and consistency checker Managing alarm messages sent by the municipality. It will represent a stress test for the entire system
In the ecosystem, there are sensors both in every house and for the whole city. These sensors send periodically information about the temperature, the luminosity, and the humidity. Both the in-house and the city sensors send the values of the sensors periodically to the ecosystem. These values are stored in a specific table of a database overwriting the previous record that was stored. The in-house sensors send information about the humidity in the house, the inside house temperature, the human presence in it, the power consumption and the water consumption of all the appliances inside it, the location where the sensor is installed (e.g., the kitchen, the bathroom, or the bedroom), the noise, and the local timestamp. Moreover, the city sensor values are collected at a specific moment using the FIWARE Ops tools (
A user can get the best recipe for him by comparing his request for a recipe with other users’ requests of using the fuzzy similarity method presented in Section
SandS recipe request similarity form.
SandS recipe request similarity resulting table.
The system is periodically querying the database and, more specifically, the collection where the sensor values are stored. Then, using the home rules, which have been added in the ecosystem, it checks whether the consistency of the ontology still holds for the new sensor values. If any of the home rules is triggered, it denotes that an inconsistency has been detected from the system for a specific appliance. This specific appliance is switched off, until none of the home rules related to this appliance are inconsistent. As it has been mentioned previously, a home rule could be triggered by both the in-house sensor value changes and the value changes detected by the SmartSantander sensors. In order to be clear, an example is presented. Figure
In-house sensor values of the noise and the human presence.
Plot of the noise levels per second of a day window
Plot of the human presence in the house, per second of a day window. “1” means that there is a human in the house and “0” means that none is in the house
Moreover, in case the system receives from a city sensor, such as the SmartSantander sensors, temperature values equal to or greater than 26°C, then the first home rule would be triggered because an inconsistency would have been detected. As a result, the house’s washing machine would be switched off. The temperature values of such an occasion are presented in Figure
SmartSantander sensor values of the temperature for a specific period in a day.
In this paper, we illustrated how the emerging semantics of the smart home environments can be captured through a novel formalism and how expert knowledge can be used to ensure semantic interoperability. User stereotypes or personas on the one hand provide flexibility, extensibility, reusability, and applicability and on the other hand knowledge management is incorporated as an efficient user and context model representation formalism. In addition, this formal, machine-processable representation is used in order to define, extract, and use a set of concepts and their fuzzy semantic relations. This user modeling approach is put into a rich smart home context representation which abstracts raw sensor data to a high-level semantic representation language in which complex home rules can be defined.
Future work includes further incorporation of user, usage, and context information, through a unified semantic representation, driving an adaptation mechanism aiming to provide a personalized service and optimizing the user experience. Among the aspects of the architecture that will be stressed through experimental validation is the computational cost and the scaling of SandS to a wider user group. Based on the SandS architecture, the cloud infrastructure ensures the optimal handling of the computational load since the intermediate processes are not computationally demanding. On the other hand, issues that may arise from the scaling of the platform application are part of the experimental validation since the load is directly correlated with the user activity. The large-scale validation at SmartSantander will provide us with useful insights about the latter.
The authors declare that they have no competing interests.
This work was supported by the European Commission under Contract FP7-317947, FIRE project, “Social & Smart.”