Complexity
Volume 2018 (2018), Article ID 9540726, 17 pages
https://doi.org/10.1155/2018/9540726

Research Article

Trust-Based Situation Awareness: Comparative Analysis of Agent-Based and Population-Based Modeling

Zara Nasar and Syed Waqar Jaffry

Punjab University College of Information Technology, Lahore, Pakistan

Correspondence should be addressed to Zara Nasar; zara.nasar@pucit.edu.pk

Received 23 December 2017; Accepted 26 March 2018; Published 8 May 2018

Academic Editor: Kevin Wong

Copyright © 2018 Zara Nasar and Syed Waqar Jaffry. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Abstract.

In recent years, to comprehend and analyze complex systems, multiagent systems modeling and simulation are being widely used across various disciplines. Two major approaches used for multiagent systems modeling and simulation are agent-based modeling (ABM) and population-based modeling (PBM). In multiagent community, it is a silent assumption that both approaches represent similar dynamics for large population size. One of the recent studies from literature has reported similar results for a model of situation awareness spread in multiagent systems. Trust is a significant factor that affects agents’ communication, and consequently it controls spread of situation awareness among agents in a multiagent system. Hence, current work firstly extends the reported model of situation awareness spread from literature, to incorporate interagent trust for both ABM and PBM. Later, these extended models are used for comparative evaluation of both approaches. Various simulation experiments for different population sizes (small and large) as well as population types (homogenous and heterogeneous) are conducted and analyzed. Results of these experiments show that for large and homogeneous population, ABM approximates behavior of PBM, but for even slightly heterogeneous population, these approaches do not produce similar results irrespective of population sizes. Thus, the current study reports that, under some conditions, ABM and PBM produce similar results for trust-based situation awareness spread in multiagent systems, but this assumption does not hold true at large.

1. Introduction

In recent years, there is an urge to comprehend complex systems in order to better understand the world. World is very complex to understand in its totality with many factors contributing towards its complexity. With the advent of sociotechnical systems, there is a dire need to analyze the relations, strengths, and weaknesses of these systems on overall environment. Thus, it is of utmost importance to understand and analyze the behavior of human beings in certain conditions, so that the emerging sociotechnical systems can serve their purpose well. Hence, practitioners and scientists are working to develop systems that can help us in understanding the underlying relations between various entities in world as well as to better comprehend and analyze the human behavior and needs. In this pursuit, multiagent systems (MAS) are on rise. MAS refer to group of multiple intelligent agents, with each having their own beliefs, desires, and intentions and each wanting to fulfill his/her goals. These agents can communicate with each other, can influence each other’s beliefs and desires, can compete and/or collaborate in order to achieve their respective goals.

The modeling and development of MAS can be carried out using variety of ways. The domain of computational modeling comprises primary concepts and constructs for modeling of real world phenomenon’s using computational modeling and simulation. In context of MAS, computational modeling offers two widely used approaches, namely, agent-based modeling (ABM) and population-based modeling (PBM), respectively. ABM tends to provide individual level insights; thus, it models changes at individual level. It supports incorporation of multiple individual factors in the system, where primary actor is an agent, and systems are formed via collection of agents. The objective of ABM is to analyze the effects of individual agents having different properties on the entire system. PBM, on the other hand, as its name suggests, focuses on providing population level/global insights regarding any phenomenon of interest. It deals with groups as whole and tends to study group dynamics at population level and their impact on the overall system. As ABM includes wide variety of parameters and offers more flexibility than that of PBM, ABM is computationally expensive than PBM and is assumed as a more realistic modeling technique. In MAS community, it is a silent assumption that if results of ABM are aggregated at population level for large population size, both ABM and PBM present similar dynamics. Thus, in this study the primary focus is to perform comparative evaluation of ABM and PBM.

In order to compare ABM and PBM, an existing study from literature is selected, which performed comparison between these two approaches as well and reported that ABM and PBM both present similar dynamics when population size is large. This existing study models the phenomenon of spread of situation awareness (SA). SA refers to flow of information regarding particular event occurrence within a group of agents. It is a perception of environment by the agent at a particular time and surrounding context followed by the projection of events in near future.

There are multiple studies that analyze and study trust dynamics in various environments and domains [1–4]. As individuals’ decisions and beliefs about surroundings and environment are affected by their trust in other agents they are communicating with, trust is one factor that affects SA. Hence, trust-based SA incorporates mutual trust of communicating agents for better SA and it is being recognized as the basis of efficient group decision-making. It is of utmost importance in safety critical systems including aviation, power plant operation, and air traffic control. A brief study covering applications of the SA in general is presented in [5]. In the last couple of years, dynamics of SA are studied in variety of ways including gaming simulators, business intelligence, and online discussions using various computational techniques [6–12]. In current study, existing model of SA is being extended to trust-based SA to make the resultant model more realistic.

Hence, in this study, primary aim is to extend computational model of SA presented in [13] to trust-based SA using ABM and PBM techniques. Keeping this in view, the key goal of the current research is to analyze the proposed model with both computational modeling paradigms, namely, ABM and PBM, using homogenous as well as heterogeneous populations. In addition, comparative analysis of ABM and PBM is also presented in the light of conducted experiments. The rest of the paper describes related background, outlines methodology opted to build the system that is an extension to a previous model proposed in [13], and briefly explains the conducted experiments and respective results against various populations, followed by conclusion and future directions.

2. Background and Literature Review

Multiagent systems (MAS) are recently on rise due to advent of sociotechnical systems. MAS are being used in various domains to better comprehend and analyze the strengths, weaknesses, and needs of human users. Some relevant studies in this regard include [14–21] that demonstrate applications of MAS in car tracking, deployment of distributed applications, land usage and land covering analysis, and serious games development regarding health, environment, and water management.

The primary research focus of the current study is comparative evaluation of the two widely used approaches for MAS modeling, namely, ABM and PBM. In this regard, there exist several studies in literature which perform comparison between these approaches. Research study presented in [22] entirely focuses on trust dynamics and performs variety of experiments using both modeling approaches followed by their comparative analysis. Another comparative study in domain of criminology is presented in [23], which studies the phenomenon of crime displacement. Several experiments with simple and complex functions are performed in this study with respect to displacement of crime. In the light of conducted experiments, study concludes that, for large population size, ABM tends to approximate the results of PBM where PBM has the advantage of being computationally efficient. Similar research study carried out in [24] is focused on epidemics and economics, which provide a comprehensive comparison on ABM and PBM approaches. It concludes that these approaches may have similar performance against certain conditions, but that is not always the case.

As far as situation awareness and trust are concerned, both these concepts are applicable only when we have multiple agents in system. Thus, in order to study relation between SA and trust, that is, trust-based SA, MAS can be used. Several studies reported in literature deal with trust-based SA. The research study carried out in [25] analyzes SA, taking interpersonal trust into account. This study models the trust-based SA using ABM. Hence, the resultant model is quite detail-oriented. In order to verify the results, air control case study was employed and results show significant effect of trust information on operation controller’s SA. The concept of trust in this study is modeled using various factors that include domain knowledge of trustee, interpersonal trust, trust in information source, and trust of an agent in its own self. The resultant model is quite complex and, consequently, cannot be modeled using PBM with such minor attention to details. Other research studies include [26, 27] that study the relation between trust and SA in the context of Diner’s Dilemma game and autonomous vehicles, respectively.

As the existing model for trust-based SA [25] cannot be readily converted into its population-based counterpart, the current study extends a relatively simple model of SA spread as presented in [13] with incorporation of interagent trust using both ABM and PBM. This existing study models the spread of SA within the group using both ABM and PBM approaches. Later, comparative analysis of both modeling approaches is also presented. In the light of conducted experiments, it concludes that both modeling approaches represent similar dynamics if the population size is large.

Therefore, in the current study, extended model of trust-based SA is being used in order to perform comparative evaluation of these modeling approaches. Moreover, experiments are performed using homogenous as well as heterogeneous populations using various values for interagent trust. Current study is an extension of our previous work [28], which is majorly focused on model formulation for trust-based SA and it performs experiments using homogenous population only.

3. Methodology

In order to perform the comparative evaluation, first of all, the existing model presented in [13] is extended to incorporate interagent trust. This section briefly covers the details regarding existing model and its proposed extension. Case study used is similar to that of original model, which belongs to Air Traffic Management domain. Case study is about a couple of flights, where some are in line and waiting for signal/command to take off. At a time, only one air vehicle is signaled for takeoff. The case focuses on a scenario when a pilot misinterprets the command and starts to take off, whereas another vehicle was already signaled for takeoff on crossing runway at the same time. Both vehicle crews were unaware that another vehicle is also on the runway. Control tower, after analyzing the situation, signaled one of the flight pilots to abort the takeoff. Pilot, in this situation, had to check whether it is possible to abort the takeoff by applying brakes or not. Eventually, after analyzing the situation, the pilot instructed copilot to stop takeoff by applying brakes. During braking, crew saw other flight flying close in with distance of five meters. Thus, a very serious accident was prevented. Further details regarding case study are presented in [29].

In the context of above study, there exist three belief states of agents and populations that include correct, incorrect, and unknown. For the sake of simplicity, model carries only one interesting phenomenon against whom these belief states are being set. This assumption is being held in original study as well. The aim is to analyze the dynamics of overall system and transition of agents/populations from one state to another over the passage of time, when interagent trust is also taken into account.

3.1. Existing Model

This section briefly covers the nomenclature and formulations of the existing model that is being followed in proposed model as well.

3.1.1. Global Properties

The existing model made use of the following six global state transition probabilities (STPs) for ABM:

(1): STP from correct to incorrect belief state,(2): STP from correct to unknown belief state,(3): STP from unknown to incorrect belief state,(4): STP from unknown to correct belief state,(5): STP from incorrect to correct belief state,(6): STP from incorrect to unknown belief state.

In the existing study, the PBM is modeled on the grounds of widely used epidemic spread model for population where each subpopulation is represented separately and their intertransition is modeled on the basis of transition rates. In order to map the ABM and PBM model, differences of respective properties in ABM model are used as state-changing parameters in PBM model. Gamma represents the transition rate and , , and refer to size of population carrying unknown, correct, and incorrect belief states. Thus, transition rates are modeled using following equations:

3.1.2. State Transition Rules

In ABM, rules to determine belief state of an agent at next time point are dependent on following inputs:

(1)agent’s own belief state at time-step ,(2)state transition probabilities (STPs mentioned in Section 3.1.1),(3)belief state of communicating agent.

Whenever two agents communicate, a random number is drawn; if its value is less than or equal to respective inter-STP of belief states, belief state of receiver agent is updated. Differential equations for PBM to compute population counts against every belief state depend on the following parameters:

(1)rate of interstate transitions,(2)population count at current time-step, against every belief state.

The respective differential equations for population-based model are as follows:

In (2), , , and represent counts of subpopulations at time-step carrying correct, incorrect, and unknown belief state, respectively.

3.2. Proposed Model

Trust is an individual factor and an agent can have different trust values for all other agents in a given environment. Trust value 0.5 is regarded as the mean trust value of an agent. Trust value greater than 0.5 shows higher trust and vice versa, where trust can reside in the range of and is modeled as nonnegative number.

In order to map the existing model, global properties that are used in existing ABM model are made local to every agent. Thus, every agent has total of six transition probabilities along with trust vector, carrying respective agent’s trust in all other agents. This setup tends to provide flexibility for generating variety of homogeneous and heterogeneous populations by means of interagent trust, transition probabilities, and combination of these two.

For PBM, in order to calculate transition rate (Gamma-), all individual transition probabilities against agents are averaged, keeping overall community trust in account. Figure 1 represents overall transitions and flow involved in proposed model. Figures 1(a) and 1(b) present the state transitions involved in PBM and ABM, respectively. Figure 1(b) extends the original model figure as presented in [13], where connections made via dashed lines represent the extensions in the proposed model. This section explains the interstate transition for ABM and PBM in extended model.

(a)

(b)

Figure 1: Interstate transition models for ABM and PBM. (a) State transition model for PBM; (b) state transition model for ABM partially adopted from [13].

3.2.1. State Transition Rules for ABM

Transition model for agent-based communication is dependent on the belief states of communicating entities along with their trust in each other. Following pseudocode presents the scenario when agent A with correct belief state (receiver) is receiving information from agent B (sender). Similar code is being used when an agent with incorrect or unknown belief state is on receiver end. The selection of sender agent is performed via randomly selecting any agent from the overall set of agents, as proposed in existing model [2]. When an agent A (receiver) communicates with agent B (sender), state transition is governed by Pseudocode 1, where returns the belief state of agent A at time-step . returns the trust of agent A on agent B, and represents STPs as described in global properties of existing model. is a uniform random number that is drawn from range . In order to keep model comparable with existing mode, factor of 2 is being multiplied with interagent trust and inter-STP. This computational model thus results in the formation of original model when interagent trust between two agents is 0.5, by making the effect of trust neutral and only using the respective inter-STP.

/Both Agent A and Agent B carry correct belief state

if Belief_State is correct and Belief_State is correct

then Belief_State is correct

/Agent A and Agent B carry correct and incorrect belief states respectively

if Belief_State is correct and Belief_State is incorrect and <= 2 trust

then Belief_State is incorrect /Agent A adopts the belief state of Agent B

if Belief_State is correct and Belief_State is incorrect and > 2 trust

then Belief_State is correct /Agent A does not adopt the belief state of Agent B

/Agent A and Agent B carry correct and unknown belief states respectively

if Belief_State is correct and Belief_State is unknown and <= 2 trust

then Belief_State is unknown /Agent A adopts the belief state of Agent B

if Belief_State is correct and Belief_State is unknown and > 2 trust

then Belief_State is correct /Agent A does not adopt the belief state of Agent B

Pseudocode 1

3.2.2. State Transition Rules for PBM

The transition equations for PBM are the same as that of the existing model. In context of PBM, difference lies in calculation of global transition probabilities only, which are computed by means of averaging transition probabilities against every agent keeping its interagent trust into account.

In (3)–(8), refers to the STP from some belief state to another belief state , at population level. in this equation represents the total number of agents/population size. represents average trust that th agent puts in the rest of community. , on the other hand, represents the average trust in overall community. refers to the inter-STP, against th agent, from some belief state to belief state , that is, local to agent . After calculating population level probabilities using (3)–(8), the following set of equations are used in order to calculate state transition rate in PBM:

In the original model, the two scenarios “nominal” and “wishful-thinking” are presented. Nominal scenario refers to the situation where mental states of agents and their respective expectations from the environment are not biased. Wishful-thinking or group thinking refers to the situation where agents tend to incorporate incorrect beliefs in order to fulfill their desires, as incorrect beliefs appear to be more wishful and desirable. Existing study carries default values of parameters against these scenarios. Default parameters and initial population settings that are used in existing model are presented in Table 1.

Table 1: Parameters and population settings for nominal and wishful-thinking scenarios.

Scenario
Parameters for ABM

Nominal
0.20
0.15
0.10
0.02
0.05
0.04

Wishful-thinking
0.15
0.10
0.10
0.02
0.05
0.01

Parameters for PBM
Initial population distributions

Unknown
Correct
Incorrect

Nominal
0.10
0.10
0.20
50%
25%
25%

Wishful-thinking
0.05
0.05
−0.01
50%
~38%
~12%

4. Experiments

Experiments conducted can be broadly classified into homogeneous and heterogeneous categories. In the context of current research study, the effect of interagent trust on spread of information is focused on; thus, the interagent trust values act as basis for categorization.

Homogeneous population refers to population where participating agents carry similar configurations; that is, they are alike. In current study, in case of homogenous population, every agent carries similar trust value for the rest of agents; that is, trust of overall community is similar to trust of an individual agent in any other. The original model can be reproduced in extended model by means of homogenous population where all agents carry mean trust value of 0.5 in each other.

Heterogeneous environments, on the other hand, deal with cases when agents in environment differ from each other; that is, agents are unalike. In extended model, heterogeneity can be involved at various levels. It can be modeled with respect to trust or inter-STPs or both. Major challenge in heterogeneous population is to deal with the variety of possibilities. For example, consider granularity of trust to be 0.1: this means that total ten possibilities can exist for trust assignment. Now as we have total of three subpopulations of different beliefs, if each subset of population is to be experimented with various assignments of trust, total possibilities will be 10³ = 1000. Consider the granularity of trust to be 0.01: in this case total possibilities for experiments become 100³ = 1000,000. Thus, exhaustively performing all possible set of experiments is not feasible due to huge set of possibilities. In other words, the level of abstraction is of huge importance in context of heterogeneous experiments.

Currently, the experiments that are conducted on heterogeneous population follow the same transition equations and model as proposed in Section 3.2. There exist multiple options to further introduce heterogeneity by means of using distributions rather than concrete trust values or by updating trust after agents communicate with each other. Current study focuses on assignment of trust values, where trust can either be low, medium, or high, as means of heterogeneity.

The rest of this section focuses on homogeneous and heterogeneous experiments that are carried out using extended model. All the experiments in this study are carried out using MATLAB.

4.1. Homogeneous Experiments

Experiments are carried out following similar experimental setup to that described in original model. The parameter settings described in Table 1 are employed in order to perform the experiments. Original study tends to experiment with two population sizes: eight and hundred, whereas simulation values are recorded up to 250 time-steps. In addition, to avoid randomness, ABM results are averaged over 1000 simulations for every experiment performed. In the original model, the results against eight agents (small population size) do not result in similar behavior against both computational modeling paradigms, whereas in case of hundred agents, both techniques generate similar behavior, as affirmed in many studies in relevant literature.

As the proposed model carries additional feature of interagent trust, simulation values are recorded up to 500 time-steps in order to check for convergence of results for ABM and PBM. Transition probabilities for both scenarios, which are computed via proposed approach for PBM, are presented in Table 2. To get an overall idea and for the sake of brevity, results are being reported on lower trust value of 0.1 and higher trust value of 0.9 against both scenarios. This section presents the experiment’s results against homogenous population, where each color-coded graph represents time-steps along -axis and population size/number of agents along -axis. In addition, each graph carries ABM results with solid line and PBM results with dashed line.

Table 2: Transition probabilities for PBM against different values of trust in both scenarios.

Trust
Nominal scenario
Wishful-thinking scenario

0.1
0.02
0.02
0.004
0.01
0.01
−0.020

0.3
0.06
0.06
0.012
0.03
0.03
−0.060

0.5
0.10
0.10
0.020
0.05
0.05
−0.010

0.7
0.14
0.14
0.028
0.07
0.07
−0.014

0.9
0.18
0.18
0.036
0.09
0.09
−0.018

The proposed approach models trust as nonnegative entity where mean trust value is 0.5. Trust value greater than 0.5 represents high trust and vice versa. With mean trust value of 0.5, the original model can be produced using the proposed formulation. Figure 2 shows the results generated via original model and the proposed model. Results show that existing model is among the subset of models that can be produced via presented approach.

(a)

(b)

(c)

(d)

Figure 2: Comparison of original and proposed model with mean trust value of 0.5 and 100 population size. (a) Nominal scenario in existing model; (b) nominal scenario in proposed model; (c) wishful-thinking scenario in existing model; (d) wishful-thinking scenario in proposed model.

In case of both nominal and wishful-thinking scenarios, for smaller population size, ABM and PBM do not approach each other, whereas, for larger population sizes, the dynamics of ABM and PBM represent similar behavior. Thus, the hypothesis carried from original study regarding dynamics of ABM and PBM with respect to population sizes is verified. Figure 3 presents the results of both computational techniques against nominal scenario with population sizes of eight and hundred, along with trust value of 0.1 and 0.9. Figure 4 focuses on the results achieved, when proposed model is used in wishful-thinking scenario.

(a)

(b)

(c)

(d)

Figure 3: Dynamics in “nominal” scenario against low and high trust values with different population sizes. (a) Trust = 0.1, population size = 8; (b) trust = 0.1, population size = 100; (c) trust = 0.9, population size = 8; (d) trust = 0.9, population size = 100.

(a)

(b)

(c)

(d)

Figure 4: Dynamics in “wishful-thinking scenario” against low and high trust values with different population sizes. (a) Trust = 0.1, ; (b) trust = 0.1, ; (c) trust = 0.9, ; (d) trust = 0.9, .

One thing to note in homogenous population results is that PBM trend remains almost consistent in small population size as well as large population size. ABM dynamics, on the other hand, gets heavily affected by the total population size. The comparative study shows that for large population sizes, ABM tends to approximate the results of PBM, where PBM is very efficient and its dynamics are independent of the population size. On the contrary, computational complexity of ABM increases with the number of agents. Thus, in order to model small population size, PBM cannot be used if individual’s biases are to be taken into account.

In addition to these insights, results also show that if trust community has high trust factor, agents having unknown belief state tend to adopt the correct/incorrect belief states, depending on the scenarios and transition rates, quite quickly. On the other hand, in case of low trust, the transition from unknown to correct/incorrect state takes relatively longer time. The primary hypothesis regarding this study about relation with SA and trust is thus confirmed.

4.2. Heterogeneous Experiments

Heterogeneous experiments are also conducted on the settings presented in Table 1. In order to conduct the experiments, trust is categorized into three groups, low, medium, and high, with values of 0.1, 0.5, and 0.9, respectively. The reason for hard-codifying the value is to have ease in analysis of results. In the current set of conducted experiments, a trust value being either low, medium, or high is assigned to subpopulation carrying a particular belief.

Total of twenty-seven configurations are possible using current experiment settings. After conducting all experiments against both scenarios, resulting graphs were firstly analyzed visually in order to look for patterns. Later averaged root mean square error (RMSE) was calculated against all belief states using temporal simulation traces of PBM and ABM for every experiment using (12). In these equations, represents total number of agents/population size. The overall trend of RMSE in both scenarios and values against all configurations are provided in Appendix A.

As affirmed in homogenous experiments, both ABM and PBM almost produce similar trends over the time. As total of these 27 configurations consist of three homogenous populations as well, the RMSE among homogenous populations was minimum compared to the rest. Thus, it further validates our experiment assumption that ABM and PBM show similar trends in homogenous populations.

An interesting finding in the light of visually analyzing results and by means of error function is that “almost homogeneous populations” also perform well in both scenarios. One intuition for this can be the similarity between these populations and homogenous populations. As these configurations are very close to homogenous configurations, respective results are better than the rest as shown in Figure 5. These cases are best performers after homogenous populations. Details of overall RMSE results against twenty-seven scenarios up to 500 time-steps are shared in Appendix A. Figure 5(a) presents the system where trust configurations are HMM; that is, unknown community has high trust whereas correct and incorrect communities have medium trust in the overall community. The configuration is coded in the order unknown, correct and incorrect. Similarly, Figure 5(b) presents the scenario when unknown and correct communities place high trust whereas incorrect community places medium trust in the overall population.

(a)

(b)

Figure 5: Results against almost homogenous populations for both scenarios. (a) HMM case against “nominal scenario”; (b) HHM case against “wishful-thinking scenario.”

The only exception to the above observation is when unknown community has low trust in the rest. If the net transitions in the model are analyzed, it gets clear that unknown community ends up opting either incorrect or correct belief states as represented in Figure 1(a). Now, if unknown community has low trust in the rest, in case of ABM, the resulting transitions would be quite slow in comparison to PBM. Due to this slow convergence in ABM, the difference between ABM and PBM is increased, though eventually they either converge to each other or make an asymptote. Figure 6 presents the scenarios where unknown community has low trust in others. Population configurations presented in Figure 6 include LHL in case of nominal scenario that carries the highest RMSE in comparison to other configurations; for wishful-thinking scenario, LLH case is included, that is, the third most erroneous configurations in terms of RMSE.

(a)

(b)

Figure 6: Results against both scenarios where unknown population has low trust in the community. (a) LHL case against “nominal scenario”; (b) LLH case against “wishful-thinking scenario.”

If both scenarios are analyzed closely, it is evident that nominal scenario favors correct belief spread and eventually majority of population will end up opting correct belief whereas wishful-thinking scenario favors incorrect belief and eventually population will carry incorrect belief. These final states can be regarded as sink. Next pattern that is observed in conducted experiments highlights the effect of low trust on sink community. Hypothesis in this regard is that as the eventual state to converge is the same as that of sink and additionally sink community has low trust in others, transition from sink to others would be very slow. Thus, earlier convergence will happen in case of ABM than in that of PBM, and overall differences would be less. Consequently, results would be better matched and overall error will be reduced. Figure 7 presents the results when sink has low trust, where correct and incorrect belief states are sinks in nominal and wishful-thinking scenarios, respectively.

(a)

(b)

Figure 7: Results against both scenarios where SINK has low trust in the community. (a) HLM case against “nominal scenario”; (b) HML case against “wishful-thinking scenario.”

Figure 8 presents the trends against both scenarios in a similar case. As both scenarios have different sinks, the configuration selected for comparison is the one with similar trust for both correct and incorrect populations. In case of nominal scenario, ABM and PBM tend to approach each other earlier than that of wishful-thinking scenario. One hypothesis in this regard is that wishful-thinking scenario has lower inter-STP than nominal scenario as presented in Table 1.

(a)

(b)

Figure 8: Results against both scenarios against similar configuration of trust. (a) LHH case against “nominal scenario”; (b) LHH case against “wishful-thinking scenario.”

Another offshoot of low inter-STPs is that, in some particular cases, where unknown community puts low trust in others, not all population eventually transit towards sink. This is because, due to lower interstate STP, the transition rate among states already gets low. In addition, as the major community that is to be transited to other communities (i.e., the unknown population) also has lower trust in population, resulting transition rate would be very low. Due to their resultant affect, the whole population does not end up following the same belief. Thus, asymptotes are formed as represented in Figure 9. Here, in this particular experiment, simulation values are recorded up to 5000 time-steps in order to present the dynamics of the overall system more clearly and precisely.

(a)

(b)

Figure 9: Asymptotes formation in both scenarios. (a) LHL case against nominal scenario; (b) LLH case against wishful-thinking scenario.

In the light of the above points, one can clearly analyze the power of ABM in contrast to PBM. Both ABM and PBM tend to model the effect of inter-STPs as reported in Appendix B. Apart from that, PBM tends to model various configurations in a similar way, whereas ABM gets affected by the individual agent’s biases towards the system in terms of both transition rate and trust as reported in Appendix A. Thus, ABM has the capability to model the dynamics of system taking into account agents’ local properties and biases. PBM, on the other hand, tends to model phenomenon at quite abstract level. Which one of them is really true in real world can only be determined by empirical validation.

5. Conclusion and Future Work

In this study, comparison of two widely used approaches to model multiagent systems (MAS) is performed, namely, agent-based modeling (ABM) and population-based modeling (PBM). ABM is assumed to be more realistic and insightful as it has capability to model individual level characteristics, but due to this very reason, it is computationally expensive. PBM is usually employed to analyze global insights, and thus it is computationally efficient as it is independent of population size. Regarding these two modeling approaches, there is a prevailing assumption in MAS community that both approaches present similar dynamics for large population. Therefore, it is of interest to determine whether or not these two modeling approaches present similar dynamics in case of large population size.

In order to perform the comparative study, trust-based situation awareness was modeled by means of extending an existing study from literature. This existing study also performs comparative evaluation of these approaches and concludes that both approaches present similar dynamics if population size is large. This study models the spread of situation awareness (SA) within a group of agents. As trust is among primary factors that can affect SA spread, current study extends this model to incorporate interagent trust, thus proposing formulation for trust-based SA through both ABM and PBM approaches.

To compare ABM and PBM, experiments are conducted using homogenous and heterogeneous populations with respect to interagent trust factor. Results show that, in case of homogenous population, both approaches present similar dynamics when population size is large as affirmed in existing literature studies.

Heterogeneous experiments’ dynamics, on the other hand, are sensitive to initial trust assignments as well as community wide trust distributions. Thus, dynamics exhibited by both approaches in heterogeneous populations tend to vary with respect to parameter configurations. In various scenarios, both approaches exhibit dynamics that are different in the start of simulation but eventually converge to each other. Moreover, some of the input configurations result in equilibrium state with respect to the number of agents against each modeling approach, but both approaches do not converge to each other. Thus, it could be concluded that, for trust-based SA, these approaches do not behave alike when population is heterogeneous.

Therefore, this study deserves further empirical work to validate which one of these approaches performs better in reality for trust-based SA models. If it is empirically validated that ABM’s dynamics are closer to reality, then community should focus on the efficient implementation of ABM models. On the other hand, if PBM’s results are found to be more realistic, then community should focus on developing and extending existing PBM models which carry additional advantage of being computationally efficient as well. After determination of empirically validated modeling approach, the respective model can be extended to incorporate further related concepts such as group hierarchy in subpopulations. Additionally, state transition rules can also be updated in order to model relatively more realism for trust-based SA.

Appendix

A. Root Mean Square Errors

This section is focused on results of root mean square error (RMSE) against heterogeneous populations. RMSE is used in order to compare the results of ABM and PBM against similar trust distributions. As total of twenty-seven configurations are possible, Table 3 lists the RMSE against every configuration for both scenarios along with mean trust. Mean trust represents the average trust of the whole community against every configuration that is primarily used in the formulation of PBM dynamics. Table entries are made in the chronological order with respect to trust configuration.

Table 3: RMSE against all configurations for both scenarios.

Trust distribution
Nominal scenario
Wishful-thinking scenario

Unknown
Correct
Incorrect
Mean trust
RMSE
Mean trust
RMSE

L
L
L
0.1
0.8688
0.1
0.5521

L
L
M
0.2
6.5583
0.148
16.3115

L
L
H
0.3
7.3172
0.196
24.518

L
M
L
0.2
22.2928
0.252
16.7516

L
M
M
0.3
13.2705
0.3
16.3012

L
M
H
0.4
12.278
0.348
24.0071

L
H
L
0.3
33.7182
0.404
17.2442

L
H
M
0.4
20.8549
0.452
18.6414

L
H
H
0.5
18.6118
0.5
24.3485

M
L
L
0.3
6.0913
0.3
9.6563

M
L
M
0.4
4.4084
0.348
19.854

M
L
H
0.5
4.3772
0.396
25.752

M
M
L
0.4
10.1467
0.452
6.7352

M
M
M
0.5
0.5007
0.5
1.5256

M
M
H
0.6
1.6807
0.548
6.5738

M
H
L
0.5
13.0625
0.604
8.0953

M
H
M
0.6
3.5929
0.652
3.7707

M
H
H
0.7
1.9148
0.7
3.9007

H
L
L
0.5
8.1663
0.5
16.032

H
L
M
0.6
3.2983
0.548
23.3535

H
L
H
0.7
3.9246
0.596
27.1409

H
M
L
0.6
10.6268
0.652
4.5747

H
M
M
0.7
1.0245
0.7
2.7398

H
M
H
0.8
1.4448
0.748
5.7717

H
H
L
0.7
12.0641
0.804
6.0571

H
H
M
0.8
2.2877
0.852
2.2345

H
H
H
0.9
0.3569
0.9
1.0679

By closely analyzing the trend of RMSE in Table 3, it is quite evident that for various trust distributions mean trust remains the same; for example, the configurations LHH, MMM, and HLL all result in mean trust value of 0.5 with different RMSEs against both scenarios. Dynamics for configurations that result in mean trust value of 0.5 against both scenarios are presented in Figure 10 (entries in boldface in Table 3). Each color-coded figure represents dynamics exhibited by PBM with dashed lines whereas results against ABM are represented with solid lines.

(a)

(b)

(c)

(d)

(e)

(f)

Figure 10: Dynamics in various configurations that result in similar mean trust against PBM against both scenarios. (a) LHH case in nominal; (b) MMM case in nominal; (c) HLL case in nominal; (d) LHH case in wishful-thinking; (e) MMM in wishful-thinking; (f) HLL in wishful-thinking.

As it is evident in Figure 10, various population configurations exhibit similar behavior in case of PBM, whereas trends vary in case of ABM depending upon the population distribution. This is because unlike PBM, ABM has the ability to incorporate and eventually project the dynamics of system, keeping agents’ properties and biases in account.

B. Effect of Transition Rates

State transition rate probabilities (STPs) play a significant role in terms of convergence. The higher the STP is, the faster the convergence will happen. In order to analyze the effect of transition rate in the proposed model, an experiment was carried out. In this experiment, traces are generated against nominal and wishful-thinking scenarios with their default transition rates as well as with the updated transition rates reduced by 50%. Figure 11 represents the trends when normal and reduced transition rates are employed against MLM configuration, that is, a configuration where unknown population have medium trust, correct population have low trust, and incorrect population have medium trust in the rest of the community. Figures 11(a) and 11(b) represent the trends in nominal scenario when original and reduced STPs are employed whereas Figures 11(c) and 11(d) show the trends in wishful-thinking scenario with original and reduced transition rates. Each color-coded figure represents dynamics exhibited by PBM with dashed lines whereas results against ABM are represented with solid lines.

(a)

(b)

(c)

(d)

Figure 11: Results against both scenarios against reduced transition rates. (a) MLM case against “nominal scenario” with original transition rates; (b) MLM case against “nominal scenario” with reduced transition rates; (c) MLM case against “wishful-thinking scenario”; (d) MLM case against “wishful-thinking scenario” with reduced transition rates.

If we analyze Figures 11(a) and 11(b), it is evident that dynamics in (a) tends to converge after approximately 250 time-steps, whereas dynamics exhibited in (b) show convergence after 500 time-steps approximately. Thus, both systems represent similar dynamics, with the primary difference of respective time required. Similar behavior is presented in Figures 11(c) and 11(d). In these figures, simulations are recorded up to 5000 time-steps in order to provide clear idea about the overall dynamics of the system. If we analyze the trend of PBM curve for correct population (one in red and represented via dashed line) in both figures, it is clear that in (c) respective curve gets to zero after approximately 1000 time-steps, whereas in (d) it takes almost double the time, that is, approximately 2000 time-steps to get a similar result. Thus, in the light of the above points, we can say that transition rate affects the overall convergence in the context of both ABM and PBM.

Disclosure

Part of this paper has been presented at the International Conference on Advancements in Computational Sciences (ICACS-2018) [28]. The current paper extends the conference paper by providing additional simulation experiments in heterogeneous population, more detailed analysis of simulation results, and a more extensive discussion regarding the behavior of agent-based and population-based modeling in various environments.

Conflicts of Interest

The authors declare no conflicts of interest.

References

R. Falcone and C. Castelfranchi, “Trust dynamics: how trust is influenced by direct experiences and by trust itself,” in Proceedings of the Third International Joint Conference on Autonomous Agents and Multiagent Systems, AAMAS 2004, pp. 740–747, New York, NY, USA, July 2004.

M. Hoogendoorn, S. W. Jaffry, and J. Treur, “Modeling dynamics of relative trust of competitive information agents,” in Proceedings of the Cooperative Information Agents XII, pp. 55–70, Prague, Czech Republic, 2008.

M. Hoogendoorn, S. W. Jaffry, and J. Treur, “An adaptive agent model estimating human trust in information sources,” in Proceedings of the 2009 IEEE/WIC/ACM International Conference on Web Intelligence and Intelligent Agent Technology, WI-IAT 2009, pp. 458–465, Milan, Italy, September 2009.

S. W. Jaffry and J. Treur, “Modelling trust for communicating agents: Agent-based and population-based perspectives,” in International Conference on Computational Collective Intelligence, vol. 6922 of Lecture Notes in Computer Science, pp. 366–377, Springer, Berlin, Germany, 2011.

N. A. Stanton, P. R. G. Chambers, and J. Piggott, “Situational awareness and safety,” Safety Science, vol. 39, no. 3, pp. 189–204, 2001.

T. Bosse, R.-J. Merk, and J. Treur, “Modelling temporal aspects of situation awareness,” in International Conference on Neural Information Processing, vol. 7663 of Lecture Notes in Computer Science, pp. 473–483, Springer, Berlin, Germany, 2012.

M. Castellanos, C. Gupta, S. Wang, U. Dayal, and M. Durazo, “A platform for situational awareness in operational BI,” Decision Support Systems, vol. 52, no. 4, pp. 869–883, 2012.

T. Bosse, K. Majdanik, K. Boersma, and K. Ingibergsdóttir, “Studying shared situation awareness by agent-based simulation,” in Proceedings of the 12th IEEE/WIC/ACM International Conference on Intelligent Agent Technology, IAT 2013, pp. 201–208, Atlanta, Ga, USA, November 2013.

J. C. Lo and S. A. Meijer, “Measuring group situation awareness in a multiactor gaming simulation: A pilot study of railway and passenger traffic operators,” Proceedings of the Human Factors and Ergonomics Society Annual Meeting, vol. 57, no. 1, pp. 177–181, 2013.

J. Li, D. Zhang, and P. Zhang, “Supporting dynamic situation awareness in online group discussion: A visualization approach,” in Proceedings of the 46th Annual Hawaii International Conference on System Sciences, HICSS 2013, pp. 470–479, Wailea, Maui, Hawaii, USA, January 2013.

G. Castellano, M. G. C. A. Cimino, A. M. Fanelli, B. Lazzerini, F. Marcelloni, and M. A. Torsello, “A multi-agent system for enabling collaborative situation awareness via position-based stigmergy and neuro-fuzzy learning,” Neurocomputing, vol. 135, pp. 86–97, 2014.

G. D'Aniello, V. Loia, and F. Orciuoli, “A multi-agent fuzzy consensus model in a Situation Awareness framework,” Applied Soft Computing, vol. 30, pp. 430–440, 2015.

T. Bosse and N. Mogles, “Spread of situation awareness in a group: Population-based vs. agent-based modelling,” in Proceedings of the 2014 IEEE/WIC/ACM International Joint Conference on Web Intelligence and Intelligent Agent Technology - Workshops, WI-IAT 2014, pp. 1117–1124, Warsaw, Poland, August 2014.

D. C. Parker, S. M. Manson, M. A. Janssen, M. J. Hoffmann, and P. Deadman, “Multi-agent systems for the simulation of land-use and land-cover change: a review,” Annals of the Association of American Geographers, vol. 93, no. 2, pp. 314–337, 2003.

F. Piette, A. El Fallah, C. Seghrouchni, P. Taillibert, and C. Dinont, “A Multi-Agent Approach for the Deployment of Distributed Applications,” in in Smart Environments, pp. 37–46, Paris, France, 2016.

J. Dai, Z. Li, and N. Ivanova, “Regional water resource allocation problem in multi-agent game: Taking the water rights transfer as example,” Advances in Intelligent Systems and Computing, vol. 502, pp. 803–815, 2017.

M. Othmani-Guibourg, A. E. Fallah-Seghrouchni, J.-L. Farges, and M. Potop-Butucaru, “Multi-agent patrolling in dynamic environments,” in Proceedings of the 2017 IEEE International Conference on Agents, ICA 2017, pp. 72–77, Beijing, China, July 2017.

J. Liu, K. M. Kockelman, P. M. Boesch, and F. Ciari, “Tracking a system of shared autonomous vehicles across the Austin, Texas network using agent-based simulation,” Transportation, vol. 44, no. 6, pp. 1261–1278, 2017.

H. Aslam, A. Sidorov, N. Bogomazov, F. Berezyuk, and J. A. Brown, “Relief camp manager: A serious game using the world health organization’s relief camp guidelines,” in European Conference on the Applications of Evolutionary Computation, vol. 10199 of Lecture Notes in Computer Science, pp. 407–417, Springer, 2017.

I.-H. Chen, “An evolutionary game study of an ecological industry chain based on multi-agent simulation: A case study of the Poyang Lake Eco-Economic zone,” Sustainability, vol. 9, no. 7, article 1165, 2017.

J. Sušnik, C. Chew, X. Domingo et al., “Multi-stakeholder development of a serious game to explore the water-energy-food-land-climate nexus: The SIM4NEXUS approach,” Water, vol. 10, no. 2, p. 139, 2018.

S. W. Jaffry and J. Treur, “Agent-based and population-based modeling of trust dynamics,” in Transactions on Computational Collective Intelligence IX, vol. 7770 of Lecture Notes in Computer Science, pp. 124–151, Springer, Berlin, Germany, 2013.

T. Bosse, C. Gerritsen, M. Hoogendoorn, S. W. Jaffry, and J. Treur, “Agent-based vs. population-based simulation of displacement of crime: A comparative study,” Web Intelligence and Agent Systems, vol. 9, no. 2, pp. 147–160, 2011.

T. Bosse, S. W. Jaffry, G. F. Siddiqui, and J. Treur, “Comparative analysis of agent-based and population-based modelling in epidemics and economics,” Multiagent and Grid Systems, vol. 8, no. 3, pp. 223–255, 2012.

R. Aydoğan, A. Sharpanskykh, and J. Lo, “A Trust-Based Situation Awareness Model,” in European Conference on Multi-Agent Systems, vol. 8953 of Lecture Notes in Computer Science, pp. 19–34, Springer, 2015.

J. O’Donovan, R. E. Jones, L. R. Marusich, Y. Teng, C. Gonzalez, and T. Höllerer, “A model-based evaluation of trust and situation awareness in the diner’s dilemma game,” in Proceedings of the 22th Behavior Representation in Modeling &amp; Simulation (BRIMS) Conference, pp. 11–14, Ottawa, Canada, 2013.

K. Sonoda and T. Wada, “Displaying system situation awareness increases driver trust in automated driving,” IEEE Transactions on Intelligent Vehicles, vol. 2, no. 3, pp. 185–193, 2017.

Z. Nasar and S. W. Jaffry, “Trust-based situation awareness: Agent-based versus population-based modeling a comparative study,” in Proceedings of the 2018 International Conference on Advancements in Computational Sciences (ICACS), pp. 1–7, Lahore, Pakistan, Feburary 2018.

T. Bosse and N. M. Mogles, “Formal analysis of aviation incidents,” in International Conference on Industrial, Engineering and Other Applications of Applied Intelligent Systems, vol. 7345 of Lecture Notes in Computer Science, pp. 371–380, Springer, Berlin, Germany, 2012.