Journal of Sensors
Volume 2016 (2016), Article ID 5656245, 9 pages
http://dx.doi.org/10.1155/2016/5656245

Research Article

Citizen Sensing for Improved Urban Environmental Monitoring

Qijun Jiang,¹ Frank Kresin,² Arnold K. Bregt,1,3 Lammert Kooistra,¹ Emma Pareschi,² Edith van Putten,⁴ Hester Volten,⁴ and Joost Wesseling⁴

¹Laboratory of Geo-Information Science and Remote Sensing, Wageningen University, P.O. Box 47, 6700 AA Wageningen, Netherlands
²Waag Society, Sint Antoniesbreestraat 69, 1011 HB Amsterdam, Netherlands
³AMS, Amsterdam Institute for Advanced Metropolitan Solutions, Mauritskade 62, 1092 AD Amsterdam, Netherlands
⁴National Institute for Public Health and the Environment (RIVM), P.O. Box 1, 3720 BA Bilthoven, Netherlands

Received 22 January 2016; Accepted 7 April 2016

Academic Editor: Kewei Zhang

Copyright © 2016 Qijun Jiang et al. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Abstract.

Citizen science is increasingly being used in diverse research domains. With the emergence and rapid development of sensor technologies, citizens potentially have more powerful tools to collect data and generate information to understand their living environment. Although sensor technologies are developing fast, citizen sensing has not been widely implemented yet and published studies are only a few. In this paper, we analyse the practical experiences from an implementation of citizen sensing for urban environment monitoring. A bottom-up model in which citizens develop and use sensors for environmental monitoring is described and assessed. The paper focuses on a case study of Amsterdam Smart Citizens Lab using NO₂ sensors for air quality monitoring. We found that the bottom-up citizen sensing is still challenging but can be successful with open cooperation and effective use of online and offline facilities. Based on the assessment, suggestions are proposed for further implementations and research.

1. Introduction

Highly dynamic environmental phenomena call for detailed and timely environmental information to support decision making. For instance, in the case of air pollution, it is essential to understand where this pollution comes from and how to reduce it [1]. For environmental disasters, such as floods [2], severe weather [3], and volcanic eruptions [4], sufficient and timely environmental information is essential for risk forecasting and early warning. To derive this information, environmental data needs to be collected.

Traditionally, environmental monitoring is conducted by official authorities which usually spend large amounts of money for high quality but expensive monitoring equipment followed by continuous labour and money investments on maintenance and calibration [5] which leads to often low spatial and temporal resolution [6]. Therefore, these data sources are often too sparse to meet the information demands from the public and organizations. For example, there are twelve air quality monitoring stations in Amsterdam operated by the Public Health Service of Amsterdam (GGD Amsterdam) and the National Institute for Public Health and the Environment (RIVM) at selected locations and most of these stations measure a limited number of air quality parameters; compared to Amsterdam, other cities in the Netherlands even have less official air quality stations (http://www.luchtmeetnet.nl/). In addition, in some developing countries, official environmental monitoring systems are completely absent [5]. For instance, during the 26/12/2004 tsunamis around the Indian Ocean, affected countries could have had enough time to avoid the disaster, if they had employed a functional alarm system earlier [7, 8].

There is a general need for flexible and affordable alternatives to complement the official or formal environmental monitoring stations. Recent developments of sensor technologies allow citizens to buy affordable sensors and electronic components like Arduino (https://www.arduino.cc/) and Raspberry Pi (https://www.raspberrypi.org/) to create sensor systems by themselves or with help from communities which provide alternative approaches to collecting environmental data [9, 10]. So-called informal sensors operated by citizens are not only raising public awareness of environmental problems from social aspects but also are potentially capable to complement the quantity and spatial-temporal resolution of the formal environmental data sources [11]. For example, after the Fukushima Daiichi accident on 11/3/2011, local citizens started using sensors, developed by a project called Safecast, to personally detect the radiation levels and shared the data to a website which gathered all these data coming from citizens. Now this has become an international community for radiation monitoring all over the world [12]. Citizens contribute to the data source and benefit not only themselves but also the larger public and policy makers with an independent source of environmental data. Similarly, weather stations operated by amateurs in the UK are rapidly growing and have become crucial data sources supplying timely and high density monitoring data for local weather information [13].

Citizen science has existed for a long time. It has been used for birds observations [14], invasive species monitoring [15], and other domains. Most of them use top-down approaches in which scientists design the research project and subsequently citizens are asked to join, mainly collecting data. Due to development of sensor technology, citizens can now use affordable sensors to monitor their living environment by themselves. They can even create their own sensor systems for their interest. However, it is still not clear how this bottom-up approach is organized; how to tackle the calibration and implementation problems; what the data quality of informal sensor networks is compared to formal sources; how the knowledge dissemination on sensor development is; and how to make sense of these data. According to previous studies, citizen science can be classified as follows: community consulting model where citizens only define the problems, community workers model where citizens are mainly involved in collecting data, and community-based participatory research model where citizens are involved in all research activities [16].

In this paper, by taking a case study focusing on air quality monitoring in the city of Amsterdam, a bottom-up citizen science approach for informal sensor environment monitoring is developed and evaluated.

2. A Bottom-Up Approach for Informal Sensing

The project named Amsterdam Smart Citizens Lab (http://waag.org/en/project/smart-citizens-lab) uses a bottom-up approach to organizing citizen sensing for urban environmental monitoring. This approach can be classified as cocreated class and is recognized as a community-based, participatory research model where citizens are involved in all steps of the project [16–18]. Within this approach, citizens are involved in each step (as shown in Figure 1) of sensing strategy together with project partners.

Figure 1: The project steps of Amsterdam Smart Citizens Lab for bottom-up citizen sensing.

The project was initiated by Waag Society, Institute for Art, Science and Technology, a pioneer in the field of digital media at Amsterdam. The approach is completely bottom-up. Waag Society is responsible for community coordination and provides a place for meetings and for sensor making in their Fablab Amsterdam, a place for makers. As organizer, Waag Society invited other partners who have expertise in different aspects to help citizens, such as Netherlands Organisation for Applied Scientific Research (TNO), Amsterdam Smart City (ASC), RIVM, and SenseMakers. Firstly, Waag Society organized a meetup called Topical BarCamp in which citizens raise their urban environment concern (issue mapping in Figure 1). According to the issue mapping from citizens and their interests, the community was divided into small groups to develop and test sensor systems for specific urban environment problems. During this period, the Fablab Amsterdam was open every Tuesday for teams to use their facilities for making prototypes. This step is called sensors making and done by an Open Hardware Bootcamp. After the prototypes were developed and tested, citizens used the sensors they developed themselves to monitor the city environment and collect data (sensing). These data were then interpreted and visualized with the help of experts. For each step, Waag Society invited experts to inspire and share their experience. In the next section, the adopted methods and results for every step of the proposed bottom-up approach are presented in more detail.

After issue mapping, four sensing topics (air quality, noise pollution, wind, and road bumpiness) were raised by citizens. For this paper, the air quality topic was selected as a case study to describe and evaluate the bottom-up citizen sensing approach. This case was inspired by previous experience within the Smart Citizen Kit (https://smartcitizen.me/) pilot in 2014. During this pilot, Amsterdam citizens measured temperature, humidity, light intensity, sound levels, carbon monoxide, and nitrogen dioxide in the city using Smart Citizen Kit 1.1 version. One outcome of the Smart Citizen Kit pilot was that the semiconductor air quality sensors (CO and NO₂) used in Smart Citizen Kit 1.1 version were not suitable for urban air quality monitoring [19]. A community was established including citizens and experts during this pilot. Some of them decide to join the Amsterdam Smart Citizens Lab to explore new sensors which can be used for urban air quality monitoring. Compared to other groups, this air quality group is relatively diverse including citizens, hardware developers, air quality experts, and university researcher.

3. Results

In this section, we present the results from each step of the bottom-up approach as indicated in Figure 1.

3.1. Issue Mapping: Topical BarCamp

At the issue mapping meeting (Figure 2), people discussed what kind of environmental problems they were concerned about and would like to solve. The meeting was organized as a Topical BarCamp which means that the content is provided by the participants. Air quality, noise pollution, wind, and road bumpiness were raised as main issues of concern. The Meetup environment (http://www.meetup.com/Amsterdam-Smart-Citizens-Lab) is used as a community platform that has functions like member registration, events organization, sharing of information, communication, and so on. Citizens were informed to join this platform during this meeting.

Figure 2: Citizens discuss environmental problems and sensing strategies during the issue mapping phase.

3.2. Sensors Making: Open Hardware Bootcamp

One aim of the air quality group is to find an air quality sensor with the proper requirements for urban air monitoring. The main three requirements are that the sensor should (1) be able to measure a pollutant relevant for urban environments, (2) be sensitive enough to measure typical ambient concentrations, and (3) be affordable. Over recent years, most of the concentration levels in the Netherlands have decreased substantially. Presently, the nitrogen dioxide levels are the most likely ones to lead to exceedances of legal threshold values. As a result, there is much emphasis on nitrogen dioxide, both from official authorities as well as from concerned citizens. Several measuring campaigns using passive NO₂ samples were undertaken during the last five years, by both municipalities and concerned citizens. Combined with the available state of the art in sensors for gasses at ambient concentration levels this has led us to focus on nitrogen dioxide concentrations. Based on these requirements and the experience of the Smart Citizen Kit pilot (http://waag.org/en/project/smart-citizen-kit), different sensor options were proposed by group members. With scientific proof [5, 20], after group discussion and comparison, the electrochemical Alphasense NO₂ sensor (NO₂-B42F) (http://www.alphasense.com/) was chosen for this experiment, according to the specification, the measurement range is 20 ppm NO₂ limit of performance warranty. The Arduino Uno or Arduino compatible microcontrollers were chosen to connect the sensor and other components such as the power supply, a real time clock (RTC), and a storage module (Figure 3). The RTC is used for timestamp; the temporal resolution can be programmed accordingly. For the measurement campaign, we measured every minute. To read the analogue signal from sensors, a high resolution analogue to digital converter (ADC) above 16 bits is required. The Arduino board only has a 10-bit ADC on board, which is not enough to accurately determine the output of the sensor that varied only a few millivolts (mV). Therefore, an external ADC was used. To reduce the noise of the data, a stable power supply is also recommended, which was not implemented. During the Open Hardware Bootcamp, the sensors were made by the citizens with assistance of experts of Fablab Amsterdam, RIVM, and Wageningen University. In total, five NO₂ sensor boxes were prepared in this phase of the project.

(a)

(b)

(c)

Figure 3: Main hardware, sensor, and different prototypes: (a) function test, (b) sensors with waterproof enclosure for outdoor test, and (c) sensor box for monitoring campaign.

To test the performance of the Alphasense NO₂ sensor, we first located four Alphasense NO₂ sensors together indoors at Waag Society as these sensors had no enclosures designed yet. Besides these Alphasense sensors, there is also a Smart Citizen Kit measuring outdoor of Waag Society. As can be seen in Figure 4, all Alphasense NO₂ sensors show good sensitivity compared to the Smart Citizen Kit, which did not show any variation of the 24 hours measurement period. However, although Alphasense NO₂ sensors indicated quite similar trends, the four sensors did not show the same concentration. Clearly, a calibration procedure is needed.

Figure 4: The calculated NO₂ concentrations measured by four sensors at the same test location (inside the building of Waag Society) and the measurement of a Smart Citizen Kit (outside Waag Society) every minute on 20/9/2015, for around 24 hours.

Next, four sensors were mounted in a common box and placed outside a window of the Waag Society building. Measurements were taken during five days and afterwards the raw outputs voltages of the sensors were converted to estimated NO₂ concentrations. For the calibration, readings from a nearby located official measuring station (Oude Schans, roughly 300 meters from the Waag Society) and one other station southwest of the centre of Amsterdam (Vondelpark, roughly 2700 meters from the Waag Society) were used. This latter station provides an indication of the city background concentration for NO₂ during the prevalent western winds.

According to “Alphasense 4-Electrode Individual Sensor Board (ISB); User Manual 085-2217,” the sensors output two voltages: the “Working Electrode” (WE) and the “Auxiliary Electrode” (AE). Both have to be corrected for a zero-offset of typically 225–245 mV. The value of the corrected AE is subtracted from the value of the corrected WE and the remaining voltage is divided by sensitivity in mV/ppm of typically 0.175–0.185. In order to get a similar set of hourly concentrations for all sensors, offsets for WE of 234 mV were combined with AE offsets between 220 and 245 mV, combined with sensitivities between 0.5 and 0.8 mV/ppm. Finally, a conversion from ppb to micrograms per cubic meter of air (μg/m³) was performed, as all Dutch concentrations are reported in μg/m³.

In the beginning of the comparison, the Alphasense sensors took several hours to become stable. With the adjustments, the concentrations show a roughly similar pattern in time as the official measuring stations (Figure 5). The concentrations show a daily pattern. The reason for the high concentrations at station “Oude Schans” between the hours 30 and 45 is not clear. The concentration values are averaged and presented in hourly values as shown in Figure 5.

Figure 5: Measured NO₂ concentrations outside Waag Society during five days. Oude Schans and Vondelpark are official stations operated by GGD Amsterdam.

In order to obtain a similar behaviour for all sensors, the parameters used to convert output voltages into concentrations varied substantially between sensors. The concentrations show a roughly similar pattern in time as the official measuring stations do.

3.3. Sensing: Citizen Data Collection

After system testing and improving, an air quality campaign test was conducted by the air quality monitoring group on 2/12/2015. After discussion on a paper map for measurement campaign location planning, the digital map was developed for online access (Figure 6). The developed sensor prototypes do not have Global Positioning System (GPS) module; to get the measurement location’s latitude and longitude coordinates, a measurement location retrieval tool was developed which can use GPS and Wi-Fi signals on the smartphone to retrieve location coordinates (Figure 7). A website was developed to host these prototype tools and visualization map. Twenty-Seven locations were selected including city background, traffic streets, and parks. Figure 8 shows a monitoring example that is close to an official monitor station (Amsterdam-Stadhouderskade) for comparison. For each location, the air quality sensor was operated for a minimum of 15 minutes, which enabled the sensors to stabilize.

Figure 6: Plan for locations in Amsterdam to perform air quality monitoring.

Figure 7: Tool to retrieve latitude and longitude of measurement locations.

Figure 8: An informal measurement point is close to an official station (Amsterdam-Stadhouderskade) for data comparison.

3.4. Understanding: Visualization and Interpretation

A typical concentration pattern is shown in Figure 9. First, a measurement was done outside (traffic); next, the sensor was transported to a new location and was put in a bag for some time before taking out and moving to next location; and, subsequently, a follow-up measurement was done at the new location (park). As can be seen in Figure 9, the NO₂ sensor shows considerable different results between in the bag and out of the bag. No large differences between results in traffic and park locations are observed. In order to evaluate the performance of Alphasense NO₂ sensors, we installed sensors close to the GGD Amsterdam official air quality monitoring stations (see Figure 8 as an example). The official station data can be downloaded from the web portal Luchtmeetnet (http://www.luchtmeetnet.nl/) and are on hourly basis (Figure 10). Compared to the official measurements, the informal Alphasense NO₂ sensor measured concentrations in minutes and shows a similar trend (Figure 10).

Figure 9: Rough comparison of measurements from one sensor.

Figure 10: Roughly the same period of the informal sensor measurement (without calibration, 1 ppb = 1.91 μg/m³ is used for the conversion) every minute (11:37–13:36) and the hourly official measurement in dashed line (11:00–14:00) at Amsterdam-Stadhouderskade.

To visualize NO₂ concentration data, an online map service (Figure 11) was developed using open source Mapbox and GitHub Pages. The concentration is colour coded according to the Dutch national standard.

Figure 11: The data visualization map prototype for NO₂ measurement campaign on 2/12/2015.

3.5. Community Analysis

As shown in Figure 12, the number of members of the Amsterdam Smart Citizens Lab has been increasing continually which means that this community received continuous attention from the public. However, according to Figure 13, the number of active participants for each community event did not increase substantially.

Figure 12: The growing number of total signed members (data collected from Meetup) compared to event participants indicated in purple bars.

Figure 13: The variation of community participation: data collected from Amsterdam Smart Citizens Lab Meetup, the actual participants may be different, especially the first meeting in which most citizens have not registered for the Meetup (see Figure 2).

4. Discussion

In this part, we first take air quality monitoring as an example to discuss the technical and operational challenges. Next, we analyse and discuss the bottom-up approach for citizen sensing.

4.1. Citizen Sensing Is Still Challenging

The Arduino board is a relatively simple and powerful tool for experimentation. However, to retrieve reliable data, professional knowledge and support are still needed. An important issue for air quality measurement is sensor data calibration and analysis. With different tests, several problems were identified, but we did not have sufficient time within this project to find solutions or to test potential solutions. The NO₂-B42F sensor is cross-sensitive to ozone, temperature, and humidity, as described in the specifications. Our tests suggest that other environmental factors may also influence the results of measurements. The indoor test as shown in Figure 4 shows that even each sensor package was calibrated individually by the sensor company; the four sensors indicated quite different concentrations. This is an obstacle for citizens, because if further calibration is needed, citizens normally do not have the facilities to do so.

This project was based on the experience of the Smart Citizen Kit pilot [19]. The general aim was to encourage citizens to measure their environment by developing sensor platforms together in a community. According to the outcomes and observations of Amsterdam Smart Citizens Lab, this complete bottom-up approach is challenging but can be successful. Still an important precondition for success is that expert communities are involved. They provide the required support to build a fully functional sensor system which can collect reliable data. Independent development would still be challenging for citizens. There are quite a lot of sensors and sensor-related electronic products, which are assumed to be “plug and play” products. To create a functional environmental sensing system is an interdisciplinary task which requires knowledge from different domains like electronics, environmental sciences, communication, information technology, design, and so on.

4.2. Bottom-Up Approach Calls for Broad Open Cooperation

The continually reduced costs and increased functions of sensors and microcomputers offer citizens possibilities to measure the environment by themselves, which has already made an impact for environment awareness and decision making. There are a lot of open hardware and software resources that can be used by citizens. Furthermore, the fab labs and the maker movement also play important role in citizen science. The fab labs provide facilities for citizens to get together to communicate, use facilities, and get help to make their sensor platforms. From this, we see organizations that support citizens sensing as an important resource. Research conducted by Balestrini et al. [21] concluded that the supporting organization is important in terms of connecting people, supplying guidelines, and helping each other. From our observation, this is confirmed in this case study. In particular, we observe that the support or involvement from professionals in pushing the community by supplying information, suggestions, and technical assistance is very valuable. Since environmental sensing by citizens is an interdisciplinary field, the experts from different domains foster the community and keep the project running.

4.3. Stimulate Citizens for a Sustainable and Growing Community: Online Plus Offline

In the Amsterdam Smart Citizens Lab, Fablab Amsterdam plays an important role for offline meetups for citizens to learn skills, work with like-minded people, and create and test prototypes. In addition, the online meetup platform is not only used to organize offline meetups but also functioned as a broader virtual community platform for information sharing and attracting more people to join the community. Even though from Figure 12 the active participants did not increase substantially compared to the continuous growing online community, it shows that citizens are interested in the project and potentially could become active participants. From the Meetup platform, citizens know what has been achieved by the Amsterdam Smart Citizens Lab community; they can join the community and keep it updated for further activities. This “online plus offline” model is a good approach to organizing citizen sensing. There are 602 fab labs in the world and the number is still growing (https://www.fablabs.io/labs). In addition, there are diverse open living labs (http://www.openlivinglabs.eu/) and hacker spaces. For online tools, besides Meetup (http://www.meetup.com/), social media like Facebook, Twitter, YouTube, and blogs can also be used for community building. Trello and Slack can be used for teamwork. If these tools and open offline physical places can be used together effectively and efficiently, this may help to stimulate citizens to create a sustainable and growing global and local community.

5. Conclusions

In the present paper, we present a bottom-up approach for citizen science to collect informal urban environmental sensor data. We found that highly sensitive electrochemical sensors potentially have better performance than semiconductor sensors for urban air quality monitoring but need appropriate hardware and software design, careful calibration, and postprocessing to deliver correct and usable data. This leads to challenges for citizens to build sensor systems from scratch. Therefore, wide cooperation from different aspects such as community building, maker spaces, fab labs, and different types of professional support for domain knowledge is essential for citizen sensing. This wide cooperation, together with effective use of online and offline facilities, can keep the citizen sensing community sustainable and growing.

6. Future Work

For future work, the focus will be on how citizens and experts can work together to optimize the citizen environmental sensing model for reliable data provision to benefit citizens, official organizations, and more importantly the whole society. For example, the data quality is a big concern for both official organizations and citizens. Official organizations can, for instance, guide and help citizens to calibrate sensors in order to improve data quality. How local governments react on citizen sensing can be another interesting research topic. In terms of the environmental monitoring, other air pollutants like particulate matter, CO, and O₃ need also to be considered.

Competing Interests

The authors declare that they have no competing interests.

Acknowledgments

The valuable support of Huib de Bats, Bob Beertema, Mirjam Bekker, Co de Boer, Pieter van Boheemen, Guillem Camprodon Pujol, Tomas Diez, Jimena Gauna, Sybrand Hekking, Christine van den Horn, Wo Meijer, Fokko Visser, and other members of Amsterdam Smart Citizens Lab is highly appreciated by the authors. The Amsterdam Smart Citizens Lab is supported by The municipality of Amsterdam and Creative Industries Fund NL. Qijun Jiang is supported by China Scholarship Council (CSC) for his Ph.D. research.

References

C. E. Kolb, S. C. Herndon, J. B. Mcmanus et al., “Mobile laboratory with rapid response instruments for real-time measurements of urban and regional trace gas and particulate distributions and emission source characteristics,” Environmental Science & Technology, vol. 38, no. 21, pp. 5694–5703, 2004.

E. Schnebele, G. Cervone, S. Kumar, and N. Waters, “Real time estimation of the calgary floods using limited remote sensing data,” Water, vol. 6, no. 2, pp. 381–398, 2014.

J. Zhang, K. Howard, C. Langston et al., “National mosaic and multi-sensor QPE (NMQ) system: description, results, and future plans,” Bulletin of the American Meteorological Society, vol. 92, no. 10, pp. 1321–1338, 2011.

G. Werner-Allen et al., “Monitoring volcanic eruptions with a wireless sensor network,” in Proceedings of the 2nd European Workshop on Wireless Sensor Networks (EWSN '05), pp. 108–120, Istanbul, Turkey, February 2005.

M. I. Mead, O. A. M. Popoola, G. B. Stewart et al., “The use of electrochemical sensors for monitoring urban air quality in low-cost, high-density networks,” Atmospheric Environment, vol. 70, pp. 186–203, 2013.

D. Hasenfratz, O. Saukh, and L. Thiele, “On-the-fly calibration of low-cost gas sensors,” in Wireless Sensor Networks: 9th European Conference, EWSN 2012, Trento, Italy, February 15–17, 2012. Proceedings, vol. 7158 of Lecture Notes in Computer Science, pp. 228–244, Springer, Berlin, Germany, 2012.

R. Samarajiva, “Mobilizing information and communications technologies for effective disaster warning: lessons from the 2004 tsunami,” New Media and Society, vol. 7, no. 6, pp. 731–747, 2005.

I. Kelman, “Warning for the 26 December 2004 tsunamis,” Disaster Prevention and Management, vol. 15, no. 1, pp. 178–189, 2006.

A. Zerger, R. A. Viscarra Rossel, D. L. Swain et al., “Environmental sensor networks for vegetation, animal and soil sciences,” International Journal of Applied Earth Observation and Geoinformation, vol. 12, no. 5, pp. 303–316, 2010.

I. F. Akyildiz, W. Su, Y. Sankarasubramaniam, and E. Cayirci, “Wireless sensor networks: a survey,” Computer Networks, vol. 38, no. 4, pp. 393–422, 2002.

P. Corke, T. Wark, R. Jurdak, W. Hu, P. Valencia, and D. Moore, “Environmental wireless sensor networks,” Proceedings of the IEEE, vol. 98, no. 11, pp. 1903–1917, 2010.

A. Hemmi and I. Graham, “Hacker science versus closed science: building environmental monitoring infrastructure,” Information Communication and Society, vol. 17, no. 7, pp. 830–842, 2014.

S. Bell, D. Cornford, and L. Bastin, “The state of automated amateur weather observations,” Weather, vol. 68, no. 2, pp. 36–41, 2013.

B. L. Sullivan, C. L. Wood, M. J. Iliff, R. E. Bonney, D. Fink, and S. Kelling, “eBird: a citizen-based bird observation network in the biological sciences,” Biological Conservation, vol. 142, no. 10, pp. 2282–2292, 2009.

D. G. Delaney, C. D. Sperling, C. S. Adams, and B. Leung, “Marine invasive species: validation of citizen science and implications for national monitoring networks,” Biological Invasions, vol. 10, no. 1, pp. 117–128, 2008.

C. C. Wilderman, “Models of community science: design lessons from the field,” in Proceedings of the Citizen Science Toolkit Conference, C. McEver, R. Bonney, J. Dickinson, S. Kelling, K. Rosenberg, and J. L. Shirk, Eds., Cornell Laboratory of Ornithology, Ithaca, NY, USA, June 2007.

M. Morzy, “ICT Services for open and citizen science,” World Wide Web, vol. 18, no. 4, pp. 1147–1161, 2015.

C. B. Cooper, J. Dickinson, T. Phillips, and R. Bonney, “Citizen science as a tool for conservation in residential ecosystems,” Ecology and Society, vol. 12, no. 2, 2007.

C. V. D. Horn and R. Boonstra, Eindrapportage Smart Citizen Kit Amsterdam Meten is Weten?Waag Society, 2014.

D. Hasenfratz, O. Saukh, C. Walser et al., “Deriving high-resolution urban air pollution maps using mobile sensor nodes,” Pervasive and Mobile Computing, vol. 16, part B, pp. 268–285, 2015.

M. Balestrini, T. Diez, P. Marshall, A. Gluhak, and Y. Rogers, “IoT community technologies: leaving users to their own devices or orchestration of engagement?” EAI Endorsed Transactions on Internet of Things, vol. 15, no. 1, article e7, 2015.