The atmosphere plays a central role in the global circulation of heat, water and volatiles, making our current life possible and convenient. It is the reservoir of the breathable gases and takes away the exhaled gases that would otherwise poison our immediate surroundings. Our civilization is closely linked with the atmosphere in a multitude of ways, whether we are considering simple agricultural communities or modern urban areas.

The CITYZER project concentrates on two closely linked aspects of the atmosphere with the focus on urban climate. On the one hand, we consider air quality, which is highly important in particular due to its possible adverse health effects. On the other hand, we consider weather phenomena, in particular those linked with precipitating weather systems. Weather events can cause episodes of high pollutant concentrations in urban areas, or they can help clean the urban air through washing and wet deposition or by bringing in cleaner air from adjacent areas. In their own right, the weather effects can be hazardous, causing infrastructural damages and loss of lives, or they may be bothersome and cause harmful friction for logistics planning and the everyday life of human society. Collecting sensor data in near real time from the area of interest, sophisticated interpolation and forecast software provides reliable information for the next hours concerning air quality and weather phenomena, which can be made available to the public or service providers via mobile application or warning systems (Harri et al., 2018; Smart & Clean, 2020).

The main objective of the project was to develop new digital services and products to support decision-making processes related to weather and air quality. The overall idea was to create a new ecosystem of services open to third-party developers, based on open data. To that end, the project set out to design and implement an IoT-based platform for collecting, refining and delivering environmental data. Accordingly, the project was closely linked to several megatrends, namely open data, big data, platform economy and digitalization.

The CITYZER ecosystem consists of three levels:

Measurement data and observation level contain the data collection and management systems like sensor networks and public or restricted data repositories.

This article concentrates on the first and the third level, leaving the details of the analysis forecast software for weather and air quality to other publications.

The system is very modular with well-defined interfaces allowing the replacement, removal or addition of different elements and their implementation in different environments from commercial hardware configurations to virtual cloud-based networks.

The six main modules of the CITYZER system architecture are presented in Fig. 2.

Observation networks consist of various sensors for all observation parameters to be included into the specific deployment together with the field data networks for forwarding the measurement data and controlling the sensors from remote supervising stations where applicable.

Depending on the needs or the deployment one or several forecast model software packages can be implemented. These might reside inside the same physical computer system as the control system or could be implemented on different platforms and linked to the control system via file-sharing technology. In the course of the project the precipitation nowcasting software for rainfall was installed and operated on the same Linux-based server as the control module. It was adapted from the modelling software developed at the Finnish Meteorological Institute as part of the RAVAKE project (Heinonen et al., 2013). The source code size was 205 kB with a size of the executable file of 180 kB. A typical run time for one complete weather forecast covering the larger Helsinki area is about 8 to 13 s. A motion vector analysis for the complete area of Finland takes about 1 min.

The air quality forecast modelling software ENFUSER (ENFUSER, 2020; Johansson et al., 2015) was developed into an operational modelling system with test implementations for the CITYZER demonstrator in a virtual Linux environment, as cloud service and on a local Windows computer. It includes detailed treatment of traffic emissions for individual roads, shipping emissions and elevated point sources such as power plants taking into account urban morphology, atmospheric stability and rain forecast information.

Sensor data can be provided in two different ways:

Public data are accessible via standard web interfaces. The CITYZER environment implements the Open Geospatial Consortium's Web Feature Service (OGC/WFS) as described in the organization's document OGC 04-094, (OGC, 2020) reference list. Most European and many worldwide weather service data are available using this standard.

The sensor data are collected in the data storage in a unified format and sorted according to data type as grid data or coordinate-based point data, day of observation and sensor providing the data.

The analysis and forecast software modules are ingesting the data needed for the forecast interval, adapting to the extended or reduced availability of sensors automatically. The forecast results are written back into the same data storage using a format compatible with the sensor data but organized into different files for easy access.

A database system keeps track of the available data and provides this information on request to user applications. These may be mobile applications allowing the public to assess the weather and air quality situation in the near future in an area of interest. The data related to the inquiry are then provided as links to the respective file in the data storage for download by the application server. This usually will also access geographic data from external providers as a platform on which the weather and air quality data are presented to the end user.

Alternative access to the forecast data was implemented using the open-source SmartMet Server interface (SmartMet, 2020) developed at the Finnish Meteorological Institute in 2013 as implementation of the INSPIRE (INSPIRE, 2020) requirements for open data access. This interface has already been used by several applications outside the framework of the project.

One critical aspect of any multi-interface system is the system and data security. Most of the sensor systems deployed in the field can usually be compromised without excessive efforts. The same is true of interfaces between service providers and end-user application systems like mobile phones. The CITYZER system architecture therefore provides hardware solutions to protecting the central data storage and forecast software resources. Any connection for incoming sensor data or requests from application servers is separated by a strict firewall without any external write access possibility. The control system sends information requests to each connected data provider at regular intervals. When new data are available it fetches these data, filters them through incoming filter software and writes the possibly re-formatted data into its internal data storage, thereby eliminating any illegal data or possible command instructions.

The availability of new sensor or model output data is indicated to the application database outside the firewall, providing a complete link to these data. All data storage files are accessible by the application database software via a hardware read-only link without any write-back possibility. In the case of a compromised database, it can be re-built on the fly from the data inside the secure data storage.

Access to the application data server is controlled via an authentication mechanism. Each possible user has to register first with this server, indicating also the type of access intended: data polling and fetching by the application or automatic push service according to rules to be defined by the user service like update time, geographical boundary parameters to be provided etc.

In the example diagram, Fig. 3, one set of air quality sensors from Pegasor is connected via a public phone data network or via an Emtele-provided LoRa (LoRa Alliance, 2020) sub-network to Vaisala Observation Network Manager NM10. Another set of air quality sensors, AQT400 series from Vaisala, is connected via a Vaisala Beacon View data access and control system to the NM10. Weather radar data are collected separately via the common Vaisala IRIS system for map generation. The connection between NM10 and the second layer control system is realized as Open Geospatial Consortiums' Web Feature Service (OGC/WFS) to make it as universally adaptable as possible. Also other public data servers are connected to the control system via OGC/WFS compatible communication standards. The abbreviations used in the flow diagram are OD for open data, AO for air quality observations, WO for weather observations, MD for model data and CRC for command flow from the run control module.

The control unit synchronizes all activities in the CITYZER system. Based on regular timing or action events, different processes are started, data are collected or the database is updated, thereby indicating to the application processes that new data are available.

The initial configuration is defined as follows – see the timing information in the leftmost column in Fig. 4.

Once a minute requests are sent via OGC protocol to the attached and registered data providers to check the availability of new data. In the case of new data these are fetched, converted if necessary and stored in the respective file system of the server according to type, sensor and time of observation. All incoming data are expected to be calibrated in standard physical units. The data sets usually also contain quality information indicating whether the related sensors were correctly calibrated or offline. Such an update is also indicated to the database system to make observational data directly available to the users, possibly initiating an alert to the application servers if so requested.

Every 5 min new weather radar data are fetched and stored in the file system, and the rain nowcasting model is activated. Its results are available within a minute and stored in a different part of the file system. This information is also forwarded to the database system. At a special resolution of 250 m the size of one radar composite file varies between 50 and 1000 kB depending on the amount and type of observed precipitation. A forecast output motion vector field covering the whole area of Finland has a typical size of 60 kB.

Every 60 min the air quality data modelling system is activated providing new hourly and 1-day forecasts of the air quality development in the covered area. These forecasts both make use of fresh observational data as newly calculated rain nowcast results anticipating air quality modifications by imminent rainfall. One typical model run for the Helsinki area takes about 35 min and generates maps with a grid size of 13 m × 13 m. These approximately 3 million grid cells contain concentration estimates for NO2 and O3, dust concentrations PM10 and PM2.5, and air quality index (AQI) estimates. For each forecast about 1000 local air quality measurements are ingested and used besides the hourly 10 km resolution weather forecast maps and the 100 m resolution/5 min time resolution rain nowcasting data.

Alternatively any analysis software package might run autonomously at certain time intervals accessing those data available at the start of a new activation.

The output of each control script is logged. Figure 4 shows the various data flow paths, where the solid arrows indicate the flow of data between the different modules, and the dashed lines represent command flows for synchronization of the different modules and control of the attached sensors where possible and needed.

In the demonstration implementation the central control module is implemented as a set of scripts activated according to specific timing rules. A typical control and time diagram is shown in Fig 4.

User applications register usually with an application service provider which provides a downloadable application for displaying the requested data and a service which generates these data according to the requirements of the end user. The application server has to register with the CITYZER database to gain access to the data. At the time of registration the strategy of accessing new data can be defined either as notification to get alerted whenever data relevant for the application are updated or as polling, where the application requests data matching certain criteria, and the database provides these data if available. In both cases the returned information provides a complete data link address from where a file with the related data can be retrieved. This scheme is shown in Fig. 5.

The system is designed to support also the generation of automatic alerts. If one or several parameters exceed pre-defined limits, an alert message can be generated and sent to the connected mobile phone or user service centre. This feature is currently implemented in the Helsinki area to re-schedule the street cleaning services in case microdust levels exceed pre-defined levels on major streets before they become a health hazard.

The rolling storage of data is divided into two separate file systems:

Local storage for incoming and cache-type data is on a local disc. The storage size of the demo version is 8 GB.

Radar data for rain nowcasting are stored in the ODIM HDF5 format (ODIM, 2020), developed by the European Meteorological Service Network EUMETNET. Data originating from the various air quality sensors and air quality model data for single geographic points are stored in the XML-based GML format (GML, 2020), also endorsed by the INSPIRE directive. Coordinate grid data are stored in NetCDF format (NetCDF, 2020).

In the local storage there is a directory “Today”, which is linked every day at midnight to the actual directory of the new day (Mon, Tue, Wed etc.). Each daily directory is structured as follows:

Nowcasting contains data for the rain nowcasting model RAVAKE.

The Linux crontab-based data fetch process of the run control module copies and links new files from these directories for notification and fetching for CITYZER database server. The fetch process checks the status of dedicated log files in the shared output directories at defined time intervals (currently 10 min) to copy new data files to storage and link them for notification.

As verification of the CITYZER ecosystem concept, all components were implemented in the Helsinki area (see also Fig. 3). Existing air quality sensor networks were augmented by additional sensors and their various data connections. Real-time data from the Finnish weather radar network were used as input for the rain forecast software. The data server, together with the modelling software and the application database system, was built using resources of the Finnish Meteorological institute. A dedicated application for web access and mobile phones was developed showing the forecast information for rain and air quality parameters projected onto a map of the larger Helsinki area and allowing also the specification of automatic alerts in case a specified parameter exceeds a pre-set limit. In preparation for the deployment of the demonstration system, several air quality measurement campaigns were executed (e.g. Hietikko et al., 2018; Järvinen et al., 2019; Kuula et al., 2019; Teinilä et al., 2019). An example of an air quality forecast map for the larger Helsinki area is shown in Fig. 6. The corresponding rain forecast map for the same area can be seen in Fig. 7, while the application was configured to show the probability of light rain. Alternatives for medium and heavy rain are also offered. An additional inbuilt service allows the end user to receive an alert via mobile phone when the probability of rain at a user-specified point exceeds a given level.

A subset of the CITYZER ecosystem concentrating on the forecasting of air quality parameters was developed into an operative service, implemented and maintained by the Finnish Meteorological Institute. It started its routine operation in the beginning of 2019 and is among other applications used by the city of Helsinki to optimize the deployment of street maintenance and cleaning resources to keep the dust level along traffic routes at a low level (HAQT, 2020). Updated air quality forecasts for the Helsinki region are also shown routinely on the electronic advertisement panels in the Helsinki metro trains (HSY Air quality map, 2020).

A modified version of the HAQT environment was developed and deployed in the Nanjing region of China; see Harri et al. (2018).

The main purpose of the EU-funded project HOPE is to empower citizens to develop their own districts. The project focuses on three different districts with varying air quality challenges: one district with heavy port-related traffic, another with street canyons and with over 40 000 daily vehicles, and the third district which is affected by main roads and wood burning at homes. The change that the project wants to achieve is that the citizens will find air quality issues easily relatable and understandable by creating a feedback loop between high-resolution hyperlocal air quality data and actions of individuals and communities (HOPE, 2020).

During the CITYZER project, students of the Haaga-Helia University of Applied Sciences in Helsinki were asked to develop mobile applications for air quality and rain forecast monitoring based on the CITYZER demonstrator implementation. Using an in-house developed learning ecosystem for business information technology, the students successfully demonstrated that new service concepts and mobile applications can be easily developed and linked to the data services provided by the CITYZER environment.