News & Events

Kunak has proven to be among the best sensor-based Air Quality stations in International Evaluations

A full comparative report on the results of AQSPEC Programme in Los Angeles, California
Air Quaility Monitoring Technology Comparative Assessment

As USEPA stated in its Air Sensor Evaluation Toolbox, “emerging air quality sensors – with general traits of being more compact, directly reading pollutants, and lower in cost than traditional methods – have a wide appeal to professional researchers, community groups, students, and citizen scientists alike. […] Since this technology is still under development, little information exists on the quality of data that these sensors produce.”

Air Quality Sensor Performance Evaluation Center (AQSPEC – http://www.aqmd.gov/aq-spec) was established by the South Coast AQM as a programme to inform the general public about the actual performance of commercially available “low-cost” sensors. The objective of AQSPEC programme is to carry out a complete characterization of the currently available “low-cost” sensors.

In this context, their main goals of the AQSPEC programme are:

  • Evaluate the performances of commercially available “low-cost” air quality sensors.
  • Provide guidance and clarity for ever-evolving technology and data interpretation.
  • Catalyse the successful evolution, development, and use of sensor technology.

“AQSPEC programme is carrying out an important task, testing and evaluating the performances of current “low-cost” sensors and systems which integrate those sensors, and giving actual information in an objective way.”

Air Quality Sensor Performance Evaluation Center (AQSPEC – http://www.aqmd.gov/aq-spec) was established by the South Coast AQM as a programme to inform the general public about the actual performance of commercially available “low-cost” sensors. The objective of AQSPEC programme is to carry out a complete characterization of the currently available “low-cost” sensors.

In this context, their main goals of the AQSPEC programme are:

  • Evaluate the performances of commercially available “low-cost” air quality sensors.
  • Provide guidance and clarity for ever-evolving technology and data interpretation.
  • Catalyse the successful evolution, development, and use of sensor technology.

“AQSPEC programme is carrying out an important task, testing and evaluating the performances of current “low-cost” sensors and systems which integrate those sensors, and giving actual information in an objective way.”

Access the full report

Access the full report

Access the full report

As it was already explained in the web article “The necessity of standards and certification programmes on air quality monitoring”, during the last decade, AQSPEC external evaluation centre has focused on determining the performance of different air quality sensors system, due to a lack of an accepted performance specification for “low-cost” sensors.

Hence, AQSPEC programme is carrying out an important task, testing and evaluating the performances of current “low-cost” sensors and systems which integrate those sensors, and giving actual information in an objective way.

The main objective of the report created is to assess the different “low-cost” sensors and air quality systems, regarding the results of the gas and particle sensors obtained in the AQSPEC evaluation.

The original AQSPEC evaluation assessment can be found in http://www.aqmd.gov/aq-spec/evaluations/field.

It has to be noted that the evaluation processes shown in this report, and in the reports of the AQSPEC web, are carried out in different seasons and under different conditions. Thus, the results shown must be treated carefully, this report is only an indicative assessment.

KUNAK AIR A10 Field Evaluation

Three Kunak devices were evaluated by the AQSPEC programme from May to July 2019. They were run side-by-side with Federal Equivalent Method (FEM) and Federal Reference Method (FRM) instruments measuring the same pollutants.

Each device reported O3, CO, NO, NO2, NOx, as well as PM1, PM2.5 and PM10, and meteorological parameters (temperature, relative humidity, pressure, wind speed and wind direction). The report of this field evaluation can be found in AQSPEC web page (http://www.aqmd.gov/docs/default-source/aq-spec/field-evaluations/kunak-air-a10—field-evaluation.pdf).

Gas and Particle sensor assessment

The gas and particle sensors are compared against different reference (FRM) and equivalence (FEM) instruments. The tests performed by AQSPEC are:

  1. Carbon monoxide (CO)
  2. Nitrogen oxides (NO2 and NO)
  3. Ozone (O3)
  4. Particulate Matter (PM1, PM2.5 and PM10)

A correlation between the sensor data and the reference data is used for AQSPEC as an evaluation method. The parameter shown in the comparative assessment is the R2 and the equation line given by AQSPEC.

It has to notice that this metric has some limitations. The data needs to follow a normal distribution, which is not always the case in real studies. Besides, R2 is insensitive to bias between the air quality system or sensor and the reference data used for the evaluation (Karagulian et al. 2019). Thus, R2 should be accompanied by another metric to measure the error between the data and the references, i.e. MAE, RMSE, or the expanded uncertainty.

Besides, the intra-model variability of the sensor is given. This is an important parameter, especially when a network is going to be deployed and it is not always considered. It shows how different devices from the same manufacturer are going to behave. Low intra-model variability means that you can use one device of the deployment to assure the good operability of all the network since all the sensors and sensor systems will behave in the same way.

Table 1 shows the date, duration, and season in which the different sensors and sensor systems were evaluated by the AQSPEC programme.

Table 1. Air quality sensors and sensor systems evaluated by AQSPEC

Air Quality SystemSensorStudy DateStudy Duration (days)Season
Kunak AIR A10Gas + PM28/04/201911/07/201974Spring/Summer
AeroqualGas22/12/201727/03/201895Winter
APISPM13/03/201914/05/201962Spring
AQMesh v4.0Gas26/06/201525/09/201591Summer
AQMesh v5.1Gas11/04/202018/06/202068Spring
AQMesh v3.0PM11/04/202018/06/202068Spring
VaisalaGas06/03/201803/05/201858Spring
Cairpol COGas22/11/201817/01/201956Winter
Cairpol NO2Gas02/05/201910/07/201969Summer
UNITECGas01/07/201531/07/201530Summer
Perkin ElmerGas + PM22/07/201525/09/201565Summer
uradPM30/10/201808/01/201970Winter
Alphasense OPC N3PM15/08/201811/10/201857Summer/Autumn
SensirionPM07/03/201914/05/201968Spring
ClarityPM15/02/201825/04/201869Winter/Spring
Purple AirPM08/12/201626/01/201749Winter
Met OnePM01/06/201515/07/201544Summer
Air BeamPM20/07/201819/09/201861Summer
Air quality EggPM25/04/201826/06/201862Spring
Sensirion NuboPM27/12/201927/02/202062Winter
TSI AirAssurePM18/12/201515/02/201659Winter
TSI BlueskyPM08/04/202015/06/202068Spring

Regarding Kunak AIR A10, it uses onboard sensors and Kunak embedded algorithm that calculates the concentrations in ppb or μg/m3 locally and in real-time, not using or needing external reference data to calculate gas or particle concentrations. Kunak algorithm corrects the interferences and artifacts typically found in raw data compensating environmental effects like temperature, humidity and pressure as well as cross-sensitivities using onboard data.

For most of the electrochemical gas sensors, humidity and temperature effect have a no-linear response (Lewis et al. 2018; Pang et al. 2018; Wei et al. 2018). The raw data is corrected by the Kunak algorithm converting the no-linear response into a linear response. However, none of the sensors were calibrated before deploying. The only calibration carried out was the factory calibration.

“The Kunak AIR A10 uses onboard sensors and embedded algorithm that calculates the concentrations in ppb or μg/m3 locally and in real-time, not using or needing external reference data to calculate gas or particle concentrations.”

Figure 1 shows the R2 values obtained for all the sensors and sensors systems evaluated, and Figure 2 shows the intra-model variability. The blue star shows the value achieved by Kunak AIR A10, being possible to compare if Kunak AIR A10 system with the rest of the evaluations carried out by AQSPEC programme. For more information, you can download the full report, in which a detailed comparison with other sensors evaluated by AQSPEC is available.

Figure 1. R2 values obtained between the different sensors evaluated. Blue start shows the R2 obtained by Kunak AIR A10.

Figure 2. Intra-model variability values obtained between the different sensors evaluated. Blue start shows the intra-model variability obtained by Kunak AIR A10 (The lower intra-model variability the better)

In the specific case of Kunak AIR A10:

  • For O3, Kunak AIR A10 achieved an averaged R2 with a slope equal to 1, meaning that the concentration values collected by Kunak O3 sensor provide very accurate data.
  • Kunak AIR A10 achieved a medium result for CO. In the case of the CO, most of the values were under the CO performance limit level un field conditions. Besides, due to a high-temperature event, up to 40ºC, the CO sensor started to overestimate the CO concentrations. At the beginning of the field evaluation, Kunak sensor systems followed the same trend and reach the same CO values as the reference, but after the high-temperature event, a sensor baseline recalibration is needed, as it is explained in Kunak Air A10 Manual, however it was not carried out during the field assessment.

Figure 3. CO 24 hour means measures, Kunak AIR A10 vs FRM

Due to R2 metric have some limitations, as it was explained above, another metric should be considered. The future European Technical Specification, “Performance evaluation of air quality sensors”, will be use the expanded uncertainty for field evaluations, in which the maximum uncertainty allowed regarding the different pollutants will be set.

Thus, this expanded uncertainty was calculated in the case of CO sensor. The future specification said that the maximum expanded uncertainty allowed for CO is 25%, and the one achieved in the case of the CO sensor with the data obtained during the AQSPEC field evaluation was, 12.20% without needed any calibration, and using the regression line calculated, it could be achieved an expanded uncertainty of 4.25%, being even lower than the 15% expanded uncertainty required for reference stations according to the EU Directive/2008/50 CE.

  • Kunak NO2 sensor obtains a low R2 value, like the rest of the sensors evaluated by the programme, except for the one that was evaluated during winter. The study was performed during summer where low concentrations of NO2 appeared, leading to a higher error. Besides, several humidity transients occurred during the study period provoking an effect on the sensors. Humidity transients’ effects in electrochemical sensors, especially in NO2 sensors, have not been studied yet. By the moment, the data obtained when a humidity transient occurs should be invalidated. However, it was not the case during the field evaluation where all the data were used for the evaluation study.
  • Only four air quality systems have been evaluated for NO sensor. Only two NO sensors obtained a high R2 value (Kunak AIR A10 R2 = 0.87), while the other two sensor systems were not able to measure NO.
  • Evaluating Kunak AIR A10 PM5 sensor against other sensors, it is on average with them, while for PM10, Kunak obtained the highest R2 when compared with the other sensors evaluated. It is noticed that some devices which obtained good results for PM2.5 were not evaluated for PM10 because the sensor used is not able to measure both PM2.5 and PM10. Kunak AIR A10 is the only sensor system which provides PM2.5 and PM10 accurate measurements.
  • Regarding the intra-model variability, Kunak AIR A10 device is the only sensor system which obtained the lowest intra-model variability values in every sensor.
figure-4

Figure 4. Intra-model variability obtained regarding the different sensors integrated into the Kunak Air A10 system.

Conclusions

It must be highlighted that the evaluation processes were performed in different seasons and under different conditions by the AQSPEC programme.

To conclude:

  • Kunak AIR A10 devices were not calibrated prior to the evaluation. It is shown how Kunak gas sensors do not need a calibration, apart from the factory calibration, while Kunak PM sensor would need an adjustment of the mass (correction factor) which depends on the particle type specific in each location, as occurs with the particle FRM and FEM instruments.
  • Kunak AIR A10 devices obtained very high correlation values for the O3 and NO sensors, medium correlation value for CO and poor for NO2 due to high temperatures, low pollutant concentrations, and in the specific case of the NO2 due to humidity transient effects. All these effects are a common problem when using electrochemical sensors and affect to all manufacturers, who are working continuously to improve their algorithms to achieve a sensor system in which these temperature and humidity effect will not be a problem in a near future.
  • Kunak AIR A10 obtained the lowest intra-model variability in all the gas and particle assessments, showing the good performance of the devices and trusting in the results when only one device could be compared with a reference station. This very low intra-model variability allows using a “Gold” device for calibration, to rely on differential measurements and to compare sensor systems in different locations.
  • Kunak PM sensor is the only sensor that can measure PM5 and PM10 accurately. Even though some sensors obtain slightly higher PM2.5 R2 values, they are not able to measure PM10. Kunak AIR A10 provides a precise PM2.5 and PM10 measures with only one PM sensor.

AQSPEC evaluation programme is doing a great effort to compare low-cost air monitoring sensors to establish performance standards, being the most complete external evaluation programme. Nevertheless, some improvements could be carried out for a more objective evaluation among the different sensors. Even though it is difficult to carry out, to get more objective results, all the sensors should be measure at least in the same season, such as winter, thus, high-temperature effects and humidity transients would be avoided. As it was mentioned before, due to the R2 limitations, another metric should be considered, i.e. RMSE, MAE, or the expanded uncertainty, as considered in the future European Technical Specification, “Performance evaluation of air quality sensors”.

To summarize, despite the environmental conditions provoking an effect on the sensors, such as high temperatures, humidity transients, which not only occur to Kunak AIR A10 sensor system, but they are common to all the low-cost sensors, Kunak AIR A10 is the best option as a multiparametric air quality monitoring system, being able to measure accurately different gases (O3, CO, NO2 and NO), particulate matter (PM1, PM2.5 and PM10) and environmental data (temperature and humidity).

Regarding Kunak Air A10, it uses onboard sensors and Kunak embedded algorithm that calculates the concentrations in ppb or μg/m3 locally and in real-time, not using or needing external reference data to calculate gas or particle concentrations. Kunak algorithm corrects the interferences and artifacts typically found in raw data compensating environmental effects like temperature, humidity and pressure as well as cross-sensitivities using onboard data.

For most of the electrochemical gas sensors, humidity and temperature effect have a no-linear response (Lewis et al. 2018; Pang et al. 2018; Wei et al. 2018). The raw data is corrected by the Kunak algorithm converting the no-linear response into a linear response. However, none of the sensors were calibrated before deploying. The only calibration carried out was the factory calibration.

Kunak Air A10 is the best option as a multiparametric air quality monitoring system, being able to measure accurately different gases (O3, CO, NO2 and NO), particulate matter (PM1, PM2.5 and PM10) and environmental data (temperature and humidity).

References

Karagulian, Federico et al. 2019. “Review of the Performance of Low-Cost Sensors for Air Quality Monitoring.” Atmosphere 10(9).

Lewis, Alastair C. et al. 2018. Wmo Low-Cost Sensors for the Measurement of Atmospheric Composition: Overview of Topic and Future Applications. http://www.wmo.int/pages/prog/arep/gaw/documents/Draft_low_cost_sensors.pdf.

Pang, Xiaobing, Marvin D. Shaw, Stefan Gillot, and Alastair C. Lewis. 2018. “The Impacts of Water Vapour and Co-Pollutants on the Performance of Electrochemical Gas Sensors Used for Air Quality Monitoring.” Sensors and Actuators, B: Chemical 266: 674–84. https://doi.org/10.1016/j.snb.2018.03.144.

Wei, Peng et al. 2018. “Impact Analysis of Temperature and Humidity Conditions on Electrochemical Sensor Response in Ambient Air Quality Monitoring.” Sensors (Switzerland) 18(2).

Access the full report

Access the full report

Access the full report