SentemQC - A novel and cost-efficient method for quality assurance and quality control of high-resolution frequency sensor data in fresh waters

作者：Sofie Gyritia Weitzmann van't Veen, Brian Kronvang, Joachim Audet, Thomas A. Davidson, Erik Jeppesen, Esben Astrup Kristensen, S. Larsen, Jane R. Laugesen, Eti Ester Levi, Anders Nielsen, Peter Mejlhede Andersen · 发表于：Open Research Europe · 年份：2024 · DOI：10.12688/openreseurope.18134.1 · 被引用次数：3 · 研究领域：Fish Ecology and Management Studies、Hydrological Forecasting Using AI、Water Quality Monitoring Technologies

The growing use of sensors in fresh waters for water quality measurements generates an increasingly large amount of data that requires quality assurance (QA)/quality control (QC) before the results can be exploited. Such a process is often resource-intensive and may not be consistent across users and sensors. SentemQC (QC of high temporal resolution sensor data) is a cost-efficient, and open-source Python approach developed to ensure the quality of sensor data by performing data QC on large volumes of high-frequency (HF) sensor data. The SentemQC method is computationally efficient and features a six-step user-friendly setup for anomaly detection. The method marks anomalies in data using five moving windows. These windows connect each data point to neighboring points, including those further away in the moving window. As a result, the method can mark not only individual outliers but also clusters of anomalies. Our analysis shows that the method is robust for detecting anomalies in HF sensor data from multiple water quality sensors measuring nitrate, turbidity, oxygen, and pH. The sensors were installed in three different freshwater ecosystems (two streams and one lake) and experimental lake mesocosms. The explored sensor data revealed anomaly percentages ranging from 0.1% to 0.8% of the cleaned datasets by SentemQC. While the sensors in this study contained relatively few anomalies (<2%), they may represent a best-case scenario in terms of use and maintenance. SentemQC allows...