The creation of a dynamic platform based on Advanced Metering Infrastructure models requires an intelligent meter data processing.
To address the issue of errors in the meter data, we follow the framework of Extreme Values Analysis to detect and remove outliers [5, 6, 7]. Specifically, the method of the mean plus or minus 2.5 standard deviations has been deployed.
Figure 2: Detecting and removing outliers in time-series of temperature values.
This method is based on the characteristics of a normal distribution for which 99% of the data appear within this range (see Figure 1). Therefore, the decision consists in removing the values that occur only in 1% of all cases. This constitutes the first step in our automated meter data cleaning.
The second step in our data cleaning process addresses the problem of missing data, either due to missing readings or outlier removals. To ensure that the meter data are reliable, these gaps must be filled. The estimation of the missing gaps is also known in the literature as imputation.