Sample-based estimation of mean electricity consumption curves for small domains
Section 5. Application to electricity consumption curves

• Previous
• Next

We will now test the methods that we have just presented to compare their performance on electricity consumption data for French residential clients.

5.1  Presentation of the data set

We worked with a data set belonging to EDF that contains electricity consumption curves for $N\mathrm{<mo>=</mo>}\text{1,905}$ French residential clients by daily interval from October 2011 to March 2012, without any missing values $(L\mathrm{<mo>=</mo>\; 177}$ points). This population is subdivided into $D\mathrm{<mo>=</mo>\; 8}$ domains corresponding to geographic areas with respective sizes of 573, 195, 304, 121, 228, 219, 45 and 220. For confidentiality purposes, we cannot describe the data set in great detail, or show the mean curves by domain.

By way of illustration, Figure 5.1 shows the appearance of the standardized curves (i.e., each curve is divided by its mean calculated over the period of time studied) for five random individuals, and Figure 5.2 shows the appearance of the first five principal components of the functional PCA created for this data set.

We see that the first component, the overall appearance of which is similar to that of the mean curve, is a “level” effect. Components two and three, which present peaks during the coldest period in February, describe the sensitivity of consumption to outside temperatures. The fourth compares “mid-season” consumption to “wintertime” consumption and, finally, the fifth shows a low at about Christmas (and about February 14).

For each individual in our population of study, we have four auxiliary variables at the individual level: contract power (in three classes), rate option (base or off-peak periods) (in the base option, the price per kWh remains constant, while the rate for off-peak periods is reduced for eight hours [referred to as off-peak]. The largest consumers tend to prefer that rate. Off-peak periods can vary from one client to another, but this factor has no impact here, as we are working on a daily interval), the previous year’s annual consumption, and the type of dwelling (apartment or single home). These auxiliary variables remain the same for all methods used in order to compare identical auxiliary information. All tests were implemented in R.

5.2  Test protocol

We compare various estimators obtained using the methods set out in this chapter, for various types of modelling (unit-level linear mixed models, linear functional regressions, regression trees, random forests). We test two versions of the unit-level linear mixed model, one by placing linear mixed models on the PCA scores, as suggested in Section 4.2, and the other by applying them directly to the values of the curves of instants of discretization.

For non-parametric methods, the forests and trees have a depth (number of levels) of 5 and a minimum size of 5 leaves. There are 40 trees in the forests. The algorithms can be applied by separating the estimation of the level of the curve and its form (standardization = “yes”) or not separating (standardization = “no”). To not multiply the possible combinations, we finally focused on the estimators listed in Table 5.1. The parameters of the regression tree and random forest models are set out in Table 5.2.

To evaluate the quality of our estimation methods, our test protocol consists of conducting a large number $E$ of sampling simulations from our original population and then estimating the mean curve for each $D\mathrm{<mo>=</mo>\; 8}$ domain based on each sample gathered by the various proposed methods. In our simulations, the eighth domain $\left(d\mathrm{<mo>=</mo>}D\mathrm{<mo>=</mo>\; 8}\right)$ will always be unsampled in order to measure the performance of our various estimators in this scenario. For each simulation, we select $n\mathrm{<mo>=</mo>\; 200}$ individuals by simple random sampling from among those in the seven sampled domains $\left(d\mathrm{<mo>=</mo>\; 1,}\dots \mathrm{,}7\right).$

Let ${\mu }_d\left(t_l\right)$ the mean curve for the domain $d$ at the instant $t_l$ and ${\widehat{\mu }}_d\left(t_l\right)$ its estimator by a given method. We calculate the relative bias of ${\widehat{\mu }}_d\left(t_l\right):$

$$\text{RB}\left({\widehat{\mu }}_d\left(t_l\right)\right)\mathrm{<mo>=</mo>\; 100}\frac{E_{\text{MC}}\left[{\widehat{\mu }}_d\left(t_l\right)\right]-{\mu }_d\left(t_l\right)}{{\mu }_d\left(t_l\right)}\mathrm{,}d\mathrm{<mo>=</mo>\; 1,}\dots \mathrm{,}D\mathrm{,}l\mathrm{<mo>=</mo>\; 1,}\dots \mathrm{,}L\mathrm{,}(5.1)$$

where $E_{\text{MC}}\left[{\widehat{\mu }}_d\left(t_l\right)\right]\mathrm{<mo>=</mo>}{\displaystyle {\sum }_{e\mathrm{<mo>=</mo>\; 1}}^E}{\widehat{\mu }}_d^{\left(e\right)}\left(t_l\right)/E$ is the Monte Carlo expectation of the estimator ${\widehat{\mu }}_d\left(t_l\right)$ with ${\widehat{\mu }}_d^{\left(e\right)}\left(t_l\right)$ the estimator of the mean curve obtained for the $e^{\text{th}}$ simulation, for $e\mathrm{<mo>=</mo>\; 1,}\dots \mathrm{,}E.$ A second indicator, known as relative efficiency (RE), is calculated as follows:

$$\text{RE}\left({\widehat{\mu }}_d\right)\left(t_l\right)\mathrm{<mo>=</mo>\; 100}\frac{{\text{MSE}}_{\text{MC}}\left({\widehat{\mu }}_d\right)\left(t_l\right)}{{\text{MSE}}_{\text{MC}}\left({\widehat{\mu }}_d^{\text{HT}}\right)\left(t_l\right)}\mathrm{,}d\mathrm{<mo>=</mo>\; 1,}\dots \mathrm{,}D-\mathrm{1,}l\mathrm{<mo>=</mo>\; 1,}\dots \mathrm{,}L\mathrm{.}(5.2)$$

where ${\text{MSE}}_{\text{MC}}\left({\widehat{\mu }}_d\left(t_l\right)\right)\mathrm{<mo>=</mo>}{\displaystyle {\sum }_{e\mathrm{<mo>=</mo>}1}^E}{\left({\widehat{\mu }}_d^{\left(e\right)}\left(t_l\right)-{\mu }_d\left(t_l\right)\right)}^2/E$ is the Monte Carlo mean square error, $d\mathrm{<mo>=</mo>\; 1,}\dots \mathrm{,}D\mathrm{,}l\mathrm{<mo>=</mo>\; 1,}\dots \mathrm{,}L\mathrm{.}$ The lower the RE indicator, the more the estimator will be considered effective. An RE of 100 corresponds to an indicator as effective as the reference estimator.

Here, the reference estimator ${\widehat{\mu }}_d^{\text{HT}}$ is the Horvitz-Thompson estimator (which, for our simple random sampling plan, is the simple mean of the curves in the domain considered); it corresponds to the model described by equation (3.1). This estimator cannot be calculated for the unsampled domain. The RE estimator is then obtained by dividing the MSE of the various estimators by the mean MSE of the Horvitz-Thompson estimator over the seven sampled domains, i.e.

$$\text{RE}\left({\widehat{\mu }}_D\right)\left(t_l\right)\mathrm{<mo>=</mo>\; 100}\frac{{\text{MSE}}_{\text{MC}}\left({\widehat{\mu }}_D\right)\left(t_l\right)}{{\overline{\text{MSE}}}_{\text{MC}}^{\text{HT}}\left(t_l\right)}\mathrm{,}l\mathrm{<mo>=</mo>\; 1,}\dots \mathrm{,}L\mathrm{,}(5.3)$$

with ${\overline{\text{MSE}}}_{\text{MC}}^{\text{HT}}\left(t_l\right)\mathrm{<mo>=</mo>}{\displaystyle {\sum }_{d\mathrm{<mo>=</mo>\; 1}}^{D-1}}{\text{MSE}}_{\text{MC}}\left({\widehat{\mu }}_d^{\text{HT}}\right)\left(t_l\right)\mathrm{,}l\mathrm{<mo>=</mo>\; 1,}\dots \mathrm{,}L\mathrm{.}$

For each indicator and each instant $t_l,$ the results obtained for the various sampled domains are then aggregated for all domains, ${\text{RB}}_{\text{ech}}\left(\widehat{\mu }\right)\left(t_l\right)\mathrm{<mo>=</mo>}\frac{1}{D-1}{\displaystyle {\sum }_{d\mathrm{<mo>=</mo>\; 1}}^{D-1}}\text{RB}\left({\widehat{\mu }}_d\right)\left(t_l\right)$ and ${\text{RE}}_{\text{ech}}\left(\widehat{\mu }\right)\left(t_l\right)\mathrm{<mo>=</mo>}\frac{1}{D-1}{\displaystyle {\sum }_{d\mathrm{<mo>=</mo>\; 1}}^{D-1}}\text{RE}\left({\widehat{\mu }}_d\right)\left(t_l\right)$ for $l\mathrm{<mo>=</mo>\; 1,}\dots \mathrm{,}L\mathrm{,}$ while the indicators obtained for the unsampled domain are used as-is.

Finally, to evaluate overall performance, we consider the mean of those indicators for all instants in the test period, while still separating the sampled domains from the unsampled domain. We also look at the calculation times of the various estimators.

5.3  Results and test conclusions

The test results of the methods are presented in Table 5.3 and illustrated in Figures 5.3 to 5.5.

For sampled domains, we see that the integration of explanatory variables in the estimate, regardless of the method used, leads to a net gain in performance: thus, for the least effective method (the estimator by calibration), the error is divided by three when explanatory variables are used.

As well, the use of our various estimators based on superpopulation models leads to an additional gain in accuracy: the RE for our various methods thus range from 15% for linear mixed models to 25% for random forests.

The linear mixed models are the most effective method, so we can assume that there are characteristics of the domains that are unexplainable using only the auxiliary variables that this type of model is able to capture. We therefore go from an RE of 25% for the linear functional regression to an RE of approximately 15% by including these random effects.

The tree and random forest methods capture non-linearities in the relationship between explanatory variables and the interest variable, which explains why these methods give better results than linear functional regressions: the RE of the various non-parametric methods are between 20% and 25%, compared to 25% for linear functional regressions. Very surprisingly, the regression tree gives better results than the random forest. We can put forth the theory that this is because our objective is to best estimate the mean curve of a series of units, not each curve individually. It is therefore possible that the tree is not as good for predicting each curve, but better at the aggregate level. As well, on this particular data set, the method gives the best results when working on raw curves, not when distinguishing between the estimation of form and level.

Projecting curves based on the PCA does not seem to lead to any significant gains in accuracy here.

The Horvitz-Thompson estimator cannot be produced on unsampled domains. The differences between the other methods are much more restricted than on the sampled domains: the random effects cannot be estimated for unsampled domains.

Finally, in Figure 5.5, we trace the mean square error of our estimators for the sampled and unsampled domains. We note that this square error is higher in the winter (January and February). This high variability could be due to a sharp drop in outside temperatures during those months, which increases the variability of heating consumption (difference in behaviour and electrical heating equipment depending on clients). The naive and calibration estimators adapt least well to this situation.

5.4  Comparison of methods and selection criteria

Each model-based method has benefits and drawbacks. Unit-level linear mixed models are the only ones that, due to random effects, make it possible for the modelling to include domain characteristics not reflected in auxiliary information. It thus seems relevant to use them when assuming that the explanatory variables do not make it possible to explain all differences between domains.

The linear functional regression ignores the random effect of the domains, so we expect it to be less effective than linear mixed models due to its construction. Finally, the two non-parametric methods allow for better modelling of the non-linear relationships between the explanatory variables and the interest variable, but on the other hand, does not make it possible to capture the differences between domains that are not reflected in the auxiliary information. They also require the availability of auxiliary information $X_i$ for each individual in the population when, in the past, we only needed mean values ${\overline{X}}_d$ for each domain in the population and $X_i$ for the sample. The choice between a parametric and non-parametric approach will therefore depend on the nature of the problem, the diversity of domains and the explanatory variables available. Be believe that neither of the two approaches is systematically preferable over the other.

A process for choosing between the two approaches could be to estimate the respective variances in the random effects and the residuals in the linear mixed models and, depending on the relative scope of those effects, moving more toward one or the other type of model. Conversely, cross-validation can be used to quantify the respective performance of the linear mixed models and the non-parametric models for predicting the aggregates of individual curves in order to direct our choice.

Among the non-parametric methods, the choice between regression trees and random forests will depend on the predictive performance of those methods on data, for the mean curves of domains. Generally, we can assume that random forests will give better results than regression trees for individual data (see Breiman et al., 1984); however, it is entirely possible that the best of the two methods for predicting each curve may not be the one that gives the best results to all domains or, at the very least, that the two methods are reduced when we consider the prediction of mean curves of individual aggregates. As well, due to their construction, random forests require a lot more calculation time than regression trees and that aspect cannot be ignored when the data sets being processed are large in size.

• Previous
• Article main page
• Next