a question about the smoothness of the trend

[hardmann]

Dear Tom:

I have a question about the decomposition. I decomposed the log of monthly electricity consumption into trend, cyclical, and seasonal components. The trend component is very smooth. After adding some intervention variables, the log-likelihood improved, and both AIC and BIC (or SBC) decreased, indicating a better fit. However, the estimated trend has become less smooth.

I believe this represents an improvement in model fit, but I’m unsure how to interpret or explain the reduced smoothness of the trend. Could you help clarify this?

Attached is data and program.

Best Regard
Hardmann

[TomDoan]

Congratulations on getting a components model to work in the first place with fully estimated variances—that doesn't happen very often. (Restricting the trend to only have a shock to the second difference is pretty standard). What exactly are your shift dummies? I can see one is a leap year, so I assume your data are monthly totals. Switching to average daily consumption might make more sense than trying to "estimate" a leap year shift. There is another one that shifts consumption between January and February and another which shifts consumption among January, February and March. What are those doing? It looks like some of the seasonal is leaking over into the trend.

Have you tried using the harmonic seasonal instead of the additive one? The harmonic tends to work better when you have something that is more weather-related than driven by the calendar.

[hardmann]

Dear Tom:

Our dummy variable is the Spring Festival, a lunar festival that typically moves between January and February. We constructed it as a three-stage model, namely before, during, and after the Spring Festival, with each stage simply lasting for 7 days. Before the Spring Festival, it increases linearly from 0 to 1; after the Spring Festival, it decreases linearly from 1 to 0; and during the Spring Festival, it remains unchanged. Then, normalization is performed to obtain three sets of dummy variables. If a single-stage model is used, only the period during the Spring Festival is needed. The Spring Festival in the lunar calendar is similar to Easter in Europe and America.

My core question is whether the estimated trend should be smoother if the model estimation improves. However, the current results seem to indicate the opposite.

Best regard
Hardmann

[TomDoan]

No. In fact, it is just the opposite (at least in practice). The likelihood maximizer usually has a very volatile trend. Most UCM's need some type of restriction on how variable the trend can be to get "reasonable" results.

[hardmann]

Dear Tom:


I have identified the cause of the problem. I should have set the intervention variable to the 'mu' option instead of the 'z' option. The intervention variable should act on the observed variable rather than the state variable.

Best regard
Hardmann

[TomDoan]

That makes sense. I was a bit puzzled though on how the timing of the holiday would be expected to change the demand for electricity.

[hardmann]

Dear Tom:

Power generation should exhibit varying changes during the three stages of the mobile festival: before, during, and after. Electricity consumption can be broadly categorized into industrial and residential consumption. Industrial consumption is further divided into continuous and discontinuous usage.
For discontinuous industrial electricity usage, there may be moderate overtime work before the festival, a complete shutdown during the festival, and a gradual return to normal levels afterward.
For continuous industrial electricity usage, the intensity may also decrease moderately during the festival.
As for residential electricity consumption, its impact is unlikely to vary significantly across the three stages.
Electricity consumption patterns are a mix of different behaviors; therefore, power generation should match overall electricity demand.
I should collect data to test these assumptions.

Best regard
Hardmann

[hardmann]

Dear Tom：

If there were no observation errors, there would seem to be no difference between assigning the intervention variable to the mu and z options. However, in reality, there is a significant difference in the estimates. If the intervention effect is in the z option, then the intervention effect is already included in the trend and is difficult to separate. For the original monthly time series, which has not been removed for seasonal and calendar components, we can decompose it into four components: trend, cycle, season, and intervention. If we use this method to filter out the seasonal and intervention components, it seems that the mu option is more appropriate. However, the estimate when the intervention variable is placed in the z option seems to be easier.

Please provide me with more detailed guidance and analysis.

Best regard
Hardmann

[TomDoan]

If you put it in the Z (as a addition to the trend state), then it will accumulate in the state (it will be an innovational shift) while you seem to be describing it as a additive shift.