Cognos Analytics

IBM Cognos Analytics Forecasting: Comparison of R4 and R5

By MARCO MAAS posted Fri January 24, 2020 05:41 AM

  

Differences between forecast results available in the IBM Cognos Analytics R4 and R5 can be encountered due to the forecasting algorithms enhancements in R5. As a result, long seasonal period in a seasonal model is often replaced by a shorter period, or it is omitted in favor of a non-seasonal model.

The following examples provide an illustration of the changes that have been introduced in Cognos Analytics R5. Some discussion is provided to help elucidate relevance of the seasonal model component.

Go Sales Data

 

Compared are differences for five time series from Sample Go Sales data. Data that is found in the Cognos Analytics file Team content>Samples>Data>Source files>SampleFile_GOSales.xls is used to generate line chart in a dashboard. Columns Year and Quarter are specified in the x-axis slot, Revenue is specified in the y-axis slot, and Product line is moved into the Color slot. When Forecasting is activated in Cognos Analytics R4 with the last time point ignored, it produces the following results.

blog1.png

 

The forecasts are clearly different in Cognos Analytics R5 for the same data as displayed below.

blog2.png

The time series in the charts above correspond to different categories of the Product line. We analyze and discuss each series separately starting with the series Personal Accessories having the highest values, and proceeding with the series displayed below it. 

Each analysis starts with display of an autocorrelation plot. The autocorrelation computes the correlation of a series with lagged versions of itself. If there is high correlation at a specific lag this is evidence of possible seasonal effects. For data that has a trend we difference it in order to remove the trend. Another tool that we use for seasonality detection is the seasonal plot. It displays a line for each seasonal period found in the data.

We compare Cognos Analytics results with Expert modeler feature for automated selection of exponential smoothing models available in the IBM SPSS Statistics. Results from corresponding ETS function in R are not discussed below because a simple exponential smoothing model with no trend and no seasonality is selected for all example series.

Capabilities for three products above are comparable when seasonality period is specified by the user. Each one selects a seasonal exponential smoothing model with specified seasonal period or an appropriate non-seasonal model. Notice that the user is not required to specify seasonal period in Cognos Analytics. Auto setting will select both appropriate seasonal period and a corresponding exponential smoothing model.

Modeling of the considered time series is limited due to small number of observed values. Each series contains only up to 14 time points, one per each quarter over the span of two and a half years.Personal Accessories

 

CA Forecasting Model

 

Trend

Seasonality

Period

R4

Additive

Multiplicative

6

R5

None

None

 

 

Autocorrelation function of the 1-differenced series on the left and a seasonal plot with period 6 on the right indicate no seasonal pattern. Values are higher towards end of the period during the first 2 cycles, but they become much larger at the beginning of the period in the last cycle.

blog3.png blog4.png



Cognos Analytics R5 selects a simple exponential smoothing model with no trend and no seasonality. Model fit and residual autocorrelation are acceptable for the simple model displayed below.

blog5.png blog6.png

 

Expert modeler in SPSS Statistics selects a more complex model that includes both trend and seasonality with period 4. It fits historic data better than the simple model, but its seasonal pattern does not match the data very well. Seasonal plot for period 4 shows no pattern except for a large increase in Q1 2017. There is no evidence in favor of the more complex seasonal model and the corresponding forecast cannot be trusted.

blog7.png blog8.png

Camping Equipment

 

CA Forecasting Model

 

Trend

Seasonality

Period

R4

Additive

Multiplicative

5

R5

Additive

None

 

 

Autocorrelation function of the 1-differenced series on the left and a seasonal plot with period 5 on the right do not indicate a relevant seasonal pattern. Cognos Analytics R4 uses a very weak and unreliable seasonal pattern. 

blog9.png blog10.png


Both Cognos Analytics R5 and Expert modeler in SPSS Statistics select the same model with additive trend and no seasonality displayed below.

blog11.png blog12.png

Golf Equipment

 

CA Forecasting Model

 

Trend

Seasonality

Period

R4

Additive

Multiplicative

4

R5

None

None

 

 

Autocorrelation function of the 1-differenced series on the left and a seasonal plot with period 4 on the right do not indicate a seasonal pattern. Cognos Analytics R4 picks up a very weak pattern mostly due to increase in the first quarter of 2016 and 2017.

blog13.png blog14.png

Cognos Analytics R5 selects a simple model with no trend and no seasonality.

blog15.png blog16.png

 

Expert modeler in SPSS Statistics selects a more complex model that includes both trend and seasonality with period 4. It corresponds to the model selected in Cognos Analytics R4. Nevertheless, fitted seasonal pattern does not match the data and residual autocorrelation displays a pattern. This model cannot be recommended for forecasting.

blog17.png blog18.png

Mountaineering Equipment

 

CA Forecasting Model

 

Trend

Seasonality

Period

R4

Additive

Multiplicative

6

R5

Additive

Multiplicative

4

 

Autocorrelation function of the 1-differenced series on the left and a seasonal plot with period 4 on the right indicate a weak seasonal pattern, mostly due to the Revenue increase in the first quarter. Cognos Analytics R4 picks up non-existent signal at 6 lags for series containing 10 observations only. This is due to 4 initial missing values extrapolated from the observed data. Such missing values are simply ignored in Cognos Analytics R5.

blog19.png blog20.png

 

Both Cognos Analytics R5 and Expert modeler in SPSS Statistics select the same model type with additive trend and seasonality below. The seasonality with period 4 seems to provide good fit to observed data.

blog21.png blog22.png

Outdoor Protection

 

CA Forecasting Model

 

Trend

Seasonality

Period

R4

Additive

Multiplicative

4

R5

Additive

None

 

 

Autocorrelation function of the 1-differenced series on the left and the seasonal plot with period 4 on the right indicate relatively weak signal at 4 lags, due to increase of Revenue in the first quarter. Cognos Analytics R4 adopts the seasonal contribution, but it is ignored in Cognos Analytics R5.

blog23.png blog24.png

 

Trend only model in Cognos Analytics R5 provides a reasonable fit to the data and there are no issues with residuals. However, negative forecast values for Revenue series are not expected, and this model provides no practical value.

blog25.png blog26.png

Both Cognos Analytics R4 and Expert modeler in SPSS Statistics select the same model type with additive trend and seasonality displayed below. The seasonality seems to provide a plausible fit to observed data. Forecast is also more realistic than for the model above because it remains in the positive range.

blog27.png blog28.png

Accidents data

Accidents data contains count of traffic accidents reported daily from January 1, 2019 to February 28, 2019. This data provides a time series with 59 observation affording a more thorough investigation compared to very short Go Sales time series.

Cognos Analytics R4 forecasting algorithm selects a model with large seasonal period 28, while a simple model with no seasonality and no trend is selected in Cognos Analytics R5 instead.

Before (Cognos Analytics R4)

blog29.png

After (Cognos Analytics R5)

blog30.png 

Seasonal period: 28 days

 

Autocorrelation function plot shows that there is a weak seasonal period possible at 28 days. Seasonal plot reveals that two full 28 days periods are somewhat comparable, but they do not present a clear pattern.

blog31.png blog32.png

 

Seasonal model with period 28 fits the observed data better than the alternative simple model. However, fitting a seasonal model with additional 28 parameters to the data containing only 59 points greatly increases its chance of overfitting the observed data.  Relatively large number of parameters penalizes the value of Akaike information criterion, and it makes seasonal model a weaker forecasting candidate than the simple model. 

The following table compares 1-step ahead forecast errors for the last 3 observed points using only earlier data for modeling in each instance. There is no advantage in using a seasonal model over a simple model based on (absolute) differences of observed and forecast values.

 

Forecast Error Comparison

Model/Date

Feb 26

Feb 27

Feb 28

Seasonal(28)

11.78

4.23

6.73

Simple

11.42

4.58

4.42

 

Notice that earlier points cannot be forecasted using a seasonal model with period 28 days because modeling seasonal data minimally requires observing two seasonal periods.

 

Seasonal period: 7 days

 

The autocorrelation function plot above shows that possible weekly pattern is detected. Weekly pattern is plausible for Accidents data and we display a seasonal plot and the corresponding box plot for period 7 to further assess its relevance. It turns out that there is a visible pattern where days 5 (Thursday) and 6 (Friday) tend to have higher values, while days 7 (Saturday) and 1 (Sunday) usually have lower number of accidents.

blog33.png blog34.png

 

The following charts displays observed Accidents data and fitted values based on a seasonal model with period 7 obtained using forecasting algorithm in Cognos Analytics R5.

blog35.png 

Fitted seasonal patterns strongly matches data during the first two weeks, but it becomes considerably weaker towards the end of the series due to the changing weekly pattern of the data.

Again, improved model fit achieved by the seasonal model is not sufficient to overcome the parameter penalty due to estimating 7 additional parameters in the seasonal model. The following table shows it has no advantage in forecasting the last three observed data points.

 

Forecast Error Comparison

Model/Date

Feb 26

Feb 27

Feb 28

Seasonal(7)

13.89

6.52

4.18

Simple

11.42

4.58

4.42

 

When comparison is extended to earlier points, the seasonal model reduces forecast error by 10% on average compared to the simple model. This indicates overall accuracy advantage for the seasonal model, but the simple model still provides robust forecast for Accident data.

Expert modeler in SPSS Statistics prefers a seasonal exponential smoothing model with period 7 to a simple exponential smoothing model due to a stronger seasonal fit for the observed data. Notice that its seasonal pattern for fitted values does not change over time.

blog36.png blog37.png

 

Still, forecast error for seasonal model in SPSS Statistics is not improved when compared to the seasonal model generated in Cognos Analytics. This points to risks of forecasting using a weak seasonal model in practice. Seasonal indices can easily change during future forecast periods.


#LearnCognosAnalytics
#home
#What'sHot
#News-BA
#News-BA-home
0 comments
4434 views

Permalink