Time Series Forecasting– Part I

*Time Series Forecasting– Part IWhat is a Time Series ?Components of Time SeriesEvaluation Methods of ForecastSmoothing Methods of Time Seriesby Duong Tuan AnhFaculty of Computer Science and EngineeringSeptember 2011*What is a Time series ?A time series is a collection of observations made sequentially in time. 05010015020025030035040045050023242526272829Examples: Financial time series, scientific time series*Time series modelsRegression modelsPredict the response over time of the variable under study to changes in one or more of the explanatory variables.Deterministic models of time seriesStochastic models of time seriesAll the three kinds of models can be used for forecasting.*Components of a time seriesThe pattern or behavior of the data in a time series has several components.Theoretically, any time series can be decomposed into:TrendCyclicalSeasonalIrregularHowever, this decomposition is often not straight-forward because these factors interact.*Trend componentThe trend component accounts for the gradual shifting of the time series to relatively higher or lower values over a long period of time.Trend is usually the result of long-term factors such as changes in the population, demographics, technology, or consumer preferences.*Seasonal componentThe seasonal component accounts for regular patterns of variability within certain time periods, such as a year.The variability does not always correspond with the seasons of the year (i.e. winter, spring, summer, fall).There can be, for example, within-week or within-day “seasonal” behavior.*Cyclical component Any regular pattern of sequences of values above and below the trend line lasting more than one year can be attributed to the cyclical component.Usually, this component is due to multiyear cyclical movements in the economy.*Evaluating Methods of forecastsForecasting method is selected - many times by intuition, previous experience, or computer resource availabilityDivide the data into two sections - an initialization part and a test partUse the forecast technique to determine the fitted values for the initialization data setUse the forecast technique to forecast the test data set and determine the forecast errorsEvaluate errors (MAD, MPE, MSD, MAPE)Use the technique, modify, or develop new model*Evaluation Methods of ForecastsThere are three measures of accuracy of the fitted models: MAPE, MAD and MSD for each of the sample forecasting and smoothing methods.For all three measures, the smaller the value, the better the fit of the model. Use these statistics to compare the fit of the different methods.MAPE (Mean Absolute Percentage Error) measure the accuracy of fitted time series values. It expresses accuracy as a percentage. |(yt-yt’)/yt| MAPE = --------------  100 (yt  0) n*MAPE, MAD, and MSDwhere yt is the actual value, yt’ is the fitted value and n is the number of observations. MAD (Mean Absolute Deviation) expresses accuracy in the same units as the data, which help conceptualize the amount of error. |yt-yt’| MAD = ---------- nwhere yt is the actual value, yt’ is the fitted value and n is the number of observations. *MAPE, MAD, and MSDMSD(Mean Squared Deviation) is a more sensitive measure of an unusually large forecast error than MAD. (yt-yt’)2 MSD = ---------- nwhere yt is the actual value, yt’ is the fitted value and n is the number of observations. *Methods of smoothing time series Arithmetic Moving AverageExponential Smoothing MethodsHolt-Winters method for Exponential SmoothingSmoothing a time series: to eliminate some of short-term fluctuations.Smoothing also can be done to remove seasonal fluctuations, i.e., to deseasonalize a time series.These models are deterministic in that no reference is made to the sources or nature of the underlying randomness in the series.The models involves extrapolation techniques.*Averaging MethodsSimple Averages - quick, inexpensive (should only be used on stationary data)Moving Average method consists of computing an average of the most recent n data values for the series and using this average for forecasting the value of the time series for the next period.Moving averages are useful if one can assume item to be forecast will stay steady over time.Series of arithmetic means – used only for smoothing, provides overall impression of data over time  (most recent n data items) Moving Average = ------------------------------------------ n*Moving average methodsWorks best with stationary data. The smaller the number, the more weight given to recent periods.A smaller number is desirable when there are sudden shifts in the level of the series.The greater the number, less weight is given to more recent periods.The larger the order of the moving average, the greater the smoothing effect. Larger n when there are wide, infrequent fluctuations in the data. By smoothing recent actual values, removes randomness.*Weighted Moving AveragesWeighted Moving Average - place more weight on recent observations. Sum of the weights needs to equal 1. Used when trend is presentOlder data usually less important (weight for period n)(Value in period n)WMA = -------------------------------------------------------- weights*Notes on Moving AveragesMA models do not provide information about forecast confidence. We can not calculate standard errors.We can not explain the stochastic component of the time series. This stochastic component creates the error in our forecast.*Exponential Smoothing MethodsSingle Exponential Smoothing (Averaging)Double Exponential Smoothing & Holt’s MethodWinter’s Model.Note: - Single Exponential Smoothing is for series without trend and without seasonal component. - Double Exponential Smoothing is for series with trend and without seasonal component. - Winter’s model is for for series with trend and seasonal component.*Single Exponential Smoothing Continually revising a forecast in light of more recent experiences. Averaging (smoothing) past values of a series in a decreasing (exponential) manner. The observations are weighted with more weight being given to the more recent observations At = αYt-1 + (1 – α) At-1 (S1) New forecast = α  (old observation) + (1- α)  old forecast Here we denote the original series by yt and the smoothed series by At. The equation can be rewritten as: At = At-1 + α(Yt –At-1)*Single Exponential Smoothing When looking at the formula – new forecast is really the old forecast plus a times the error in the old forecastTo get started, we need a smoothing constant a, an initial forecast, and an actual value. We can use the first actual as the forecast value or we can average the first n observations. The smoothing constant serves as the weighting factor. When a is close to 1, the new forecast will include a substantial adjustment for any error that occurred in the preceding forecast. When a is close to 0, the new forecast is very similar to the old forecast.*Single Exponential Smoothing (cont.)The smoothing constant α is not an arbitrary choice - but generally falls between 0.1 and 0.5. If we want predictions to be stable and random variation smoothed, use a small a. If we want a rapid response, a larger a value is required.*Why Exponential?At = Yt-1 + (1- )At-1 At-1 = Yt-2 + (1- )At-2 At-2 = Yt-3 + (1- )At-3 At = Yt-1 + (1- ) Yt-2 + (1- ) 2Yt-3 + . + (1 - ) kYt-k+1k decreases exponentially.*The small a here smooths the data.*The large a in this example responds quickly to the data.*TrackingUse a tracking signal (measure of errors over time) and setting limits. For example, if we forecast 10 periods, count the number of negative and positive errors. If the number of positive errors is substantially less or greater than n/2, then the process is out of control. Can also use 95% prediction interval (1.96 * sqrt (MSE)). If the forecast error is outside of the interval, use a new optimal a.Looking back at the .1 single exponential smoothing: 1.96*sqrt(24261) = +-305 Observation #21 is out-of-control. We need to re-evaluate alpha level because this technique is biased. *Exponential Smoothing Adjusted for Trend: Holt’s methodIn some situations, the observed data are trending and contain information that allows the anticipation of future upward movement. In that case, a linear trend forecast function is needed.Holt’s smoothing method allows for evolving local linear trend in a time series and can be used to forecast. When there is a trend, an estimate of the current slope and the current level is required.*Holt’s MethodHolt’s method uses two coefficients. a is the smoothing constant for the levelb is the trend smoothing constant - used to remove random error.Advantage of Holt’s method: it provides flexibility in selecting the rates at which the level and trend are tracked.*Equations in Holt’s methodThe exponentially smoothed series, or the current level estimate: At = Yt + (1- )(At-1 + Tt-1) (S2)The trend estimate: Tt = (At – At-1)+(1- )Tt-1 (S3)Forecast p periods into the future: Y’t+p = At + pTtwhere At = new smoothed value (estimate of current level) Yt = new actual value at time t. Tt = trend estimate Y’t+p = forecast for p periods into the future.  = smoothing constant for the level  = smoothing constant for trend estimateHow to initiate Holt’s methodTo get started, initial values for A and T in equation (S2) and (S3) must be determined. One approach is to set A1 to Y1 and T1 to zero.The second approach is to use the average of the first five or six observations as A1. T1 is then estimated by the slope of a line that is fit to these five or six observations.Holt’s methodHolt exponential smoothing with parameters  = 1.0 and  = 0.099 for time series of electricity consumption.*Winter’s MethodWinters’ method is an easy way to account for seasonality when data have a seasonal pattern.It extends Holt’s Method to include an estimate for seasonality.a is the smoothing constant for the levelb is the trend smoothing constant - used to remove random error.g smoothing constant for seasonality This formula removes seasonal effects. The forecast is modified by multiplying by a seasonal index. *Winter’s MethodThe four equations used in Winters’ (multiplication) smoothing are:The smoothed series or level estimate: At = Yt /St-s+ (1- )(At-1 + Tt-1)The trend estimate: Tt = (At – At-1)+(1- ) Tt-1The seasonality estimate: St = Yt/At + (1- )St-sForecast p periods into the future: Y’t+p = (At + pTt)St-s+pwhere At = new smoothed value (estimate of current level) Yt = new actual value at time t. Tt = trend estimate Y’t+p = forecast for p periods into the future. Tt = trend estimate  = smoothing constant for the level  = smoothing constant for trend estimate  = smoothing constant for seasonality estimate p = periods to be forecast into the future s = length of seasonalityWINTERS’ METHODIs also called TRIPLE EXPONENTIAL SMOOTHING )How to initiate Winter’s methodTo begin the Winter’s method, the initial values for the smoothed series At, the trend Tt and the seasonal indices St must be set.One approach is to set the first estimate of At to Y1. The trend is estimated to 0 and the seasonal indices are each set to 1.0. *Winter’s Method