Sina Bolouri
Back to Science articles

Introduction to Time Series

From Observations to a Modelable Time Series

Why stationarity matters, how transformations remove changing structure, and what differencing does to the information in a series.

Article 1Time-Series Analysisbeginner16-20 minutespartially-verified
time seriesforecastingdata sciencestationaritydifferencingseasonalitytrendstochastic processesr

Introduction: The Problem Before the Method

A time series is often introduced as a list of observations indexed by time: temperature by hour, revenue by quarter, vibration amplitude by second, or package volume by shift. That definition is accurate, but it hides the main analytical difficulty. The observations are not merely arranged in time; their order can contain dependence, persistence, trends, seasonal repetition, changing variability, and structural breaks.

The practical task is therefore not simply to fit a line through a sequence. It is to decide what kind of data-generating mechanism could plausibly have produced the observed path and whether that mechanism is stable enough to learn from.

This distinction separates time-series data from a time-series model. The data are one realized sequence, such as (x_1,\ldots,x_n). A model is a stochastic process ({X_t}): a collection of random variables whose joint behavior describes many sequences that could have occurred. Forecasting uses the observed realization to estimate a model and then extends the model's conditional distribution into the future (Brockwell & Davis, 2016; Shumway & Stoffer, 2025).

That move from one observed path to a reusable probabilistic model requires some form of stability. Stationarity provides one of the most important versions of that stability.

The Core Idea

A stationary process behaves according to time-invariant probabilistic rules. This does not mean that every observation is constant or that the series cannot move sharply. It means that the probabilistic properties used for modeling do not systematically depend on the calendar position.

For most introductory linear time-series methods, the relevant condition is weak stationarity:

  1. (E[X_t]=\mu) does not depend on (t).
  2. (\operatorname{Var}(X_t)=\gamma(0)<\infty) does not depend on (t).
  3. (\operatorname{Cov}(X_t,X_{t+h})=\gamma(h)) depends on the lag (h), not on the specific time (t).

A stronger condition, strict or strong stationarity, requires the entire joint distribution to remain unchanged after a common time shift. Weak stationarity is less demanding and is sufficient for much of classical ARMA analysis because those methods are built primarily from means, variances, and covariances (Brockwell & Davis, 2016).

Why the Concept Exists

Suppose a series has a steadily rising mean. An average calculated from the first half then describes a different operating regime from an average calculated from the second half. Likewise, if seasonal amplitude grows over time, one variance estimate cannot represent the whole series.

Without stability, repeated observations are not straightforward replications of the same underlying relationship. Model parameters can become averages over incompatible regimes rather than estimates of a persistent mechanism.

Stationarity makes learning from the past defensible because it permits the analyst to pool information across time. It supports questions such as:

  • How strongly does the present relate to one period ago?
  • Does a shock decay or remain permanently embedded?
  • Is the residual variance reasonably constant?
  • Can coefficients estimated from earlier observations remain relevant later?

Stationarity is therefore not a decorative assumption. It is the condition that makes many covariance-based summaries interpretable.

Background and Prerequisites

A useful conceptual decomposition writes an observed series as

Xt=mt+st+Yt,X_t = m_t + s_t + Y_t,

where:

  • (m_t) is a slowly changing level or trend,
  • (s_t) is a repeating seasonal component,
  • (Y_t) is a remaining component that may be modeled as stationary.

This additive representation is not universally correct, but it separates three different questions:

  1. Is the typical level changing?
  2. Does a pattern repeat at a known period?
  3. After those structures are removed, is there stable serial dependence left?

A multiplicative pattern is more appropriate when seasonal variation scales with the level. A common remedy is to transform first, for example with a logarithm, so that multiplicative effects become approximately additive.

Assumptions

The observation interval has meaning

A lag of one must represent a consistent interval: one minute, one day, one quarter, or one cycle. Irregular sampling changes the interpretation of lag-based dependence and often requires explicit handling.

The transformation is scientifically acceptable

Differencing changes the target from a level to a change. A logarithm changes additive differences into approximate relative differences. These operations are useful only when their transformed interpretation is meaningful.

The underlying regime is sufficiently stable

Differencing can remove deterministic or stochastic trend, but it does not repair every structural change. A machine replacement, policy change, sensor recalibration, or market shock may divide the series into genuinely different regimes.

Seasonal period is known or defensible

Seasonal differencing at lag (d) assumes that (d) corresponds to a real repeating cycle. Choosing (d=12) for monthly data is reasonable only when annual recurrence is plausible.

How It Works

The backshift operator (B) is defined by

BXt=Xt−1.BX_t=X_{t-1}.

Ordinary first differencing is

∇Xt=(1−B)Xt=Xt−Xt−1.\nabla X_t=(1-B)X_t=X_t-X_{t-1}.

The operation removes a constant linear trend. If

Xt=a+bt+Yt,X_t=a+bt+Y_t,

then

∇Xt=b+∇Yt.\nabla X_t=b+\nabla Y_t.

The level (a) disappears, and the time-varying term (bt) becomes the constant (b). The transformed series may therefore have a stable mean even when the original level rises or falls.

Seasonal differencing at period (d) is

∇dXt=(1−Bd)Xt=Xt−Xt−d.\nabla_d X_t=(1-B^d)X_t=X_t-X_{t-d}.

For monthly observations with annual seasonality, (d=12). Each month is compared with the same month in the previous year, so a stable annual seasonal effect cancels.

The transformations can be combined:

∇∇dXt=(1−B)(1−Bd)Xt.\nabla\nabla_d X_t=(1-B)(1-B^d)X_t.

This is useful when both a changing level and seasonal repetition remain.

The conceptual workflow is:

The important point is that transformation is iterative. One does not difference until the plot looks noisy and then declare success. Each operation should have a stated purpose and should be followed by a new inspection.

The Practical Procedure

1. Establish the time index

Confirm frequency, missing timestamps, duplicates, aggregation rules, and whether the interval is regular.

2. Plot the original series

Look for changing level, seasonal cycles, increasing amplitude, outliers, discontinuities, and long stretches with different behavior.

3. Separate changing variance from changing mean

A log or Box-Cox transformation addresses scale-dependent variability. Differencing primarily addresses changing level and persistent integration. They are not interchangeable.

4. Apply the smallest defensible transformation

Begin with one operation tied to one observed problem. For example, log-transform a positive series whose seasonal amplitude grows with its level.

5. Difference only when needed

Use first differencing for level persistence or trend-like behavior. Use seasonal differencing when repeated seasonal structure survives.

6. Replot and diagnose

Inspect the transformed series, its autocorrelation, and eventually its fitted residuals. Stationarity is a modeling judgment supported by evidence, not a visual label alone.

7. Preserve the inverse transformation

Forecasts must often be returned to the original scale. Store the last observed levels and seasonal values needed to undo differencing, and account for bias when back-transforming nonlinear transformations.

Mathematical or Technical Foundation

A random walk illustrates why differencing matters:

Xt=Xt−1+Wt,X_t=X_{t-1}+W_t,

where (W_t) is white noise with mean zero and variance (\sigma^2).

Repeated substitution gives

Xt=X0+∑j=1tWj.X_t=X_0+\sum_{j=1}^{t}W_j.

Therefore,

Var⁡(Xt)=Var⁡(X0)+tσ2,\operatorname{Var}(X_t)=\operatorname{Var}(X_0)+t\sigma^2,

which grows with time. The process is not weakly stationary. But differencing gives

∇Xt=Wt,\nabla X_t=W_t,

which is stationary under the white-noise assumptions.

This explains why a shock to a random walk is permanent in the level but temporary in the differenced representation. It also explains why a highly persistent stationary AR(1) process can visually resemble a random walk: finite samples may not make the distinction obvious. Formal unit-root testing is useful later, but even those tests do not replace domain judgment.

Worked Example

Source-derived example: Lake Huron levels

The uploaded material uses annual Lake Huron water-level data to show a series with an apparent changing level. First differencing replaces each annual level with the year-to-year change:

x <- LakeHuron
dx <- diff(x)

plot(x, main = "Lake Huron level")
plot(dx, main = "Annual change in Lake Huron level")

The transformed plot is more plausibly centered around a stable mean. The operation does not claim that the physical system has no long-term dynamics; it changes the modeling question from “What is the level?” to “How does the level change from one year to the next?”

Source-derived example: monthly temperature

For monthly temperature data with annual recurrence, lag-12 differencing compares each month with the same month one year earlier:

seasonal_change <- diff(nottem, lag = 12)
plot(seasonal_change)

A repeating seasonal mean can disappear while short-run fluctuations remain.

Original explanatory example: condition monitoring

Assume a vibration sensor records one RMS value per hour. The level rises gradually as a bearing degrades, while a daily production schedule creates a 24-hour cycle.

A reasonable exploratory sequence is:

  1. Verify that missing hours are handled.
  2. Plot the raw RMS series.
  3. Consider a log transform if variance rises with level.
  4. Apply lag-24 differencing to remove the daily operating cycle.
  5. Apply ordinary differencing only if a changing level remains.
  6. Fit a dependence model to the transformed series.
  7. Retain the original level as a separate health indicator because differencing can hide the absolute severity of degradation.

The last step matters: a transformation useful for forecasting changes may be unsuitable for alarm thresholds based on physical magnitude.

Interpreting the Results

A transformed series that appears stationary supports, but does not prove, the modeling assumption. Useful signs include:

  • a stable center,
  • no obvious deterministic trend,
  • approximately stable variance,
  • seasonal structure no longer dominating,
  • an autocorrelation pattern that decays rather than remaining near one for many lags.

The result should not be interpreted as evidence that the system is physically unchanged. Stationarity is relative to the representation, timescale, variables, and observation window.

Real-World Applications

In predictive maintenance, differencing can isolate changes in vibration or temperature from persistent baseline levels. In demand forecasting, seasonal differencing can separate year-over-year change from recurring calendar effects. In software monitoring, transforming request counts can stabilize scale before modeling residual dependence. In robotics, a stationary residual process can support anomaly detection after planned routes, shift schedules, or charging cycles have been modeled separately.

The same idea applies to portfolio telemetry: raw cumulative counts are often nonstationary, while rates, increments, or deviations from a seasonal baseline may be modelable.

Common Misunderstandings

“Stationary means constant”

A stationary process can fluctuate dramatically. The requirement concerns invariant distributions or moments, not a flat plot.

“Differencing always removes trend”

Differencing removes certain trend structures. It does not automatically handle nonlinear drift, structural breaks, changing seasonality, or evolving variance.

“A stationary-looking plot proves stationarity”

Plots are essential but limited. Short samples can make a unit-root process look mean-reverting or a persistent stationary process look nonstationary.

“More differencing is safer”

Over-differencing can create unnecessary negative autocorrelation, amplify noise, increase forecast uncertainty, and make the model harder to interpret.

“A log transform and differencing do the same thing”

A log transform primarily changes scale and converts multiplicative relationships toward additive ones. Differencing removes components that persist across adjacent or seasonal observations.

Limitations and Failure Modes

Differencing discards observations at the beginning of the series and changes the quantity being modeled. Seasonal differencing can be costly when data are short because each difference spans an entire period. Missing values can propagate through multiple differences.

Structural breaks require separate treatment. If a sensor was recalibrated, the difference at the change point becomes an artificial spike. If seasonality evolves, fixed seasonal differencing may leave residual seasonal dependence. If the data are bounded, count-valued, intermittent, or irregularly sampled, Gaussian linear methods may be a poor fit even after transformation.

Alternatives and Trade-Offs

A deterministic regression trend is preferable when the trend has a stable interpretable form and extrapolation is defensible. Seasonal indicators or Fourier terms can model recurring structure without seasonal differencing. State-space structural models allow level and trend to evolve explicitly. STL and related decomposition methods can handle changing seasonal patterns more flexibly. ARIMA is useful when differencing yields a process with linear autocorrelation structure.

The choice depends on whether trend and seasonality should be removed before modeling or represented as evolving components inside the model.

Connection to Broader Topics

Stationarity leads directly to autocovariance and autocorrelation, because those summaries assume relationships depend on lag rather than calendar position. Differencing leads to ARIMA, where the “I” denotes integration and the parameter (d) records the number of ordinary differences. Seasonal differencing extends the same logic to SARIMA.

State-space models provide another perspective: instead of forcing the observed level to be stationary, they let hidden level, trend, and seasonal states evolve while modeling the innovations.

Connection to Portfolio or Learning

This concept is particularly relevant to condition-based maintenance and KineticNode telemetry. A relevant portfolio application would be to maintain both:

  • an absolute health view, where physical levels and thresholds remain visible; and
  • a modeling view, where known schedules, trends, and seasonal patterns are removed before forecasting or anomaly detection.

For Monograph semantic retrieval, this article should precede articles on ACF/PACF, ARMA, ARIMA, and exponential smoothing.

Key Takeaways

  1. A time-series dataset is one observed path; a time-series model describes a probability law over possible paths.
  2. Weak stationarity stabilizes the mean, variance, and lag-based covariance structure needed by many classical models.
  3. A log or Box-Cox transformation addresses scale-dependent variability, while differencing addresses persistent changes in level or seasonality.
  4. Ordinary differencing models changes between adjacent observations; seasonal differencing models changes relative to the same position in a previous cycle.
  5. The smallest defensible transformation is preferable because over-differencing can add noise and obscure interpretation.
  6. Stationarity is a property of a chosen representation and time window, not proof that the underlying physical system never changes.
  7. Transformations must be reversible or otherwise accounted for when forecasts are returned to the original scale.

Review Questions

  1. Why does a changing mean make a single autocovariance function difficult to interpret?
  2. What practical difference exists between modeling (X_t) and modeling (\nabla X_t)?
  3. Under what pattern would a logarithm be more appropriate than an additional difference?
  4. How can over-differencing appear in an autocorrelation plot?
  5. Why might a predictive-maintenance system retain both raw levels and differenced features?
  6. What evidence would suggest a structural break rather than ordinary nonstationarity?

Further-Learning Path

Random variables and covariance
→ provide the probability language needed to define stochastic processes.

Stationarity and transformations
→ establish a stable representation for lag-based modeling.

Autocovariance, ACF, and PACF
→ describe how dependence changes with lag.

ARMA and ARIMA models
→ encode stationary dependence and differencing in a parametric model.

Residual diagnostics and forecast evaluation
→ test whether the transformed model has captured the available structure.

Suggested Related Monograph Articles

  1. Understanding Dependence in Time Series — continuation. Introduces autocovariance, autocorrelation, linear filters, and sample correlograms.
  2. ARMA Models as Dynamic Filters — continuation. Explains autoregressive and moving-average mechanisms, causality, and invertibility.
  3. Unit Roots and the Augmented Dickey-Fuller Test — deeper theory. Distinguishes stochastic trend from strong but stationary persistence.
  4. Seasonal Modeling: Indicators, Fourier Terms, STL, and SARIMA — comparison. Compares ways to represent recurring structure.
  5. Preprocessing Industrial Sensor Streams for Forecasting — practical application. Applies transformation choices to condition-monitoring data.

References

Author not identified. (n.d.). Introduction to time series [Course notes, Modules 1-2]. Full citation details could not be confirmed.

Brockwell, P. J., & Davis, R. A. (2016). Introduction to time series and forecasting (3rd ed.). Springer. https://doi.org/10.1007/978-3-319-29854-2

Hyndman, R. J., & Athanasopoulos, G. (2021). Forecasting: Principles and practice (3rd ed.). OTexts. https://otexts.com/fpp3/

Shumway, R. H., & Stoffer, D. S. (2025). Time series analysis and its applications: With R examples (5th ed.). Springer. https://doi.org/10.1007/978-3-031-70584-7

R Core Team. (n.d.). R documentation. R Foundation for Statistical Computing. https://stat.ethz.ch/R-manual/R-devel/library/stats/html/00Index.html

Research and Verification Notes

  • Uploaded material used: M1L1 for the distinction between observations, stochastic-process models, and forecasting; M1L2 for the introductory R workflow; M1L3 for stationarity, decomposition, ordinary differencing, seasonal differencing, LakeHuron, nottem, and JohnsonJohnson examples.
  • Authoritative verification: Brockwell and Davis (2016) supported the definitions of stationarity, transformations, and ARIMA framing. Shumway and Stoffer (2025) supported the stochastic-process and modern modeling context. Hyndman and Athanasopoulos (2021) supported the practical transformation workflow.
  • Clarifications added: Stationarity was separated from constancy; log transformation was distinguished from differencing; structural breaks and over-differencing were added as explicit failure modes.
  • Potentially outdated source content: The uploaded RStudio setup instructions reference links accessed in 2024. Installation instructions were not reproduced.
  • Code status: The R snippets are concise adaptations of standard base-R usage. They were not executed in the present environment because R was unavailable.
  • Missing metadata: The course-note author, institution, and formal publication year were not identifiable from the PDFs.
  • Recommended review: Confirm preferred Monograph conventions for displaying Mermaid and LaTeX, and execute code in the target R environment before publication.