Prophet and modern forecasting libraries

The big idea

Suppose your company needs forecasts for 300 metrics: sign-ups per country, tickets per support queue, orders per warehouse. A statistician could hand-tune an ARIMA model for each, but there is one statistician and 300 metrics. The people who know the business, that Black Friday doubles orders or that a pricing change in March bent the growth curve, usually do not know what a unit root is.

ProphetAn open-source forecasting library from Meta that fits an additive model of trend with changepoints, Fourier seasonality and holiday effects., released as open source by Facebook (now Meta) in 2017, was built for this gap. Instead of asking for p, d and q, it asks questions an analyst can answer: Where might the growth rate have changed? Which holidays matter? Is the seasonality weekly, yearly, or both? It treats forecasting as fitting a curve made of readable parts, then extending the curve forward.

How it works

Step 1: Trend + seasonality + holidays

Prophet's paper (Taylor and Letham, "Forecasting at scale") writes the series as a sum:

where $g$ is the trend, $s$ the periodic seasonality, $h$ the holiday effects and $\epsilon$ noise. This is the additive decomposition of Time series fundamentals turned into a forecasting model: each part is a function of the date, so to forecast you evaluate the functions at future dates.

Step 2: The trend and its changepoints

The default trend is piecewise linear. Prophet places many candidate changepointsA date at which a piecewise trend is allowed to change its slope; Prophet places many candidates and shrinks most of their changes to zero., by default 25 potential changepoints uniformly in the first 80 percent of the history, and fits a slope change at each. A sparse prior shrinks most of those changes to zero, so only real bends survive. The changepoint_prior_scale setting (default 0.05) controls how flexible the trend is: raise it and the trend follows the data more closely, at the risk of chasing noise.

For metrics that cannot grow forever, such as users in a market of fixed size, there is a logistic growth option with a capacity you set.

Step 3: Seasonality as a Fourier series

Each seasonal pattern is a sum of sines and cosines of the period $P$:

N is the Fourier order: N terms mean 2N coefficients to fit. The defaults are order 10 for yearly and 3 for weekly seasonality. Higher orders allow sharper, faster-changing shapes and more overfitting. You can add any other period with add_seasonality, such as a monthly payday cycle.

Step 4: Holidays and extra regressors

Holidays break seasonality: Thanksgiving moves around November, and Easter moves between March and April. You give Prophet a table of holiday names and dates, past and future, optionally with lower_window and upper_window to cover the days around them. add_country_holidays fills in public holidays for a country from the holidays package. add_regressor adds your own known-in-advance inputs, such as planned promotions or prices.

Step 5: Uncertainty

The forecast table has yhat plus yhat_lower and yhat_upper, an 80 percent interval by default (interval_width changes it). The main source of that uncertainty is the trend: Prophet allows for future trend changes, assuming "the future will see similar trend changes as the history". Seasonal uncertainty is only included with full Bayesian sampling (mcmc_samples), which takes minutes instead of seconds. The documentation itself warns that these intervals rest on the past behaving like the future.

Going deeper

Code: holidays make the difference

import logging
for name in ["cmdstanpy", "prophet", "prophet.plot"]: logging.getLogger(name).disabled = True
import numpy as np, pandas as pd
from prophet import Prophet   # prophet 1.x
np.random.seed(0)             # Prophet samples its uncertainty intervals with numpy's global RNG

# Synthetic daily shop sales 2021-2023: trend that speeds up in mid-2022, weekend lift,
# a summer bump, Thanksgiving and Christmas closures, and a pre-Christmas rush. The truth is known.
rng = np.random.default_rng(3)
ds = pd.date_range("2021-01-01", "2023-12-31", freq="D")
t = np.arange(len(ds))
y = 200 + 0.05 * t + 0.10 * np.maximum(t - 540, 0)                      # changepoint at day 540
y += np.where(ds.dayofweek >= 5, 30, 0) + 15 * np.sin(2 * np.pi * (ds.dayofyear - 100) / 365.25)
thanks = ds.isin(pd.to_datetime(["2021-11-25", "2022-11-24", "2023-11-23"]))
xmas = (ds.month == 12) & (ds.day == 25)
rush = (ds.month == 12) & (ds.day >= 18) & (ds.day <= 23)
y = y - 120 * thanks - 200 * xmas + 40 * rush + rng.normal(0, 8, len(ds))
df = pd.DataFrame({"ds": ds, "y": y})
train, test = df[df.ds < "2023-10-01"], df[df.ds >= "2023-10-01"]
special = test.ds.isin(ds[thanks | xmas | rush])
mae = lambda f, m=slice(None): np.mean(np.abs(np.asarray(f)[m] - test.y.values[m]))

rush_df = pd.DataFrame({"holiday": "xmas_rush", "ds": pd.to_datetime(["2021-12-18", "2022-12-18", "2023-12-18"]),
                        "lower_window": 0, "upper_window": 5})
results = {"seasonal naive (last year)": df.set_index("ds").y.shift(364).loc[test.ds].values}
for label, use_holidays in [("Prophet, no holidays", False), ("Prophet + holidays", True)]:
    m = Prophet(holidays=rush_df if use_holidays else None, daily_seasonality=False)
    if use_holidays:
        m.add_country_holidays(country_name="US")          # Thanksgiving, Christmas Day, ...
    m.fit(train)
    fc = m.predict(test[["ds"]])
    results[label] = fc.yhat.values
    if use_holidays:
        row = fc[fc.ds == "2023-12-25"].iloc[0]
        print(f"Christmas 2023: forecast {row.yhat:.0f}, 80% interval {row.yhat_lower:.0f} to {row.yhat_upper:.0f}, "
              f"actual {test[test.ds == '2023-12-25'].y.iloc[0]:.0f}")
        print(f"learned effects: Christmas Day {row['Christmas Day']:.0f}, rush (Dec 18) "
              f"{fc[fc.ds == '2023-12-18'].iloc[0]['xmas_rush']:.0f}; true -200 and +40")

print(f"\nMAE, Oct-Dec 2023 (all {len(test)} days / {special.sum()} holiday and rush days):")
for label, f in results.items():
    print(f"  {label:27s} {mae(f):6.1f} / {mae(f, special.values):6.1f}")

Christmas 2023: forecast 88, 80% interval 78 to 97, actual 92
learned effects: Christmas Day -207, rush (Dec 18) 24; true -200 and +40

MAE, Oct-Dec 2023 (all 92 days / 8 holiday and rush days):
  seasonal naive (last year)    58.0 /   71.6
  Prophet, no holidays          12.6 /   71.5
  Prophet + holidays             6.4 /    8.9

The data are synthetic daily shop sales for 2021 to 2023 built from known parts: a trend that speeds up in mid-2022, a weekend lift, a summer bump, closures on Thanksgiving and Christmas Day (-120 and -200) and a December 18 to 23 shopping rush (+40). Every model trains up to September 2023 and forecasts October to December.

• Seasonal naive (same weekday 52 weeks earlier) has an MAE of 58.0: the trend sped up, so last year is far too low.
• Prophet without holidays gets the trend and seasons right (12.6 overall) but misses every special day: 71.5 on the 8 holiday and rush days.
• Prophet with holidays (US public holidays plus a custom "xmas_rush" window) cuts that to 8.9 and the overall MAE to 6.4. It learned a Christmas Day effect of -207 against the true -200. The rush effect comes out at 24 against a true 40: with only two past Decembers to learn from, an effect spread over six days is hard to pin down.

This is Prophet's sweet spot: daily data, strong weekly and yearly patterns, a trend change, and holidays an analyst can list.

When to use it, and its limits

Meta's announcement lists the series Prophet is meant for: "hourly, daily, or weekly observations with at least a few months (preferably a year) of history", strong multiple seasonalities, known irregular holidays, missing data or outliers, historical trend changes and saturating growth.

Its limits follow from the design:

• No autoregression. Prophet fits a curve of the date. It does not use yesterday's error to adjust today's forecast, so after a shock it keeps forecasting the curve, while ARIMA or exponential smoothing react.
• Trend extrapolation. The last fitted slope is extended indefinitely. Over long horizons a small slope error grows into a large one.
• Garbage holidays in, garbage out. Its biggest advantage depends on an analyst supplying the right special dates.
• Not automatically the most accurate. Prophet made forecasting easy, not unbeatable. On many series exponential smoothing, ARIMA or a gradient-boosted model is as accurate or better, so always backtest it against seasonal naive and a classical model ( Backtesting and forecast evaluation ).

The wider library landscape

• statsmodels (Python): ARIMA, SARIMAX, exponential smoothing, STL, statistical tests.
• fable (R): tidy automatic ETS and ARIMA across many series, from the Forecasting: Principles and Practice authors.
• statsforecast, sktime and Darts (Python): unified interfaces over classical, ML and deep models, built for many series and backtesting.
• Gradient boosting with lag features and global deep models: ML and deep learning for forecasting .

Trade-offs

Common mistakes

• Leaving holidays out and then blaming Prophet for missing Black Friday.
• Turning changepoint_prior_scale up until the trend fits every wiggle, then extrapolating that noise.
• Forecasting years ahead from a few months of daily data: yearly seasonality cannot be learned from less than a year.
• Reading the 80 percent interval as 95 percent, or as covering shocks unlike anything in the history.
• Skipping the baseline comparison because the plot looks plausible.

In the wild

Check yourself

Recap