Today I needed a quick refresher on Bayesian credible intervals vs. frequentist confidence intervals. They might seem similar—both produce numeric ranges — but they have fundamentally different interpretations:

Bayesian Credible Interval

  • Treats the unknown parameter as a random variable with a probability distribution.

  • A 95% credible interval [4, 6] means, "Given our data and prior, there's a 95% probability the true parameter lies between 4 and 6."

  • You can confidently say, "I'm 95% sure it's in this interval."

Frequentist Confidence Interval

  • Treats the unknown parameter as fixed but the interval itself as random.

  • A 95% confidence interval [4, 6] means, "If we repeated this experiment many times, about 95% of such intervals would contain the true parameter."

  • You cannot directly state a 95% probability for a single interval; it's about the method, not the specific outcome.

Practical example in Python

Here's a quick demonstration to clarify:

import numpy as np
from scipy import stats

# Simulated data
np.random.seed(0)

# Generate some data
#   loc=5 = mean of the normal distribution to 5
#   scale=1 = standard deviation of the distribution to 1
#   size=100 = number of samples to generate
data = np.random.normal(loc=5, scale=1, size=100)

# Frequentist confidence interval
mean, sigma = np.mean(data), np.std(data, ddof=1)
conf_interval = stats.t.interval(0.95, len(data)-1, loc=mean, scale=sigma/np.sqrt(len(data)))
print(f"Frequentist 95% CI: {conf_interval}")

# Bayesian credible interval (assuming known variance and a normal prior)
prior_mean, prior_sigma = 0, 10  # weak prior
posterior_mean = (prior_mean/prior_sigma**2 + len(data)*mean/sigma**2) / (1/prior_sigma**2 + len(data)/sigma**2)
posterior_sigma = np.sqrt(1 / (1/prior_sigma**2 + len(data)/sigma**2))
credible_interval = stats.norm.interval(0.95, loc=posterior_mean, scale=posterior_sigma)
print(f"Bayesian 95% credible interval: {credible_interval}")

Approximately outputs:

Frequentist 95% CI: (4.85881, 5.26080)
Bayesian 95% credible interval: (4.86076, 5.25782)

Why It Matters

Imagine predicting sales between $4K - $6K:

  • Bayesian: "We're 95% sure sales fall in this exact range."
  • Frequentist: "Our method is reliable; across many predictions, 95% will contain the true sales, but we can't guarantee this specific one."

Key Takeaways

  • Credible intervals provide a direct probabilistic interpretation of the parameter.

  • Confidence intervals guarantee long-term reliability of the procedure, not the specific interval.

  • Though numerically similar, these intervals communicate uncertainty differently and serve distinct purposes.