Bayesian Credible Intervals: A Worked Example, Computed From Scratch

Bayesian vs. frequentist statistics covers the conceptual contrast between a credible interval and a confidence interval — this guide picks up where that leaves off, walking through an actual credible interval computed from scratch, using the same prior-times-likelihood machinery behind Bayes' theorem, just applied to estimating a continuous parameter rather than a single yes/no event.

The Setup: Estimating a Proportion

Suppose the goal is estimating an unknown true proportion p (a conversion rate, a disease prevalence, a coin's bias) from binomial data — a fixed number of trials, each a success or failure. The Bayesian approach starts with a prior distribution over p — a probability distribution representing what's believed about p before seeing any data — and updates it using the observed data to produce a posterior distribution, from which a credible interval is then read directly.

For a proportion, the standard, mathematically convenient choice of prior is the Beta distribution, because combining a Beta prior with binomial data produces another Beta distribution as the posterior — a "conjugate prior" relationship that makes the update a matter of simple arithmetic on the Beta distribution's two parameters, rather than requiring complex computation.

Here a₀ and b₀ are the prior's own two parameters, and the update simply adds the observed successes to a₀ and the observed failures to b₀.

Worked Example: A Neutral (Uninformative) Prior

Problem: Starting with no strong prior belief about a website's conversion rate — represented by a Beta(1,1) prior, which is mathematically identical to a flat, uniform distribution across every value from 0 to 1 — 20 visitors are observed, of whom 8 convert. Find the posterior distribution and a 95% credible interval.

Posterior = Beta(1+8, 1+12) = Beta(9, 13).

The posterior mean is a₀/(a₀+b₀) = 9/(9+13) ≈ 0.409 — close to, but not exactly, the raw sample proportion of 8/20=0.40; the small difference comes from the prior's own (here, minimal) influence blending in with the observed data.

Reading the 2.5th and 97.5th percentiles directly off the Beta(9,13) distribution gives the 95% credible interval: (0.218, 0.616). This interval has the direct, intuitive interpretation a confidence interval doesn't formally support: given the prior and the observed data, there's a 95% probability the true conversion rate lies between 21.8% and 61.6%.

Comparing to the Equivalent Frequentist Confidence Interval

Computing an ordinary 95% confidence interval for a proportion on the same data (p̂=0.40, n=20) gives approximately (0.185, 0.615) — numerically very close to the Bayesian credible interval above, differing mainly in the lower bound. This convergence is expected: with a genuinely neutral (uninformative) prior, the Bayesian and frequentist results tend to align closely, exactly as Bayesian vs. frequentist statistics explains — the real divergence between the two approaches shows up specifically when an informative prior is used instead.

Worked Example: An Informative Prior

Problem: Now suppose there's genuine prior reason to expect a low conversion rate — perhaps historical data from similar past campaigns — represented by a Beta(2,8) prior (a distribution centered around 2/(2+8)=0.20). Using the identical observed data (8 conversions out of 20 visitors), find the new posterior and credible interval.

Posterior = Beta(2+8, 8+12) = Beta(10, 20).

Posterior mean = 10/(10+20) ≈ 0.333 — noticeably lower than the neutral-prior result of 0.409, pulled toward the informative prior's belief that the true rate is likely lower. The 95% credible interval is (0.179, 0.508) — shifted downward and somewhat narrower than the neutral-prior version, reflecting the added information (and added certainty) the informative prior contributes on top of the data alone.

This comparison makes the practical consequence of prior choice concrete: the exact same observed data (8/20 conversions) produces genuinely different posterior estimates and credible intervals depending on what was believed before the data arrived — a feature, not a flaw, of the Bayesian approach, as long as the prior itself is defensible and transparently reported alongside the result.

How the Credible Interval Is Actually Found

For a Beta posterior specifically, the 95% credible interval shown above is the equal-tailed interval — the range between the posterior distribution's own 2.5th and 97.5th percentiles, leaving exactly 2.5% of the posterior probability in each excluded tail. Software with a Beta-distribution quantile function computes these directly; there's no simple closed-form shortcut the way there is for a normal-distribution-based confidence interval, which is why Bayesian intervals are typically computed with statistical software rather than a lookup table.

Real-World Uses

Clinical trial monitoring: Bayesian credible intervals are increasingly used in clinical trials specifically because they allow a trial's data to be evaluated at any point during data collection with a directly interpretable probability statement, rather than requiring the fixed, pre-specified sample size a frequentist confidence interval's guarantees depend on.

A/B testing dashboards: many modern A/B testing platforms report results as credible intervals or direct posterior probabilities ("there's a 92% probability variant B converts better than variant A") specifically because that framing is more directly interpretable to a non-statistician than a p-value or confidence interval, tying back to the practical convenience discussed in Bayesian vs. frequentist statistics.

Combining historical data with a new small study: the informative-prior example above illustrates a genuinely practical use case — when a small amount of new data alone wouldn't support a confident conclusion, a well-justified prior built from relevant historical data lets a credible interval draw on that accumulated evidence directly, rather than starting from scratch each time.

Common Mistakes

Choosing an informative prior after seeing the data, to nudge the result toward a preferred conclusion. A prior must be set based on genuine external evidence or belief formed before the current data is examined — choosing it afterward to steer the posterior defeats the entire logical structure of the method and amounts to a form of the same p-hacking risk covered in one-tailed vs. two-tailed tests.

Interpreting a frequentist confidence interval as if it were a credible interval. Even when the two intervals are numerically close (as in the neutral-prior example above), their formal interpretations remain genuinely different — a confidence interval is a statement about the reliability of the interval-construction procedure over repeated sampling, not a direct probability statement about where the true parameter lies.