Prediction interval

• Prediction intervals are always wider than confidence intervals, because they incorporate both the error in the estimate for (the reducible error) and the uncertainty as to how much an individual point will differ from the population regression plane (the irreducible error).

Reference: P 104, An introduction to statistical learning with application in R.

• There are two popular statistical methods used to generate prediction intervals in regression tasks:
  1. Quantile regression
  2. Conformal prediction
  3. Conformalized quantile regression

Reference: https://munin.uit.no/bitstream/handle/10037/21914/thesis.pdf?sequence=2&isAllowed=y

Confidence interval

• As a result, to express uncertainty in our knowledge after an experiment, the frequentist approach uses a "confidence interval" -- a range of values designed to include the true value of the parameter with some minimum probability, say 95%.

Credible interval

• Bayesian approaches formulate the problem differently. Instead of saying the parameter simply has one (unknown) true value, a Bayesian method says the parameter's value is fixed but has been chosen from some probability distribution -- known as the prior probability distribution. (Another way to say that is that before taking any measurements, the Bayesian assigns a probability distribution, which they call a belief state, on what the true value of the parameter happens to be.) This "prior" might be known (imagine trying to estimate the size of a truck, if we know the overall distribution of truck sizes from the DMV) or it might be an assumption drawn out of thin air. The Bayesian inference is simpler -- we collect some data, and then calculate the probability of different values of the parameter GIVEN the data. This new probability distribution is called the "a posteriori probability" or simply the "posterior." Bayesian approaches can summarize their uncertainty by giving a range of values on the posterior probability distribution that includes 95% of the probability -- this is called a "95% credibility interval."

Reference:

1. https://machinelearningmastery.com/prediction-intervals-for-machine-learning/
2. https://machinelearningmastery.com/confidence-intervals-for-machine-learning/
3. https://stats.stackexchange.com/questions/2272/whats-the-difference-between-a-confidence-interval-and-a-credible-interval