Posterior Probability and Bayesian Hypothesis testing

Let’s assume there some people in a group.

20% are from France= $P(F)$

30% are from UK

50% are from Canada

50% of french are boys= $P(B | F)$

30% of English are boys

25% of Canadians are boys

Obviously If we choose a person randomly the probability of choosing a french person is 20%.

If I tell you that the chosen person is a boy, What would be the probability of the chosen to be french.

Posterior probability is a revised probability that takes into account new available information.

From the symmetry we know

$P(A\wedge B)= {P(B \wedge A)}$

$P(A|B) {P(B)}= {P(B | A)\, P(A)}$

Posterior Probability $=P(A|B) = \frac{P(B | A)\, P(A)}{P(B)}= \frac{P(B | A)}{P(B)}P(A)$

We can also interpret that as

$Likelihood (A tohappenif B)=P(A|B) = \frac{P(B | A)}{P(B)}P(A)= \frac{Likelihood (B tohappenif A)}{P(B)}P(A)$

So we can consider $\frac{P(B | A)}{P(B)}= \frac{Likelihood (B tohappenif A)}{P(B)}$ as the correction for prior probability $P(A)$ which is called “Prior probability”.

as the result of the other information we have about a condition which is known to be the case.

In the example above we are interested in $P(F|B) = \frac{P(B | F)\, P(F)}{P(B)}= \frac{P(B | F)}{P(B)}P(F)=\frac{.50}{?}.20$

If we knew the total proportion of boys the answer would be easy but we don’t.

However,

$P(B)= P(B \wedge F)+P(B \wedge U)+P(B \wedge C)$

$P(B)= P(F)P(B | F)+P(U) P(B | U)+P(C) P(B | C)$

therefore even without knowing the Probability of B, if we know about B’s posterior probabilities, we can answer the question.

$P(F|B) = \frac{P(B | F)\, P(F)}{P(B)}= \frac{P(B | F)}{P(B)}P(F)=\frac{.50}{(.20*.50)(.30*.30)(.50*.25)}.20$

2013 (c) Amir H. Ghaseminejad

=================================================================================

http://en.wikipedia.org/wiki/Bayes’_theorem

Here is a nice proof for the fundamental conditional probability formula from:

http://www.dartmouth.edu/~chance/teaching_aids/books_articles/probability_book/Chapter4.pdf

Hypothesis testing for Bayes:

–We have a hypotheses H0 (null),

-We have Alternative Hypothesis H1

–We have data (Y)

–We want to check if the model that we have (H1) fits our data and doesn’t fit H0 (accept H1 / reject H0) or

our data fits (H0) (Fail to reject H0)

Inferential statistics:

what is the probability that we can reject H0 and accept H1 at some level of significance (alpha, P)

Bayes:

We get some evidence for the model (“likelihood”) and then can even compare “likelihoods” of different models

We want to know the probability of Hypothesis to be true if Observation is observed.

$P(H0|O) = \frac{P(O | H0)\, P(H0)}{P(O)}= \frac{P(O | H0)}{P(O \wedge H0)+P(O \wedge H1) }P(H0)=\frac{P(O | H0)}{P(O | H0)P(H0))+P(O |H1)P(H1)) }P(H0)$