Bayesian t-test hypothesis testing for two independent groups

This method is based on the book "Bayesian Cognitive Modeling: A Practical Course" by Michael Lee and Eric-Jan Wagenmakers. Additionally, a published scientific article can be found here. Some of the code may have been changed in order to make the application of the method easier. If you want to learn more about the model and the code you can read the book.

Both groups should have their data normally distributed.

The procedure is being done using R and OpenBugs.

Input
Set the first three lines according to your setup and data. You also need to set the file that contains the OpenBUGS model which you can find at the end of this page. If you wish to change the priors you can, just remember that you need to adjust the prior for hypothesis 2 and hypothesis 3 in order apply only for positive or negative numbers. Also you can change the itterations and the burnin if you want to improve your results.

Output
The hypotheses for this test are as follows:


 * H0 - δ = 0 means that there is no difference between the two groups
 * H1 - δ <>0 means that there is a difference between the two groups
 * H2 - δ < 0 means that group 2 has larger values than group 1
 * H3 - δ > 0 means that group 1 has larger values than group 2

The output produces a set of results in text such as these along with the probability distribution plots for each one of them. Both are useful for making a decision about your hypothesis. As an example, you can use the graph and determine if at the point δ=0 your posterior(your results after the data) are higher or lower than the prior(your initial belief). If the posterior is higher than the prior at δ=0 then it reinforces the fact that the null hypothesis(H0) is probably true. If the posterior is lower than the prior then the data weakens your belief that the null hypothesis is true.

You determined which hypothesis is more likely based on the Bayes Factor. The way to interpret a general Bayes Factor is the following. If a Bayes factor is denoted as BFxy then you say that the data are n times more likely under Hx than Hy. An example based on the BF10 would be that the data would be 0.53 times more likely under H1 than H0. If we use the BF01 then we would say that the data are 1.87 times more likely under H0 than H1(which is more meaningful). Basically, one version of Bayes factor(e.g. BF01) is the inverted version of the other(e.g BF10).

You determine how strong is your evidence based on the Bayes factor by looking at the table here.

In the first two cases the evidence is "Barely worth mentioning" for H0. But, the third result (7.67) is considered "Substantial" evidence in favor of H0, indicating that when we consider if group 1 has a bigger effect than group 2, there is substantial evidence to say that is unlikely(providing proof for H0).

When publishing you are going to have to report also the process you obtained your results and the numbers of itterations for the MCMC test along with the burnin value and the chains(in this case 3).

The model specifications are: