Bayesian analyses

Uit Methodologiewinkel
Ga naar: navigatie, zoeken

What is Bayesian Statistics?

What are the general principles of Bayesian analysis? First, Bayesians describe their beliefs for a specific model or hypothesis with probability distributions. Second, the observed data of an experiment are used to update this prior belief and becomes posterior information of belief. In Bayesian hypothesis testing, we compare the null and the alternative hypothesis by means of their plausibility under the observed data. The test decision is made by evaluating the so called Bayes factor.

The Bayes factor quantifies the change from the prior odds to the posterior odds and states how many times the data are more likely under one hypothesis compared to the other. For example, a Bayes factor for the alternative versus the null hypothesis of 2 (i.e., BF10 = 2) means, that the data are twice as likely to have occurred under the alternative, than under the null hypothesis. Note that the Bayes factor for the null versus the alternative hypothesis BF01 is simply BF01 = 1/BF10.

The Bayes factor has a continuous scale, but coming from a statistical practice in which effects are categorized in significant and non-significant it can be helpful to summarize the Bayes factor in terms of discrete categories of evidential strength. Jeffries proposed the following classification scheme:

Interpretation categories for the Bayes Factor according to Jeffries (1961)

Bayes Factor BF01 Interpretation
>100 Decicive evidence for H1
30 - 100 Very strong evidence for H1
10 - 30 Strong evidence for H1
3 - 10 Substantial evidence for H1
1 - 3 Anecdotal evidence for H1
1 No evidence
1/3 - 1 Anecdotal evidence for H0
1/10 - 1/3 Substantial evidence for H0
1/30 - 1/10 Strong evidence for H0
1/100 - 1/30 Very strong evidence for H0
< 1/100 Decisive evidence for H0

Copied from Lee, M. D., & Wagenmakers, E. J. (2014). Bayesian cognitive modeling: A practical course. Cambridge University Press.

When do you need Bayesian statistics? Waarvoor gebruik je dit?

The advantage of Bayesian statistic is that we are able to quantify support in favor of and against the stated null hypothesis. That means that we can make informative statements in the analysis even if the effect is absent (the effect is non-significant). Deriving conclusions about the null hypothesis are particular valuable when we are dealing with:

  1. Replication studies. If an effect is not replicating it does not necessarily mean that the design was inadequate – it could simply mean that the original effect was mistakenly assumed to exist. If we collect evidence in favor for the null hypothesis, this can be taken as ‘evidence for replication failure’.
  2. Testing effects that seem surprising or counterintuitive. Some effects seem very questionably and are unlikely to be replicated. With classical statistics, however, it is difficult to make statements implying that the effect probably only occurred by chance. With Bayesian analysis we can state how strongly the data supported the null hypothesis.
  3. Testing a null effect or the absence of an effect. With the usual statistical analyses, there is no way to find out if a null hypothesis is true or not – we are only able to test the alternative hypothesis. Bayesian analyses can allow us to collect evidence in favor for a null effect or the absence of an effect; which means that we are more flexible with the models and hypotheses we are testing.

A second advantage is the so-called sequential updating. In classical statistics you have to set a sampling design before you are collecting any data; which means that we have to conduct a power analysis to see how many participants we have to assess. In Bayesian analysis this is different. We do not need a fixed sampling plan before we collect the data and we do not need to do a power analysis. We can simply collect as many participants as we need until we get a convincing Bayes factor. The reason is as follows: In the classical framework, you would always get a significant result even if the null hypothesis is true, if you would test an infinite number of participants. This is why you need to set the number beforehand. In the Bayesian framework, if the null hypothesis is true, you would collect more and more evidence in favor for the null hypothesis with a growing number of participants. That means, that you cannot ‘cheat’ with Bayesian statistics.

How can you perform Bayesian analyses?

The basic Bayesian analysis – like Bayesian correlations, T-Tests and ANOVAs – can easily be performed in JASP. JASP is a free statistical software and has an intuitive interface just like SPSS, which makes it easy to use. All output, graphs and tables are displayed in APA style and can be simply copy pasted into the thesis. The user guide on the JASP website explains how JASP is downloaded and installed on your computer. Additionally, it explains the basics you need to know for the analysis, for example the required file format, the possible variable categories etc:
https://jasp-stats.org/user-guide/
A short introduction of the main features of JASP is shown in this video:
https://www.youtube.com/watch?v=feorVor1weo

Example: The Kitchen Roll experiment.

To illustrate the usage of JASP, we will perform Bayesian T-Test on the implemented Kitchen Roll dataset. The story behind the data is as follows: Topolinski and Sparenberg (2012) sought to demonstrate that clockwise movements induce an orientation towards the future and novelty. For that, participants had to turn kitchen rolls either clockwise or counterclockwise. Then they filled out the openness to experience subscale from the NEO PI-R . The hypothesis was that that participants who turned the rolls clockwise reported more “openness to experience” than those that turned the rolls counterclockwise. Wagenmakers and colleagues (2015) attempted to replicate the results and included dummy data of this experiment as T-Test example in JASP.

How can we perform a Bayesian analysis with JASP? Hoe uit te voeren in JASP?

For our example analysis, we test the hypothesis, that participants who turn kitchen rolls clockwise report higher openness to experience than participants who turn the rolls counterclockwise. The mean score of the openness to experience subscale mean_NEO serves as our dependent variable. Additionally, we define the direction of rotation Rotation as grouping variable.

To load the dataset in JASP we click on File --> Open --> Examples --> Kitchen Rolls. What we see the dataset, in which each row represents one participant and each column one variable.

Bayes1.png

Classic Independent T-Test

  • Next, we will perform a standard independent T-Test. For that we click on T-Test --> Independent Samples T-Test. We drag the dependent and group variables in the field Dependent variable and Grouping Variable. Then, we select the intended analysis under the subheading Tests select under Hypothesis, whether we have a directed or none-directed hypothesis. We want to perform a standard Students T-Test, with a directed hypothesis. That means that we propose that the group that rotates the kitchen rolls clockwise will have higher openness to experience scores than the group that rotates the kitchen rolls counterclockwise. Note that the hypothesis is also specified under the T-Test table of the Results window.
  • Select possible additional statistics that are necessary for your report in the field Additional Statistics. We chose the mean difference between the groups and the effect sizes. Those statistics are now displayed in the results table as well.

Bayes2.png

  • Under the subheading Assumptions, we can check whether our data meet the assumption of normality and equal variances. Therefore, we click on the tests you wish to perform. The test results will show up immediately in the results window. Note that we can not manipulate our data in JASP; if we want to transform variables we need to use different programs like SPSS, Excel or R.

Bayes3.png

How do we report the classical T-Test? Rapporteren conclusive

With t(100) = -0.754, p = .774 the proposed effect is non-significant. That means, that the null hypothesis, that the counterclockwise group reports equal or lower openness to experience scores cannot be rejected. With the traditional way of analyzing the data, we don’t have any information about whether our data actually supports the null hypothesis or not. To include this information, we need to do the Bayesian equivalent to the classical T-Test.

Bayesian Independent T-Test

Next, we will perform the Bayesian version of the Independence T-Test. We click on T-Tests --> Bayesian Independent Samples T-Test and select again the mean_NEO as dependent and Rotation as grouping variable. The results for the Bayesian T-Test and the Bayes Factor BF10 now displayed in the Results window.

Bayes4.png

How do we report Bayes factors? Rapporteren conclusie

The Bayes factor of BF10 = 0.129 indicates substantial evidence for the null hypothesis. More precise, it means that the data are 1/BF10 = 7.77 times more likely to have occurred under the null than under the alternative hypothesis.

References

  • Jeffries, H. (1961). Theory of probability.
  • Topolinski, S., & Sparenberg, P. (2012). Turning the Hands of Time Clockwise Movements Increase Preference for Novelty. Social Psychological and Personality Science, 3(3), 308-314.
  • Wagenmakers, E. J., Beek, T. F., Rotteveel, M., Gierholz, A., Matzke, D., Steingroever, H., ... & Gronau, Q. F. (2015). Turning the hands of time again: a purely confirmatory replication study and a Bayesian analysis. Frontiers in psychology, 6, 494.