Supplemental Materials for Brunner and Schimmack's

How replicable is psychology?

• Current draft of the main paper
• Technical Supplement: Derivations and technical details
• R functions including the p-curve, p-uniform, maximum likelihood and z-curve estimation functions.
• Simulations: The simulations are organized into "studies," which in some cases are pieces of the large factorial experiments reported in the paper. Each study consists of a simulation phase and an analysis phase. The code in the simulation phase just writes a plain text data file with one line for each input set of k significant test statistics. In the analysis phase, we read the raw data file and produced tables of results. All the computation was done with R.
  1. Study 1: Chi-squared tests, heterogeneity in sample size only
     • Simulation Code
     • Results: Analysis Code and Tables
       • df = 1
       • df = 3
       • df = 5
  2. Study 2: F-tests, heterogeneity in sample size only
     • Simulation Code
     • Results: Analysis Code and Tables
       • Numerator df = 1
       • Numerator df = 3
       • Numerator df = 5
  3. Study 3: Chi-squared tests, heterogeneity in sample size and effect size, gamma effect size
     • Simulation Code
     • Results: Analysis Code and Tables
       • df = 1
       • df = 3
       • df = 5
  4. Study 4: F-tests, heterogeneity in sample size and effect size, gamma effect size
     • Simulation Code
     • Results: Analysis Code and Tables
       • Numerator df = 1
       • Numerator df = 3
       • Numerator df = 5
  5. Study 5: Full heterogeneity (sample size, effect size, F versus chi-squared, df)
     • Simulation Code
     • Results: Analysis Code and Tables
  6. Study 6: Bootstrap confidence interval for z-curve
     • Simulation Code
     • Results: Analysis Code and Tables
  7. Study 7: Complex heterogeneity, with correlated sample size and effect size, beta effect size. Just F-tests with numerator df=1
     • Simulation Code
     • Results: Analysis Code and Tables