meta-analysis packages for the R statistical software program

Gene V. Glass (1976) introduced the term meta-analysis to refer to “the statistical analysis of a large collection of analysis results from individual studies for the purpose of integrating the findings” (p. 3). This set of statistical techniques has grown in popularity, and meta-analysis is now the standard tool for providing transparent, objective, and replicable summaries of research findings in the social sciences, medicine, education, and other fields (Hunt, 1997).

Glass, G. V. (1976). Primary, secondary, and meta-analysis. Educational Researcher, 5(10), 3-8.

Hunt, M. (1997). How science takes stock: The story of meta-analysis. New York: Russell Sage.

There are currently 5 related packages for conducting a meta-analysis in :

1. compute.es

2. MAc

3. MAc GUI

4. MAd

5. MAd GUI

MAc (Meta-Analysis with Correlations) and MAd (Meta-Analysis with Mean Differences) are integrated meta-analysis packages for conducting a research synthesis. One of the unique features of these packages is in their integration of user-friendly functions to complete the majority of statistical steps involved in a meta-analysis. It uses recommended procedures as described in The Handbook of Research Synthesis and Meta-Analysis (Cooper, Hedges, & Valentine, 2009).

The meta-analyst can begin their project by using the package functions to derive the necessary effect size from the primary studies, when statistics other than the ones needed are reported (e.g., t-test, p-value, or Cohen’s d can be converted to r). Then, the analyst can aggregate all within-study effect sizes (while accounting for within-study correlations among outcome measures and eliminating any dependencies in the dataset), calculate omnibus effect sizes under a fixed and random effects model, and assess for significant moderators (categorical and continuous, single and multi-predictor models) in the dataset. Finally, the meta-analyst can use one of several user-friendly graphical functions to visually represent their data in an elegant manner.

The compute.es package contains several functions for calculating and converting various statistics, such as a t-test or p-value and sample size, to effect size estimates of d (mean difference), g (unbiased estimate of d), r (correlation coefficient), z’ (Fisher’s z), and log odds ratio. The variance of these estimates are also computed. These calculation/conversion functions are a particularly useful resource during the preliminary stages of a meta-analytic project when deriving effect sizes from reported data.

No problem! These meta-analysis packages are also available for use with a Graphical User Interface (GUI), similar to SPSS. This enables the user to conduct a meta-analysis in a familiar menu-driven (i.e., ‘point and click’) GUI environment, while having the full statistical capabilities of R and these meta-analysis packages.

Links to GUIs:

MAc GUI:

Link on CRAN. The Reference manual is here.

MAd GUI:

Link on CRAN. The Reference manual is here.

*The MA* GUIs are powered by John Fox’s Rcmdr package, which is a GUI for the R program.

compute.es

* Download latest version from CRAN.

* The Reference manual can be found here.

Meta-Analysis with Correlations (MAc):

* Download latest version from CRAN.

* The Reference manual can be found here.

GUI for MAc:

* Download latest version from CRAN.

* The Reference manual can be found here.

Meta-Analysis with Mean Differences (MAd):

* Download latest version from CRAN.

* The Reference manual can be found here.

GUI for MAd:

* Download latest version from CRAN.

* The Reference manual can be found here.

The MA* packages have integrated functions to facilitate the meta-analytic process at nearly every analytical stage.

There are currently five broad areas of analysis that the MA* packages target:

1. Computations to Calculate Effect Size Estimates

2. Within-Study Aggregation of Effect Sizes

3. Fixed and Random Effects Omnibus Analysis

4. Moderator Analyses

5. Graphics

Sample Data

# Example data for mean differences. The following examples using the 'MAd' package:
 
id <- c(1, 1:19)
n.1 <- c(10,20,13,22,28,12,12,36,19,12,36,75,33,121,37,14,40,16,14,20)
n.2 <- c(11,22,10,20,25,12,12,36,19,11,34,75,33,120,37,14,40,16,10,21)
g <- c(.68,.56,.23,.64,.49,-.04,1.49,1.33,.58,1.18,-.11,1.27,.26,.40,.49,
       .51,.40,.34,.42,1.16)
var.g <- c(.08,.06,.03,.04,.09,.04,.009,.033,.0058,.018,.011,.027,.026,.0040,
           .049,.0051,.040,.034,.0042,.016)
mod <- factor(c(rep(c(1,1,2,3),5)))
mods2 <- c(rep(1:5,4))
df <- data.frame(id, n.1, n.2, g, var.g, mod, mods2)  # df = name of the sample data

Computing Effect Sizes from Primary Studies

install.packages('MAd') # install the MAd package
 
library(MAd)  # loads the MAd package which is required to run these functions
 
# computation to derive d (standardized mean difference):
# For example, suppose the primary study reported an log odds ratio for different
# proportions between 2 groups. Then, running:
 
lor_to_d(.9070, .0676)
 
# where the reported log odds ratio lor = .9070 and variance = .0676 will output the
# standardized mean difference d and its variance var.d for use in
# the meta-analysis.

Within-Study Aggregation of Effect Sizes

# This fuction has automated (i.e., will compute for all studies simultaneously) the process of aggregating
# within-study effect sizes while taking into account the correlations among the 
# outcome measures (Gleser & Olkin 2009; Gleser & Olkin 2009; Hedges & Olkin, 1985; Rosenthal
# et al., 2006). The default correlation between within-study effect sizes is set at .50
# (Wampold et al., 1997) and will compute the aggregated effect size for all studies. This
# default of .50 is adjustable. MAd aggregation functions implement Gleser & Olkin (1994; 2009)
# recommendations for aggregating dependent correlated effect sizes.
 
library(MAd)  # loads the MAd package which is required to run these functions
 
# aggregate to 1 id per study (independent sample)
 
agg(id = id, g = g, var = var.g, n.1 = n.1, n.2 = n.2, data = df) # aggregation function
 
# where df = data.frame with columns for id, g (unbiased standardized
# mean difference), var.g (variance of g), n.1 (sample size of group
# one), and n.2 (sample size of comparison group) with multiple rows per
# study. Outputs an aggregated data.frame with 1 effect size per study.
 
# aggregate by id & a moderator (non-independent sample)
 
temp <- agg(id = id, g = g, var = var.g, n.1 = n.1, n.2 = n.2, mod = mod, data=df)
 
temp
 
# This function below will randomly select one within
# study level of the moderator (if there are more than one) and output an
# independent sample. Replace temp with the name of your data.
 
do.call(rbind, lapply(split(temp, temp$id),
function(.data) .data[sample(nrow(.data), 1),]))
 
 

Fixed and Random Effects Omnibus Analysis

library(MAd)  # loads the MAd package which is required to run these functions
 
omni(g = g, var = var.g, data = df, type="weighted", method = "random")  # function
 
# where df = data.frame with columns for id, g (unbiased standardized
# mean difference), var.g (variance of g), n.1 (sample size of group
# one), and n.2 (sample size of comparison group).
# Outputs omnibus effect size (under fixed and random effects),
# variance, confidence intervals, I^2, etc.

Moderator Analyses

library(MAd)  # loads the MAd package which is required to run these functions
 
# Random effects single predictor (categorical) moderator analysis 
 
macat(g = g, var= var.g, mod = mod, data = df, method= "random")
 
# multi-predictor moderator analysis (or single predictor continuous mod)
 
# first, install metafor (this function requires Dr. Viechtbauer's package)
 
install.packages('metafor', dependencies = TRUE)
 
# Examples
 
# Random Effects
 
rand <- mareg(g~ mod + mods2, var = var.g, method = "REML", data = df)
 
summary(rand)  # obtain more detailed information 
 
# Fixed Effects
 
mareg(g~ mod + mods2, var = var.g, method = "FE", data = df)
 
# where the formula: g ~ mod + mods2
# is the criterion variable (effect size) predicted by ('~') mod and mods2.
# the argument'var' is calling for the variance of the effect size and the 
# method 'REML' =  restricted maximal likelihood (random effects) & 'FE = fixed effects

Graphics

library(MAd)  # loads the MAd package which is required to run these functions
library(ggplot2)  # Dr. Wickham's R graphics package is required to run these functions
 
plotcon(g = g, var = var.g, mod = mod, data = df, method= "random",
modname= "Moderator")  # function

Export Neatly Formatted Tables to Word

# install R2wd
 
install.packages('R2wd', dependencies = TRUE)
 
library(R2wd)  # load the R2wd package
 
# mareg fuction
 
temp <- mareg(g~ mod + mods2, var = var.g, method = "REML", data = df)
 
# Export data to Word in formatted table
 
wd(temp, get = TRUE, new = TRUE)
 
 

* GUI for compute.es package (computes effect sizes for a variety of statistics reported in primary data)

* Integrate ICC function for inter-rater agreement with continuous moderators

* Diagnostic plots

Please add!

Anyone is free to contribute their knowledge to these packages. Contributions can be brief (e.g., ideas for functions that you would like included in these packages) or involved (e.g., submitting functions that can be included in the packages, for which credit will be given to individuals who contribute) and all are certainly appreciated!

**If you have used one of the MA* packages or compute.es , feel free to list your study here.

1.Del Re, AC, Fluckiger, C., Wampold, B. E, & Horvath, A. O.(submitted).Therapist effects in the alliance-outcome relationship: A restricted maximum likelihood meta-analysis.

2. Horvath, A.O., Del Re, AC, Flückiger, C., Symonds, D. (2011). The alliance. In J. C. Norcross & B. E. Wampold. Relationships that work. New York, NY: Oxford University Press.

3. Horvath, A.O., Del Re, AC, Flückiger, C., Symonds, D. (2011). The working alliance in psychotherapy. Psychotherapy Theory, Research, Practice, Training.

4. Farber, B. A., & Doolin, E. M. (2011). Positive regard. In J. C. Norcross (Ed.). Psychotherapy relationships that work. New York: Oxford University Press.

5. Horvath, A.O., Del Re, AC, Fluckiger, C., & Symonds, D. (2010). Alliance in individual psychotherapy. In J. C. Norcross (Eds.), Evidence-Based Therapy Relationships (Vol. National Registry of Evidence-based Programs and Practices pp. 3). Available from http://www.nrepp.samhsa.gov/Norcross.aspx

6. Fluckiger, C., Del Re, AC, Znoj, H., Wampold, B. E, Jarg, U., & Caspar, F.(In Press). Valuing patients perspective and the effects on the therapeutic alliance: A randomized controlled adjunctive instruction. Journal of Counseling Psychology.

7. Fluckiger, C., Del Re, AC, Wampold, B. E., & Horvath, A. O. (In Press). The therapeutic alliance in the context of empirically supported treatments: A longitudinal meta-analysis. Journal of Counseling Psychology.

8. Wampold, B. E., Budge, S. L., Laska, K. M., Del Re, AC, Baardseth, T. P., Fluckiger, C., Kivlighan, M., & Gunn, W. (In Press). Evidence-based treatments for depression and anxiety: Are they more effective than treatment-as-usual? Clinical Psychology Review.

9. Hoyt, W. T., Del Re, AC, & Larson, D. O. (submitted). The efficacy of grief treatment revisited: A meta-analysis.

10. Panos, S. E., Del Re, AC, Thames A. D., Levine, A. J., Strei, V., Castellon, S., & Hinkin, C. H. (submitted). Adherence to antiretroviral therapy in the treatment of HIV disease: A multi-predictor longitudinal mixed-effects analysis.