The hmetad package is designed to fit the meta-d’ model
for confidence ratings (Maniscalco &
Lau, 2012, 2014). The
hmetad package uses a Bayesian modeling approach, building
on and superseding previous development of the Hmeta-d toolbox (Fleming, 2017). A key advance is
implementation as a custom family in the brms package, which itself
provides a friendly interface to the probabilistic programming language
Stan.
This provides major benefits:
R
formulasbrms (e.g.,
tidybayes, ggdist, bayesplot,
loo, posterior, bridgesampling,
bayestestR)hmetad is available via CRAN and can be installed
using:
install.packages("hmetad")Alternatively, you can install the development version of
hmetad from GitHub with:
# install.packages("pak")
pak::pak("metacoglab/hmetad")Let’s say you have some data from a binary decision task with ordinal confidence ratings:
#> # A tibble: 1,000 × 5
#> trial stimulus response correct confidence
#> <int> <int> <int> <int> <int>
#> 1 1 0 0 1 2
#> 2 2 1 1 1 4
#> 3 3 1 1 1 2
#> 4 4 0 0 1 4
#> 5 5 0 1 0 1
#> 6 6 1 1 1 2
#> 7 7 0 0 1 2
#> 8 8 1 0 0 1
#> 9 9 1 1 1 1
#> 10 10 0 1 0 3
#> # ℹ 990 more rowsYou can fit an intercepts-only meta-d’ model using
fit_metad:
library(hmetad)
m <- fit_metad(N ~ 1,
data = d,
prior = prior(normal(0, 1), class = Intercept) +
set_prior(
"normal(0, 1)",
class = c(
"dprime", "c",
"metac2zero1diff", "metac2zero2diff", "metac2zero3diff",
"metac2one1diff", "metac2one2diff", "metac2one3diff"
)
)
)#> Family: metad__4__normal__absolute__multinomial
#> Links: mu = log
#> Formula: N ~ 1
#> Data: data.aggregated (Number of observations: 1)
#> Draws: 4 chains, each with iter = 2000; warmup = 1000; thin = 1;
#> total post-warmup draws = 4000
#>
#> Regression Coefficients:
#> Estimate Est.Error l-95% CI u-95% CI Rhat Bulk_ESS Tail_ESS
#> Intercept -0.04 0.14 -0.35 0.23 1.00 3115 3092
#>
#> Further Distributional Parameters:
#> Estimate Est.Error l-95% CI u-95% CI Rhat Bulk_ESS Tail_ESS
#> dprime 1.05 0.08 0.90 1.22 1.00 4058 2832
#> c -0.04 0.04 -0.12 0.04 1.00 3950 3312
#> metac2zero1diff 0.47 0.03 0.41 0.54 1.00 5121 3320
#> metac2zero2diff 0.49 0.04 0.42 0.57 1.00 5262 2913
#> metac2zero3diff 0.56 0.05 0.47 0.66 1.00 5578 3139
#> metac2one1diff 0.50 0.04 0.44 0.57 1.00 4059 3253
#> metac2one2diff 0.52 0.04 0.45 0.60 1.00 5753 3401
#> metac2one3diff 0.46 0.04 0.37 0.55 1.00 5955 2555
#>
#> Draws were sampled using sampling(NUTS). For each parameter, Bulk_ESS
#> and Tail_ESS are effective sample size measures, and Rhat is the potential
#> scale reduction factor on split chains (at convergence, Rhat = 1).Now let’s say you have a more complicated design, such as a within-participant manipulation:
#> # A tibble: 5,000 × 7
#> # Groups: participant, condition [50]
#> participant condition trial stimulus response correct confidence
#> <int> <int> <int> <int> <int> <int> <int>
#> 1 1 1 1 0 0 1 4
#> 2 1 1 2 1 1 1 4
#> 3 1 1 3 0 0 1 3
#> 4 1 1 4 0 1 0 3
#> 5 1 1 5 0 0 1 3
#> 6 1 1 6 0 0 1 4
#> 7 1 1 7 0 0 1 4
#> 8 1 1 8 0 0 1 1
#> 9 1 1 9 0 1 0 1
#> 10 1 1 10 1 0 0 3
#> # ℹ 4,990 more rowsTo account for the repeated measures in this design, you can simply adjust the formula to include participant-level effects:
m <- fit_metad(
bf(
N ~ condition + (condition | participant),
dprime + c +
metac2zero1diff + metac2zero2diff + metac2zero3diff +
metac2one1diff + metac2one2diff + metac2one3diff ~
condition + (condition | participant)
),
data = d, init = "0",
prior = prior(normal(0, 1)) +
set_prior("normal(0, 1)",
dpar = c(
"dprime", "c",
"metac2zero1diff", "metac2zero2diff", "metac2zero3diff",
"metac2one1diff", "metac2one2diff", "metac2one3diff"
)
)
)#> Family: metad__4__normal__absolute__multinomial
#> Links: mu = log; dprime = identity; c = identity; metac2zero1diff = log; metac2zero2diff = log; metac2zero3diff = log; metac2one1diff = log; metac2one2diff = log; metac2one3diff = log
#> Formula: N ~ condition + (condition | participant)
#> dprime ~ condition + (condition | participant)
#> c ~ condition + (condition | participant)
#> metac2zero1diff ~ condition + (condition | participant)
#> metac2zero2diff ~ condition + (condition | participant)
#> metac2zero3diff ~ condition + (condition | participant)
#> metac2one1diff ~ condition + (condition | participant)
#> metac2one2diff ~ condition + (condition | participant)
#> metac2one3diff ~ condition + (condition | participant)
#> Data: data.aggregated (Number of observations: 50)
#> Draws: 4 chains, each with iter = 2000; warmup = 1000; thin = 1;
#> total post-warmup draws = 4000
#>
#> Multilevel Hyperparameters:
#> ~participant (Number of levels: 25)
#> Estimate Est.Error
#> sd(Intercept) 0.49 0.52
#> sd(condition) 0.31 0.32
#> sd(dprime_Intercept) 1.22 0.22
#> sd(dprime_condition) 0.75 0.14
#> sd(c_Intercept) 1.16 0.17
#> sd(c_condition) 0.74 0.11
#> sd(metac2zero1diff_Intercept) 0.11 0.10
#> sd(metac2zero1diff_condition) 0.07 0.06
#> sd(metac2zero2diff_Intercept) 0.11 0.11
#> sd(metac2zero2diff_condition) 0.07 0.07
#> sd(metac2zero3diff_Intercept) 0.10 0.09
#> sd(metac2zero3diff_condition) 0.07 0.06
#> sd(metac2one1diff_Intercept) 0.10 0.10
#> sd(metac2one1diff_condition) 0.07 0.08
#> sd(metac2one2diff_Intercept) 0.14 0.11
#> sd(metac2one2diff_condition) 0.11 0.08
#> sd(metac2one3diff_Intercept) 0.13 0.12
#> sd(metac2one3diff_condition) 0.07 0.06
#> cor(Intercept,condition) -0.43 0.57
#> cor(dprime_Intercept,dprime_condition) -0.94 0.03
#> cor(c_Intercept,c_condition) -0.94 0.03
#> cor(metac2zero1diff_Intercept,metac2zero1diff_condition) -0.41 0.56
#> cor(metac2zero2diff_Intercept,metac2zero2diff_condition) -0.34 0.56
#> cor(metac2zero3diff_Intercept,metac2zero3diff_condition) -0.37 0.56
#> cor(metac2one1diff_Intercept,metac2one1diff_condition) -0.42 0.54
#> cor(metac2one2diff_Intercept,metac2one2diff_condition) -0.32 0.55
#> cor(metac2one3diff_Intercept,metac2one3diff_condition) -0.21 0.67
#> l-95% CI u-95% CI Rhat
#> sd(Intercept) 0.02 2.02 1.18
#> sd(condition) 0.01 1.26 1.19
#> sd(dprime_Intercept) 0.89 1.74 1.02
#> sd(dprime_condition) 0.56 1.07 1.06
#> sd(c_Intercept) 0.88 1.52 1.04
#> sd(c_condition) 0.55 0.97 1.04
#> sd(metac2zero1diff_Intercept) 0.00 0.35 1.05
#> sd(metac2zero1diff_condition) 0.00 0.24 1.02
#> sd(metac2zero2diff_Intercept) 0.00 0.42 1.10
#> sd(metac2zero2diff_condition) 0.00 0.27 1.05
#> sd(metac2zero3diff_Intercept) 0.01 0.35 1.02
#> sd(metac2zero3diff_condition) 0.00 0.24 1.02
#> sd(metac2one1diff_Intercept) 0.00 0.41 1.01
#> sd(metac2one1diff_condition) 0.00 0.29 1.04
#> sd(metac2one2diff_Intercept) 0.01 0.41 1.07
#> sd(metac2one2diff_condition) 0.01 0.30 1.04
#> sd(metac2one3diff_Intercept) 0.00 0.45 1.02
#> sd(metac2one3diff_condition) 0.00 0.24 1.03
#> cor(Intercept,condition) -1.00 0.88 1.17
#> cor(dprime_Intercept,dprime_condition) -0.98 -0.87 1.02
#> cor(c_Intercept,c_condition) -0.98 -0.88 1.11
#> cor(metac2zero1diff_Intercept,metac2zero1diff_condition) -0.99 0.86 1.03
#> cor(metac2zero2diff_Intercept,metac2zero2diff_condition) -0.99 0.88 1.03
#> cor(metac2zero3diff_Intercept,metac2zero3diff_condition) -0.99 0.87 1.02
#> cor(metac2one1diff_Intercept,metac2one1diff_condition) -0.99 0.85 1.03
#> cor(metac2one2diff_Intercept,metac2one2diff_condition) -0.98 0.85 1.09
#> cor(metac2one3diff_Intercept,metac2one3diff_condition) -0.99 0.95 1.13
#> Bulk_ESS Tail_ESS
#> sd(Intercept) 16 11
#> sd(condition) 15 13
#> sd(dprime_Intercept) 401 1149
#> sd(dprime_condition) 56 591
#> sd(c_Intercept) 115 689
#> sd(c_condition) 146 575
#> sd(metac2zero1diff_Intercept) 56 1006
#> sd(metac2zero1diff_condition) 235 896
#> sd(metac2zero2diff_Intercept) 28 283
#> sd(metac2zero2diff_condition) 769 1288
#> sd(metac2zero3diff_Intercept) 510 1522
#> sd(metac2zero3diff_condition) 728 1099
#> sd(metac2one1diff_Intercept) 478 1206
#> sd(metac2one1diff_condition) 128 550
#> sd(metac2one2diff_Intercept) 1167 1470
#> sd(metac2one2diff_condition) 465 791
#> sd(metac2one3diff_Intercept) 535 1083
#> sd(metac2one3diff_condition) 599 1076
#> cor(Intercept,condition) 17 30
#> cor(dprime_Intercept,dprime_condition) 395 1404
#> cor(c_Intercept,c_condition) 26 73
#> cor(metac2zero1diff_Intercept,metac2zero1diff_condition) 119 1441
#> cor(metac2zero2diff_Intercept,metac2zero2diff_condition) 143 1646
#> cor(metac2zero3diff_Intercept,metac2zero3diff_condition) 828 1474
#> cor(metac2one1diff_Intercept,metac2one1diff_condition) 657 1283
#> cor(metac2one2diff_Intercept,metac2one2diff_condition) 607 1508
#> cor(metac2one3diff_Intercept,metac2one3diff_condition) 22 77
#>
#> Regression Coefficients:
#> Estimate Est.Error l-95% CI u-95% CI Rhat Bulk_ESS
#> Intercept 0.48 0.31 -0.14 1.02 1.14 19
#> dprime_Intercept 0.33 0.26 -0.19 0.86 1.06 398
#> c_Intercept -0.10 0.23 -0.56 0.35 1.03 230
#> metac2zero1diff_Intercept -1.02 0.12 -1.27 -0.79 1.02 243
#> metac2zero2diff_Intercept -1.04 0.14 -1.30 -0.77 1.02 219
#> metac2zero3diff_Intercept -1.09 0.15 -1.38 -0.82 1.03 126
#> metac2one1diff_Intercept -1.07 0.13 -1.33 -0.84 1.02 167
#> metac2one2diff_Intercept -0.90 0.13 -1.16 -0.65 1.05 2504
#> metac2one3diff_Intercept -0.95 0.14 -1.21 -0.68 1.01 343
#> condition -0.27 0.17 -0.60 0.03 1.11 24
#> dprime_condition 0.38 0.17 0.06 0.71 1.03 175
#> c_condition 0.10 0.15 -0.19 0.41 1.04 194
#> metac2zero1diff_condition 0.06 0.08 -0.09 0.21 1.02 179
#> metac2zero2diff_condition 0.01 0.09 -0.17 0.17 1.04 82
#> metac2zero3diff_condition 0.01 0.09 -0.17 0.19 1.01 945
#> metac2one1diff_condition 0.01 0.08 -0.15 0.17 1.01 623
#> metac2one2diff_condition -0.09 0.08 -0.26 0.07 1.05 2112
#> metac2one3diff_condition -0.03 0.09 -0.20 0.14 1.04 1436
#> Tail_ESS
#> Intercept 19
#> dprime_Intercept 734
#> c_Intercept 601
#> metac2zero1diff_Intercept 1465
#> metac2zero2diff_Intercept 2002
#> metac2zero3diff_Intercept 2022
#> metac2one1diff_Intercept 1872
#> metac2one2diff_Intercept 2062
#> metac2one3diff_Intercept 2198
#> condition 46
#> dprime_condition 682
#> c_condition 604
#> metac2zero1diff_condition 1865
#> metac2zero2diff_condition 1828
#> metac2zero3diff_condition 1569
#> metac2one1diff_condition 1991
#> metac2one2diff_condition 2074
#> metac2one3diff_condition 1557
#>
#> Draws were sampled using sampling(NUTS). For each parameter, Bulk_ESS
#> and Tail_ESS are effective sample size measures, and Rhat is the potential
#> scale reduction factor on split chains (at convergence, Rhat = 1).