behavioral-data-analysis

Overview

behavioral-data-analysis covers the practical statistics of human performance data: reaction times, accuracy, choices, and ratings from psychology experiments. It focuses on the preprocessing decisions that make or break RT analyses (trimming, transformations), the mixed-effects models that have replaced repeated-measures ANOVA, and robust inference for the skewed, outlier-prone data behavioral science produces.

When to use

• Preprocessing RT data: outlier handling, trimming vs transformations, speed-accuracy trade-offs.
• Accuracy/proportion data: logistic mixed models instead of ANOVA on percentages.
• Repeated-measures designs: linear mixed-effects models with crossed random effects.
• Choice data: logistic regression, drift-diffusion modeling basics.
• Rating scale data: ordinal models vs treating Likert as continuous.
• Power and inference: contrasts, emmeans, multiplicity control.
• Reproducible analysis pipelines for behavioral datasets.

Core concepts

• RT distributions are skewed. Reaction times are never normal — they're ex-Gaussian-ish with a long right tail. Options: log or inverse transform, or model directly (GLMM with Gamma/inverse- Gaussian family, or ex-Gaussian models). Don't just trim 2 SD and run ANOVA and hope.
• Outlier handling, preregistered. Common rules: drop RTs <200 ms (anticipations) and >2-3 s or >3 SD from the participant's condition mean; or use robust methods. Whatever the rule, decide before analysis and report how many trials were lost. Never tune the cutoff to the result.
• Speed-accuracy trade-off. Faster responses are often less accurate. Analyze RT and accuracy together; a "faster" condition with more errors may show nothing at all. Consider inverse efficiency or, better, drift-diffusion modeling to separate drift rate from caution.
• Mixed-effects models. RT ~ condition + (1 + condition | subject) + (1 + condition | item) — crossed random effects for subjects and items handle the repeated-measures structure that ANOVA mangles. Maximal random-effects structure justified by the design (Barr et al.); simplify only on convergence grounds, transparently.
• Accuracy needs logistic models. glmer(accuracy ~ condition + (1|subject), family=binomial). ANOVA on proportions violates every assumption (bounded, heteroscedastic). Report odds ratios or predicted probabilities, not just p-values.
• Contrasts, not omnibus tests. A significant 3-level ANOVA tells you almost nothing. Specify planned contrasts (treatment vs control, linear trend) that test the hypothesis directly.
• Ordinal data. Likert ratings are ordinal — consider cumulative link mixed models (ordinal::clmm) rather than pretending 1-7 is interval. In practice linear models are often robust, but check with an ordinal model as sensitivity analysis.
• Multiplicity. Families of post-hoc comparisons need correction (Tukey, Holm, FDR). Exploratory model fishing across transformations and subsets is p-hacking — preregister the pipeline.

Practical workflow

1. Audit. Trial counts per cell, missing data, impossible values, participant-level summaries. Plot RT distributions per condition before any modeling.
2. Preprocess (preregistered). Apply exclusion rules; transform or choose model family based on the distribution you see in pilot/blind data.
3. Model. LMM for RT (transformed or Gamma GLMM), binomial GLMM for accuracy, CLMM for ratings. Maximal random effects; check convergence and singularity.
4. Check. Residual plots, random-effect distributions, influence diagnostics. Refit without influential participants as sensitivity analysis.
5. Inference. Planned contrasts with CIs (emmeans); report effect sizes in original units (ms differences, percentage points) alongside standardized ones.
6. Report. Exclusion counts, transformation/model family, random-effects structure, convergence notes, and the analysis script. Follow the preregistration; label deviations.

Example (R sketch):

library(lme4)
d <- subset(d, rt > 0.2 & rt < 3)                 # preregistered exclusions
m <- lmer(log(rt) ~ condition + (1 + condition | subject) + (1 | item), data = d)
m_acc <- glmer(acc ~ condition + (1 | subject) + (1 | item), data = d, family = binomial)
emmeans::emmeans(m, pairwise ~ condition)

Common pitfalls

• ANOVA on raw RTs or on accuracy proportions.
• Post-hoc outlier trimming tuned to produce significance.
• Ignoring the speed-accuracy trade-off (reporting RT effects with unreported accuracy costs).
• Random-intercepts-only models when the design needs random slopes (inflated Type I error).
• Treating participants as fixed or averaging away item variance ("language-as-fixed-effect").
• Uncorrected post-hoc t-tests after every ANOVA.
• Analyzing only correct-trial RTs without checking whether errors differ by condition.