ab_test.R in R

Simulate an A/B test and report the effect with a t-test.

#!/usr/bin/env Rscript
# Simulate an A/B test and report the effect with a t-test.

set.seed(2026)

simulate <- function(n, mean_a, mean_b, sd = 15) {
  list(
    control = rnorm(n, mean = mean_a, sd = sd),
    variant = rnorm(n, mean = mean_b, sd = sd)
  )
}

report <- function(groups) {
  test <- t.test(groups$variant, groups$control)
  lift <- mean(groups$variant) - mean(groups$control)
  cat("control mean:", round(mean(groups$control), 2), "\n")
  cat("variant mean:", round(mean(groups$variant), 2), "\n")
  cat("lift:        ", round(lift, 2), "\n")
  cat("95% CI:      ", round(test$conf.int, 2), "\n")
  cat("p-value:     ", format.pval(test$p.value, digits = 3), "\n")
  if (test$p.value < 0.05) {
    cat("verdict: significant at alpha = 0.05\n")
  } else {
    cat("verdict: not significant\n")
  }
}

args <- commandArgs(trailingOnly = TRUE)
n <- if (length(args) > 0) as.integer(args[1]) else 200
report(simulate(n, mean_a = 100, mean_b = 106))

How it works

1. rnorm generates control and variant samples.
2. t.test gives the interval and p-value.
3. The verdict prints against alpha = 0.05.

Keywords and builtins used here

• cat
• commandArgs
• else
• function
• if
• length
• list
• mean
• report
• rnorm
• round
• simulate

The run, in numbers

Lines

30

Characters to type

921

Tokens

246

Three-star pace

105 tpm

At the three-star pace of 105 tokens a minute, this run takes about 141 seconds.

Type this snippet

Step 2 of 2 in Encore, step 11 of 11 in Statistics.

← Previous