log_digest.R in R

Functional R: parse log lines with stringr, fold them with purrr, report.

suppressPackageStartupMessages({
  library(dplyr)
  library(purrr)
  library(stringr)
  library(tibble)
  library(lubridate)
})

LINES <- c(
  "2026-07-30T09:14:02 INFO  serve: started on 8080",
  "2026-07-30T09:14:07 DEBUG db: loaded 1223 steps",
  "2026-07-30T09:15:31 WARN  ingest: snippet too long: big.R",
  "2026-07-30T09:16:02 ERROR serve: 500 on /api/results",
  "2026-07-30T09:16:03 ERROR serve: 500 on /api/results",
  "not a log line at all",
  "2026-07-30T09:17:44 INFO  serve: 200 on /healthz"
)

PATTERN <- paste0(
  "^(?<stamp>[0-9T:-]+)\\s+",
  "(?<level>DEBUG|INFO|WARN|ERROR)\\s+",
  "(?<logger>[a-z]+):\\s+(?<message>.*)$"
)

parse_line <- function(line) {
  matched <- str_match(line, PATTERN)
  if (is.na(matched[1, 1])) stop("unparsed: ", line)
  tibble(
    at = ymd_hms(matched[1, "stamp"], tz = "UTC"),
    level = matched[1, "level"],
    logger = matched[1, "logger"],
    message = matched[1, "message"]
  )
}

safe_parse <- safely(parse_line)
attempts <- map(LINES, safe_parse)
failed <- keep(attempts, ~ !is.null(.x$error))
records <- attempts |>
  map("result") |>
  compact() |>
  list_rbind()

cat(sprintf("%d lines, %d parsed, %d skipped\n",
            length(LINES), nrow(records), length(failed)))

cat("\n--- by level ---\n")
print(records |>
        count(level, sort = TRUE) |>
        mutate(share = round(n / sum(n), 3)))

cat("\n--- by logger ---\n")
print(records |>
        summarise(lines = n(),
                  errors = sum(level == "ERROR"),
                  first = min(at),
                  .by = logger) |>
        arrange(desc(errors)))

cat("\n--- repeated errors ---\n")
print(records |>
        filter(level == "ERROR") |>
        count(message, sort = TRUE) |>
        filter(n > 1))

span <- reduce(records$at, function(acc, at) {
  c(min(acc[[1]], at), max(acc[[2]], at))
}, .init = c(records$at[[1]], records$at[[1]]))
cat(sprintf("\nspan %s seconds\n",
            as.numeric(difftime(span[[2]], span[[1]], units = "secs"))))

How it works

1. safely keeps one bad line from stopping the run.
2. map, keep and list_rbind do the work without a loop.
3. The digest is built by reduce, one line at a time.

Keywords and builtins used here

• arrange
• c
• cat
• compact
• count
• desc
• difftime
• filter
• function
• if
• keep
• length
• library
• list_rbind
• map
• max
• min
• mutate
• n
• nrow
• paste0
• print
• reduce
• round
• safely
• sprintf
• stop
• str_match
• sum
• summarise
• suppressPackageStartupMessages
• tibble
• ymd_hms

The run, in numbers

Lines

70

Characters to type

1793

Tokens

419

Three-star pace

110 tpm

At the three-star pace of 110 tokens a minute, this run takes about 229 seconds.

Type this snippet

Step 1 of 1 in Encore, step 10 of 10 in Functional & text tools.

← Previous