Counting in parallel in Rust

Merging per-thread maps beats fighting over one shared lock.

use rayon::prelude::*;
use std::collections::HashMap;

fn main() {
    let words: Vec<&str> = "the quick brown fox the lazy dog the end"
        .split_whitespace()
        .collect();

    let counts = words
        .par_iter()
        .copied()
        .fold(HashMap::new, |mut acc: HashMap<&str, u32>, word| {
            *acc.entry(word).or_insert(0) += 1;
            acc
        })
        .reduce(HashMap::new, |mut a, b| {
            for (word, n) in b {
                *a.entry(word).or_insert(0) += n;
            }
            a
        });

    let mut pairs: Vec<(&&str, &u32)> = counts.iter().collect();
    pairs.sort_by(|a, b| b.1.cmp(a.1).then(a.0.cmp(b.0)));
    println!("{:?}", &pairs[..3]);
}

How it works

1. fold builds a map per thread with no contention.
2. reduce merges the partial maps at the end.
3. A single Mutex<HashMap> would serialize the whole run.

Keywords and builtins used here

• HashMap
• Vec
• fn
• for
• in
• let
• main
• mut
• str
• u32
• use

The run, in numbers

Lines

26

Characters to type

555

Tokens

212

Three-star pace

110 tpm

At the three-star pace of 110 tokens a minute, this run takes about 116 seconds.

Type this snippet

Step 2 of 2 in Reductions, step 5 of 9 in Data parallelism with rayon.

← Previous Next →