Splitting Monorepo Tests into Parallel CI Shards

Your affected set is already minimal but the remaining test suite still takes too long on one runner. This page shows how to shard the suite across parallel jobs so wall-clock time drops roughly linearly with runner count.

Exact symptoms and error messages

A single test job runs the whole affected suite serially:

Test Files  212 passed (212)
     Tests  3841 passed (3841)
  Duration  14m 06s

The work is already scoped to affected packages — it is simply too much for one machine.

Root cause analysis

Once affected detection and caching have removed redundant work, the remaining cost is genuine and serial. Sharding addresses it by partitioning the test set deterministically across runners so each executes a disjoint slice. This composes with the affected filter from CI/CD Pipeline Optimization for Monorepos — shard the changed set, not the whole repo.

Sharding is the last lever precisely because it does not remove work — it only spreads it. Affected detection and caching reduce the amount of work; sharding reduces the wall-clock time of whatever work remains by running disjoint slices in parallel. Applying sharding before the other two is a common mistake: you end up parallelizing the full suite on every commit, paying N runners' startup cost to run tests that affected would have skipped entirely.

The correctness subtlety is partition determinism. A shard must always contain the same tests given the same input, or a flaky partition can hide a failure on one run and surface it on another. Framework-native sharding (Vitest's --shard, Jest's --shard) partitions deterministically by test file, which is why it is preferable to ad-hoc splitting by directory that can drift as files are added.

Resolution and configuration patch

  test:
    strategy:
      fail-fast: false
      matrix:
        shard: [1, 2, 3, 4]
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
        with: { fetch-depth: 0 }
      - run: pnpm install --frozen-lockfile --ignore-scripts
      - run: pnpm vitest run --shard=${{ matrix.shard }}/4 --filter='...[origin/main]'

Each runner executes one quarter of the affected tests; a final required check gates on all shards passing.

CLI validation and debug commands

# Run one shard locally to confirm partitioning
pnpm vitest run --shard=1/4
# Verify shards are balanced by timing each
for s in 1 2 3 4; do time pnpm vitest run --shard=$s/4; done

Prevention and CI guardrails

• Set fail-fast: false so one shard's failure does not cancel the others.
• Gate merges on a single required check that depends on every shard.
• Keep shard count proportional to suite size — over-sharding wastes runner startup time.
• Shard the affected set, never the entire repo, so small changes stay cheap.

Choosing a shard count that actually helps

More shards is not automatically faster. Each shard pays a fixed cost — checkout, install, warm-up — before it runs a single test, so past a certain point adding shards just multiplies overhead. The sweet spot is where per-shard runtime approaches that fixed cost but does not drop below it.

Measure before committing to a number. Time the full affected suite, divide by your target per-shard runtime (say, two to three minutes of actual testing), and use that as the shard count. Re-check it as the suite grows, because a count that was right at 200 tests wastes runners at 50 and under-parallelizes at 800. The honest metric is total billed runner-minutes versus wall-clock saved: sharding is worth it only while the wall-clock win justifies the extra runner-minutes the overhead costs.

Balancing shards so one isn't the bottleneck

Wall-clock time is set by the slowest shard, not the average, so an unbalanced partition wastes the parallelism you paid for. If shard 3 holds all the slow integration tests, the other shards finish early and idle while it grinds on. Framework sharding partitions by file count, which is a good first approximation but ignores per-file runtime.

For suites with a few known-slow files, either split those into their own shard deliberately or use a runner that partitions by timing data from previous runs. The check is simple: time each shard and look at the spread. A healthy split has shards finishing within a small window of each other; a large spread means the partition is uneven and the fix is to rebalance where the slow tests land, so no single shard becomes the pipeline's bottleneck.

Frequently Asked Questions

How many shards should I use?

Enough that per-shard time approaches fixed overhead (checkout + install), but not so many that startup cost dominates. Four to eight is typical; measure and adjust.

Does sharding change test isolation?

No — each shard runs a disjoint subset in its own process, so tests remain isolated. Just avoid tests that depend on shared external state, which can collide across parallel runners.

How many shards should I use?

Enough that per-shard test time approaches the fixed overhead (checkout + install), but not so many that overhead dominates. Time the affected suite, divide by a two-to-three-minute target, and re-measure as the suite grows.

Why is one shard much slower than the others?

Framework sharding splits by file count, not runtime, so a shard that happens to hold the slow integration tests becomes the bottleneck. Isolate known-slow files or use timing-based partitioning to rebalance.

Should sharding come before or after affected detection?

After. Affected and caching reduce how much work exists; sharding parallelizes what remains. Shard the affected set, never the whole repo, or you pay N runners to run tests affected would have skipped.

Related

• CI/CD Pipeline Optimization for Monorepos — the overview for fast monorepo pipelines.

← CI/CD Pipeline Optimization for Monorepos