Reproducing Flaky Tests Tips and Tricks

Use a slower machine

Some time-sensitive test may be easier to reproduce on slower machines.

-resource_class: large
+resource_class: medium

Repeat suspicious operations

If you suspect any production or test code, repeat it to maximize the probability of failure:

--let somethingRisky = await somethingRisky();
++for (let i = 0; i < 100; i++) {
++    await somethingRisky();
++}
++let somethingRisky = await somethingRisky();

Repeat suspicious test

Just copy paste the test multiple times or wrap it in a big loop (beware of timeouts - the reason why I just copy paste.)

Open multiple draft PRs

Test the whole CI and make sure tests do not interact with other runs by running CI multiple times on cloned branches.

git switch -c debug-flaky-1
git push origin debug-flaky-1

git switch -c debug-flaky-2
git push origin debug-flaky-2

# [...]

# Open a draft PRs

Increase parallelism and disable test-splitting

(This one is circle-ci specific.)

Circle-ci can run tests in parallel, which will increase the likelyhood of finding the error. You also need to disable test splitting if any.

my-tests:
    parallelism: 10
    steps:
    # ...