- After a TeamCity build is completed, the runtime log file is stored in the Artifacts section for each build.
- Each new build runs a process that issues an API call to TeamCity. This API call looks for the latest, successful build and downloads the log file. That request something like this:
HTTParty.get("#{team-city-uri}/builds/#{parameters-that-identify-a-completed-successful-build}/artifacts/content/parallel_rspec_runtime.log")
- After fetching the file, we read the file, remove any uwanted noise caused by stdout, and write the results to a new file.
- When calling
parallel_tests
, we pass the new file(1). Example:parallel_rspec --group-by runtime --runtime-log /logs/parallel_rspec_runtime.log -- path_to_spec_files
(1) Word of caution, Parallel Tests states that "when a runtime log is filled" the default group_by
strategy is runtime
. However, after digging into the source of the gem, we found that when a flag for an unrelated feature was passed, if the gem was running in default mode, the strategy would be set to “filesize”. More details here: https://github.com/grosser/parallel_tests/blob/9bc92338e2668ca4c2df81ba79a38759fcee2300/lib/parallel_tests/cli.rb#L305
To prevent any surprises and ensure you're using the runtime
strategy, it's best to be explicit and include --group-by runtime
.
The parallel_test flag --only-group
supports splitting your test across different machines. Currently, our build process runs the RSpec test suite on one machine that has ~89 CPUs. We use the --only-group
flag to assign a set of specs to each CPU. Example below:
parallel_rspec
--only-group ENV['CPU_NUMBER']
--group-by runtime
--runtime-log /logs/parallel_rspec_runtime.log
-- path_to_spec_files
Using parallel_tests
to split the test files is deterministic, so we can tell each CPU (or in the future, each machine, perhaps in your case, each "worker") which "group" of tests to run.
💛 Hope that helps!
Thanks for this. I use Travis and have been able to "share" the runtime log by using the shared cache mechanism. The issue with that though is that each worker gets a copy of the cache, updates it, then pushes it back to storage, so in essence, the last worker wins and it's cache is the last to save, overwriting updates to the runtime log of any of the other workers.
I'm looking into a new "workspaces" feature of travis where it's more like a shared drive between all current workers in a build, but it's still an alpha feature.