Skip to content

Instantly share code, notes, and snippets.

@robandpdx
Last active January 30, 2023 20:32
Show Gist options
  • Save robandpdx/aad10f5fde67bf0998e30c08a4ef2bdc to your computer and use it in GitHub Desktop.
Save robandpdx/aad10f5fde67bf0998e30c08a4ef2bdc to your computer and use it in GitHub Desktop.

PyTorch workflow testing

In order to iterate faster on workflows it is necessary to setup an environment to run the test.

  1. Launch a g5.4xlarge instance in aws with Deep Learning AMI GPU PyTorch 1.10.0 (Ubuntu 20.04) 20221003 ami-09b3fda578ad20138
  2. Generate an ssh key pair and add it to your github account.
  3. Clone pytorch/rl repo to /home/ec2-user/github/rl
  4. Use the following script to start the container and get a shell...
#!/bin/bash

docker run -it --gpus all --rm -v /home/ubuntu/github/rl:/work pytorch/conda-builder:cuda11.6 /bin/bash
  1. In the container shell, run the following...
export PYTHON_VERSION=3.8
export CUDA_ARCH_VERSION=11.6
export CU_VERSION="cu${CUDA_ARCH_VERSION:0:2}${CUDA_ARCH_VERSION:3:1}"

echo "PYTHON_VERSION: $PYTHON_VERSION"
echo "CU_VERSION: $CU_VERSION"
        
cd /work

## setup_env.sh
./.circleci/unittest/linux/scripts/setup_env.sh

## install.sh
./.circleci/unittest/linux/scripts/install.sh

## run_test.sh
./.circleci/unittest/linux/scripts/run_test.sh

## post_process.sh
./.circleci/unittest/linux/scripts/post_process.sh
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment