Created
September 8, 2021 18:30
-
-
Save mGalarnyk/a972f41c4c150f22a8cf0bb71d445d75 to your computer and use it in GitHub Desktop.
Gist originally from Introducing Ray Lightning: Multi-node PyTorch Lightning Training made easy: https://www.anyscale.com/blog/introducing-ray-lightning-multi-node-gpu-training-for-pytorch-lightning-made
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
cluster_name: ml | |
# Cloud-provider specific configuration. | |
provider: | |
type: aws | |
region: us-west-2 | |
availability_zone: us-west-2a,us-west-2b | |
# How Ray will authenticate with newly launched nodes. | |
auth: | |
ssh_user: ubuntu | |
head_node: | |
InstanceType: p3.8xlarge | |
ImageId: latest_dlami | |
# You can provision additional disk space with a conf as follows | |
BlockDeviceMappings: | |
- DeviceName: /dev/sda1 | |
Ebs: | |
VolumeSize: 100 | |
worker_nodes: | |
InstanceType: p3.2xlarge | |
ImageId: latest_dlami | |
file_mounts: { | |
"/path1/on/remote/machine": "/path1/on/local/machine", | |
} | |
# List of shell commands to run to set up nodes. | |
setup_commands: | |
- pip install -U ray-lightning |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment