Create a Cluster with Spack Binary Cache

package	install command	working?
openfoam	spack install --no-check-signature --cache-only openfoam	✅
gromacs	spack install --no-check-signature --cache-only gromacs	✅
ior	spack install --no-check-signature --cache-only ior	✅
osu-micro-benchmarks	spack install --no-check-signature --cache-only osu-micro-benchmarks	✅

1. Create a Cluster, I used the following config:

Important You must have s3_read_resource = arn:aws:s3:::*

[aws]
aws_region_name = ${AWS_DEFAULT_REGION}

[global]
cluster_template = default
update_check = false
sanity_check = true

[cluster default]
key_name = ${AWS_DEFAULT_REGION}
vpc_settings = public
base_os = alinux2
ebs_settings = myebs
compute_instance_type = c5.18xlarge
master_instance_type = c5n.2xlarge
cluster_type = ondemand
placement_group = DYNAMIC
placement = compute
max_queue_size = 8
initial_queue_size = 0
disable_hyperthreading = true
scheduler = slurm
s3_read_resource = arn:aws:s3:::*

[vpc public]
vpc_id = ${vpc_id}
master_subnet_id = ${master_subnet_id}
compute_subnet_id = ${compute_subnet_id}

[ebs myebs]
shared_dir = /shared
volume_type = gp2
volume_size = 20

[aliases]
ssh = ssh {CFN_USER}@{MASTER_IP} {ARGS}

2. Install Spack

sudo su 
export SPACK_ROOT=/shared/spack
mkdir -p $SPACK_ROOT
git clone https://github.com/spack/spack $SPACK_ROOT
cd $SPACK_ROOT
git checkout releases/v0.15
echo "export SPACK_ROOT=$SPACK_ROOT" > /etc/profile.d/spack.sh
echo "source $SPACK_ROOT/share/spack/setup-env.sh" >> /etc/profile.d/spack.sh
exit
source /etc/profile.d/spack.sh
sudo chown -R $USER:$USER $SPACK_ROOT

Verify the install:

$ spack -V
0.15.4

4. Add the environment

$ sudo su
$ mkdir -p $SPACK_ROOT/var/spack/environments/aws
$ cat <<EOF >> $SPACK_ROOT/var/spack/environments/aws/spack.yaml
spack:
  view: false
  concretization: separately

  config:
    install_tree: /shared/spack/opt/spack
    build_jobs: 4

    packages:
      all:
        providers:
          blas:
          - openblas
          mpi:
          - mpich
        variants: +mpi
      mpich:
        variants: ~wrapperrpath netmod=ofi
        version:
          - 3.2.1
      binutils:
        variants: +gold+headers+libiberty~nls
        version:
          - 2.33.1
      cmake:
        version:
          - 3.17.3
      openfoam:
        variants: +paraview
        version:
          - 2006
      paraview:
        variants: +qt+python3
        version:
          - 5.8.0
      qt:
          variants: +opengl
      ncurses:
          variants: +termlib
      sqlite:
          variants: +column_metadata
      hdf5:
          variants: +hl
      mesa:
        variants: swr=avx,avx2
        version:
          - 18.3.6
      llvm:
        version:
          - 6.0.1
      hwloc:
        version:
          - 1.11.11

  mirrors: { "mirror": "s3://spack-mirrors/amzn2-e4s" }
EOF
exit

5. Activate the environment

$ spack env list
aws
$ spack env activate aws
$ spack concretize
$ spack find

6. Install Python 3 and Boto3

sudo yum install -y python3
sudo pip3 install boto3

7. Install packages!

spack install --no-check-signature ior

FAQs

1. My packages are installing from source - help!

Note: patchelf always installs from source - this is because it's a spack dependency and not a package dependency.

There's 2 reasons why this may happen:

a. No access to the S3 mirror. Run:

$ aws s3 ls s3://spack-mirrors/amzn2-e4s
An error occurred (AccessDenied) when calling the ListObjectsV2 operation: Access Denied

If you see "Access Denied" then add the AmazonS3ReadOnlyAccess to your instance's iam profile, or add s3_read_resource = arn:aws:s3:::* to your cluster's config and run update.

b. Environment isn't activated

Check spack env list and make sure aws is in green, if not run spack env activate aws