Skip to content

Instantly share code, notes, and snippets.

@pwighton

pwighton/20211021-build-samseg-atlas-for-dorit-on-nike-take4.md

Created January 27, 2022 19:31
Show Gist options
  • Save pwighton/ce55ccee1fe20abc890ba44ccbad1f3c to your computer and use it in GitHub Desktop.
Save pwighton/ce55ccee1fe20abc890ba44ccbad1f3c to your computer and use it in GitHub Desktop.
Download ZIP
Building a samseg atlas and including corpus callosum as a WM label
Raw
20211021-build-samseg-atlas-for-dorit-on-nike-take4.md

Trying to make the atlas for Dorit on Nike

  • On Nike
  • With same parameters as original samseg atlas
  • Using Koens trick stiffnes 1.0 -> 0.1 trick
  • Using edgeCollapseEncouragmentFactor.txt=1.025 on restart
  • Using full set of subjects
  1. ssh to nike
bash

export SAMSEG_ATLAS_BUILD_BINARY='/autofs/cluster/gerenuk/pwighton/samseg/install/gems/bin/kvlBuildAtlasMesh'
export ITK_GLOBAL_DEFAULT_NUMBER_OF_THREADS=12
export SAMSEG_UPSAMPLING_STEPS=4
export SAMSEG_XYZ_MESH_SIZE="6 7 6"
export SAMSEG_STIFFNESS=1.0
export SAMSEG_LOGDIR=/autofs/cluster/gerenuk/pwighton/samseg/dorit-atlas/20subs-upsample4-xyz676-stiff-1.0-trick
export SAMSEG_LAUNCHDIR=/autofs/cluster/gerenuk/pwighton/samseg/dorit-atlas/20subs-upsample4-xyz676-stiff-1.0-trick--launch
export SAMSEG_ATLAS_INFILES="\
  /autofs/cluster/gerenuk/pwighton/samseg/dorit-atlas/fsa.990317_vc876/mri/seg_edited10+xcerebral+1cc.mgz \
  /autofs/cluster/gerenuk/pwighton/samseg/dorit-atlas/fsa.990326_vc891/mri/seg_edited10+xcerebral+1cc.mgz \
  /autofs/cluster/gerenuk/pwighton/samseg/dorit-atlas/fsa.981102_vc604/mri/seg_edited10+xcerebral+1cc.mgz \
  /autofs/cluster/gerenuk/pwighton/samseg/dorit-atlas/fsa.981112_vc623/mri/seg_edited10+xcerebral+1cc.mgz \
  /autofs/cluster/gerenuk/pwighton/samseg/dorit-atlas/fsa.981204_vc660/mri/seg_edited10+xcerebral+1cc.mgz \
  /autofs/cluster/gerenuk/pwighton/samseg/dorit-atlas/fsa.990111_vc716/mri/seg_edited10+xcerebral+1cc.mgz \
  /autofs/cluster/gerenuk/pwighton/samseg/dorit-atlas/fsa.990119_vc740/mri/seg_edited10+xcerebral+1cc.mgz \
  /autofs/cluster/gerenuk/pwighton/samseg/dorit-atlas/fsa.990128_vc764/mri/seg_edited10+xcerebral+1cc.mgz \
  /autofs/cluster/gerenuk/pwighton/samseg/dorit-atlas/fsa.990210_vc792/mri/seg_edited10+xcerebral+1cc.mgz \
  /autofs/cluster/gerenuk/pwighton/samseg/dorit-atlas/fsa.990715_vc1131/mri/seg_edited10+xcerebral+1cc.mgz \
  /autofs/cluster/gerenuk/pwighton/samseg/dorit-atlas/fsa.990729_vc1168/mri/seg_edited10+xcerebral+1cc.mgz \
  /autofs/cluster/gerenuk/pwighton/samseg/dorit-atlas/fsa.990903_vc1253/mri/seg_edited10+xcerebral+1cc.mgz \
  /autofs/cluster/gerenuk/pwighton/samseg/dorit-atlas/fsa.991025_vc1379/mri/seg_edited10+xcerebral+1cc.mgz \
  /autofs/cluster/gerenuk/pwighton/samseg/dorit-atlas/fsa.991102_vc1401/mri/seg_edited10+xcerebral+1cc.mgz \
  /autofs/cluster/gerenuk/pwighton/samseg/dorit-atlas/fsa.991109_vc1420/mri/seg_edited10+xcerebral+1cc.mgz \
  /autofs/cluster/gerenuk/pwighton/samseg/dorit-atlas/fsa.991110_vc1425/mri/seg_edited10+xcerebral+1cc.mgz \
  /autofs/cluster/gerenuk/pwighton/samseg/dorit-atlas/fsa.991113_vc1439/mri/seg_edited10+xcerebral+1cc.mgz \
  /autofs/cluster/gerenuk/pwighton/samseg/dorit-atlas/fsa.991113_vc1440/mri/seg_edited10+xcerebral+1cc.mgz \
  /autofs/cluster/gerenuk/pwighton/samseg/dorit-atlas/fsa.991122_vc1479/mri/seg_edited10+xcerebral+1cc.mgz"

mkdir -p $SAMSEG_LAUNCHDIR
cd $SAMSEG_LAUNCHDIR
wget https://raw.githubusercontent.com/freesurfer/freesurfer/dev/distribution/FreeSurferColorLUT.txt

pbsubmit \
  -n 12 \
  -q nike \
  -c "$SAMSEG_ATLAS_BUILD_BINARY \
        $SAMSEG_UPSAMPLING_STEPS \
        $SAMSEG_XYZ_MESH_SIZE \
        $SAMSEG_STIFFNESS \
        $SAMSEG_LOGDIR \
        $SAMSEG_ATLAS_INFILES"

Check

qstat | grep -w pwighton jobinfo 817373 top ls -la $SAMSEG_LOGDIR

After CurrentMeshCollection0.gz had been written

  1. Kill the job
qdel 817373
  1. Copy CurrentMeshCollection0.gz to $SAMSEG_LAUNCHDIR
cp $SAMSEG_LOGDIR/CurrentMeshCollection0.gz $SAMSEG_LAUNCHDIR/explicitStartCollection.gz
  1. Set edge collapse factor to what Koen used for origin atlas
echo 1.025 > $SAMSEG_LAUNCHDIR/edgeCollapseEncouragmentFactor.txt
  1. Change env vars:
  • SAMSEG_STIFFNESS is now 0.1
  • SAMSEG_LOGDIR is now:
    • /autofs/cluster/gerenuk/pwighton/samseg/dorit-atlas/20subs-upsample4-xyz676-stiff-1.0-trick-restart
  • everything else is the same
export SAMSEG_ATLAS_BUILD_BINARY='/autofs/cluster/gerenuk/pwighton/samseg/install/gems/bin/kvlBuildAtlasMesh'
export ITK_GLOBAL_DEFAULT_NUMBER_OF_THREADS=12
export SAMSEG_UPSAMPLING_STEPS=4
export SAMSEG_XYZ_MESH_SIZE="6 7 6"
export SAMSEG_STIFFNESS=0.1
export SAMSEG_LOGDIR=/autofs/cluster/gerenuk/pwighton/samseg/dorit-atlas/20subs-upsample4-xyz676-stiff-1.0-trick-restart
export SAMSEG_LAUNCHDIR=/autofs/cluster/gerenuk/pwighton/samseg/dorit-atlas/20subs-upsample4-xyz676-stiff-1.0-trick--launch
export SAMSEG_ATLAS_INFILES="\
  /autofs/cluster/gerenuk/pwighton/samseg/dorit-atlas/fsa.990317_vc876/mri/seg_edited10+xcerebral+1cc.mgz \
  /autofs/cluster/gerenuk/pwighton/samseg/dorit-atlas/fsa.990326_vc891/mri/seg_edited10+xcerebral+1cc.mgz \
  /autofs/cluster/gerenuk/pwighton/samseg/dorit-atlas/fsa.981102_vc604/mri/seg_edited10+xcerebral+1cc.mgz \
  /autofs/cluster/gerenuk/pwighton/samseg/dorit-atlas/fsa.981112_vc623/mri/seg_edited10+xcerebral+1cc.mgz \
  /autofs/cluster/gerenuk/pwighton/samseg/dorit-atlas/fsa.981204_vc660/mri/seg_edited10+xcerebral+1cc.mgz \
  /autofs/cluster/gerenuk/pwighton/samseg/dorit-atlas/fsa.990111_vc716/mri/seg_edited10+xcerebral+1cc.mgz \
  /autofs/cluster/gerenuk/pwighton/samseg/dorit-atlas/fsa.990119_vc740/mri/seg_edited10+xcerebral+1cc.mgz \
  /autofs/cluster/gerenuk/pwighton/samseg/dorit-atlas/fsa.990128_vc764/mri/seg_edited10+xcerebral+1cc.mgz \
  /autofs/cluster/gerenuk/pwighton/samseg/dorit-atlas/fsa.990210_vc792/mri/seg_edited10+xcerebral+1cc.mgz \
  /autofs/cluster/gerenuk/pwighton/samseg/dorit-atlas/fsa.990715_vc1131/mri/seg_edited10+xcerebral+1cc.mgz \
  /autofs/cluster/gerenuk/pwighton/samseg/dorit-atlas/fsa.990729_vc1168/mri/seg_edited10+xcerebral+1cc.mgz \
  /autofs/cluster/gerenuk/pwighton/samseg/dorit-atlas/fsa.990903_vc1253/mri/seg_edited10+xcerebral+1cc.mgz \
  /autofs/cluster/gerenuk/pwighton/samseg/dorit-atlas/fsa.991025_vc1379/mri/seg_edited10+xcerebral+1cc.mgz \
  /autofs/cluster/gerenuk/pwighton/samseg/dorit-atlas/fsa.991102_vc1401/mri/seg_edited10+xcerebral+1cc.mgz \
  /autofs/cluster/gerenuk/pwighton/samseg/dorit-atlas/fsa.991109_vc1420/mri/seg_edited10+xcerebral+1cc.mgz \
  /autofs/cluster/gerenuk/pwighton/samseg/dorit-atlas/fsa.991110_vc1425/mri/seg_edited10+xcerebral+1cc.mgz \
  /autofs/cluster/gerenuk/pwighton/samseg/dorit-atlas/fsa.991113_vc1439/mri/seg_edited10+xcerebral+1cc.mgz \
  /autofs/cluster/gerenuk/pwighton/samseg/dorit-atlas/fsa.991113_vc1440/mri/seg_edited10+xcerebral+1cc.mgz \
  /autofs/cluster/gerenuk/pwighton/samseg/dorit-atlas/fsa.991122_vc1479/mri/seg_edited10+xcerebral+1cc.mgz"
  1. re-launch
cd $SAMSEG_LAUNCHDIR

Run the above pbsubmit command with -T to get the qsub command Also adding -p so the script is called from $SAMSEG_LAUNCHDIR

pbsubmit -T -p -n 12 -q nike -c "$SAMSEG_ATLAS_BUILD_BINARY $SAMSEG_UPSAMPLING_STEPS $SAMSEG_XYZ_MESH_SIZE $SAMSEG_STIFFNESS $SAMSEG_LOGDIR $SAMSEG_ATLAS_INFILES"

returns:

qsub -V -S /bin/sh   -q nike  -l nodes=1:ppn=12,vmem=84gb -r n    /pbs/pwighton/pbsjob_7

Then run pbsubmit, using -l and passing the above params plus walltime=8:0:0:0 for 8 days:

pbsubmit -p -l nodes=1:ppn=12,vmem=84gb,walltime=8:0:0:0 -q nike -c "$SAMSEG_ATLAS_BUILD_BINARY $SAMSEG_UPSAMPLING_STEPS $SAMSEG_XYZ_MESH_SIZE $SAMSEG_STIFFNESS $SAMSEG_LOGDIR $SAMSEG_ATLAS_INFILES"
  1. Inspect

qstat | grep -w pwighton jobinfo 817374 top ls -la $SAMSEG_LOGDIR

Killing this job then looking at the stdout (cat /pbs/pwighton/pbsjob_8.o817374) confirms that explicitStartCollection is being used, so re-launch

Got an explicitStartCollection, but it's stiffness does not match: 1 vs. 0.1
    difference is 0.9
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment