Skip to content

Instantly share code, notes, and snippets.

@hiattp
Last active March 6, 2020 09:14
Show Gist options
  • Save hiattp/f2b8eca4a160c5f266e3 to your computer and use it in GitHub Desktop.
Save hiattp/f2b8eca4a160c5f266e3 to your computer and use it in GitHub Desktop.
Sidekiq Config for Elastic Beanstalk
# Sidekiq interaction and startup script
files:
"/opt/elasticbeanstalk/hooks/appdeploy/post/50_restart_sidekiq.sh":
mode: "000755"
content: |
#!/bin/bash
. /opt/elasticbeanstalk/hooks/common.sh
. /opt/elasticbeanstalk/support/envvars
set -xe
EB_SCRIPT_DIR=$(/opt/elasticbeanstalk/bin/get-config container -k script_dir)
EB_CONFIG_APP_CURRENT=$(/opt/elasticbeanstalk/bin/get-config container -k app_deploy_dir)
EB_CONFIG_APP_LOGS=$(/opt/elasticbeanstalk/bin/get-config container -k app_log_dir)
EB_CONFIG_APP_PIDS=$(/opt/elasticbeanstalk/bin/get-config container -k app_pid_dir)
. $EB_SCRIPT_DIR/use-app-ruby.sh
BUNDLE=`which bundle`
SIDEKIQ=`which sidekiq`
cd $EB_CONFIG_APP_CURRENT
# For 2 instances of Sidekiq this can be increased or decreased,
# just make sure it is updated here and below. Also, if decreasing
# these values, be sure to terminate processes at previously available
# indices (e.g. kill sidekiq-3.pid if going from 3 to 2 processes).
# for i in `seq 1 2`
for i in `seq 1`
do
PIDFILE=$EB_CONFIG_APP_PIDS/sidekiq-$i.pid
# Stop current Sidekiq processes
if [ -f $PIDFILE ]
then
SIDEKIQ_LIVES=$(/bin/ps -o pid= -p `cat $PIDFILE`)
if [ -z $SIDEKIQ_LIVES ]
then
rm -rf $PIDFILE
else
kill -TERM `cat $PIDFILE`
fi
fi
# Boot Sidekiq process
$BUNDLE exec $SIDEKIQ \
-e production \
-P $PIDFILE \
-C $EB_CONFIG_APP_CURRENT/config/sidekiq.yml \
-L $EB_CONFIG_APP_LOGS/sidekiq.log \
-d
done
"/opt/elasticbeanstalk/hooks/appdeploy/pre/03_quiet_sidekiq.sh":
mode: "000755"
content: |
#!/bin/bash
. /opt/elasticbeanstalk/support/envvars
EB_CONFIG_APP_PIDS=$(/opt/elasticbeanstalk/bin/get-config container -k app_pid_dir)
# For 2 instances of Sidekiq this can be increased or decreased,
# just make sure it is updated here and below.
# for i in `seq 1 2`
for i in `seq 1`
do
PIDFILE=$EB_CONFIG_APP_PIDS/sidekiq-$i.pid
# quiet any running instance
if [ -f $PIDFILE ]
then
SIDEKIQ_LIVES=$(/bin/ps -o pid= -p `cat $PIDFILE`)
if [ -z $SIDEKIQ_LIVES ]
then
rm -rf $PIDFILE
else
kill -USR1 `cat $PIDFILE`
fi
fi
done
@syntaxTerr0r
Copy link

Thank for this! Any idea why i'm getting this during an eb deploy:

[2015-10-13T14:13:56.938Z] INFO  [30659] - [Application update/AppDeployStage1/AppDeployPostHook/50_restart_sidekiq] : Activity execution failed, because: bundler: command not found: sidekiq
  Install missing gem executables with `bundle install` (ElasticBeanstalk::ExternalInvocationError)
caused by: bundler: command not found: sidekiq
  Install missing gem executables with `bundle install` (Executor::NonZeroExitStatus)

But if I manually try to eb ssh then cd /var/app/current && bundle exec sidekiq sidekiq is starting well?

EDIT:
The question is: Why does the startup script can't play with installed gems?

Thanks for any help.

@rmiadaira
Copy link

i had the same issue, any solution?

@phthhieu
Copy link

Hmm, the same with me :( any help guys?

@3lackRos3
Copy link

Did anyone find a fix for this?

@heygambo
Copy link

I'm new to AWS Beanstalk and looking for a way how to autoscale sidekiq workers with beanstalk.

3 Questions:

  1. Does this setup run sidekiq on the app instance or on a seperated sidekiq environment?
  2. Is it possible to autoscale this?
  3. Is the problem from above solved?

Best, Gambo

@dharmdip
Copy link

dharmdip commented Sep 27, 2016

Problem is EB is not checking stack ruby version and complains bundle with this error

Activity execution failed, because: bundler: command not found: sidekiq Install missing gem executables withbundle install(ElasticBeanstalk::ExternalInvocationError) caused by: bundler: command not found: sidekiq Install missing gem executables withbundle install(Executor::NonZeroExitStatus)

need to tell ruby version where gems are installed by adding this line or manage by environment variables.

. /opt/elasticbeanstalk/support/scripts/use-app-ruby.sh

Hope this solves issue.

@fagiani
Copy link

fagiani commented Aug 23, 2018

@dharmdip I see that very line on the script above and it breaks anyway. with set -xe it is possible to see on the logs that it's actually setting the right ruby version. I am also facing this issue recently as it didn't use to happen.

Any other clues? Thanks!

@fagiani
Copy link

fagiani commented Aug 24, 2018

@syntaxTerr0r @rmiadaira @phthhieu @3lackRos3 After some struggle I've come to conclusion that this issue was caused because of a bad state on the AWS Elastic Beanstalk environment.

In my case it started to happen after I modified the instance type for the environment and it had to recreate all EC2 machines. It also happened when I cloned the environment. So the solution for this was recreate a fresh environment, set the variables and redeploy the code and it got back working.

Hope that helps you too!

@jamesst20
Copy link

jamesst20 commented Oct 8, 2019

Sidekiq 6.0 breaking changes
The script 50_restart_sidekiq.sh will now run forever :

[root@ip-172-31-9-59 post]# ./50_restart_sidekiq.sh
starting sidekiq
`/home/webapp` is not a directory.
Bundler will use `/tmp/bundler/home/ec2-user' as your home directory temporarily.
ERROR: PID file creation was removed in Sidekiq 6.0, please use a proper process supervisor to start and manage your services
ERROR: Logfile redirection was removed in Sidekiq 6.0, Sidekiq will only log to STDOUT
ERROR: Daemonization mode was removed in Sidekiq 6.0, please use a proper process supervisor to start and manage your services
2019-10-08T18:15:49.379Z pid=29234 tid=gpaiypvxa INFO: Running in ruby 2.5.6p201 (2019-08-28 revision 67796) [x86_64-linux]
2019-10-08T18:15:49.379Z pid=29234 tid=gpaiypvxa INFO: See LICENSE and the LGPL-3.0 for licensing details.
2019-10-08T18:15:49.379Z pid=29234 tid=gpaiypvxa INFO: Upgrade to Sidekiq Pro for more features and support: http://sidekiq.org
2019-10-08T18:15:49.379Z pid=29234 tid=gpaiypvxa INFO: Booting Sidekiq 6.0.0 with redis options {:id=>"Sidekiq-server-PID-29234", :url=>nil}

This will hang your deploys : Daemonization mode was removed in Sidekiq 6.0, please use a proper process supervisor to start and manage your services

@degendra
Copy link

degendra commented Mar 6, 2020

Sidekiq 5.0+
Note: The quiet signal used to be USR1 but was changed to TSTP in Sidekiq 5.0.
TSTP should be used instead of "USR1" on L79

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment