+86 13541016684Mon. - Fri. 10:00-22:00

Forcefully Aborting a ElasticBeanStalk Deployment

Forcefully Aborting a ElasticBeanStalk Deployment

Forcefully Aborting a ElasticBeanStalk Deployment

Are you impatient? Do you want to get your code deployed immediately and skipped Code Review and Quality Assurance testing? If the answer is yes and you use Elastic Bean Stalk on AWS then this little tip will save you loads of time. Lets Forcefully abort a deployment to ElasticBeanStalk.

We are currently in the development phase of a new prototype portal where I am designing and building a RESTful API that is housed on EBS, and since I am the sole DevOps, Developer, Cloud Engineer / everything else, I don’t have a lot of time for waiting for eb abort to work its wonders.

If you deploy using eb deploy and notice after the fact that your service has moved from an Ok to Severe state but don’t want to wait the extra couple minutes to be able to deploy a fixed version do the following:

Have access to your ElasticBeanStalk Instance

If you don’t have access then you’ll just have to wait for the abort command to finish through; otherwise ssh into the instance.

  1. ssh beanstalk.instance l ec2user i .ssh/SomeKey.pem

Find the deployment process

  1. # ps auwxf | grep appdeploy -B3
  2. root 28610 0.0 0.0 158352 3100 pts/0 S 14:37 0:00 \_ su
  3. root 28611 0.0 0.0 115324 3464 pts/0 S 14:37 0:00 \_ bash
  4. root 7807 0.0 0.0 117292 2488 pts/0 R+ 19:03 0:00 \_ ps auwxf
  5. root 7808 0.0 0.0 110460 2160 pts/0 S+ 19:03 0:00 \_ grep color=auto appdeploy B3
  6. root 2566 0.2 0.6 599320 26372 ? Ssl 2015 122:15 python /opt/aws/bin/cfnhup
  7. root 7640 0.0 0.0 115184 3076 ? S 19:02 0:00 \_ /bin/bash e /opt/elasticbeanstalk/bin/commandprocessor
  8. root 7641 2.4 0.6 230300 26600 ? Sl 19:02 0:00 \_ /opt/elasticbeanstalk/lib/ruby/bin/ruby /opt/elasticbeanstalk/lib/ruby/bin/commandprocessor
  9. root 7790 1.5 0.3 203748 13824 ? S 19:03 0:00 \_ python /opt/elasticbeanstalk/hooks/appdeploy/enact/01flip.py

Kill it with fire!

  1. kill 9 7790 # This is the 01flip.py script

Fair warning: There were no negative consequences outside of logging that the process had failed (See Next); but I don’t know what this process is doing. I assume since its called 01flip.py that it is taking /opt/python/current and flipping it w/ the object in S3. But alas I don’t have time to look at the source.

Events on AWS

  1. INFO: Environment update is starting.
  2. INFO: Deploying new version to instance(s).
  3. INFO: Environment health has transitioned from Ok to Info. Command is executing on all instances.
  4. WARN: User initiated abort was received, however the current step of the operation in progress is not cancellable. The current operation will be aborted as soon as the noncancellable step(s) complete.
  5. WARN: Environment health has transitioned from Info to Severe. 100.0 % of the requests are failing with HTTP 5xx. Insufficient request rate (24.0 requests/min) to determine application health. 25.0 % of the requests to the ELB are failing with HTTP 5xx. Insufficient request rate (2.0 requests/min) to determine application health (6 minutes ago). Command is executing on all instances. ELB health is failing or not available for all instances.
  6. ERROR: [Instance: i0f4b4ab9] Command failed on instance. Return code: null Output: httpd: stopped
  7. httpd: started
  8. httpd RUNNING pid 7444, uptime 0:00:03.
  9. Hook /opt/elasticbeanstalk/hooks/appdeploy/enact/01flip.py failed. For more detail, check /var/log/ebactivity.log using console or EB CLI.
  10. INFO: Command execution completed on all instances. Summary: [Successful: 0, Failed: 1].
  11. WARN: User initiated abort was received. Canceling the current operation.
  12. ERROR: Failed to deploy application.
  13. ERROR: During an aborted deployment, some instances may have deployed the new application version. To ensure all instances are running the same version, redeploy the appropriate application version.
  14. Events (safe to Ctrl+C)

Deploy your code back to EBS

  1. $ eb deploy
  2. Creating application version archive “app-48b2-160201_131124″.
  3. Uploading MyAPI/app48b2160201_131124.zip to S3. This may take a while.
  4. Upload Complete.
  5. INFO: Environment update is starting.
  6. INFO: Deploying new version to instance(s).
  7. INFO: Environment health has transitioned from Ok to Info. Command is executing on all instances.
  8. INFO: New application version was deployed to running EC2 instances.
  9. INFO: Environment update completed successfully.