T-Systems-Claim-Logo
Search
Lightbulbs lying on a purple background and one of them is on

EC2 Cost Optimization through Night Time Shutdowns on AWS

Save money with our step-by-step guide. Learn how to auto-shutdown your instances. 

August 20 2021Aleksa Perovic

Identifying cost saving potential

For building and testing applications, cloud providers’ pay-as-you-go pricing models offer tremendous flexibility. But leaving applications to run 24/7 can be costly. This is especially true for larger enterprises, where the costs of applications are often not billed individually, making it harder to spot unnecessary expenditure.  As an AWS partner, we’ve identified how to manage unneeded EC2 (Amazon Elastic Compute Cloud) instances and save costs by shutting them down overnight, explained here step-by-step.  

Scheduling instances on AWS

A hand putting a coin into a jar filled with coins and a hand typing on a calculator

The AWS Solutions Library includes an Instance Scheduler: a serverless tool for automatically starting and stopping EC2 and RDS instances. You can use the tool to define multiple schedules for different use cases. For example, you could have schedules for: 

  • Applications running Monday to Friday during office hours 
  • Running instances for one hour in the evening seven days a week for analytics workloads 

In our customers’ environments, the Instance Scheduler is managed centrally, enabling applications to consume it without operational overhead. The Solution is very user friendly, as the end-user only needs to tag the instances with the appropriate schedule tag.


Diagram of the AWS Instance Scheduler

AWS Instance Scheduler

Scaling the AWS Instance Scheduler

Diagram of the AWS Instance Scheduler

The AWS Instance Scheduler is a scalable solution, making it equally suitable for large AWS Organizations with multiple accounts as well as single account environments. Large organizations using the tool require the following adjustments: 

  1. Roll out of an AWS role to all accounts, which the Instance Scheduler can assume for starting and stopping instances.
  2. The role in each account must be added to the central DynamoDB Table of the Instance Scheduler.
  3. The Lambda function – which handles the API calls for starting and stopping – must be adjusted to manage thousands of instances. This adjustment requires increasing memory allocation as well as decreasing the invoking frequency to handle the load. 

Tagging newly created EC2 instances

Now that you have adjusted the Instance Scheduler, you must tag the EC2s with the proper schedule. In organizations with hundreds of accounts, this is not straightforward. You may find that account owners do not tag their instances consistently or at all. To address this, we deployed the following automation for a client:  

  1. In each project account, an EventBridge Rule is rolled out (previously called CloudWatch Events). It is triggered when CloudTrail records the API Call "RunInstances".
  2. The event is published on the creation of an EC2 instance and forwarded to an event bus in a central account, then on to an Auto-Tag Lambda.
  3. The Lambda investigates the incoming event and filters out instances that are: 
    -    part of an Autoscaling Group
    -    created by AWS services like EKS/ECS, Beanstalk, Cloud9, etc.
    -    already tagged with a schedule
  4. For all instances that should be shut down overnight, the Lambda assumes a role in the respective project account and tags the newly created EC2 instance.

Auto-tagging of EC2 instances

Tagging existing EC2 instances 

Now that all new instances are tagged automatically, the next challenge are the existing EC2s. These are the ones really driving the costs, so they need to be included in the night-time saving. We solved this problem for our enterprise customer by informing all project account owners that we would centrally tag instances in non-production accounts on a specific date. 

On the due day, we invoked a central Lambda similar to the Auto-Tag Lambda. It assumed a role in each project account and tagged existing instances with the same filtering mechanism mentioned. An opt-out schedule was also defined and communicated beforehand to project account owners in case they needed to run their instances 24/7.

Conclusion 

The Instance Scheduler automates starting and stopping instances on AWS. With some modifications, it is also possible to use the tool in big AWS organizations to shut down instances overnight and realize measurable cost savings. This solution provides lower cost savings in environments with a high number of autoscaling groups or the previously mentioned AWS services (EKS/ECS, Beanstalk, Cloud9). With our enterprise customer example, although several Autoscaling Groups were in use, we reduced the costs of EC2 instances by 15%. 

If you enjoyed this blog post, please stay tuned for future blogs to learn about other ways to optimize your cloud costs.  

About the author
IM-Perovic-Aleksa (1)

Aleksa Perovic

Cloud Solution Architekt, T-Systems International GmbH

Show profile and articles

You might also be interested in:

Do you visit t-systems.com outside of Germany? Visit the local website for more information and offers for your country.