Stop and Start Amazon RDS instances by schedule with Terraform, Lambda and CloudWatch - Part 1

Table of Content

Introducing

As you might know, AWS billings at the end of the month can be slightly frightening, but there are many cases how to reduce the cost of using AWS infrastructure. You can configure AutoScalingGroup and shutdown the instances at night, or use SpotInstances, also you can stop/start any instance by schedule with Lambda and CloudWatch. The last one we will be used in this tutorial. In our case, we have three environments: dev, stage and prod. When we only migrated to AWS, our infrastructure was running 24/7, that was not an efficient waste of money.

Such as we use dev and stage environment only in development phases and pre deploying realise, there was no need to keeping them running up in the night. We’ve decided to stop RDS instances in the evening and run them up in the morning. I forgot about one great tool, which is named Terraform. All our infrastructure is described in Terraform configuration, so we don’t afraid forgot something during the time, and at any time we can destroy our infrastructure and spin it up.

I read a lot of article and most of them use Python as a programming language to describe Lambda function. I don’t know Python, but I know Node.js, so I decided to combine all information and implement it on Node.js. Here I’ll describe only how to scheduling RDS instance but as you will see you can implement scheduler in the same way for EC2 instances and so on.

How it works. We’ll implement RdsScheduler which will start and run out RDS instances by schedule, and for this CloudWatch in pair with Lambda function will be used.

I found this scaffolding which helped me get a sense of the concept.

In the first part, I’ll describe only Terraform configuration and in a second one, show the code of Lambda function.

Implementation

My Terraform module path is modules/scheduler. Create in this folder a file main.tf, this will entrypoint to our module.

archive_file

Terraform’s archive_file is universal data source which can be used to archive code for Lambda. There are a few ways to define the files inside the archive and also several attributes to consume the archive.

It construction is really simple. You define the data source with the zip type and an output filename that is usually a temporary file:

# define a zip archive
data "archive_file" "zip" {
    type        = "zip"
    output_path = "/${path.module}/aws-stop-start-rds.zip"
    # ... define the source
}

After it we can use the output_path for the file itself and the output_base64sha256 (or one of the similar functions) to detect changes to the contents:

# use the outputs
resource "..." {
    filename = data.archive_file.zip.output_path
    source_code_hash = data.archive_file.zip.output_base64sha256
}

Full example of our archive_file is

data "archive_file" "zip" {
  type = "zip"
  output_path = "${path.module}/aws-stop-start-rds.zip"
  source_dir  = "${path.module}/package/"
}

Take a note that it will be automatically archive all the files from ${path.module}/package/ to ${path.module}/aws-stop-start-rds.zip after running terraform apply.

aws_iam_role

Create a role that will have access to operate with our RDS instances.

data "aws_iam_policy_document" "policy" {
  statement {
    sid = ""
    effect = "Allow"

    principals {
      identifiers = ["lambda.amazonaws.com"]
      type = "Service"
    }

    actions = [
      "sts:AssumeRole",
    ]
  }
}

The AWS Security Token Service (STS) is a web service that enables you to request temporary, limited-privilege credentials for AWS Identity and Access Management (IAM) users or for users that you authenticate (federated users).

resource "aws_iam_role" "this" {
  name = "${var.name}-scheduler-lambda"
  assume_role_policy = data.aws_iam_policy_document.policy.json
}

aws_iam_role_policy

Provides an IAM role policy to. Declarate which access has our Lambda role.

Access to operate with RDS instances

data "aws_iam_policy_document" "rds_policy" {
  statement {
    sid = ""
    effect = "Allow"

    resources = ["*"]

    actions = [
      "rds:ListTagsForResource",
      "rds:DescribeDBClusters",
      "rds:StartDBCluster",
      "rds:StopDBCluster",
      "rds:DescribeDBInstances",
      "rds:StartDBInstance",
      "rds:StopDBInstance"
    ]
  }
}

resource "aws_iam_role_policy" "schedule_rds" {
  name  = "${var.name}-rds-custom-policy-scheduler"
  role  = aws_iam_role.this.id
  policy = data.aws_iam_policy_document.rds_policy.json
}

Access to operate with CloudWatch

resource "aws_iam_role_policy" "schedule_cloudwatch" {
  name  = "${var.name}-cloudwatch-custom-policy-scheduler"
  role  = aws_iam_role.this.id

  policy = <<EOF
{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Action": [
                "cloudwatch:DescribeAlarms",
                "cloudwatch:DisableAlarmActions",
                "cloudwatch:EnableAlarmActions",
                "cloudwatch:ListTagsForResource"
            ],
            "Effect": "Allow",
            "Resource": "*"
        }
    ]
}
EOF
}

Access to loggin information

locals {
  lambda_logging_policy = {
    "Version": "2012-10-17",
    "Statement": [
      {
        Action: [
          "logs:CreateLogStream",
          "logs:PutLogEvents"
        ],
        Resource: "arn:aws:logs:*:*:*",
        Effect: "Allow"
      }
    ]
  }
}

resource "aws_iam_role_policy" "lambda_logging" {
  name   = "${var.name}-lambda-logging"
  role   = aws_iam_role.this.id
  policy = jsonencode(local.lambda_logging_policy)
}

NOTE: As you can see we use the same aws_iam_role_policy construction, but here I described different way to declaration policy. Feel free use any of them.

aws_lambda_function

It’s a heart of what we’re trying to implement here.

resource "aws_lambda_function" "this" {
  function_name = var.name

  handler = "index.handler"
  role = aws_iam_role.this.arn
  runtime = "nodejs12.x"
  timeout = "600"

  filename         = data.archive_file.zip.output_path
  source_code_hash = data.archive_file.zip.output_base64sha256

  environment {
    variables = {
      AWS_REGIONS               = var.aws_regions == null ? data.aws_region.current.name : join(", ", var.aws_regions)
      SCHEDULE_ACTION           = var.schedule_action
      RESOURCES_TAGS            = jsonencode(var.resources_tags)
      RDS_SCHEDULE              = var.rds_schedule
    }
  }
}

handler – the module-name.export value in your function. For example, "index.handler" calls exports.handler in index.js.
runtime – the identifier of funtion’s runtime
environment – the Lambda environment’s configuration settings. For example, in Lambda function you can get a variable as process.env.SCHEDULE_ACTION and so on.

CloudWatch

You need to keep invoking Lambda. A simple trick you can use in AWS is to use a CloudWatch Events scheduler to regularly call your Lambda function by schedule.

# CLOUDWATCH EVENT
resource "aws_cloudwatch_event_rule" "this" {
  name                = "trigger-lambda-scheduler-${var.name}"
  description         = "Trigger lambda scheduler"
  #schedule_expression = "cron(0 07 ? * MON *)"
}

resource "aws_cloudwatch_event_target" "this" {
  arn  = aws_lambda_function.this.arn
  rule = aws_cloudwatch_event_rule.this.name
}

resource "aws_lambda_permission" "this" {
  statement_id  = "AllowExecutionFromCloudWatch"
  action        = "lambda:InvokeFunction"
  principal     = "events.amazonaws.com"
  function_name = aws_lambda_function.this.function_name
  source_arn    = aws_cloudwatch_event_rule.this.arn
}

# CLOUDWATCH LOG
resource "aws_cloudwatch_log_group" "this" {
  name              = "/aws/lambda/${var.name}"
  retention_in_days = 14
}

RDS Scheduler

I don’t want to oversupplied the article with information, so I’ll implement only "Hello, World!" example that you can play with it.

// 'Hello World' nodejs12.x runtime AWS Lambda function
exports.handler = (event, context, callback) => {
  console.log('Hello, logs!');
  callback(null, 'great success');
};

NOTE. You should call calback and pass here the API response or else valuable information that will help to go through logs.

How to test AWS Lambda function you can read in this article

Running

Lambda’s configuration is ready, add module in your Terraform configuration

module "db" {
  source  = "terraform-aws-modules/rds/aws"
  identifier = "${terraform.workspace}-${var.identifier}-db"

  #...

  tags = {
    Owner       = var.db_username
    Environment = terraform.workspace
    ToStop = true
  }

  #...
}

# RDS stop/start scheduler
module "rds-stop-nightly" {
  source                         = "modules/scheduler"
  name                           = "${terraform.workspace}-${var.name}-stop-rds"
  aws_regions                    = [var.aws_region]

  cloudwatch_schedule_expression = "cron(0 17 ? * MON-FRI *)"
  schedule_action                = "stop"

  rds_schedule                   = "true"

  resources_tags = [
    {
      key = "ToStop"
      value = "true"
    },
    {
      key = "Environment"
      value = terraform.workspace
    }]
}

module "rds-stop-daily" {
  source                         = "../modules/scheduler"
  name                           = "${terraform.workspace}-${var.name}-start-rds"
  aws_regions                    = [var.aws_region]

  cloudwatch_schedule_expression = "cron(0 07 ? * MON-FRI *)"
  schedule_action                = "start"
  rds_schedule                   = "true"

  resources_tags = [
    {
      key = "ToStop"
      value = "true"
    },
    {
      key = "Environment"
      value = terraform.workspace
    }]
}

What you should take note, it’s resources_tags. It’s how we will define which instance should be scheduling. As you can see, Lambda function and RDS instance configured with the same tags: ToStop, Environment. It’ll allow RdsScheduler to find out only instances which match the filters. This is a great thing that can help to you setup your environment granularly.

I guess you’ve already seen cloudwatch_schedule_expression option in configuration, and yes it is cron expression that we like to use in Linux for running scripts by schedule. You can read more about it on AWS article "Schedule Expressions for Rules".

All is ready to deploy Lambda to AWS

terraform init
terrafrom apply

How to create a сomplete RdsScheduler in Node.js I will describe in the next article.