Service Catalog Version 0.78.1

ECS Deploy Runner

View SourceFiltered Release Notes

Overview

This service deploys ECS Deploy Runner, the central component of Gruntwork Pipelines.

Gruntwork Pipelines architecture

Gruntwork Pipelines is a code framework and approach that enables you to use your preferred CI tool to set up an end-to-end pipeline for infrastructure code (Terraform) and app code packaged in multiple formats, including container images (Docker) and Amazon Machine Images (AMIs built with Packer).

Features

• Set up a Terraform and Terragrunt pipeline based on best practices
• Run deployments using Fargate or EC2 tasks on the ECS cluster
• Configure the pipeline for building Packer and Docker images and for running plan and apply operations
• Grant fine-grained permissions for running deployments with minimum necessary privileges
• Stream output from the pipeline to CloudWatch Logs
• Protect secrets needed by the pipeline using AWS Secrets Manager
• Use KMS grants to allow the ECS task containers to access shared secrets and encrypted images between accounts
• Easily upgrade Terraform versions with Terraform version management support

Under the hood, this is all implemented using Terraform modules from the Gruntwork terraform-aws-ci repo.

Learn

note

This repo is a part of the Gruntwork Service Catalog, a collection of reusable, battle-tested, production ready infrastructure code. If you’ve never used the Service Catalog before, make sure to read How to use the Gruntwork Service Catalog!

Core concepts

• For a comprehensive guide to Gruntwork Pipelines, refer to How to configure a production-grade CI-CD workflow for infrastructure code.
• For an overview of how the various parts fit together to form the complete pipeline, refer to the ECS Deploy Runner Core Concepts.
• The rest of the docs within the ecs-deploy-runner module in the terraform-aws-ci repository may also help with context.
• The ECS Deploy Runner standard configuration is a shortcut for setting up the ecs-deploy-runner module in a manner consistent with Gruntwork recommendations.

Deploy

Non-production deployment (quick start for learning)

If you just want to try this repo out for experimenting and learning, check out the following resources:

• examples/for-learning-and-testing folder: The examples/for-learning-and-testing folder contains standalone sample code optimized for learning, experimenting, and testing (but not direct production usage).

Production deployment

If you want to deploy this repo in production, check out the following resources:

• shared account ecs-deploy-runner configuration in the for-production folder: The examples/for-production folder contains sample code optimized for direct usage in production. This is code from the Gruntwork Reference Architecture, and it shows you how we build an end-to-end, integrated tech stack on top of the Gruntwork Service Catalog.

Reference

Inputs

• ami_builder_config — Configuration options for the ami-builder container of the ECS deploy runner stack. This container will be used for building AMIs in the CI/CD pipeline using packer. Set to null to disable this container.

• cloudwatch_log_group_for_ec2_kms_key_id — The ID (ARN, alias ARN, AWS ID) of a customer managed KMS Key to use for encrypting log data.

• cloudwatch_log_group_for_ec2_retention_in_days — The number of days to retain log events in the log group. Refer to https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/cloudwatch_log_group#retention_in_days for all the valid values. When null, the log events are retained forever.

• cloudwatch_log_group_for_ec2_tags — Tags to apply on the CloudWatch Log Group, encoded as a map where the keys are tag keys and values are tag values.

• container_cpu — The default CPU units for the instances that Fargate will spin up. The invoker allows users to override the CPU at run time, but this value will be used if the user provides no value for the CPU. Options here: https://docs.aws.amazon.com/AmazonECS/latest/developerguide/AWS_Fargate.html#fargate-tasks-size.

• container_default_launch_type — The default launch type of the ECS deploy runner workers. This launch type will be used if it is not overridden during invocation of the lambda function. Must be FARGATE or EC2.

• container_max_cpu — The maximum CPU units that is allowed to be specified by the user when invoking the deploy runner with the Lambda function.

• container_max_memory — The maximum memory units that is allowed to be specified by the user when invoking the deploy runner with the Lambda function.

• container_memory — The default memory units for the instances that Fargate will spin up. The invoker allows users to override the memory at run time, but this value will be used if the user provides no value for memory. Options here: https://docs.aws.amazon.com/AmazonECS/latest/developerguide/AWS_Fargate.html#fargate-tasks-size.

• docker_image_builder_config — Configuration options for the docker-image-builder container of the ECS deploy runner stack. This container will be used for building docker images in the CI/CD pipeline. Set to null to disable this container.

• docker_image_builder_hardcoded_args — Unlike hardcoded_options, this is used for hardcoded positional args and will always be passed in at the end of the args list.

• docker_image_builder_hardcoded_options — Which options and args to always pass in alongside the ones provided by the command. This is a map of option keys to args to pass in. Each arg in the list will be passed in as a separate option. This will be passed in first, before the args provided by the user in the event data.

• ec2_worker_pool_configuration — Worker configuration of a EC2 worker pool for the ECS cluster. An EC2 worker pool supports caching of Docker images, so your builds may run faster, whereas Fargate is serverless, so you have no persistent EC2 instances to manage and pay for. If null, no EC2 worker pool will be allocated and the deploy runner will be in Fargate only mode. Note that when this variable is set, this example module will automatically lookup and use the base ECS optimized AMI that AWS provides.

• iam_groups — List of AWS IAM groups that should be given access to invoke the deploy runner.

• iam_roles — List of AWS IAM roles that should be given access to invoke the deploy runner.

• iam_users — List of AWS IAM usernames that should be given access to invoke the deploy runner.

• kms_grant_opt_in_regions — Create multi-region resources in the specified regions. The best practice is to enable multi-region services in all enabled regions in your AWS account. This variable must NOT be set to null or empty. Otherwise, we won't know which regions to use and authenticate to, and may use some not enabled in your AWS account (e.g., GovCloud, China, etc). To get the list of regions enabled in your AWS account, you can use the AWS CLI: aws ec2 describe-regions.

• name — Name of this instance of the deploy runner stack. Used to namespace all resources.

• private_subnet_ids — List of IDs of private subnets that can be used for running the ECS task and Lambda function.

• shared_secrets_enabled — If true, this module will create grants for a given shared secrets KMS key. You must pass a value for shared_secrets_kms_cmk_arn if this is set to true. Defaults to false.

• shared_secrets_kms_cmk_arn — The ARN of the KMS CMK used for sharing AWS Secrets Manager secrets between accounts.

• should_create_cloudwatch_log_group_for_ec2 — When true, precreate the CloudWatch Log Group to use for log aggregation from the EC2 instances. This is useful if you wish to customize the CloudWatch Log Group with various settings such as retention periods and KMS encryption. When false, the CloudWatch agent will automatically create a basic log group to use.

• snapshot_encryption_kms_cmk_arns — Map of names to ARNs of KMS CMKs that are used to encrypt snapshots (including AMIs). This module will create the necessary KMS key grants to allow the respective deploy containers access to utilize the keys for managing the encrypted snapshots. The keys are arbitrary names that are used to identify the key.

• terraform_applier_config — Configuration options for the terraform-applier container of the ECS deploy runner stack. This container will be used for running infrastructure deployment actions (including automated variable updates) in the CI/CD pipeline with Terraform / Terragrunt. Set to null to disable this container.

• terraform_planner_config — Configuration options for the terraform-planner container of the ECS deploy runner stack. This container will be used for running infrastructure plan (including validate) actions in the CI/CD pipeline with Terraform / Terragrunt. Set to null to disable this container.

• vpc_id — ID of the VPC where the ECS task and Lambda function should run.

Outputs

• cloudwatch_log_group_name — Name of the CloudWatch Log Group used to store the log output from the Deploy Runner ECS task.

• default_ecs_task_arn — AWS ARN of the default ECS Task Definition. Can be used to trigger the ECS Task directly.

• ecs_cluster_arn — AWS ARN of the ECS Cluster that can be used to run the deploy runner task.

• ecs_task_arns — Map of AWS ARNs of the ECS Task Definition. There are four entries, one for each container in the standard config (docker-image-builder ; ami-builder ; terraform-planner ; terraform-applier).

• ecs_task_execution_role_arn — ECS Task execution role ARN

• ecs_task_families — Map of the families of the ECS Task Definition that is currently live. There are four entries, one for each container in the standard config (docker-image-builder ; ami-builder ; terraform-planner ; terraform-applier).

• ecs_task_iam_roles — Map of AWS ARNs and names of the IAM role that will be attached to the ECS task to grant it access to AWS resources. Each container will have its own IAM role. There are four entries, one for each container in the standard config (docker-image-builder ; ami-builder ; terraform-planner ; terraform-applier).

• ecs_task_revisions — Map of the current revision of the ECS Task Definition that is currently live. There are four entries, one for each container in the standard config (docker-image-builder ; ami-builder ; terraform-planner ; terraform-applier).

• invoke_policy_arn — The ARN of the IAM policy that allows access to the invoke the deploy runner.

• invoker_function_arn — AWS ARN of the invoker lambda function that can be used to invoke a deployment.

• security_group_allow_all_outbound_id — Security Group ID of the ECS task