Jesse's Software Engineering Blog
AWS CodePipeline Continuous Deployment
AWS CodePipeline offers a continuous deployment pipeline which integrates well with serverless architectures. It offers infrastructure creation via AWS CloudFormation, code integration with GitHub (or other) code repositories, building and testing with AWS CodeBuild, and deployment with AWS CloudFormation (or AWS CodeDeploy). CodePipeline allows for various stages in the pipeline which correlate to steps such as building, testing, packaging, etc. Each stage can have multiple actions which are run in parallel. Artifacts can be passed from stage to stage allowing for a mix of sequential and parallel processing. Here is a great step by step tutorial on a serverless CodePipeline setup from AWS. I also put together a simple sample repo.
Setting up a CodePipeline is pretty straight forward through the console. The tutorial above provides the complete pipeline as a CloudFormation template, which is a better approach than manually configuring the pipeline through the console. This would allow for the templating of stacks, with a goal of not needing to manually spin up any of the infrastructure.
Defining the code source allows for GitHub, AWS CodeCommit, or S3. The GitHub integration is simple. AWS will sync with your GitHub account, although there is currently only support for triggering deploys on git branches, not tags. The code source can be configured to be triggered automatically on code update, triggered via CloudWatch events or other internal AWS events (i.e. SNS), or can be triggered manually.
For the build step, there’s the option to use AWS CodeBuild or integrate with a third party tool like Jenkins. From a macro level, CodeBuild is very similar to CircleCI. You select the Docker image to build on, either a default image from AWS or your own from ECR, DockerHub, etc., and provide a buildspec.yml which has phases, or hooks, allowing for commands or scripts to be run at various points during the build process. For this write up, the build is simply generating a CloudFormation template from an AWS serverless application model. It’s important to note that the CodeUri path is relative, this is so that as part of the deployment CloudFormation will analyze the local code bundle for changes to deploy. The buildspec.yml also has an artifacts key which defines what files will be passed on to the next stage in the pipeline. In this case, the generated CloudFormation template will be used in future stages.
The deployment stage is done in two sequential actions. The first action is to create a change set. A change set represents changes to the stack, which is the collection of resources defined in the original SAM as well as any changes to the code base. The second action is to deploy the change set out. An advantage of using CloudFormation for the deployments is that we are encapsulating our infrastructure in code and providing continuous deployment for both our code as well as our infrastructure updates. This helps curb the need for any manual infrastructure changes and keeps infrastructure in an centralized, organized manner.
AWS CodePipeline is a great tool for incorporating continuous deployment into AWS services. The combination of SAM and CloudFormation allows for easy integration of code based infrastructure into the CD pipeline. One thing to consider is the cost structure. Tools like CircleCI charge per build container, where as CodePipeline charges per pipeline. Depending on your org, this could significantly affect cost. A couple of other things I noticed:
- The build in AWS containers were relatively far behind (as is Lambda). For example, cannot build on NodeJS 8.x unless using a custom image in which case all of the AWS commands and permissions would need to be configured.
- CircleCI 2.0 offers a feature for dependency caching, which allows for speeding up subsequent builds. I did not incorporate it but it appears that CodePipeline offers similar functionality.