What is AWS infrastructure as code?

The Enhanced Studio Pack contains all the tools you need to do world-class development in a single, pre-configured environment. You can deploy everything you need with the AWS Infrastructure as Code required to support large projects, big files, and all your team members.

Infrastructure as Code (IaC) is a key DevOps concept that is essential in the Data Science world when we’re building and defining production level workloads. IaC allows developers to manage a project’s infrastructure as software. This enables developers to easily maintain and configure changes within a project’s resources and architecture. While similar to traditional scripting, IaC allows for developers to use declarative language to provision resources. There’s numerous IaC tools that are available such as Terraform, Chef, Puppet, and Ansible. For today’s demonstration we’ll be using CloudFormation which is specific for AWS resources. Through the article you will understand how to maintain all your resources within one software file as well as understand the benefits in regards to speed that IaC brings to the table. Without IaC, costs and time for manual deployment of different infrastructures can skyrocket, by maintaining your infrastructure as software you’re able to easily and quickly test various deployments from a central source. In this article, we’ll explore an example of manually provisioning resources vs deploying a CloudFormation script to create a REST API and a serverless Lambda function on AWS.

NOTE: For those of you new to AWS (it’s good to have some experience to fully understand the article), make sure you make an account at the following link if you want to follow along. Make sure to also have the AWS CLI installed to work with the example. I’ll also provide a list of services we’ll be using along with more in-depth definitions. If you’re already familiar with these services, feel free to skip ahead to the parts you’re interested in.

Table of Contents

1. AWS Services
2. Example Overview
3. Manual Deployment
4. Deploying with CloudFormation
5. Entire Code & Conclusion
6. Additional Resources/References

1. AWS Services

AWS API Gateway (API GW): Service that allows for developers to create, publish, and monitor secure RESTful and Socket APIs. We’ll be using this service to create our REST API.

AWS Lambda: A serverless computing service, that allows developers to run code without managing or provisioning servers. We will be using this service to setup a sample serverless function on the backend integrated with our REST API.

Identity Access and Management (IAM): Lets you manage access of AWS services through permissions and roles. We will be creating a role for our Lambda function to be able to access API GW.

AWS CLI: To work with AWS resources and services you can use the CLI they provide rather than the Console for easy access.

AWS SAM: An abstraction of CloudFormation that helps build serverless applications, check out the SAM CLI for more information.

2. Example Overview

For the article we’ll be building a REST API with API Gateway that is integrated with a serverless backend Lambda function that works with GET and POST requests from our API. The first step will detail how to manually build and deploy these resources through the AWS console, while the second step walks through using CloudFormation to automate and repeat the exact same process.

3. Manual Deployment

For manual deployment we have to work within the AWS console. This in itself is already a bad developer habit as we can’t work from our local IDE and it’s hard to track changes/development within the console in a bigger project. For the first step we will go and create a sample Lambda function.

Screenshot by Author

Make sure to create a role with basic permissions, if you want your Lambda to work with a different service such as Comprehend or such make sure to give permissions for that service. For more about IAM roles and permissions check out the following resource.

Here we have our sample “Hello World” lambda function that should return the shown statement if properly integrated with API Gateway.

Now that we have our Lambda function configured we want to setup our REST API to make sure it can interface with the Lambda function so we go to Amazon API Gateway. Click create API and choose REST API out of the options that are provided.

Creating REST API (Screenshot by Author)

Now we go and create a GET method from the Actions dropdown bar and make sure to point our REST API towards our Lambda function.

Integrate REST API with Lambda (Screenshot by Author)

We can now deploy our API to test if its properly integrated with our Lambda function (pick whatever name you want for stage we will be using “prod”).

Screenshot by Author

After you’ve deployed your API you should be able to see a URL on the prod stage and if you access this URL you should see your Lambda function in action returning “Hello World” as we described in the function.

Screenshot by Author

Lambda Function Return (Screenshot by Author)4. Deploying with CloudFormation

Now that didn’t seem too bad at first, it was a few steps that took a few minutes, but let’s say you had more than just one method, more than just one API, and more than just one developer. How are you supposed to track all these resources and any changes centrally. What if you want to adjust your Lambda function to integrate with another API? Do you have to delete this current method or API manually? This would require developers having cross account access, permissions and being able to see and provision each others resources in varying accounts when working with each other. All of this can lead to a large waste of time, cost, and resources so we will use AWS CloudFormation to deploy this very same example within minutes while also giving the developer flexibility and freedom to adjust their infrastructure with a simple script.

So first, how does CloudFormation work? We will be using a YAML file to declare and provision our resources that we deploy to AWS to create a CloudFormation stack that contains all the resources necessary for our project. The template we will be using is known as SAM template which is an abstraction of CloudFormation with the same underlying functionality but more geared for serverless applications with less YAML code required. For those non-familiar with YAML, think of it as similar to JSON (CloudFormation uses both) with both having the functionality to build large declarative configuration files. First we go to our local code editor rather than the console and spin up the exact same Lambda function. Along with this we will create a template.yaml file that will contain our infrastructure.

File Structure for Project (Screenshot by Author)

In our helloworld.py we can put the exact same Python code we had in the console.

Now we can define our API Gateway and Lambda function in our template.yaml file. To first build this file we need to add some statements that are common to all SAM templates.

Common to SAM templates

Now we can add something known as “Globals” to our CloudFormation template file. Globals are common configurations for resources that you will be deploying, rather than specifying the same information for the different resources we can establish them as global for that specific resource type. For this use case we will be doing it for Lambda Functions as an example even though we only have one.

Now we can focus on our Resources which is a Lambda Function and REST API, we define these under the Resources tag in our template.yaml file. To access the docs to follow along for defining serverless functions on CloudFormation look at the following link.

Here we define some of the parameters for creating a Lambda function. For the event we are creating a REST API as that is what is triggering our Lambda function. There’s an array of other parameters that you can specify for your serverless function such as CodeURI, Description, and more that you can follow along in the CloudFormation docs. Generally the best way to approach creating a template file is to go to the CloudFormation docs and see the parameters/language accepted for creating a template file for your specific service/resource.

Now we can deploy our template file, to do this we will use the AWS CLI to run two commands.

Deploy CFN template

After running the first bash command, you should see a sam template file created which is what we use in the second command to create our CloudFormation stack.

Screenshot by Author

After running both commands in a few minutes you should see stack successfully created in the CLI and we can verify this by going to CloudFormation in the console.

CloudFormation Stack Created (Screenshot by Author)

Here we see all the resources we provisioned through code successfully deployed and created through a simple template.yaml file. To further verify this we can click on the API created and test the URL as we did in our manual deployment to ensure the same working functionality after deploying the API

API & Lambda from CFN template (Screenshot by Author)

To cleanup we can run a delete stack command through the CLI to destroy all resources created for this demonstration. Run the following command and you should see a delete of all resources in progress.

Deleting Stack (Screenshot by Author)5. Entire Code & Conclusion

GitHub - RamVegiraju/Serverless-CFN: Sample CFN stack for a small demo serverless application

Sample CFN stack for a small demo serverless application - GitHub - RamVegiraju/Serverless-CFN: Sample CFN stack for a…

github.com

To replicate the example and access the code for the full demo check out the repository above. At first, CloudFormation can seem intimidating with a weird yaml syntax and language and there definitely is a learning curve with mastering CloudFormation. But if you establish a level of comfort and familiarity with the service it helps you build and manage incredibly powerful applications in a simple efficient manner from one central template file. While this comparison shows the difference in ease and efficiency in deploying manually vs IaC it is even more evident when your project grows and you have multiple developers working on one project. For the next steps see how seamlessly CloudFormation helps build CI/CD pipelines in AWS in this article.

I hope this article has been helpful for anyone trying to understand more about IaC, CloudFormation or AWS in general. Feel free to connect with me on LinkedIn or follow me on Medium for more of my writing. Share any thoughts or feedback, thank you for reading!

Is CloudFormation infrastructure as code?

What is terraform infrastructure as code?

Is infrastructure as code part of DevOps?

What is infrastructure as code example?