AWS service models and provision resources based on a template in an automated and secure way

This blog post was updated on 7/21/2020 to reflect recent changes to how AWS Service Catalog obtains outputs from provisioned products. For more information see Provisioned product outputs are now available in AWS Service Catalog.

You can use AWS Service Catalog to create preconfigured products that your developers can launch. In a large organization, it’s typical for a cross-functional team like a Cloud Center of Excellence (CCOE) to maintain the catalog for the organization. An AWS Service Catalog product can contain one or more AWS resources. Many customers use AWS Service Catalog to restrict access to resources, such as AWS APIs, using a launch constraint. Launch constraints allow an AWS Service Catalog end user to launch an AWS Service Catalog product without requiring elevated permissions to AWS resources.

An AWS Service Catalog product also allows a CCOE to enforce configuration standards within a customer’s products, while granting development teams flexibility to customize AWS resources using parameters. Despite the flexibility of parameters, it is difficult for a CCOE to determine every development team’s architecture patterns and requirements. Accommodating the broad requirements of development teams leads to a large portfolio of AWS Service Catalog products that need to be maintained.

Recently, we announced AWS CloudFormation support for AWS Service Catalog products. In this blog post, I’ll walk you through how to leverage this new feature to provide development teams the freedom to create complex architectures. The building blocks for development teams will be AWS Service Catalog products configured as simple hardened components. Using simple components will help the CCOE reduce their workload while implementing preventative controls that manage an organization’s risk.

Why wouldn’t I simply use native AWS CloudFormation or native AWS Service Catalog to provision resources?

For many customers, the CCOE is responsible for maintaining the AWS environment. This includes establishing security, governance, and operating controls that allow the business to leverage AWS at scale while managing risk. Typically, the CCOE owns the whitelisting process that approves the use of AWS services. The output of this process is a directive control—namely, the configuration standards and guidelines that development teams must follow when they use the AWS service. Establishing these controls takes time because the CCOE must evaluate each AWS service.

Standards and guidelines are a great start, but ultimately the business needs to ensure that directive controls are followed by creating preventative, detective, and reactive controls. The most important of these controls are preventative controls. Why allow bad things to happen if we can prevent them? The easiest preventative control is simply not giving a user permission to use an AWS service. Ideally, your preventative controls are more nuanced than this because these controls can prevent the business benefits of an AWS service from being realized.

Next, you need to decide where to place the preventative control. Mature AWS customers leverage continuous integration/continuous deployment (CI/CD) pipelines and AWS CloudFormation to deploy into AWS. Like the applications they support, CI/CD pipelines come in all shapes and sizes. In the short term, fitting preventative controls into each development team’s CI/CD pipeline is time consuming and potentially disruptive. The diagram below shows where you would place preventative controls in a simplistic CI / CD pipeline.

A simple component configured with preventative controls in Service Catalog streamlines developer adoption of these hardened AWS service configurations, while providing the flexibility for developers to design their own architectures. The diagram below shows the shift of preventative controls from the existing CI / CD pipeline into a Service Catalog product.

To further highlight the advantages of using CloudFormation support for AWS Service Catalog products, I’ll review common methods for deploying AWS resources with this new feature:

1. Native AWS CloudFormation
2. Native AWS Service Catalog products
3. AWS CloudFormation support for AWS Service Catalog products

Using AWS CloudFormation to provision AWS resources

The following diagram illustrates web and application tiers deployed using native AWS CloudFormation. The AWS CloudFormation template describes the AWS resources in the colored box above it.

Pros and cons of this method:

• Pros
  • The CCOE does not need to write infrastructure as code for development teams.
  • Developers write infrastructure as code for their applications; this gives them the flexibility to create architectures that meet their needs. This model is easier to scale than having a central team write all the infrastructure as code.
• Cons
  • Developers require permissions to directly access APIs for resources they are creating.
    • It is difficult to implement preventative controls for resource configuration.
    • Enterprises may view this as too much risk, especially when they are early in their cloud adoption journey.
  • Developers need to understand the configuration options for each AWS resource.
  • The CCOE needs to create detective and reactive controls to ensure that AWS services are used in a manner consistent with the company’s standards.

Using AWS Service Catalog to provision AWS resources

The following diagram illustrates web and application tiers deployed using a single AWS Service Catalog product.

Pros and cons of this method:

• Pros
  • The CCOE is able to restrict access to AWS APIs, while still providing developers with the ability to provision AWS resources.
  • It’s easy to ensure that AWS resources adhere to security / governance standards.
  • Developers can launch AWS Service Catalog products from the console without writing any code.
  • Developers do not need to understand the configuration options for each AWS resource.
• Cons
  • There’s less flexibility for developers. Developers can’t use AWS CloudFormation to describe their infrastructure.
  • The CCOE needs to create a product per architecture.
    • This may not meet developer requirements.
    • The complexity and size of the AWS Service Catalog product is increased.
    • Workload for the CCOE is increased.

Using AWS CloudFormation support to provision AWS Service Catalog products

This diagram illustrates combining the previous two provisioning methods to deploy the web and application tiers using AWS CloudFormation support for AWS Service Catalog products.

Pros and cons of this method:

• Pros
  • Restricts access to AWS APIs, while still providing developers with ability to provision AWS resources.
  • Easy to ensure that AWS resources adhere to security / governance standards.The CCOE is responsible for creating simple components instead of complete architectures.
  • Developers write their own infrastructure as code for their applications. Using parameters, they can customize the simple components to meet their needs.
• Cons
  • The CCOE needs to provide development teams with AWS Service Catalog resources IDs.
  • Developers need to understand the configuration options for each AWS resource.

How to create simple component products in AWS Service Catalog

Before a development team can launch an AWS Service Catalog product, the CCOE will need to create the AWS Service Catalog portfolio and products. Here is an outline of the steps I used to set up my Service Catalog environment and share it with an end user:

1. Create a portfolio for your product.
2. Create the Application Load Balancer (ALB) product.
3. Create the Auto Scaling group product.
4. Add both products to your “Dev” portfolio.
5. Create launch constraints for the Application Load Balancer product.
6. Create launch constraints for the Auto Scaling group product.
7. Grant the appropriate user, group, or role permissions to the portfolio.

In this section, I’ll provide the commands and the input parameters for each command. I’m using the cli-input-json parameter to improve the readability of the parameters for each command. If you did not know the format for the parameter files, you could use the generate-cli-skeleton parameter with each command to obtain it (example: aws servicecatalog create-portfolio –generate-cli-skeleton).

Step 1: Create a portfolio for your product

Now let’s create a portfolio called Development Whitelisted Services. Make sure to capture the portfolio ID that is returned. I’ve highlighted this in the output section that follows.

Command:

File: Portfolio_params.json

Output:

Step 2: Create the Application Load-Balancer product

When you create the product you will be required to specify the Amazon S3 location of the AWS CloudFormation template that describes the AWS resources the product will create when launched. Make sure to save the ProductId and the ProvisioningArtifactDetail ID from this step. These values are required inputs for the AWS CloudFormation template that launches these products. I’ve highlighted these values in the output section that follows.

Command:

File: ALB_product_params.json

Output:

Step 3: Create the Auto Scaling group product

When you create the product, you will be required to specify the Amazon S3 location of the AWS CloudFormation template that describes the AWS resources the product will create when launched. Make sure to save the ProductId and the ProvisioningArtifactDetail ID from this step. These values will be required inputs for the AWS CloudFormation template that creates launches these products. I’ve highlighted these values below in the output.

Command:

File: ASG_product_params.json

Output:

Step 4: Add both products to my “Dev” portfolio

I need to associate my products with my portfolio. Access to products is granted at the portfolio level. I’m using the portfolio ID from step 1 and the product IDs from steps 2 and 3.

Commands:

File: associate_alb_params.json

File: associate_asg_params.json

Output:

No text is returned if the command is successful.

Step 5: Create launch constraints for the Application-Load-Balancer product

The launch constraint associates an AWS Identity and Access Management (IAM) role that contains the permissions necessary to launch the product. By leveraging this feature, developers will not require permissions to AWS APIs. I’m assuming the role has been previously created.

Command:

File: create-launch-constraint_alb_params.json

Output:

Step 6: Create launch constraints for the Auto Scaling group product

The launch constraint associates an IAM role that contains permissions necessary to launch the product. I’m assuming the role has been previously created

Command:

File: create-launch-constraint_asg_params.json

Output:

Step 7: Grant the appropriate user, group, or role permissions to the portfolio

Before users can launch the products I need to grant them permissions.

Command:

File: associate_principal_params.json

Output:

No text is returned if the command is successful.

How to obtain the product ID or provisioning artifact ID from the CLI

If you didn’t capture the product ID or provisioning artifact ID when you created your product, you can use the following commands to find them. Notice for each command I am using the –query option to reduce the size of the response. The first command helps me find the product ID.

Command

Output

This second command helps me find the provisioning artifact ID. I’ll need to use the product ID I obtained earlier (prod-6w72oh4zxhhhq) as an input for this command. The describe-product-as-admin command will return multiple artifacts, once for each product version. The query parameter can help reduce the output.

Command

Output

The CCOE will need to provide the development teams the product ID, provisioning artifact ID, and the list of parameters for each product. Keep in mind that the provisioning artifact ID is going to change with each update of the product. These values will be unique to each AWS account/Region the development teams uses. You will want to implement a scalable process that avoids development teams having to ask the CCOE for these values. Many customers use an AWS Lambda-backed custom resource to discover resource values based on parameters passed into the custom resource. Here is an example of an AWS Lambda function that can be invoked as a custom resource.  It returns the product Id and provision artifact ID back to AWS CloudFormation.  To keep my example simple, I’m going to pass these values in using AWS CloudFormation parameters.

Using the AWS Service Catalog simple component products to create the infrastructure

I’m going to use a single AWS CloudFormation template to launch my AWS Service Catalog products. I’m not explicitly including tags in this template. I would expect they would be passed during the create-stack API call or configured as part of the AWS Service Catalog product. As mentioned earlier, product ID and product artifact IDs are required inputs. I’m passing these in as parameters. In a real-world scenario there may be dependencies between the components we would want to consider.

In this example I’m going to reference the target group Amazon Resource Name (ARN) created in the ALB template when I create my Auto Scaling group.

Snippet from the ALB product’s CloudFormation output section: In this snippet I am outputting the ALB Target Group ARN.   I will pass this value to the Auto Scaling product.

Snippet from resource section of the CloudFormation template that deploys the solution: I can use the !GetAtt intrinsic function to obtain the ALB Target Group ARN created by AWS Service Catalog product and pass it into the Auto Scaling group parameters.

The following is the complete AWS CloudFormation template I am using to create my infrastructure from the AWS Service Catalog simple component products.

Source Code

The solution described in this post including AWS CloudFormation templates, source code, deployment script and documentation can be download from GitHub.

Conclusion

Thanks for taking the time to read this blog post. I’ve shown you how AWS CloudFormation support for AWS Service Catalog provides you with the capability to preventatively implement security and governance controls in your AWS Service Catalog products, while granting developers the flexibility to create architectures that meet their applications’ requirements.

About the Authors