_
_
It is not uncommon for us at RapDev to encounter customers who are unable to share their configuration management code with us because it contains sensitive information. Gaining access to at least some of the infrastructure code is important in our engagements, as we help customers deploy the Datadog Agent within their environments. When clients are unable to safely share their infrastructure code, it often leads to delays in the project timelines. In order to pierce that veil and help our customers secure their code, I’d like to introduce AWS Secrets Manager.
AWS Secrets Manager is a service provided by Amazon Web Services that helps in managing, accessing, and storing sensitive data such as passwords, API keys, and credentials. It enables users to rotate, manage, and retrieve database credentials, API keys, and other secrets throughout their lifecycle, keeping sensitive data out of source code repositories. The great thing about Secrets Manager is that once you’ve created the secret object in AWS and set up your configuration, you can rotate and change your secrets without ever having to update your infrastructure code.
In this guide I’ll go over two different plugins, one for Puppet and the other for Ansible, that will enable secret lookups in AWS Secrets Manager.
The first plugin is called “hiera-aws-secretsmanager” and it is developed by Salesforce. It integrates with Hiera, which is a built-in key/value lookup tool for Puppet. The full installation instructions are included in the README file, but the basic steps are as follows.
Installation
From your puppet master server
puppetserver gem install aws-sdk-secretsmanager
AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY environment variables.
$HOME/.aws/credentials file in the puppet service user’s home directory.
secretsmanager:GetSecretValue
secretsmanager:ListSecrets
I encourage you to read through the full README file for more information and instructions on how to integrate with statsd, should that be something your organization uses.
The module does not need to be included in any puppet manifests to work, you simply need to create a secret using the aws cli and then use the puppet lookup function to call it in your puppet manifest.
1. Create a secret file in json format.
vim /tmp/secret.json
2. Enter a string like the following example and save the file. The string must be double quoted.
"password123!"
3. Run the following aws cli command to create a secret in the secrets manager. Change the region to match your environment.
aws secretsmanager create-secret --region us-west-2 --name puppet/production/webservers==prod==secret_password --secret-string file:///tmp/secret.json
The --name parameter must follow the same conventions as this example. Let's break it down:
--name puppet/production/webservers==secret_password
But if it needed to be called from the webservers::prod class the name would need to be;
--name puppet/production/webservers==prod==secret_password
4. One method of getting the secrets into your Puppet code would be to perform a lookup of the secret in the manifest file and assign it to a variable. Something like this would go in the manifests/init.pp file. It will either lookup the secret or it will output "AWS Secrets lookup failure" if it cannot be found.
$secret_password = lookup('webserver::secret_password', undef, undef, 'AWS Secrets lookup failure')
From there, you can simply use it like any other variable inside your manifest files, like so
5. Another example, this time setting different secrets based on the target hostname.
-- Manifest Example (.pp file) --
-- Template Example (.erb file) --
In addition to using variables that are defined in the manifest files, templates can also use the scope lookup function to pull the secret in from any module elsewhere in your puppet code. Here, an AWS credentials file is defined, pulling in the “dev_aws_key” variable defined in the “accounts” module.
Ansible is much easier to configure for secret lookups than Puppet. Of course everyone knows about Ansible Vault, it’s built-in solution for securely storing secrets, but for many that’s not always an ideal solution. For one, it still has to be accessible to Ansible at run time and for many that means keeping it within the same code repository. In addition, you may need to juggle multiple different vault files for different environments and maintaining these files when passwords need to be changed can be a management headache. Once again, enter AWS Secrets Manager.
Installation
To set up the amazon.aws.secretsmanager_secret lookup plugin in Ansible:
ansible-galaxy collection install amazon.aws
sudo pip3 install boto3 boto
In this example we’re using environment variables to provide the access to query Secrets Manager in the us-east-1 region. See the official documentation for more examples.
In conclusion, integrating AWS Secrets Manager into your Puppet and Ansible automation workflows represents a significant step towards enhancing the security and manageability of your infrastructure. By centralizing the storage and management of secrets, AWS Secrets Manager not only simplifies the handling of sensitive data but also fortifies your security posture with features like automatic rotation and fine-grained access controls. The ability to seamlessly integrate with automation tools like Puppet and Ansible allows for more streamlined and secure operations, ensuring that credentials and secrets are dynamically and securely retrieved as needed, without ever exposing them in code or logs. This approach not only adheres to best practices in cloud security but also aligns with a DevOps philosophy that emphasizes automation, efficiency, and reliability.
