Overview
As companies progress down the path toward their goals, they will adapt strategies when business conditions change to ensure they do not drift away from the main vision. In recent years, organizations have adapted to using Secrets Management solutions to centrally store, manage, and tightly control access to sensitive credentials like passwords, API keys, and encryption certificates.
Your secrets are precious and you will defend them like a Rhino on an African safari as shown below (from a picture I took in the Gnorognoro Crater):
IBM acquired Hashicorp Vault which is a leader in the secrets management space and our team developed a built-in integration with Vault for our IBM DevOps Deploy solution to benefit our many clients. Some of the key reasons for using the combined capabilities of DevOps Deploy and Vault are to:
- Stop manual update of secure properties: deployment processes can now fetch credentials dynamically from Hashicorp Vault instead of having to periodically update the secret property values in IBM DevOps Deploy either manually or through scripting.
- Eliminate secret sprawl: this helps to replace scattered, unmonitored credentials across multiple clouds and servers with a single source of truth.
- Track access with audit logs: both IBM DevOps Deploy and Hashicorp Vault log every read, write, and authentication attempts to satisfy security compliance and regulatory standards.
This blog provides an easy-to-follow example of how you can integrate IBM DevOps Deploy with Hashicorp Vault.
Create a Free Vault Instance
To try out this integration, you can setup a free Vault instance in the Hashicorp portal online. I followed these steps:
1. Navigate in your browser to https://portal.cloud.hashicorp.com/sign-in
2. Create an account (I linked my account to my github account)
3. You will create a default-project when your account is created, and you will be logged in.
4. In the project dashboard that appears, click on ‘Vault Dedicated’ as shown below:
5. Click the ‘Create cluster’ button at the top right
6. After your Vault cluster is created, you will see a message like this one:
7. You can copy the code snippets highlighted above to a terminal window and run them to verify you can connect to the instance properly. Here is an example:
We now have a Vault instance we can work with to test the built-in integration that IBM DevOps Deploy provides with Hashicorp Vault.
Create a Vault Secret Store
Our first step is to create a Vault Secret Store in IBM DevOps Deploy. To do this, follow these steps:
1. In a web browser, navigate to your DevOps Deploy web console and login.
2. Click on ‘Secret Stores’ in the left-hand navigation as shown below:
3. Click on ‘Create Vault Secret Store’ to create your new Secret Store
4. In the Manage Vault Secret Store dialog window that appears, enter a name of your choice as shown below along with an optional description:
5. Copy and paste the URL and Namespace from your Vault instance that was created in the Hashicorp web portal. Here are the values as shown:
6. Click Save
Add your AppRole to the Vault Secret Store
Now that we have a Secret Store created, we now need to authenticate to the instance using one of the following options:
- Vault AppRole
- LDAP authentication methods
- Use of an SSL/TLS certificate
We added support for LDAP authentication methods to further enhance security and provide flexibility in user authentication in our 8.1.0 release. We added support for certificate-based authentication in our 8.1.2 release bringing even more flexibility for our clients.
For this blog, I am going to create an AppRole for authentication.
I followed these steps:
1. In the Secret Stores page, click on your newly created secret store as shown below:
2. Click the ‘Create Vault AppRole’ button as shown below:
3. In the ‘Manage Vault AppRole’ dialog that appears, provide your AppRole Name, AppRole Id, and Secret Id as shown below:
The AppRole ID and Secret ID can be found on the Vault web console for the cluster you created as shown below:
Copy and paste these values appropriately.
For the name, provide any name that you like. It is best to provide the name in all lowercase letters.
4. Click Save
5. Notice that the MyAppRole was created and shown in the table as shown below:
NOTE: the name is converted to all lowercase characters
6. You can Test your connection to the Vault Secret Store once the AppRole is created. To do this, click the three dots to the far right of you new AppRole as shown below:
7. Select ‘Test’ from the pulldown menu as shown above.
8. In the ‘Test Vault App Role’ dialog that appears, provide the secret path to your secure property in vault. In my case, it is ‘secret/sample-secret:first-secret’ which you can see in the output of the script we ran earlier:
9. Therefore, I enter “secret/sample-secret:first-secret” in the property field as shown below:
10. Click Test.
11. You should see ‘Success’ in the Alert dialog that appears as shown below:
We can now review a DevOps Deploy example that leverages the Vault integration to pull secrets at runtime of a process execution.
Referencing a Vault-based property using a Shell Step
Now that we have a successful connection to our Vault secret store, we can pull secrets from Vault at the time of a deployment using IBM DevOps Deploy. To do this, you will use a unique property reference that begins with ${p:secret:vault:}. Here is the full syntax for this unique property reference for Vault:
${p:secret:vault:<secret-store-name>:<approle-name>:<path-to-secret>:<secret>}
For our example we have configured, the property reference would translate to this:
${p:secret:vault:vault-cluster:myapprole:secret/sample-secret:first-secret}
Using this syntax, you can add a shell step in a component process to echo the value that is fetched at runtime from Vault. By default, since it is a secret, you will not see the value in your step log. It will look like this:
Whenever a deployment process runs, DevOps Deploy will automatically authenticate against Vault using your credentials, retrieve the secret payload, isolate the value, and securely inject it into the executing step context.
Referencing a Vault secret using an Environment Property
Many of our clients use secure properties at the Environment scope in IBM DevOps Deploy for passwords or for other secrets. Prior to this Vault integration, an update to a property value was required on a regular interval that required manual entry in the web console or running a script to update the passwords using our CLI or REST APIs.
If you manage the update of secrets in Hashicorp Vault, this requirement to manually update secure properties in IBM DevOps Deploy is no longer needed.
Here is an example of an environment property that references a secret in Vault:
This allows your current steps that reference these environment properties to run as they always have without any modification. For example, here is a shell step that references the environment property that in turn pulls the secret value from Vault:
echo ${p:environment/my-secure-property}
Conclusion
Integration with secrets management solutions is a common use case in the world of DevOps and release automation. We have many customers using IBM DevOps Deploy today integrated with both Hashicorp Vault and CyberArk as examples. Integrating secrets management with IBM DevOps Deploy allows you to fetch sensitive information like database passwords, API keys, and tokens dynamically at deployment time rather than hard coding them or storing them statically in DevOps Deploy properties.
If you have interest in integrating with other Secrets Management solutions, please let us know. You can open an Aha Idea enhancement request (RFE) using this link to IBM Aha. We are interested in adding new integrations that make sense and your inputs are very important.
I want to wish our customers all the best and thank you for your continued support and investment in IBM DevOps Deploy! Happy Deploying to all!