Using HashiCorp Vault & Spring Cloud Vault to handle Spring Boot App Key/Value Secrets - Medium

Spring Cloud | Hashicorp Vault

A step-by-step guide to managing Spring Boot App Key/Value Secrets with HashiCorp Vault and Spring Cloud Vault

Photo by Sai De Silva on Unsplash

In this article, we'll guide you through the implementation of a simple Spring Boot application named spring-boot-vault-kv-secret. This application will expose an endpoint that, upon receiving a secret key in a request, will provide the corresponding secret value. The secrets themselves will be securely stored in HashiCorp Vault. In order to communicate with Vault, we will use Spring Cloud Vault dependency.

HashiCorp Vault is an open-source tool designed for secret management, data protection, and access control. It provides a secure way to store and access sensitive information such as passwords, API keys, and encryption keys. Vault supports various authentication methods and dynamic secrets, ensuring that applications can securely access secrets without directly exposing them.

To leverage Vault, we'll walk you through the initialization, unsealing, and configuration steps using the Vault HTTP API. We'll utilize HashiCorp Consul as the storage backend for Vault. Once the Vault setup is complete, we'll proceed to store two Key/Value secrets within it.

So, let's get started!

Prerequisites

If you would like to follow along, you must have Java 17+ and Docker installed on your machine. Also, we will use jq to parse the JSON responses from cUrl commands.

Creating Spring Boot app

Let's create a Spring Boot application using Spring Initializr.

The application name will be spring-boot-vault-kv-secret and the dependencies needed are: Spring Web and Vault Configuration.

We will use the Spring Boot version 3.1.6 and Java 17. Here is the link that contains all the setup mentioned previously.

Click the GENERATE button to download a zip file. Unzip the file to a preferred folder and then open the spring-boot-vault-kv-secret project in your IDE.

Create the controller package

In order to keep our code organized, inside the com.example.springbootvaultkvsecret root package, let's create the controller package.

Create the SecretController class

In controller package, let's create the SecretController class with the content below:

package com.example.springbootvaultkvsecret.controller;
import org.springframework.core.env.Environment;
import org.springframework.util.StringUtils;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.PathVariable;
import org.springframework.web.bind.annotation.RestController;
@RestController
public class SecretController {
private final Environment environment;
public SecretController(Environment environment) {
        this.environment = environment;
    }
@GetMapping("/secrets/{key}")
    public String getSecretMessage(@PathVariable String key) {
        String value = environment.getProperty(key);
        return StringUtils.hasText(value) ? value : "Secret not found!";
    }
}

The SecretController class serves as a straightforward REST controller implementing the getSecretMessage method. This method responds to requests at the "/secrets/{key}" endpoint, fetching the corresponding value and returning it if present, or responding with "Secret not found!" otherwise.

Update application.properties

Let's update the application.properties with the following content below:

spring.application.name=spring-boot-vault-kv-secrets
spring.cloud.vault.authentication=APPROLE
spring.cloud.vault.app-role.role-id=my-app-role-id
spring.cloud.vault.uri=http://localhost:8200
spring.config.import=vault://
logging.level.org.springframework.vault=DEBUG

A brief explanation about the properties:

• spring.application.name: Specifies the name of the Spring Boot application;
• spring.cloud.vault.authentication: Sets the authentication method to use for connecting to Vault. Here, it's configured to use the AppRole authentication method;
• spring.cloud.vault.app-role.role-id: Provides the Role ID for the AppRole authentication method. The application uses this ID during the authentication process with Vault;
• spring.cloud.vault.uri: Specifies the URI of the Vault server;
• spring.config.import: Configures the application to import additional configuration properties from Vault. The "vault://" prefix indicates that the configuration source is Vault;
• logging.level.org.springframework.vault: Sets the logging level for the "org.springframework.vault" package to DEBUG. This configuration provides detailed debug logs for interactions with the Vault server, aiding in troubleshooting.

Setting Up Docker Compose services

Create the Vault Config file

In spring-boot-vault-kv-secret folder, create a folder called vault. Inside it folder, create a subfolder called config. Finally, inside config, create a file called config.hcl. Within it, add the content below:

backend "consul" {
  address = "consul:8500"
  path = "vault/"
}
listener "tcp" {
  address = "0.0.0.0:8200"
  tls_disable = "true"
}

This configuration employs Consul as the storage backend, establishing a connection to Consul at "consul:8500" and storing data in the "vault/" path within Consul.

For listener configuration, it actively listens for incoming connections on all available interfaces at port 8200, opting to disable TLS for simplicity.

Create the Docker Compose file

Inside the spring-boot-vault-kv-secret folder, create the docker-compose.yml file with the following content:

version: "3.8"
services:
vault:
    image: hashicorp/vault:1.15.2
    container_name: vault
    restart: unless-stopped
    ports:
      - "8200:8200"
    cap_add:
      - IPC_LOCK
    volumes:
      - ./vault:/my/vault
    command: "vault server -config=/my/vault/config/config.hcl"
    healthcheck:
      test: "nc -z -v localhost 8200"
consul:
    image: hashicorp/consul:1.17.0
    container_name: consul
    restart: unless-stopped
    ports:
      - "8400:8400"
      - "8500:8500"
      - "8600:53/udp"
    healthcheck:
      test: "curl -f http://localhost:8500/v1/status/leader || exit 1"

Start Docker Compose services

In a terminal and inside spring-boot-vault-kv-secret folder, run the following command to start the Docker Compose services:

docker compose up -d

Configuring Vault

In order to configure Vault, we will use its exposed HTTP API. For it, open a terminal and let's run a couple of commands.

Initialize Vault

In order to initialize HashiCorp Vault, let's run the following commands:

VAULT_KEYS=$(curl -X PUT -s http://localhost:8200/v1/sys/init -d '{ "secret_shares": 1, "secret_threshold": 1 }' | jq .)
VAULT_KEY1=$(echo $VAULT_KEYS | jq -r '.keys_base64[0]')
VAULT_ROOT_TOKEN=$(echo $VAULT_KEYS | jq -r '.root_token')

The first line sends an initialization request to Vault, setting up with the value 1 for both secret_shares and secret_threshold. It captures the server response in VAULT_KEYS.

In the second line, it retrieves the first encryption key from the Vault initialization response and stores it in VAULT_KEY1.

Finally, in the third line, it grabs the root token from the Vault initialization response and stores it in VAULT_ROOT_TOKEN. This token has administrative privileges in Vault.

Important: the VAULT_KEY1 and VAULT_ROOT_TOKEN will be used on the next requests to Vault.

Unseal Vault

Every initialized HashiCorp Vault server starts in the sealed state. To unseal, we must have the threshold number of unseal keys. As in the previous step, we set up with the value 1 for both secret_shares and secret_threshold, we should use only the VAULT_KEY1 to unseal Vault. The API call is the following:

curl -i -X PUT http://localhost:8200/v1/sys/unseal \
  -d '{ "key": "'${VAULT_KEY1}'" }'

Check Vault Status

We can run the API call below to check the Vault status:

curl -i http://localhost:8200/v1/sys/init

Enable KV Secrets Engine

The following request will enable a new secret engine of type "kv" at the "secret" path.

curl -i -X POST -H "X-Vault-Token: ${VAULT_ROOT_TOKEN}" \
  http://localhost:8200/v1/sys/mounts/secret \
  -d '{"type": "kv"}'

Enable AppRole Auth Method

With the request below, we are enabling the "approle" authentication method, allowing applications to use AppRole-based authentication to obtain tokens for accessing secrets in Vault.

curl -i -X POST -H "X-Vault-Token: ${VAULT_ROOT_TOKEN}" \
  http://localhost:8200/v1/sys/auth/approle \
  -d '{"type": "approle"}'

Set KV Secret Policy

The following command will create a new policy called "my-app-kv-policy", which grants read access to paths under "secret/". Policies in Vault are used to control access to secrets based on defined rules.

curl -i -X POST -H "X-Vault-Token:${VAULT_ROOT_TOKEN}" \
  http://localhost:8200/v1/sys/policy/my-app-kv-policy \
  -d '{"policy":"path \"secret/*\" {policy=\"read\"}"}'

Create AppRole with Policy

With the request below, we will create a new AppRole role named "my-app-user", associating it with the "my-app-kv-policy" policy, and allowing any IP address to authenticate without requiring a secret ID.

curl -i -X POST -H "X-Vault-Token: ${VAULT_ROOT_TOKEN}" \
  http://localhost:8200/v1/auth/approle/role/my-app-user \
  -d '{"policies": [ "my-app-kv-policy" ], "bound_cidr_list": "0.0.0.0/0", "bind_secret_id": false}'

Set Role ID for the AppRole

The following command will set the Role ID for the AppRole "my-app-user" to "my-app-role-id". The Role ID is a crucial piece of information that client applications will use during the authentication process to obtain an authentication token. The "my-app-role-id" is set in the Spring Boot application.properties file.

curl -i -X POST -H "X-Vault-Token: ${VAULT_ROOT_TOKEN}" \
  http://localhost:8200/v1/auth/approle/role/my-app-user/role-id \
  -d '{"role_id": "my-app-role-id"}'

Create Key/Value Secrets

With the following command, we configure HashiCorp Vault to write key-value pairs to the path "secret/spring-boot-vault-kv-secrets".

Note: we are using the Spring Boot application name to define the secret path. For more, check this.

curl -i -X POST -H "X-Vault-Token: ${VAULT_ROOT_TOKEN}" \
  http://localhost:8200/v1/secret/spring-boot-vault-kv-secrets \
  -d '{"message1": "This is the 1st message!", "message2": "This is the 2nd message!"}'

The POST request payload specifies the following key-value pairs to be written to Vault:

• A key-value pair where the key is "message1," and the value is "This is the 1st message!"
• Another key-value pair where the key is "message2," and the value is "This is the 2nd message!"

Demonstration

Start Spring Boot app

In terminal and inside the spring-boot-vault-kv-secret root folder, run the command below:

./mvnw clean spring-boot:run

In another terminal, let's try to retrieve the secret value of the message1 key secret:

curl -i localhost:8080/secrets/message1

It should return:

HTTP/1.1 200
...
This is the 1st message!

Next, let's try to retrieve the secret value of the message2 key secret:

curl -i localhost:8080/secrets/message2

The response should be:

HTTP/1.1 200
...
This is the 2nd message!

In case, we request the value of a secret that does not exist:

curl -i localhost:8080/secrets/message3

The response returned will be:

HTTP/1.1 200
...
Secret not found!

Shutdown

In the terminal where the spring-boot-vault-kv-secret is running, press Ctrl+C to stop the application.

To stop the Docker Compose services, in a terminal and inside the spring-boot-vault-kv-secret root folder, run the following command:

docker compose down -v

Conclusion

In this tutorial, we implemented a simple Spring Boot app called spring-boot-vault-kv-secret. This app has an endpoint that gives you secret values when you provide a secret key. We stored these secrets securely in HashiCorp Vault, and we used the Spring Cloud Vault tool to communicate with Vault.

We walked through the steps of setting up Vault, using HashiCorp Consul as a reliable storage backend. After configuring Vault, we stored two secrets in it.

To wrap things up, we started the application and used the endpoint to get the secret values. This tutorial is a practical guide to help you integrate HashiCorp Vault into your Spring Boot apps for secure secret management.

Additional Readings

Using HashiCorp Vault & Spring Cloud Vault to obtain Dynamic MySQL Credentials

A step-by-step guide to retrieve Dynamic MySQL Username and Password with HashiCorp Vault and Spring Cloud Vault

medium.com

How to Rotate Expired Spring Cloud Vault Relational DB Credentials Without Restarting the App

Presenting a solution for rotating MySQL dynamic credentials when the Maximum Lease TTL is reached

medium.com

Support and Engagement

If you enjoyed this article and would like to show your support, please consider taking the following actions:

• 👏 Engage by clapping, highlighting, and replying to my story. I'll be happy to answer any of your questions;
• 🌐 Share my story on Social Media;
• 🔔 Follow me on: Medium | LinkedIn | Twitter;
• ✉️ Subscribe to my newsletter, so you don't miss out on my latest posts.