Manage your secrets with HashiCorp Vault

How to run HashiCorp Vault in production

Security is important like never before. Keeping your secrets safe should be a top priority. This blog shows you how to get started in production.

In a software delivery pipeline, there are several environments involved and thus many types of secrets. We create our pipelines as code, but of course we cannot store our secrets in version control. Here at Praqma, we had a customer who raised the question of securing credentials for a container cluster used for building and testing.

Enter HashiCorp Vault, an encrypted key store with mechanisms to control access to the stored values. Values can be everything from passwords, certificates, URLs to other sensitive data. Vault supports a variety of backends to authenticate users, like GitHub, AWS, LDAP and Radius.

It is really easy to try out Vault, using what they call dev-mode. Things get a bit more tricky when you want to put it into production. We will show you how to setup Vault in production mode and control access to the content.

Let’s get started

We love containers at Praqma, so it would be nothing but strange to not run Vault in a container. Our setup is based on the official Docker image for Vault maintained by HashiCorp, and a free web UI called djenriquez/vault-ui. Both can be found on the Docker Hub.

A simple Dockerfile for Vault would look like this:

FROM vault:0.7.0 
MAINTAINER Praqma

And for the UI:

FROM djenriquez/vault-ui:2.1.0 
MAINTAINER Praqma

Place these in separate folders called vault and vault-ui, and we are ready to build and push to the Docker Hub:

docker build -t praqma/vault:0.7.0 vault/. 
docker build -t praqma/vault-ui:2.0.2 vault-ui/. 

docker push praqma/vault:0.7.0 
docker push praqma/vault-ui:2.0.2

When done, we have two images in the Docker Hub, ready to use in our organization.

Prepare the network and configuration

The Vault UI needs to connect to the container running Vault, so let’s create a network:

docker network create vault-network

You can verify the network is created by running:

docker network ls 
NETWORK ID      NAME             DRIVER    SCOPE 
7b332ca38df5    bridge           bridge    local 
60f40b5e80f7    vault-network    bridge    local

Next we create a configuration file. We need to specify a location for the backend storage, in this case a file, which address range Vault should listen to and whether or not TLS should be disabled.

Create a file called vault.conf with the following content:

backend "file" { 
  path = "/vault/file/vaultsecrets" 
} 

listener "tcp" { 
   address = "0.0.0.0:8200" 
   tls_disable = 1  
}

We choose to put the storage file at /vault/file inside the container, as this is one of the volumes defined by the base container image. This way we can mount it on the host if we want to.

We tell Vault to listen on all network interfaces on port 8200, and to turn off TLS as we don’t have any certificates. When you do, add two lines like this to the listener section:

tls_cert_file = "/root/server.crt" 
tls_key_file = "/root/server.key"

Remember to copy your certificates into the container at the location you specify.

Start the Vault container

Now we are ready to start our Vault container:

docker run -d  
  --hostname vault  
  --network vault-network  
  --privileged  
  --name vault  
  --cap-add=IPC_LOCK  
  -v vault.conf:/vault/config/vault.conf  
  -v /opt/persistant_storage/vault/:/vault/file/ \
  praqma/vault:0.7.0 server -config /vault/config/vault.conf 

Vault should be running, and you can confirm this by running:

docker logs vault

You should get output like this:

==> Vault server configuration: 
                      Cgo: disabled 
               Listener 1: tcp (addr: "0.0.0.0:8200", cluster address: "0.0.0.0:8201", tls: "disabled") 
                Log Level: info 
                    Mlock: supported: true, enabled: true
                  Storage: file 
                  Version: Vault v0.7.0 
              Version Sha:614deacfca3f3b7162bbf30a36d6fc7362cd47f0 

==> Vault server started! Log data will stream in below:

Hooray, Vault is running. Now let’s initialize it, so we can start using it.

Initialization

lock

Start a terminal session inside the Vault container:

docker exec -it vault /bin/sh

Set an environment variable to tell the Vault client where to connect, and then initialize Vault:

export VAULT_ADDR=http://127.0.0.1:8200 
vault init

From this you will get five keys and a root token. When Vault starts up, it is sealed, preventing access to any data. The keys generated are used to unseal Vault and you need at least three keys to do so.

Unseal Key 1: z+LgaNbDbejE+hxqq9Q87vjBBfQH6VII1Ey/vmiH+Q4B 
Unseal Key 2: Zmy5jsn/+LQCMKzaM97imgLOQ2XIBHrmms6Oyxdz5eoC 
Unseal Key 3: xFEtXOUM6w4JMG+4tf4VaXu/bjoMbQx68/YKzMxO5G4D 
Unseal Key 4: MKsgyCt935cPcKzvNRPE+0MKOveD0GYvCh7ktsTJvp0E 
Unseal Key 5: kpa0GgeOzC0EcG+NszMzCDp7F6hHuRCzYyZgsR/0vxkF
Initial Root Token: 9cfb29e7-0fde-cc24-df72-76ee59f5c712 

Vault initialized with 5 keys and a key threshold of 3. Please securely distribute the above keys. When the Vault is re-sealed, restarted, or stopped, you must provide at least 3 of these keys to unseal it again. 

Vault does not store the master key. Without at least 3 keys, your Vault will remain permanently sealed.

You unseal Vault by using three of the keys like this:

/ # vault unseal z+LgaNbDbejE+hxqq9Q87vjBBfQH6VII1Ey/vmiH+Q4B
Sealed: true 
Key Shares: 5 
Key Threshold: 3 
Unseal Progress: 1 
Unseal Nonce: 88d38704-415f-b945-e00e-747a51aa8f14 
/ # vault unseal Zmy5jsn/+LQCMKzaM97imgLOQ2XIBHrmms6Oyxdz5eoC
Sealed: true Key Shares: 5 Key Threshold: 3 Unseal Progress: 2 Unseal Nonce: 88d38704-415f-b945-e00e-747a51aa8f14 
/ # vault unseal xFEtXOUM6w4JMG+4tf4VaXu/bjoMbQx68/YKzMxO5G4D Sealed: false 
Key Shares: 5 
Key Threshold: 3 
Unseal Progress: 0 
Unseal Nonce: 

Now that Vault is unsealed, we can authenticate by using the root token:

/ # vault auth 9cfb29e7-0fde-cc24-df72-76ee59f5c712 
Successfully authenticated! You are now logged in. 
token: 9cfb29e7-0fde-cc24-df72-76ee59f5c712 
token_duration: 0 
token_policies: [root] 

And we are in.

We need to create a policy for our team to use. This will give them the permissions they need when fetching data from Vault.

In the container, create a policy file in HashiCorp’s HCL format:

vi /vault/praqma-policy.hcl

With the following content:

path "*"{ 
    capabilities = [ "create", "read", "update", "list" ] 
}

This will give people access to everything related to these operations, but for example not deleting secrets. This is for demonstration purposes and you should adjust according to your security standards.

Then create the policy:

/ # vault policy-write praqma /vault/praqma-policy.hcl 
Policy 'praqma' written. 

Now we will enable a GitHub backend so our employees to use their GitHub account to login to the Vault UI.

/ # vault auth-enable github 
Successfully enabled 'github' at 'github'! 

Set the organization that a GitHub user must be member of:

/ # vault write auth/github/config organization=Praqma 
Success! Data written to: auth/github/configTie the login from GitHub to the policy we created:

/ # vault write auth/github/map/teams/default value=praqma 
Success! Data written to: auth/github/map/teams/default

In this example we used the team default. You can have multiple teams with different policies, like ops and dev, to get more fine grained access level control.

Start the Vault UI container

Vault UI

Exit the Vault container, so we can start the UI container.

We need to specify two environment variables: VAULT_URL_DEFAULT points to the Vault server and VAULT_AUTH_DEFAULT specifies the default authentication method, in this case GitHub:

 docker run -d \
  -p 8000:8000 \
  --network vault-network \
  --hostname vault-ui \
  -e VAULT_URL_DEFAULT="http://vault:8200" \
  -e VAULT_AUTH_DEFAULT="GITHUB" \
  --name vault-ui \
  praqma/vault-ui:2.0.2

Running docker ps should show you two running containers: vault and vault-ui:

CONTAINER ID        IMAGE             COMMAND                  CREATED 
STATUS              PORTS         NAMES 

5941ad1aaeb8   praqma/vault-ui:2.0.2  "/entrypoint.sh np..."  5 seconds ago Up 4 seconds  0.0.0.0:8000-&gt;8000/tcp  vault-ui 

d0d126ac150e   praqma/vault:0.7.0   "docker-entrypoint..."   19 minutes ago    Up 19 minutes   8200/tcp vault 

We see that Vault listens on port 8200 and Vault-UI on port 8000. Go to http://localhost:8000 and you should get the Vault-UI login screen.

Now go to github.com, click settings under your profile picture and then Personal access tokens in the left menu. Click on the button “Generate new token”, give the token a name and choose the permissions you want to give this token (You need at least read permissions).

Insert the token in the Vault-UI to login. You should now see the UI.

Click on the “secret” menu item in the left menu and click the “new secret” button. Give the secret a name and a secret value (by clicking on the square next to “(empty object)”) and then click Save. The new item should appear. Try and click the trash can in the right. You will be told “Error, Access denied”, as defined in our policy from earlier.

Summary

We got started with Vault, beyond dev-mode. We saw how to run and configure the Vault server and a UI where we can get an overview.

Published: Jun 13, 2017

Updated: Mar 26, 2024

Cloud Security