Storing Files in Vault as a K/V Pair

Sometimes I find myself needing to stash a couple files away somewhere safe. Here is a quick trick to that I use to store files in my Hashicorp Vault instance.

Create a gzip’d tarball of the directory of files.

tar -cvzf secrets.tar.gz secrets_dir/*

This next line does two things. The first thing is it base64 encodes the tarball, turning the binary representation of the file into a text representation of your file. The second part of this command sends the file into Vault at the path kv/secret_tar under the key filecontents.

vault kv put kv/secret_tar filecontents="$(base64 secrets.tar.gz)"

The file now rests securely in Vault. When we need the file again, do something like this:

base64 --decode <(vault kv get -format=json kv/secret_tar | jq -r .data.data | jq -r .[]) > retrieved_secrets.tar.gz

There is a lot going on here. This command leverages a trick called process substitution “<()” in BASH. Vault outputs data in a table format by default, which is nice to look at but a poor format for automation. Using the Vault CLI flag -format=json along with jq, we are able to extract the base64-encoded file from vault. Pass the string from Vault into base64 --decode to translate the file back into its binary representation.

tar xvzf retrieved_secrets.tar.gz

Finally untar the directory and the original files will be available for consumption.