hashipi

Raspberry Pi Test Cluster for HashiCorp Vault, Nomad and Consul
git clone https://git.in0rdr.ch/hashipi.git
Log | Files | Refs | README

commit 41d23bbc383bdf55eccd31ef13532948f2df70b2
parent 992c47b39e7df83d21f03cc347231cb06d9ad9cf
Author: Andreas Gruhler <agruhl@gmx.ch>
Date:   Mon, 27 Jul 2020 12:31:38 +0200

init

Diffstat:
AREADME.md | 81+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Abootstrap.sh | 80+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Aconsul-tls.sh | 46++++++++++++++++++++++++++++++++++++++++++++++
Aconsul.sh | 123+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Ahashi-pi.json | 168+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Ahosts/pi0.json | 16++++++++++++++++
Ahosts/pi1.json | 14++++++++++++++
Ahosts/pi2.json | 14++++++++++++++
Aimg/HashiPi-small.jpg | 0
Anomad.sh | 110+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Avault-tls.sh | 53+++++++++++++++++++++++++++++++++++++++++++++++++++++
Avault.sh | 159+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
12 files changed, 864 insertions(+), 0 deletions(-)

diff --git a/README.md b/README.md @@ -0,0 +1,81 @@ +# HashiPi + +A RaspberryPi test cluster for HashiCorp Vault and Nomad with Consul storage backend. + +![HashiPi](./img/HashiPi-small.jpg) + +## ARM Builder +These Packer files use the [Packer builder for ARM](https://github.com/mkaczanowski/packer-builder-arm). Follow the instructions in the repo to setup the builder and use the provided [examples](https://github.com/mkaczanowski/packer-builder-arm/tree/master/boards) to modify to your liking. + +## Install Custom Nomad Binary + +Copy the [pre-built binary](https://github.com/in0rdr/nomad-arm6l) for the target architecture to the `./bin/nomad` path. The binary file can have an arbitrary name, it need not be set to "nomad". Then, in the Packer JSON, set the `nomad_binary_path` to the relative path of the file, e.g., `nomad_binary_path='./bin/nomad'`. Then, the custom binary is used in the installation stage for Nomad (provisionier [`nomad.sh`](./nomad.sh)) in place of the binary from the [HashiCorp release page](releases.hashicorp.com/nomad/). + +## Self-signed TLS Certificates +### Consul +The steps to create a set of self-signed certificates for Consul are not fully automated to have control over the certificate generation process. + +It would be a minor modification to add the necessary Packe provisioning steps to the JSON file, however, a `consul` on the Packer build host is required to create the initial CA certificate (trust anchor): + +```bash +# create consul CA +mkdir -p tls/consul && cd tls +consul tls ca create +``` + +Then run the script from the projects root directory to create a new set of certificates in the directory `./tls/consul/certs`: +```bash +./consul-tls.sh +``` + +The script can be run after each flashed Raspberry Pi image to create a fresh set of certificates for "server", "client" and "cli" usage. + +### Vault + +To create a new self-signed CA certificate for Vault: +```bash +# create Vault self-signed CA certificate in ./tls/vault/ +./vault-tls.sh +``` + +A new self-signed server certificate is created in the Vault provisioning stage. + +The Packer JSON supports a few arguments for Vault server certificates: +```json + "vault_tls_ca_cert": "./tls/vault/ca/vault_ca.pem", + "vault_tls_ca_key": "./tls/vault/ca/vault_ca.key", + "vault_tls_subj_alt_name": "" +``` + +* `vault_tls_ca_cert`: The path of the CA certificate on the Packer build host, e.g., created with `./vault-tls.sh` +* `vault_tls_ca_key`: The path of the CA key on the Packer build host, e.g., created with `./vault-tls.sh` +* `vault_tls_subj_alt_name`: Comma seperated list of Subject Alternative Names (SAN) for the self-signed certificates, e.g., `DNS:vault.example.com` + + +## Consul ACL's (for Vault) + +tbd + +## Consul Encryption + +Generate a new consul encryption key and set as `consul_encrypt` Packer variable: +```bash +consul keygen +``` + +## Authorized Keys +Copy the contents of an openssh pubkey to `authorized_keys` Packer variable. + +## Troubleshooting + +```bash +* Post-processor failed: unexpected EOF +``` +Ensure that the `flash_device_path` exists or comment/disable the [flasher `post-processor`](https://github.com/mkaczanowski/packer-post-processor-flasher) + +To [write the resulting image file to the sd card](https://www.raspberrypi.org/documentation/installation/installing-images/linux.md) with `dd` without the "flasher" post-processor: + +```bash +sudo dd bs=4M if=HashiPi-pi1.img of=/dev/sda status=progress conv=fsync +``` + diff --git a/bootstrap.sh b/bootstrap.sh @@ -0,0 +1,79 @@ +#!/usr/bin/env bash +# +# Packer shell provisioner for Arch Linux on Raspberry Pi +# +# Based on: +# * https://github.com/mkaczanowski/packer-builder-arm/blob/master/boards/raspberry-pi/archlinuxarm.json +# * https://github.com/bcomnes/raspi-packer + +# set -o errexit +# set -o nounset +set -o xtrace + +# Initialize pacman keyring +# https://archlinuxarm.org/platforms/armv6/raspberry-pi +# https://archlinuxarm.org/platforms/armv8/broadcom/raspberry-pi-3 +pacman-key --init +pacman-key --populate archlinuxarm + +# Enable network connection +mv /etc/resolv.conf /etc/resolv.conf.bck +echo "nameserver 8.8.8.8" > /etc/resolv.conf + +# Sync packages +pacman -Syy --noconfirm +pacman -S parted man sudo unzip inetutils jq docker --noconfirm + +# Disable software rng and enable docker +systemctl disable haveged +systemctl enable docker + +# Set up no-password sudo +echo '%wheel ALL=(ALL) NOPASSWD: ALL' > /etc/sudoers.d/wheel + +# Set up localization: +# https://wiki.archlinux.org/index.php/Installation_guide#Localization +sed -i 's/#en_US.UTF-8 UTF-8/en_US.UTF-8 UTF-8/g' /etc/locale.gen +locale-gen +echo 'LANG=en_US.UTF-8' > /etc/locale.conf + +# Install script to resize fs +mv /tmp/resizerootfs.service /etc/systemd/system/ +chmod +x /tmp/resizerootfs +mv /tmp/resizerootfs /usr/sbin/ +systemctl enable resizerootfs.service + +# Set hostname +echo "${HOSTNAME}" > /etc/hostname + +# Resolve hostname +cat << EOF >> /etc/hosts +127.0.0.1 localhost +::1 localhost +127.0.1.1 $HOSTNAME.localdomain $HOSTNAME +EOF + +# Disable password auth +sed -i 's/#PasswordAuthentication yes/PasswordAuthentication no/g' /etc/ssh/sshd_config + +# Create user +useradd -m "${USERNAME}" +usermod -aG wheel "${USERNAME}" + +# Delete default user alarm:alarm +userdel -r alarm + +# Disable root login root:root +# https://wiki.archlinux.org/index.php/Sudo#Disable_root_login +passwd -l root + +# Setup ssh keys +mkdir "/home/${USERNAME}/.ssh" +touch "/home/${USERNAME}/.ssh/authorized_keys" +cat << EOF > "/home/${USERNAME}/.ssh/authorized_keys" +$AUTHORIZED_KEYS +EOF + +chown -R $USERNAME "/home/${USERNAME}/.ssh" +chmod 700 "/home/${USERNAME}/.ssh" +chmod 600 "/home/${USERNAME}/.ssh/authorized_keys" +\ No newline at end of file diff --git a/consul-tls.sh b/consul-tls.sh @@ -0,0 +1,45 @@ +#!/usr/bin/env bash +# +# Creates a set of certificates for use with HashiCorp Consul +# https://learn.hashicorp.com/consul/datacenter-deploy/deployment-guide + +# set -o errexit +# set -o nounset +# set -o xtrace + +# https://www.shellhacks.com/yes-no-bash-script-prompt-confirmation/ +read -p "Do you want to generate a new set of Consul certicates in the directory \"./tls/consul/\" [y/N]? " -n 1 -r +echo # (optional) move to a new line +if [[ ! $REPLY =~ ^[Yy]$ ]] +then + exit 1 +fi + +# Set working dir +CONSUL_TLS_BASE_PATH="${CONSUL_TLS_BASE_PATH:-./tls/consul/}" +mkdir -p "$CONSUL_TLS_BASE_PATH" +cd "$CONSUL_TLS_BASE_PATH" + +# Cleanup previously generated certificates +rm -rf certs + +# Define cert types +crt_types=("server" "cli" "client") + +for type in "${crt_types[@]}"; do + # Create certificate + consul tls cert create -${type} + cert="dc1-$type-consul-0.pem" + key="dc1-$type-consul-0-key.pem" + + # Show fingerprint + openssl x509 -in $cert -fingerprint -noout + + # Drop index + mv $cert "dc1-${type}-consul.pem" + mv $key "dc1-${type}-consul-key.pem" +done; + +# Move to certs folder +mkdir certs +mv dc1-* certs/ +\ No newline at end of file diff --git a/consul.sh b/consul.sh @@ -0,0 +1,122 @@ +#!/usr/bin/env bash +# +# Packer shell provisioner for HashiCorp Consul on Raspberry Pi +# https://learn.hashicorp.com/consul/datacenter-deploy/deployment-guide + +# set -o errexit +# set -o nounset +set -o xtrace + +CONSUL_URL="https://releases.hashicorp.com/consul" + +cd "/home/${USERNAME}" + +# Download Consul binary and checksums +curl -sS -O "${CONSUL_URL}/${CONSUL_VERSION}/consul_${CONSUL_VERSION}_linux_${CONSUL_ARCH}.zip" +curl -sS -O "${CONSUL_URL}/${CONSUL_VERSION}/consul_${CONSUL_VERSION}_SHA256SUMS" +curl -sS -O "${CONSUL_URL}/${CONSUL_VERSION}/consul_${CONSUL_VERSION}_SHA256SUMS.sig" + +# Verify signature and zip archive +gpg --import "hashicorp.asc" +gpg --verify "consul_${CONSUL_VERSION}_SHA256SUMS.sig" "consul_${CONSUL_VERSION}_SHA256SUMS" +shasum -a 256 -c "consul_${CONSUL_VERSION}_SHA256SUMS" --ignore-missing + +# Install binary +unzip "consul_${CONSUL_VERSION}_linux_${CONSUL_ARCH}.zip" +chown root: consul +mv consul /usr/local/bin/ +consul --version + +# Move uploaded tls files +mkdir -p /opt/consul/tls +mv /tmp/tls/* /opt/consul/tls/ + +# Consul system user +useradd --system --home /etc/consul.d --shell /bin/false consul +chown --recursive consul: /opt/consul + +# Change ownership and permissions for tls certs +chown consul: /opt/consul/tls/*.pem +chmod 640 /opt/consul/tls/*.pem +chmod 644 /opt/consul/tls/dc1-{cli,client}* +chmod 644 /opt/consul/tls/consul-agent-ca.pem + +# Create Consul config files +mkdir -p /etc/consul.d + +cat << EOF > /etc/consul.d/consul.hcl +datacenter = "dc1" +data_dir = "/opt/consul" +encrypt = "$CONSUL_ENCRYPT" + +retry_join = [ + $(printf '%s\n' $CONSUL_RETRY_JOIN) +] + +ports { + server = 8300 + serf_lan = 8301 + serf_wan = -1 + http = -1 + https = 8501 + dns = 8600 +} + +performance { + raft_multiplier = 1 +} +EOF + +cat << EOF > /etc/consul.d/server.hcl +server = true +bootstrap_expect = 3 + +# Auto-encrypt RPC + +# "verify rpc only", because ui=true +ui = true +verify_incoming = false +verify_incoming_rpc = true +verify_outgoing = true +verify_server_hostname = true + +ca_file = "/opt/consul/tls/consul-agent-ca.pem" +cert_file = "/opt/consul/tls/dc1-server-consul.pem" +key_file = "/opt/consul/tls/dc1-server-consul-key.pem" +EOF + +# Configure systemd service unit +cat << EOF > /etc/systemd/system/consul.service +[Unit] +Description="HashiCorp Consul - A service mesh solution" +Documentation=https://www.consul.io/ +Requires=network-online.target +After=network-online.target +ConditionFileNotEmpty=/etc/consul.d/consul.hcl + +[Service] +Type=notify +User=consul +Group=consul +ExecStart=/usr/local/bin/consul agent -config-dir=/etc/consul.d/ +ExecReload=/usr/local/bin/consul reload +KillMode=process +Restart=on-failure +LimitNOFILE=65536 + +[Install] +WantedBy=multi-user.target +EOF + +systemctl enable consul + +# Configure .bashrc +cat << EOF >> .bashrc + +complete -C /usr/local/bin/consul consul + +export CONSUL_HTTP_ADDR="https://127.0.0.1:8501" +export CONSUL_CACERT="/opt/consul/tls/consul-agent-ca.pem" +export CONSUL_CLIENT_CERT="/opt/consul/tls/dc1-cli-consul.pem" +export CONSUL_CLIENT_KEY="/opt/consul/tls/dc1-cli-consul-key.pem" +EOF +\ No newline at end of file diff --git a/hashi-pi.json b/hashi-pi.json @@ -0,0 +1,167 @@ +{ + "variables": { + "hostname": "HashiPi0", + "username": "pi", + "authorized_keys": "", + "img_url": "http://os.archlinuxarm.org/os/ArchLinuxARM-rpi-latest.tar.gz", + "img_name": "raspi.img", + "flash_device_path": null, + "img_size": "3G", + "consul_version": "1.8.0", + "consul_arch": "armelv5", + "consul_encrypt": null, + "consul_tls_ca": null, + "consul_tls_certs": null, + "consul_retry_join": "\"HashiPi0\", \"HashiPi1\", \"HashiPi2\"", + "nomad_version": "0.12.0", + "nomad_binary_path": "./bin", + "nomad_arch": "arm", + "vault_version": "1.5.0", + "vault_arch": "arm", + "vault_tls_ca_cert": "./tls/vault/ca/vault_ca.pem", + "vault_tls_ca_key": "./tls/vault/ca/vault_ca.key", + "vault_tls_subj_alt_name": "" + }, + "sensitive-variables": [ + "consul_encrypt", + "consul_tls_server_key", + "consul_tls_client_key", + "consul_tls_cli_key" + ], + "builders": [{ + "type": "arm", + "file_urls" : ["{{ user `img_url` }}"], + "file_checksum_url": "{{ user `img_url` }}.md5", + "file_checksum_type": "md5", + "file_unarchive_cmd": ["tar", "-xzf", "$ARCHIVE_PATH", "-C", "$MOUNTPOINT"], + "file_target_extension": "tar.gz", + "image_build_method": "new", + "image_path": "{{ user `img_name` }}", + "image_size": "{{ user `img_size` }}", + "image_type": "dos", + "image_partitions": [ + { + "name": "boot", + "type": "c", + "start_sector": "8192", + "filesystem": "vfat", + "size": "256M", + "mountpoint": "/boot" + }, + { + "name": "root", + "type": "83", + "start_sector": "532480", + "filesystem": "ext4", + "size": "0", + "mountpoint": "/" + } + ], + "qemu_binary_source_path": "/usr/bin/qemu-arm-static", + "qemu_binary_destination_path": "/usr/bin/qemu-arm-static" + }], + "provisioners": [ + { + "type": "file", + "source": "packer-builder-arm/scripts/resizerootfs/resizerootfs", + "destination": "/tmp/resizerootfs" + }, + { + "type": "file", + "source": "packer-builder-arm/scripts/resizerootfs/resizerootfs.service", + "destination": "/tmp/resizerootfs.service" + }, + { + "type": "shell", + "script": "bootstrap.sh", + "environment_vars": [ + "HOSTNAME={{ user `hostname` }}", + "USERNAME={{ user `username` }}", + "AUTHORIZED_KEYS={{ user `authorized_keys` }}" + ] + }, + { + "type": "file", + "source": "hashicorp.asc", + "destination": "/home/{{ user `username` }}/hashicorp.asc" + }, + { + "type": "shell", + "inline": ["mkdir /tmp/tls"] + }, + { + "type": "file", + "source": "{{ user `consul_tls_ca` }}", + "destination": "/tmp/tls/consul-agent-ca.pem" + }, + { + "type": "file", + "source": "{{ user `consul_tls_certs` }}", + "destination": "/tmp/tls/" + }, + { + "type": "shell", + "script": "consul.sh", + "remote_folder": "/home/{{ user `username` }}", + "environment_vars": [ + "USERNAME={{ user `username` }}", + "CONSUL_VERSION={{ user `consul_version` }}", + "CONSUL_ARCH={{ user `consul_arch` }}", + "CONSUL_ENCRYPT={{ user `consul_encrypt` }}", + "CONSUL_RETRY_JOIN={{ user `consul_retry_join` }}" + ] + }, + { + "type": "file", + "source": "./bin", + "destination": "/home/{{ user `username` }}/" + }, + { + "type": "shell", + "script": "nomad.sh", + "remote_folder": "/home/{{ user `username` }}", + "environment_vars": [ + "USERNAME={{ user `username` }}", + "NOMAD_VERSION={{ user `nomad_version` }}", + "NOMAD_BINARY_PATH={{ user `nomad_binary_path` }}", + "NOMAD_ARCH={{ user `nomad_arch` }}" + ] + }, + { + "destination": "/tmp/vault_ca.pem", + "source": "{{ user `vault_tls_ca_cert` }}", + "type": "file" + }, + { + "destination": "/tmp/vault_ca.key", + "source": "{{ user `vault_tls_ca_key` }}", + "type": "file" + }, + { + "type": "shell", + "script": "vault.sh", + "remote_folder": "/home/{{ user `username` }}", + "environment_vars": [ + "USERNAME={{ user `username` }}", + "HOSTNAME={{ user `hostname` }}", + "VAULT_VERSION={{ user `vault_version` }}", + "VAULT_ARCH={{ user `vault_arch` }}", + "VAULT_TLS_CA_CERT=/tmp/vault_ca.pem", + "VAULT_TLS_CA_KEY=/tmp/vault_ca.key", + "VAULT_TLS_SUBJ_ALT_NAME={{ user `vault_tls_subj_alt_name` }}" + ] + }, + { + "type": "shell", + "inline": ["mv /etc/resolv.conf.bck /etc/resolv.conf"] + } + ], + "post-processors": [ + { + "type": "flasher", + "device": "{{ user `flash_device_path` }}", + "block_size": "4096", + "interactive": true + } + ] +} +\ No newline at end of file diff --git a/hosts/pi0.json b/hosts/pi0.json @@ -0,0 +1,15 @@ +{ + "hostname": "pi0", + "username": "pi", + "authorized_keys": "", + "img_url": "http://os.archlinuxarm.org/os/ArchLinuxARM-rpi-latest.tar.gz", + "img_name": "HashiPi-pi0.img", + "flash_device_path": "/dev/sda", + "consul_arch": "armelv5", + "consul_encrypt": "", + "consul_tls_ca": "./tls/consul/consul-agent-ca.pem", + "consul_tls_certs": "./tls/consul/certs/", + "consul_retry_join": "\"pi0\", \"pi1\", \"pi2\"", + "nomad_arch": "linux_armv6", + "nomad_binary_path": "./bin/nomad" +} +\ No newline at end of file diff --git a/hosts/pi1.json b/hosts/pi1.json @@ -0,0 +1,13 @@ +{ + "hostname": "pi1", + "username": "pi", + "authorized_keys": "", + "img_url": "http://os.archlinuxarm.org/os/ArchLinuxARM-rpi-2-latest.tar.gz", + "img_name": "HashiPi-pi1.img", + "flash_device_path": "/dev/sda", + "consul_arch": "armhfv6", + "consul_encrypt": "", + "consul_tls_ca": "./tls/consul/consul-agent-ca.pem", + "consul_tls_certs": "./tls/consul/certs/", + "consul_retry_join": "\"pi0\", \"pi1\", \"pi2\"" +} +\ No newline at end of file diff --git a/hosts/pi2.json b/hosts/pi2.json @@ -0,0 +1,13 @@ +{ + "hostname": "pi2", + "username": "pi", + "authorized_keys": "", + "img_url": "http://os.archlinuxarm.org/os/ArchLinuxARM-rpi-2-latest.tar.gz", + "img_name": "HashiPi-pi2.img", + "flash_device_path": "/dev/sda", + "consul_arch": "armhfv6", + "consul_encrypt": "", + "consul_tls_ca": "./tls/consul/consul-agent-ca.pem", + "consul_tls_certs": "./tls/consul/certs/", + "consul_retry_join": "\"pi0\", \"pi1\", \"pi2\"" +} +\ No newline at end of file diff --git a/img/HashiPi-small.jpg b/img/HashiPi-small.jpg Binary files differ. diff --git a/nomad.sh b/nomad.sh @@ -0,0 +1,109 @@ +#!/usr/bin/env bash +# +# Packer shell provisioner for HashiCorp Nomad on Raspberry Pi +# https://www.nomadproject.io/docs/install/production/deployment-guide + +# set -o errexit +# set -o nounset +set -o xtrace + +NOMAD_URL="https://releases.hashicorp.com/nomad" + +cd "/home/${USERNAME}" + +if [[ "$NOMAD_BINARY_PATH" != "./bin" ]]; then + # Use custom binary from previous file provisioner + mv "$NOMAD_BINARY_PATH" ./nomad +else + # Download Nomad binary and checksums + curl -sS -O "${NOMAD_URL}/${NOMAD_VERSION}/nomad_${NOMAD_VERSION}_linux_${NOMAD_ARCH}.zip" + curl -sS -O "${NOMAD_URL}/${NOMAD_VERSION}/nomad_${NOMAD_VERSION}_SHA256SUMS" + curl -sS -O "${NOMAD_URL}/${NOMAD_VERSION}/nomad_${NOMAD_VERSION}_SHA256SUMS.sig" + + # Verify signature and zip archive + gpg --import "hashicorp.asc" + gpg --verify "nomad_${NOMAD_VERSION}_SHA256SUMS.sig" "nomad_${NOMAD_VERSION}_SHA256SUMS" + shasum -a 256 -c "nomad_${NOMAD_VERSION}_SHA256SUMS" --ignore-missing + + # Extract binary + unzip "nomad_${NOMAD_VERSION}_linux_${NOMAD_ARCH}.zip" +fi + +# Fix ownership and install binary +chown root: nomad +mv nomad /usr/local/bin/ + +# Check version +nomad --version + +# Create Nomad data directory +mkdir -p /opt/nomad + +# Create Nomads config files +mkdir -p /etc/nomad.d +chmod 700 /etc/nomad.d + +cat << EOF > /etc/nomad.d/nomad.hcl +datacenter = "dc1" +data_dir = "/opt/nomad" + +consul { + address = "127.0.0.1:8501" + ssl = true + ca_file = "/opt/consul/tls/consul-agent-ca.pem" + cert_file = "/opt/consul/tls/dc1-server-consul.pem" + key_file = "/opt/consul/tls/dc1-server-consul-key.pem" +} +EOF + +# this instance acts as a Nomad client agent +cat << EOF > /etc/nomad.d/client.hcl +client { + enabled = true +} +EOF + +# ..and as a Nomad server agent +# +# https://www.nomadproject.io/docs/configuration +# Note that it is strongly recommended not to operate a node as both client and server, +# although this is supported to simplify development and testing. +cat << EOF > /etc/nomad.d/server.hcl +server { + enabled = true + bootstrap_expect = 3 +} +EOF + +# Configure systemd service unit +cat << EOF > /etc/systemd/system/nomad.service +[Unit] +Description=Nomad +Documentation=https://nomadproject.io/docs/ +Wants=network-online.target +After=network-online.target + +[Service] +ExecReload=/bin/kill -HUP +ExecStart=/usr/local/bin/nomad agent -config /etc/nomad.d +KillMode=process +KillSignal=SIGINT +LimitNOFILE=infinity +LimitNPROC=infinity +Restart=on-failure +RestartSec=2 +StartLimitBurst=3 +StartLimitIntervalSec=10 +TasksMax=infinity + +[Install] +WantedBy=multi-user.target +EOF + +systemctl enable nomad + +# Configure .bashrc +cat << EOF >> .bashrc + +complete -C /usr/local/bin/nomad nomad +EOF +\ No newline at end of file diff --git a/vault-tls.sh b/vault-tls.sh @@ -0,0 +1,52 @@ +#!/usr/bin/env bash +# +# Creates a set of certificates for use with HashiCorp Vault +# https://learn.hashicorp.com/vault/operations/ops-deployment-guide + +# set -o errexit +# set -o nounset +# set -o xtrace + + +# Read server hostnames +# VAULT_SERVER_NAMES="${1:?Error: specify server names as input param, e.g., \`./vault-tls.sh \"pi0 pi1 pi2\"\`}" + +# echo $VAULT_SERVER_NAMES +# echo "${#VAULT_SERVER_NAMES[@]}" + +# VAULT_SAN="${2}" + +# echo "Running the script with:" +# echo " Vault server names (CN): $VAULT_SERVER_NAMES" +# echo " Vault service name (SAN): $VAULT_SAN" +# echo + +# https://www.shellhacks.com/yes-no-bash-script-prompt-confirmation/ +# read -p "Do you want to generate a new set of Vault certicates in the directory \"./tls/vault/\" [y/N]? " -n 1 -r +read -p "Do you want to generate a new Vault CA certicate in the directory \"./tls/vault/\" [y/N]? " -n 1 -r +echo # (optional) move to a new line +if [[ ! $REPLY =~ ^[Yy]$ ]] +then + exit 1 +fi + +# Set working dir +VAULT_TLS_BASE_PATH="${CONSUL_TLS_BASE_PATH:-./tls/vault/}" +mkdir -p "$VAULT_TLS_BASE_PATH" +cd "$VAULT_TLS_BASE_PATH" + +# Cleanup previously generated certificates +rm -rf certs ca +mkdir -p certs ca + +# Create CA cert +CA_CONFIG=" +[ req ] +distinguished_name = dn +[ dn ] +[ ext ] +basicConstraints = critical, CA:true, pathlen:1 +keyUsage = critical, digitalSignature, cRLSign, keyCertSign +" +openssl req -config <(echo "$CA_CONFIG") -new -newkey rsa:2048 -nodes \ + -subj "/CN=Snake Root CA" -x509 -extensions ext -keyout "./ca/vault_ca.key" -out "./ca/vault_ca.pem" +\ No newline at end of file diff --git a/vault.sh b/vault.sh @@ -0,0 +1,158 @@ +#!/usr/bin/env bash +# +# Packer shell provisioner for HashiCorp Vault on Raspberry Pi +# https://learn.hashicorp.com/vault/operations/ops-deployment-guide + +# set -o errexit +# set -o nounset +set -o xtrace + +VAULT_URL="https://releases.hashicorp.com/vault" + +cd "/home/${USERNAME}" + +# Download Vault binary and checksums +curl -sS -O "${VAULT_URL}/${VAULT_VERSION}/vault_${VAULT_VERSION}_linux_${VAULT_ARCH}.zip" +curl -sS -O "${VAULT_URL}/${VAULT_VERSION}/vault_${VAULT_VERSION}_SHA256SUMS" +curl -sS -O "${VAULT_URL}/${VAULT_VERSION}/vault_${VAULT_VERSION}_SHA256SUMS.sig" + +# Verify signature and zip archive +gpg --import "hashicorp.asc" +gpg --verify "vault_${VAULT_VERSION}_SHA256SUMS.sig" "vault_${VAULT_VERSION}_SHA256SUMS" +shasum -a 256 -c "vault_${VAULT_VERSION}_SHA256SUMS" --ignore-missing + +# Install binary +unzip "vault_${VAULT_VERSION}_linux_${VAULT_ARCH}.zip" +chown root: vault +mv vault /usr/local/bin/ +vault --version + +# Create Vault config directories +mkdir -p /etc/vault.d/tls +cd /etc/vault.d/tls + +# Vault system user +useradd --system --home /etc/vault.d --shell /bin/false vault + +# Specify CSR parameters for server key +${VAULT_TLS_SUBJ_ALT_NAME:+", $VAULT_TLS_SUBJ_ALT_NAME"} +SERVER_CONFIG=" +[ req ] +commonName = $HOSTNAME +distinguished_name = dn +req_extensions = ext +[ dn ] +CN = Common Name +[ ext ] +subjectAltName = DNS:$HOSTNAME $VAULT_TLS_SUBJ_ALT_NAME +keyUsage=critical,digitalSignature,keyAgreement +" +# Create new private key and CSR +openssl req -config <(echo "$SERVER_CONFIG") -subj "/CN=${HOSTNAME}" -extensions ext -out "${HOSTNAME}.csr" -new -newkey rsa:2048 -nodes -keyout "${HOSTNAME}.key" +# Sign the CSR +openssl x509 -extfile <(echo "$SERVER_CONFIG") -extensions ext -req -in "${HOSTNAME}.csr" -CA "$VAULT_TLS_CA_CERT" -CAkey "$VAULT_TLS_CA_KEY" -CAcreateserial -out "${HOSTNAME}.pem" -days 365 +# Show fingerprint +openssl x509 -in "${HOSTNAME}.pem" -fingerprint -noout + +# Cleanup CA key +rm -rf "$VAULT_TLS_CA_KEY" + +# Change permissions for tls certs +chmod 640 *.key +chmod 644 *.pem + +# Concatenate CA and server certificate +cat "$VAULT_TLS_CA_CERT" >> "${HOSTNAME}.pem" + +# Trust the CA +mv "$VAULT_TLS_CA_CERT" /etc/ca-certificates/trust-source/anchors/ +update-ca-trust + +# Allow usage of mlock syscall without root +setcap cap_ipc_lock=+ep /usr/local/bin/vault + +cat << EOF > /etc/vault.d/vault.hcl +ui = true + +listener "tcp" { + address = "0.0.0.0:8200" + tls_cert_file = "/etc/vault.d/tls/$HOSTNAME.pem" + tls_key_file = "/etc/vault.d/tls/$HOSTNAME.key" + tls_disable_client_certs = true +} + +# HA advertisement addresses +# +# https://www.vaultproject.io/docs/configuration#high-availability-parameters +# https://www.vaultproject.io/docs/concepts/ha#client-redirection + +# API_ADDR for client redirection (fallback, if request forwarding is disabled) +api_addr = "https://vault.wolke4.org:8200" +# CLUSTER_ADDR: Vault listens for server-to-server cluster requests +cluster_addr = "https://vault.wolke4.org:8201" + +storage "consul" { + address = "https://127.0.0.1:8501" + path = "vault/" + #token = "tbd" + tls_ca_file = "/opt/consul/tls/consul-agent-ca.pem" + tls_cert_file = "/opt/consul/tls/dc1-client-consul.pem" + tls_key_file = "/opt/consul/tls/dc1-client-consul-key.pem" +} +EOF + +chmod 640 /etc/vault.d/vault.hcl + +# Configure systemd service unit +cat << EOF > /etc/systemd/system/vault.service +[Unit] +Description="HashiCorp Vault - A tool for managing secrets" +Documentation=https://www.vaultproject.io/docs/ +Requires=network-online.target +After=network-online.target +ConditionFileNotEmpty=/etc/vault.d/vault.hcl +StartLimitIntervalSec=60 +StartLimitBurst=3 + +[Service] +User=vault +Group=vault +ProtectSystem=full +ProtectHome=read-only +PrivateTmp=yes +PrivateDevices=yes +SecureBits=keep-caps +AmbientCapabilities=CAP_IPC_LOCK +Capabilities=CAP_IPC_LOCK+ep +CapabilityBoundingSet=CAP_SYSLOG CAP_IPC_LOCK +NoNewPrivileges=yes +ExecStart=/usr/local/bin/vault server -config=/etc/vault.d/vault.hcl +ExecReload=/bin/kill --signal HUP $MAINPID +KillMode=process +KillSignal=SIGINT +Restart=on-failure +RestartSec=5 +TimeoutStopSec=30 +StartLimitInterval=60 +StartLimitIntervalSec=60 +StartLimitBurst=3 +LimitNOFILE=65536 +LimitMEMLOCK=infinity + +[Install] +WantedBy=multi-user.target +EOF + +systemctl enable vault + +# Configure .bashrc +cat << EOF >> "/home/${USERNAME}/.bashrc" + +complete -C /usr/local/bin/vault vault +export VAULT_ADDR="https://$HOSTNAME:8200" +EOF + +# Change ownership for config directory +chown -R vault: /etc/vault.d/ + +echo 0 +\ No newline at end of file