Introduction
This document provides a comprehensive architectural overview of the system, using a number of different architectural views to depict different aspects of the system. It is intended to capture and convey the significant architectural decisions that have been made on the system.
VAULT Deployment Architecture
In each DC, Vault will be deployed with the following setup:
- integrated storage (Raft backend)
- cluster with 1 active node, 2 standby nodes
- auto-unseal using transit secrete engine
Deployment
Environment
| Host | IP | Node |
|---|---|---|
| pln-n1-eso4sap1 | 204.104.46.242 | Cluster Node |
| pln-n1-eso4sap2 | 204.104.46.243 | Cluster Node |
| pln-n1-eso4sap3 | 204.104.46.244 | Cluster Node |
| pln-n1-eso4sap4 | 204.104.248.31 | Auto-unseal Node |
Vault Installation and configuration
Run below command to install vault to the four servers detailed above, and enable vault server as a service
- Run "yum install vault"
- Run "systemctl enable vault"
- Put "p4s_vault_cert.pem", "p4s_vault_key.pem" and "vault_ca.pem" to directory /opt/vault/tls
- edit configure file in /etc/vault.d/vault/hcl in three servers
ui = true
disable_mlock = true
storage "raft" {
path = "/var/vault/data"
node_id = "<%= @node_name%>"
}
listener "tcp" {
address = "0.0.0.0:8200"
cluster_address = "0.0.0.0:8201"
tls_cert_file = "/opt/vault/tls/p4s_vault_cert.pem"
tls_key_file = "/opt/vault/tls/p4s_vault_key.pem"
tls_disable_client_certs = true
}
api_addr = "https://<%= @node_ip%>:8200"
cluster_addr = "https://<%= @node_ip%>:8201"
<%= @node_name%> is node name in cluster, you can pick up any name here, here used host name of server.
<%= @node_ip%> is server IP
Vault server initiallization and cluster configuraion
Go to Server 1: pln-n1-eso4sap1 and run commands
- systemctl start vault
- export VAULT_ADDR="https://204.104.46.242:8200"
- export VAULT_SKIP_VERIFY=true
- vault operator init -key-shares=3 -key-threshold=2
This command will generate three unseal keys and a root token. Please keep these values in secret place, and will be used later** - export VAULT_TOKEN=s.vlyrBOoTjpBfU6K1MR2VYeA5(Replace token with gerneated by up step)
- vault secrets enable kv-v2
Go to Server 2: pln-n1-eso4sap2 and run commands
- systemctl start vault
- export VAULT_ADDR="https://204.104.46.243:8200"
- export VAULT_SKIP_VERIFY=true
- vault operator raft join -leader-ca-cert=@/opt/vault/tls/vault_ca.pem https://204.104.46.242:8200
- vault operator unseal zMLoNgWylWjsAWUQa6ZHbb43zvLhs3WBNhvk3Enpp9Q=(Replace unseal key with gerneated by up step)
- vault operator unseal Xlu+UajvWu9RV6sziFh0D6q/xHUeXEYYH0UWdkAtejMI(Replace unseal key with gerneated by up step)
Go to Server 3: pln-n1-eso4sap3 and run commands
- systemctl start vault
- export VAULT_ADDR="https://204.104.46.244:8200"
- export VAULT_SKIP_VERIFY=true
- vault operator raft join -leader-ca-cert=@/opt/vault/tls/vault_ca.pem https://204.104.46.242:8200
- vault operator unseal zMLoNgWylWjsAWUQa6ZHbb43zvLhs3WBNhvk3Enpp9Q=(Replace unseal key with gerneated by up step)
- vault operator unseal Xlu+UajvWu9RV6sziFh0D6q/xHUeXEYYH0UWdkAtejMI(Replace unseal key with gerneated by up step)
Configure "Re-Join" Section
- edit configure file in /etc/vault.d/vault/hcl and update content to "storage" section for each servers
pln-n1-eso4sap1
storage "raft" {
path = "/var/vault/data"
node_id = "pln-n1-eso4sap1"
retry_join {
leader_api_addr = "https://204.104.46.243:8200"
leader_ca_cert_file = "/opt/vault/tls/vault_ca.pem"
}
retry_join {
leader_api_addr = "https://204.104.46.244:8200"
leader_ca_cert_file = "/opt/vault/tls/vault_ca.pem"
}
}
pln-n1-eso4sap2
storage "raft" {
path = "/var/vault/data"
node_id = "pln-n1-eso4sap2"
retry_join {
leader_api_addr = "https://204.104.46.242:8200"
leader_ca_cert_file = "/opt/vault/tls/vault_ca.pem"
}
retry_join {
leader_api_addr = "https://204.104.46.244:8200"
leader_ca_cert_file = "/opt/vault/tls/vault_ca.pem"
}
}
pln-n1-eso4sap3.env01.mcloud.entsvcs.net
storage "raft" {
path = "/var/vault/data"
node_id = "pln-n1-eso4sap3"
retry_join {
leader_api_addr = "https://204.104.46.242:8200"
leader_ca_cert_file = "/opt/vault/tls/vault_ca.pem"
}
retry_join {
leader_api_addr = "https://204.104.46.243:8200"
leader_ca_cert_file = "/opt/vault/tls/vault_ca.pem"
}
}
Configure Auto-unseal
Run the following commands to initial the unseal server pln-n1-eso4sap4:
systemctl start vault
export VAULT_ADDR="https://204.104.248.31:8200"
export VAULT_SKIP_VERIFY=true
vault operator init -key-shares=3 -key-threshold=2
This command will generate three unseal keys and a root token. Please keep these values in secret place, and will be used laterexport VAULT_TOKEN=s.vlyrBOoTjpBfU6K1MR2VYeA5(Replace token with gerneated by up step)
vault secrets enable kv-v2
Configure Auto-unseal Key Provider
- Enable audit log
vault audit enable file file_path=/var/log/vault/vault-audit.log
- Enable transit secrete engine, this can also be configured via UI
vault secrets enable transit
- Create an encryption key - autounseal, this can also be configured via UI
vault write -f transit/keys/autounseal
- Create a policy (autounseal) which enable user to update the autounseal key
$ tee autounseal.hcl <<EOF
path "transit/encrypt/autounseal" {
capabilities = [ "update" ]
}
path "transit/decrypt/autounseal" {
capabilities = [ "update" ]
}
EOF
vault policy write autounseal autounseal.hcl
- Generate auto-unseal token
vault write auth/token/create policies=autounseal
the current lease ttl can be checked using the following command
vault read sys/auth/token/tune
this vault can be set using
vault write sys/auth/token/tune max_lease_ttl=9000h
vault write sys/auth/token/tune default_lease_ttl=9000h
*9000h for 1 year
token generated here will be passed to the unseal server.
Configure Auto-unseal Server
Auto-unseal can either be configured before server/cluster is initialed or after.
- Configure auto-unseal in server's configuration file, for example, /etc/vault.d/vault/hcl**
auto-unseal configuration
ui = true
disable_mlock = true
storage "file" {
path = "/var/vault/data"
}
listener "tcp" {
address = "0.0.0.0:8200"
tls_cert_file = "/opt/vault/tls/p4s_vault_cert.pem"
tls_key_file = "/opt/vault/tls/p4s_vault_key.pem"
tls_disable_client_certs = true
}
seal "transit"`{
address = "https://204.104.248.31:8200"
token = "s.lqEgk3aB5rN5xyRmsKm7FnNV"
disable_renewal = "false"
key_name = "autounseal"
mount_path = "transit/"
tls_skip_verify = "true"
}
- If server is already initialed:
- for each server in the Vault cluster, shut down Vault service
systemctl stop vault
- for each server in the Vault cluster, start Vault service and run unseal with migrate option
systemctl start vault
vault operator unseal -migrate.
This will migrate the previous unseal mechanism to transit key auto-unseal, and the previous unseal keys will be migrated as the recovery keys. The migration process can only be completed when all nodes in the Vault cluster are restarted and unsealed, otherwise all nodes will remain in seal status.
