ラズパイk8sにVault serverを構築
こちら を参考にラズパイk8sにHAモードでvaultをインストールしてみます。
シークレット管理ツールとしてはSealed Secretsがありますが、
そちらも以前構築してますので
こちら
をご確認ください。
この手順でインストールされるVault Agent Injectorの使い方は
こちら
を参照ください。
環境
ubuntu@k8s1:~$ k get node -o wide
NAME STATUS ROLES AGE VERSION INTERNAL-IP EXTERNAL-IP OS-IMAGE KERNEL-VERSION CONTAINER-RUNTIME
k8s1 Ready control-plane,etcd,master 306d v1.28.10+rke2r1 192.168.0.51 <none> Ubuntu 24.04 LTS 6.8.0-1013-raspi containerd://1.7.11-k3s2
k8s2 Ready worker 184d v1.28.10+rke2r1 192.168.0.52 <none> Ubuntu 24.04.1 LTS 6.8.0-1013-raspi containerd://1.7.11-k3s2
k8s3 Ready worker 306d v1.28.10+rke2r1 192.168.0.53 <none> Ubuntu 24.04 LTS 6.8.0-1013-raspi containerd://1.7.11-k3s2
アップグレードしなきゃなー
consulインストール
ドキュメントわかりにくいですがHAモードで構築するにはConsulストレージバックエンドが必要でした。
こちら
を参考に先にhelmでインストールします。
ubuntu@k8s1:~$ helm repo add hashicorp https://helm.releases.hashicorp.com
"hashicorp" has been added to your repositories
ubuntu@k8s1:~$ helm search repo hashicorp/consul
NAME CHART VERSION APP VERSION DESCRIPTION
hashicorp/consul 1.7.0 1.21.0 Official HashiCorp Consul Chart
上書き用のvalues.yamlを準備します。
特に説明が必要な部分はないですね。
global:
logJSON: true
metrics:
enabled: true
enableAgentMetrics: true
server:
storageClass: "local-path"
extraConfig: |
{
"log_level": "TRACE"
}
ui:
service:
enabled: true
ingress:
enabled: true
hosts:
- host: consul.tsuchinokometal.com
インストールします。
ubuntu@k8s1:~$ helm install consul hashicorp/consul --create-namespace -n consul --values override-consul-values.yaml
NAME: consul
LAST DEPLOYED: Sun May 18 14:01:09 2025
NAMESPACE: consul
STATUS: deployed
REVISION: 1
NOTES:
Thank you for installing HashiCorp Consul!
Your release is named consul.
To learn more about the release, run:
$ helm status consul --namespace consul
$ helm get all consul --namespace consul
Consul on Kubernetes Documentation:
https://www.consul.io/docs/platform/k8s
Consul on Kubernetes CLI Reference:
https://www.consul.io/docs/k8s/k8s-cli
インストールできました。
ubuntu@k8s1:~$ helm list -n consul
NAME NAMESPACE REVISION UPDATED STATUS CHART APP VERSION
consul consul 1 2025-05-18 14:01:09.457879847 +0900 JST deployed consul-1.7.0 1.21.0
ubuntu@k8s1:~$ k get pod,svc,ingress,pvc -n consul
NAME READY STATUS RESTARTS AGE
pod/consul-consul-connect-injector-6fdb9bb58c-54sgl 1/1 Running 1 (103s ago) 2m50s
pod/consul-consul-server-0 1/1 Running 0 2m50s
pod/consul-consul-webhook-cert-manager-5646f84cd5-djxw2 1/1 Running 0 2m50s
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
service/consul-consul-connect-injector ClusterIP 10.43.84.53 <none> 443/TCP 2m51s
service/consul-consul-dns ClusterIP 10.43.59.23 <none> 53/TCP,53/UDP 2m51s
service/consul-consul-server ClusterIP None <none> 8500/TCP,8502/TCP,8301/TCP,8301/UDP,8302/TCP,8302/UDP,8300/TCP,8600/TCP,8600/UDP 2m51s
service/consul-consul-ui ClusterIP 10.43.30.229 <none> 80/TCP 2m51s
NAME CLASS HOSTS ADDRESS PORTS AGE
ingress.networking.k8s.io/consul-consul-ui <none> consul.tsuchinokometal.com 192.168.0.52,192.168.0.53 80 2m50s
NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGE
persistentvolumeclaim/data-consul-consul-consul-server-0 Bound pvc-6f3ff8be-dca3-4605-9698-b362ad9c1742 10Gi RWO local-path 2m50s
ingressでuiにもログインできました。
vaultインストール
ではこちら を参考にVaultをインストールします。
上書き用のvalues.yamlを準備します。
injector:
logFormat: "json"
server:
logFormat: "json"
ingress:
enabled: true
hosts:
- host: vault.tsuchinokometal.com
dataStorage:
enabled: true
storageClass: "local-path"
auditStorage:
enabled: true
storageClass: "local-path"
ha:
enabled: true
replicas: 2
raft:
enabled: true
config: |
ui = true
listener "tcp" {
tls_disable = 1
address = "[::]:8200"
cluster_address = "[::]:8201"
}
storage "consul" {
path = "vault"
address = "consul-consul-server.consul.svc.cluster.local:8500"
}
ui:
enabled: true
serviceType: "LoadBalancer"
serverTelemetry:
serviceMonitor:
enabled: true
こちらの通り addressはインストールしたconsulのserviceを指定します。
storage "consul" {
path = "vault"
address = "consul-consul-server.consul.svc.cluster.local:8500"
}
以下は有効化しないと こちら と同じようにvault-activeのラベルがつかなかったです。
server:
ha:
raft:
enabled: true
uiはingressにできないんですかね。
ui:
enabled: true
serviceType: "LoadBalancer"
ではインストールします。
ubuntu@k8s1:~$ helm install vault hashicorp/vault --create-namespace -n vault --values override-vault-values.yaml
NAME: vault
LAST DEPLOYED: Sun May 18 14:08:08 2025
NAMESPACE: vault
STATUS: deployed
REVISION: 1
NOTES:
Thank you for installing HashiCorp Vault!
Now that you have deployed Vault, you should look over the docs on using
Vault with Kubernetes available here:
https://developer.hashicorp.com/vault/docs
Your release is named vault. To learn more about the release, try:
$ helm status vault
$ helm get manifest vault
無事インストールできましたが、podのREADYが0/1ですね。
これは
封印解除
しないといけないようです。
ubuntu@k8s1:~$ helm list -n vault
NAME NAMESPACE REVISION UPDATED STATUS CHART APP VERSION
vault vault 1 2025-05-18 14:08:08.155445071 +0900 JST deployed vault-0.30.0 1.19.0
ubuntu@k8s1:~$ k get pod,svc,ingress,pvc -n vault
NAME READY STATUS RESTARTS AGE
pod/vault-0 0/1 Running 0 89s
pod/vault-1 0/1 Running 0 89s
pod/vault-agent-injector-564c65f99b-vs5zb 1/1 Running 0 90s
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
service/vault ClusterIP 10.43.118.99 <none> 8200/TCP,8201/TCP 91s
service/vault-active ClusterIP 10.43.5.127 <none> 8200/TCP,8201/TCP 91s
service/vault-agent-injector-svc ClusterIP 10.43.45.190 <none> 443/TCP 91s
service/vault-internal ClusterIP None <none> 8200/TCP,8201/TCP 91s
service/vault-standby ClusterIP 10.43.26.185 <none> 8200/TCP,8201/TCP 91s
service/vault-ui LoadBalancer 10.43.169.80 192.168.0.231 8200:31058/TCP 91s
NAME CLASS HOSTS ADDRESS PORTS AGE
ingress.networking.k8s.io/vault <none> vault.tsuchinokometal.com 192.168.0.52,192.168.0.53 80 90s
NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGE
persistentvolumeclaim/audit-vault-0 Bound pvc-6e68a008-1d69-49e2-b4a7-1037dc3891c1 10Gi RWO local-path 90s
persistentvolumeclaim/audit-vault-1 Bound pvc-21d6cd73-b956-403b-a688-6dc3ccb3f292 10Gi RWO local-path 89s
persistentvolumeclaim/data-vault-0 Bound pvc-7e66d483-7b17-4734-9780-985dc7b809f5 10Gi RWO local-path 90s
persistentvolumeclaim/data-vault-1 Bound pvc-159e9d7c-eae5-4b38-a0e4-f4427b6e50e8 10Gi RWO local-path 89s
初期化します。
ubuntu@k8s1:~$ kubectl -n vault exec -ti vault-0 -- vault operator init
Unseal Key 1: uK4wZweqwCWDk9m/SdGp3oNUYhOdoYbO/sI8MRwgxnxy
Unseal Key 2: 1gTmK1oJnbJWXxOd9yaL2gmWW6J7jxKaJ/gyvEUOrgoK
Unseal Key 3: kyQrZC2sJyEIrZCB3+PAlk/DjPUePl4Xme6rmnLI3mDF
Unseal Key 4: Iv20iP3O7yRPc83jP9ABbzS0GndBeqtsKDaOfBcdRWyL
Unseal Key 5: R0YGOyL4G8HpjmXdBjb20VMeAbgxhycMPMzK9CayaGrC
Initial Root Token: hvs.mjf26MSeASXpVFlaFVIMFfd0
Vault initialized with 5 key shares and a key threshold of 3. Please securely
distribute the key shares printed above. When the Vault is re-sealed,
restarted, or stopped, you must supply at least 3 of these keys to unseal it
before it can start servicing requests.
Vault does not store the generated root key. Without at least 3 keys to
reconstruct the root key, Vault will remain permanently sealed!
It is possible to generate new unseal keys, provided you have a quorum of
existing unseal keys shares. See "vault operator rekey" for more information.
発行されたキーのどれかを使ってunsealします。
ubuntu@k8s1:~$ kubectl -n vault exec -ti vault-0 -- vault operator unseal uK4wZweqwCWDk9m/SdGp3oNUYhOdoYbO/sI8MRwgxnxy
Key Value
--- -----
Seal Type shamir
Initialized true
Sealed true
Total Shares 5
Threshold 3
Unseal Progress 1/3
Unseal Nonce fec99d0c-7374-c690-bc3e-befd1f143030
Version 1.19.0
Build Date 2025-03-04T12:36:40Z
Storage Type raft
Removed From Cluster false
HA Enabled true
ubuntu@k8s1:~$ kubectl -n vault exec -ti vault-0 -- vault operator unseal 1gTmK1oJnbJWXxOd9yaL2gmWW6J7jxKaJ/gyvEUOrgoK
Key Value
--- -----
Seal Type shamir
Initialized true
Sealed true
Total Shares 5
Threshold 3
Unseal Progress 2/3
Unseal Nonce fec99d0c-7374-c690-bc3e-befd1f143030
Version 1.19.0
Build Date 2025-03-04T12:36:40Z
Storage Type raft
Removed From Cluster false
HA Enabled true
ubuntu@k8s1:~$ kubectl -n vault exec -ti vault-0 -- vault operator unseal kyQrZC2sJyEIrZCB3+PAlk/DjPUePl4Xme6rmnLI3mDF
Key Value
--- -----
Seal Type shamir
Initialized true
Sealed false
Total Shares 5
Threshold 3
Version 1.19.0
Build Date 2025-03-04T12:36:40Z
Storage Type raft
Cluster Name vault-cluster-68c1942f
Cluster ID 90d160e5-5c7c-2af8-4c9b-c8bd53d0be4b
Removed From Cluster false
HA Enabled true
HA Cluster https://vault-0.vault-internal:8201
HA Mode active
Active Since 2025-05-18T05:14:07.009565476Z
Raft Committed Index 37
Raft Applied Index 37
vault-1も同様にunsealしようとしたらエラーになりました。
ubuntu@k8s1:~$ kubectl -n vault exec -ti vault-1 -- vault operator unseal kyQrZC2sJyEIrZCB3+PAlk/DjPUePl4Xme6rmnLI3mDF
Error unsealing: Error making API request.
URL: PUT http://127.0.0.1:8200/v1/sys/unseal
Code: 400. Errors:
* Vault is not initialized
command terminated with exit code 2
vault-0は無事1/1になりましたね。
ubuntu@k8s1:~$ kubectl get pods -l app.kubernetes.io/name=vault -A
NAMESPACE NAME READY STATUS RESTARTS AGE
vault vault-0 1/1 Running 0 7m27s
vault vault-1 0/1 Running 0 7m27s
確認したところvault-1がクラスタにいないですね。
ubuntu@k8s1:~$ kubectl -n vault exec -ti vault-0 -- sh
/ $ vault login
Token (will be hidden):
Success! You are now authenticated. The token information displayed below
is already stored in the token helper. You do NOT need to run "vault login"
again. Future Vault requests will automatically use this token.
Key Value
--- -----
token hvs.mjf26MSeASXpVFlaFVIMFfd0
token_accessor D7yE3k34Zh8MLY0FzEKgpV6d
token_duration ∞
token_renewable false
token_policies ["root"]
identity_policies []
policies ["root"]
/ $ vault operator raft list-peers
Node Address State Voter
---- ------- ----- -----
b1d79f77-ff32-0078-49db-ea4034fc8061 vault-0.vault-internal:8201 leader true
/ $ vault operator members
Host Name API Address Cluster Address Active Node Version Upgrade Version Redundancy Zone Last Echo
--------- ----------- --------------- ----------- ------- --------------- --------------- ---------
vault-0 http://10.42.219.31:8200 https://vault-0.vault-internal:8201 true 1.19.0 1.19.0 n/a n/a
こちら を参考にRaftクラスターにvault-1を参加させます。
ubuntu@k8s1:~$ kubectl -n vault exec -ti vault-1 -- vault operator raft join http://vault-0.vault-internal:8200
Key Value
--- -----
Joined true
ubuntu@k8s1:~$ kubectl -n vault exec -ti vault-0 -- vault operator raft list-peers
Node Address State Voter
---- ------- ----- -----
b1d79f77-ff32-0078-49db-ea4034fc8061 vault-0.vault-internal:8201 leader true
ubuntu@k8s1:~$ kubectl -n vault exec -ti vault-0 -- vault operator members
Host Name API Address Cluster Address Active Node Version Upgrade Version Redundancy Zone Last Echo
--------- ----------- --------------- ----------- ------- --------------- --------------- ---------
vault-0 http://10.42.219.31:8200 https://vault-0.vault-internal:8201 true 1.19.0 1.19.0 n/a n/a
ubuntu@k8s1:~$ kubectl -n vault exec -ti vault-1 -- vault operator unseal kyQrZC2sJyEIrZCB3+PAlk/DjPUePl4Xme6rmnLI3mDF
Key Value
--- -----
Seal Type shamir
Initialized true
Sealed true
Total Shares 5
Threshold 3
Unseal Progress 1/3
Unseal Nonce 03107f2f-2ede-1f4f-d6e0-b8e173aa2db7
Version 1.19.0
Build Date 2025-03-04T12:36:40Z
Storage Type raft
Removed From Cluster false
HA Enabled true
ubuntu@k8s1:~$ kubectl -n vault exec -ti vault-1 -- vault operator unseal Iv20iP3O7yRPc83jP9ABbzS0GndBeqtsKDaOfBcdRWyL
Key Value
--- -----
Seal Type shamir
Initialized true
Sealed true
Total Shares 5
Threshold 3
Unseal Progress 2/3
Unseal Nonce 03107f2f-2ede-1f4f-d6e0-b8e173aa2db7
Version 1.19.0
Build Date 2025-03-04T12:36:40Z
Storage Type raft
Removed From Cluster false
HA Enabled true
ubuntu@k8s1:~$ kubectl -n vault exec -ti vault-1 -- vault operator unseal R0YGOyL4G8HpjmXdBjb20VMeAbgxhycMPMzK9CayaGrC
Key Value
--- -----
Seal Type shamir
Initialized true
Sealed true
Total Shares 5
Threshold 3
Unseal Progress 0/3
Unseal Nonce n/a
Version 1.19.0
Build Date 2025-03-04T12:36:40Z
Storage Type raft
Removed From Cluster false
HA Enabled true
ubuntu@k8s1:~$ kubectl -n vault exec -ti vault-0 -- vault operator raft list-peers
Node Address State Voter
---- ------- ----- -----
b1d79f77-ff32-0078-49db-ea4034fc8061 vault-0.vault-internal:8201 leader true
56b071f2-30e3-978a-d2f6-17ced40299ce vault-1.vault-internal:8201 follower false
ubuntu@k8s1:~$ kubectl -n vault exec -ti vault-0 -- vault operator members
Host Name API Address Cluster Address Active Node Version Upgrade Version Redundancy Zone Last Echo
--------- ----------- --------------- ----------- ------- --------------- --------------- ---------
vault-1 http://10.42.109.81:8200 https://vault-1.vault-internal:8201 false 1.19.0 1.19.0 n/a 2025-05-18T05:23:38Z
vault-0 http://10.42.219.31:8200 https://vault-0.vault-internal:8201 true 1.19.0 1.19.0 n/a n/a
3回目の結果がおかしいですが、無事両方READYが1/1になりました。
ubuntu@k8s1:~$ kubectl get pods -l app.kubernetes.io/name=vault -A
NAMESPACE NAME READY STATUS RESTARTS AGE
vault vault-0 1/1 Running 0 16m
vault vault-1 1/1 Running 0 16m
vault cliインストール
こちら
の手順でmacOSにインストールします。
mac側でingressのhostを名前解決できるようにしておいてください。
tokenはinit実行時のtokenです。
% export VAULT_ADDR=http://vault.tsuchinokometal.com
% vault login
Token (will be hidden):
Success! You are now authenticated. The token information displayed below
is already stored in the token helper. You do NOT need to run "vault login"
again. Future Vault requests will automatically use this token.
Key Value
--- -----
token hvs.mjf26MSeASXpVFlaFVIMFfd0
token_accessor D7yE3k34Zh8MLY0FzEKgpV6d
token_duration ∞
token_renewable false
token_policies ["root"]
identity_policies []
policies ["root"]
% vault operator members
Host Name API Address Cluster Address Active Node Version Upgrade Version Redundancy Zone Last Echo
--------- ----------- --------------- ----------- ------- --------------- --------------- ---------
vault-1 http://10.42.109.81:8200 https://vault-1.vault-internal:8201 false 1.19.0 1.19.0 n/a 2025-05-18T05:26:43Z
vault-0 http://10.42.219.31:8200 https://vault-0.vault-internal:8201 true 1.19.0 1.19.0 n/a n/a
% vault operator raft list-peers
Node Address State Voter
---- ------- ----- -----
b1d79f77-ff32-0078-49db-ea4034fc8061 vault-0.vault-internal:8201 leader true
56b071f2-30e3-978a-d2f6-17ced40299ce vault-1.vault-internal:8201 follower true
uiも試します。
LoadBalancerのIPでアクセスします。
ログイン時のtokenもinit実行時のtokenです。
ログインできました。
ubuntuにvault cliインストール
ちょっと触ってみましたがkubectlと一緒に使えると便利なことに気づいたので
コントロールプレーンノードにインストールすることにしました。
こちら
からバイナリをダウンロードします。
$ wget https://releases.hashicorp.com/vault/1.19.0/vault_1.19.0_linux_arm64.zip
$ unzip vault_1.19.0_linux_arm64.zip
$ sudo mv vault /usr/local/bin/
$ vault version
Vault v1.19.0 (7eeafb6160d60ede73c1d95566b0c8ea54f3cb5a), built 2025-03-04T12:36:40Z
毎回認証するのがめんどくさいので環境変数を設定します。
export VAULT_ADDR='http://vault.tsuchinokometal.com'
export VAULT_TOKEN='hvs.mjf26MSeASXpVFlaFVIMFfd0'
これで使いやすくなると思います。
$ vault status
Key Value
--- -----
Seal Type shamir
Initialized true
Sealed false
Total Shares 5
Threshold 3
Version 1.19.0
Build Date 2025-03-04T12:36:40Z
Storage Type raft
Cluster Name vault-cluster-68c1942f
Cluster ID 90d160e5-5c7c-2af8-4c9b-c8bd53d0be4b
Removed From Cluster false
HA Enabled true
HA Cluster https://vault-1.vault-internal:8201
HA Mode active
Active Since 2025-05-31T05:00:59.777268674Z
Raft Committed Index 2638
Raft Applied Index 2638