homelab_kubernetes

GitOps repo for the homelab Kubernetes cluster, synced by Flux from ./clusters/homelab.

Secrets (SOPS + age)

Secret manifests in this repo are encrypted with SOPS using an age key pair, per .sops.yaml at the repo root. The corresponding private key lives only in the cluster (as the sops-age Secret in flux-system, managed by the Homelab_infra Terraform repo) and in a Vaultwarden backup — never in this repo.

The file on disk is always encrypted

A secret manifest here — in your editor, in git log, browsing on the remote — is always ciphertext. It never gets decrypted in place; that would defeat the point of committing it to git at all. Seeing ENC[...] values when you open one of these files is correct, not a bug.

Decryption only happens in two places, both transient, neither of which writes plaintext back into the repo:

1. Locally, on demand:
   • sops -d path/to/secret.yaml — prints the decrypted content to your terminal. The file on disk is untouched.
   • sops path/to/secret.yaml (no flags) — decrypts into a temp buffer, opens it in $EDITOR, then re-encrypts and overwrites the file when you save and exit. This is how you edit an existing secret.
2. Inside the cluster, on every Flux reconcile: kustomize-controller fetches the ciphertext from git, decrypts it in memory using the sops-age key, and applies the decrypted result to the Kubernetes API. It never writes anything back to git — only the live cluster object (kubectl get secret ... -o yaml) reflects the decrypted value.

Adding a new secret

# 1. Write the plaintext manifest - do not commit this version
cat > path/to/secret.yaml <<'EOF'
apiVersion: v1
kind: Secret
metadata:
  name: example
  namespace: default
type: Opaque
stringData:
  key: real-value
EOF

# 2. Encrypt it in place
sops -e -i path/to/secret.yaml

# 3. Commit and push the now-encrypted file
git add path/to/secret.yaml
git commit -m "add example secret"
git push

Flux picks it up on its next reconcile (or force one immediately: annotate the GitRepository/Kustomization in flux-system with reconcile.fluxcd.io/requestedAt=$(date +%s)).

Path matters: only files under ./clusters/homelab are synced at all (per the Kustomization in clusters/homelab/flux-system/gotk-sync.yaml) - anything outside that path is invisible to Flux regardless of encryption.

Restoring the age key on a new machine

The private key has no source of truth other than the Vaultwarden secure note it was backed up to at generation time. Restore the full age.agekey file content from there into %AppData%\sops\age\keys.txt (SOPS's default lookup location on Windows) to decrypt/edit secrets locally again. Never regenerate with age-keygen - a fresh key can't decrypt anything already encrypted with the old one.

Planned improvements

• Zigbee network key isn't backed up in git. The real network key/PAN ID/extended PAN ID (restored from the old Docker Zigbee2MQTT instance during migration) only lives on the zigbee2mqtt-data PVC now. A full disaster-recovery rebuild (not just a pod restart) would regenerate a fresh random network via the ConfigMap template, requiring every Zigbee device to be re-paired. Should move advanced.network_key/pan_id/ ext_pan_id into a SOPS-encrypted Secret, same pattern already used for the Postgres/Authentik credentials.

• Open WebUI doesn't map Authentik groups to roles. Right now the first person to ever log in becomes Open WebUI admin (its own bootstrap behavior, unrelated to Authentik), and everyone after that lands as pending until manually approved in Open WebUI's admin panel - Authentik group membership (e.g. a "read-only" group) currently has no effect at all. To fix: add groups to OAUTH_SCOPES (and confirm Authentik's provider actually includes a groups claim), then set ENABLE_OAUTH_ROLE_MANAGEMENT=true, OAUTH_ROLES_CLAIM=groups, and OAUTH_ADMIN_ROLES=<admin-group-name> (optionally OAUTH_ALLOWED_ROLES to gate login by group entirely) on the openwebui Deployment.

• No CrowdSec bouncer in front of public apps. The old Docker Nextcloud had a CrowdSec bouncer plugin on its Traefik router (community threat- intelligence-based IP blocking). There's no equivalent in the Gateway API/Cilium setup yet - Traefik's plugin system doesn't carry over directly, would need a separate CrowdSec deployment plus whatever Envoy/Cilium-compatible bouncer integration exists (if any) or an alternative enforcement point. Not started - bigger standalone project.

• Vaultwarden SSO considered, deliberately skipped for now. Vaultwarden never lets SSO replace the master password (client always prompts for it separately to decrypt the vault), and SSO_SIGNUPS_MATCH_EMAIL only auto-links accounts that don't already have a private key - Alfred/ Cindy/Cor all have real migrated vault data, so it wouldn't apply to any of them. Net effect for the three real users would just be an extra login gate before the real credential check, not actual SSO convenience - only future brand-new accounts would get real benefit. Revisit if that defense-in-depth angle (Authentik's own MFA/audit logging gating access to the vault) becomes worth the setup effort later.