claude/infrastructure
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

K8s security hardening + scaling to 16 CPU / 64GB / 1TB #65

Merged
claude merged 2 commits from k8s-hardening-scaling into main 2026-02-14 09:44:26 +01:00
Conversation 13 Commits 2 Files Changed 2 +45 -27
claude commented 2026-02-14 09:32:19 +01:00
Owner
Copy Link

Security Hardening

  • REMOVED DNAT rules for K8s API (6443) and ArgoCD (30443) — were open to ANY IP
  • REMOVED Proxmox FW rules for 6443/30443 (no longer needed)
  • Access to K8s API and ArgoCD now exclusively via SSH tunnel (k8s-tunnel.service)
  • Monitoring DNAT (9200-9202) restricted to control plane IP only
  • FORWARD chain protection added on bare_srv_1

Scaling

Node CPU RAM Disk IP
k8s-master 4 16 GB 100 GB 10.10.10.200
k8s-worker-01 6 24 GB 450 GB 10.10.10.201
k8s-worker-02 (NEW) 6 24 GB 450 GB 10.10.10.202
Total 16 64 GB 1 TB

Notes

  • Existing VMs will be resized in-place (restart required for CPU/RAM)
  • Disk grow + growpart/resize2fs needed after apply
  • New worker needs kubeadm join after cloud-init completes
  • FW resource renamed: k8s_api_access → k8s_monitoring_access
## Security Hardening - **REMOVED** DNAT rules for K8s API (6443) and ArgoCD (30443) — were open to ANY IP - **REMOVED** Proxmox FW rules for 6443/30443 (no longer needed) - Access to K8s API and ArgoCD now exclusively via SSH tunnel (`k8s-tunnel.service`) - Monitoring DNAT (9200-9202) restricted to control plane IP only - FORWARD chain protection added on bare_srv_1 ## Scaling | Node | CPU | RAM | Disk | IP | |------|-----|-----|------|----| | k8s-master | 4 | 16 GB | 100 GB | 10.10.10.200 | | k8s-worker-01 | 6 | 24 GB | 450 GB | 10.10.10.201 | | k8s-worker-02 (NEW) | 6 | 24 GB | 450 GB | 10.10.10.202 | | **Total** | **16** | **64 GB** | **1 TB** | | ## Notes - Existing VMs will be resized in-place (restart required for CPU/RAM) - Disk grow + growpart/resize2fs needed after apply - New worker needs kubeadm join after cloud-init completes - FW resource renamed: `k8s_api_access` → `k8s_monitoring_access`
claude added 1 commit 2026-02-14 09:32:20 +01:00
K8s security hardening + scaling to half bare_srv_1
Some checks failed
PR Checks / tofu-checks (pull_request) Failing after 3s
Details
1/1 projects planned successfully.
e43f4dfc90 
Security:
- Remove DNAT/FW rules for K8s API (6443) and ArgoCD (30443)
- Access now via SSH tunnel (k8s-tunnel.service on control plane)
- Keep monitoring DNAT (9200-9202) restricted to control plane IP

Scaling:
- k8s-master: 4 CPU, 16GB RAM, 100GB disk
- k8s-worker-01: 6 CPU, 24GB RAM, 450GB disk
- k8s-worker-02: 6 CPU, 24GB RAM, 450GB disk (NEW)
- Total: 16 CPU, 64GB RAM, 1TB disk (half of bare_srv_1)
claude commented 2026-02-14 09:32:26 +01:00
Author
Owner
Copy Link

Ran Plan for project: production dir: environments/production workspace: default

Show Output 
OpenTofu used the selected providers to generate the following execution
plan. Resource actions are indicated with the following symbols:
+ create
- destroy
-/+ destroy and then create replacement

OpenTofu will perform the following actions:

  # proxmox_virtual_environment_firewall_rules.k8s_api_access will be destroyed
  # (because proxmox_virtual_environment_firewall_rules.k8s_api_access is not in configuration)
- resource "proxmox_virtual_environment_firewall_rules" "k8s_api_access" {
      - id        = "rule-1674445101" -> null
      - node_name = "georgeops" -> null

      - rule {
          - action  = "ACCEPT" -> null
          - comment = "K8s API from control plane (DNAT to k8s-master)" -> null
          - dport   = "6443" -> null
          - enabled = true -> null
          - pos     = 0 -> null
          - proto   = "tcp" -> null
          - source  = "78.109.17.180" -> null
          - type    = "in" -> null
        }
      - rule {
          - action  = "ACCEPT" -> null
          - comment = "ArgoCD UI from control plane (DNAT to k8s-master)" -> null
          - dport   = "30443" -> null
          - enabled = true -> null
          - pos     = 1 -> null
          - proto   = "tcp" -> null
          - source  = "78.109.17.180" -> null
          - type    = "in" -> null
        }
      - rule {
          - action  = "ACCEPT" -> null
          - comment = "k8s-master node_exporter (DNAT)" -> null
          - dport   = "9200" -> null
          - enabled = true -> null
          - pos     = 2 -> null
          - proto   = "tcp" -> null
          - source  = "78.109.17.180" -> null
          - type    = "in" -> null
        }
      - rule {
          - action  = "ACCEPT" -> null
          - comment = "k8s-worker-01 node_exporter (DNAT)" -> null
          - dport   = "9201" -> null
          - enabled = true -> null
          - pos     = 3 -> null
          - proto   = "tcp" -> null
          - source  = "78.109.17.180" -> null
          - type    = "in" -> null
        }
    }

  # proxmox_virtual_environment_firewall_rules.k8s_monitoring_access will be created
+ resource "proxmox_virtual_environment_firewall_rules" "k8s_monitoring_access" {
      + id        = (known after apply)
      + node_name = "georgeops"

      + rule {
          + action  = "ACCEPT"
          + comment = "k8s-master node_exporter (DNAT)"
          + dport   = "9200"
          + enabled = true
          + pos     = (known after apply)
          + proto   = "tcp"
          + source  = "78.109.17.180"
          + type    = "in"
        }
      + rule {
          + action  = "ACCEPT"
          + comment = "k8s-worker-01 node_exporter (DNAT)"
          + dport   = "9201"
          + enabled = true
          + pos     = (known after apply)
          + proto   = "tcp"
          + source  = "78.109.17.180"
          + type    = "in"
        }
      + rule {
          + action  = "ACCEPT"
          + comment = "k8s-worker-02 node_exporter (DNAT)"
          + dport   = "9202"
          + enabled = true
          + pos     = (known after apply)
          + proto   = "tcp"
          + source  = "78.109.17.180"
          + type    = "in"
        }
    }

  # module.k8s_node["k8s-master"].proxmox_virtual_environment_file.cloud_init must be replaced
-/+ resource "proxmox_virtual_environment_file" "cloud_init" {
      + file_modification_date = (known after apply)
      ~ file_name              = "ci-k8s-master.yaml" -> (known after apply)
      + file_size              = (known after apply)
      + file_tag               = (known after apply)
      ~ id                     = "local:snippets/ci-k8s-master.yaml" -> (known after apply)
        # (5 unchanged attributes hidden)

      ~ source_raw {
          ~ data      = <<-EOT # forces replacement
                #cloud-config
                # K8s node cloud-init — installs containerd + kubeadm + node_exporter
                # kubeadm init/join is NOT run here — done manually after boot
                
                hostname: k8s-master
                manage_etc_hosts: true
                disable_root: false
                
                users:
                  - name: root
                    ssh_authorized_keys:
                      - ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIDO+Y8ns0RgUfR21POlIVsHD+Lp+x7cUBupqXsyMeVNZ claude@control-plane
                    shell: /bin/bash
                
                package_update: true
                packages:
                  - apt-transport-https
                  - ca-certificates
                  - curl
                  - gnupg
              +   - conntrack
                
                write_files:
                  # Kernel modules for K8s networking
                  - path: /etc/modules-load.d/k8s.conf
                    content: |
                      overlay
                      br_netfilter
                
                  # Sysctl for K8s networking
                  - path: /etc/sysctl.d/99-kubernetes.conf
                    content: |
                      net.bridge.bridge-nf-call-iptables = 1
                      net.bridge.bridge-nf-call-ip6tables = 1
                      net.ipv4.ip_forward = 1
                
                  # containerd config — systemd cgroup driver (required for kubeadm)
                  - path: /etc/containerd/config.toml
                    content: |
                      version = 2
                      [plugins."io.containerd.grpc.v1.cri".containerd.runtimes.runc]
                        runtime_type = "io.containerd.runc.v2"
                      [plugins."io.containerd.grpc.v1.cri".containerd.runtimes.runc.options]
                        SystemdCgroup = true
                
                  # node_exporter systemd unit
                  - path: /etc/systemd/system/node_exporter.service
                    content: |
                      [Unit]
                      Description=Prometheus Node Exporter
                      After=network.target
                
                      [Service]
                      User=node_exporter
                      ExecStart=/usr/local/bin/node_exporter
                      Restart=on-failure
                      RestartSec=5
                
                      [Install]
                      WantedBy=multi-user.target
                
                runcmd:
                  # ── Kernel modules ──
                  - modprobe overlay
                  - modprobe br_netfilter
                  - sysctl --system
                
                  # ── Disable swap (required for K8s) ──
                  - swapoff -a
                  - sed -i '/swap/d' /etc/fstab
                
                  # ── Install containerd from Docker repo ──
                  - install -m 0755 -d /etc/apt/keyrings
                  - curl -fsSL https://download.docker.com/linux/ubuntu/gpg -o /etc/apt/keyrings/docker.asc
                  - chmod a+r /etc/apt/keyrings/docker.asc
                  - echo "deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.asc] https://download.docker.com/linux/ubuntu $(. /etc/os-release && echo $VERSION_CODENAME) stable" > /etc/apt/sources.list.d/docker.list
                  - apt-get update
                  - apt-get install -y containerd.io
                  - mkdir -p /etc/containerd
                  - systemctl restart containerd
                  - systemctl enable containerd
                
                  # ── Install kubeadm, kubelet, kubectl (v1.31) ──
                  - curl -fsSL https://pkgs.k8s.io/core:/stable:/v1.31/deb/Release.key | gpg --dearmor -o /etc/apt/keyrings/kubernetes-apt-keyring.gpg
                  - echo "deb [signed-by=/etc/apt/keyrings/kubernetes-apt-keyring.gpg] https://pkgs.k8s.io/core:/stable:/v1.31/deb/ /" > /etc/apt/sources.list.d/kubernetes.list
                  - apt-get update
                  - apt-get install -y kubelet kubeadm kubectl
                  - apt-mark hold kubelet kubeadm kubectl
                
                  # ── Install node_exporter for monitoring ──
                  - useradd --no-create-home --shell /bin/false node_exporter
                  - curl -fsSL https://github.com/prometheus/node_exporter/releases/download/v1.10.2/node_exporter-1.10.2.linux-amd64.tar.gz -o /tmp/node_exporter.tar.gz
                  - tar xzf /tmp/node_exporter.tar.gz -C /tmp
                  - cp /tmp/node_exporter-1.10.2.linux-amd64/node_exporter /usr/local/bin/
                  - chown node_exporter:node_exporter /usr/local/bin/node_exporter
                  - rm -rf /tmp/node_exporter*
                  - systemctl daemon-reload
                  - systemctl enable --now node_exporter
                
                  # ── Signal cloud-init completion ──
                  - touch /var/lib/cloud/instance/k8s-ready
            EOT
            # (2 unchanged attributes hidden)
        }
    }

  # module.k8s_node["k8s-master"].proxmox_virtual_environment_vm.k8s_node must be replaced
-/+ resource "proxmox_virtual_environment_vm" "k8s_node" {
      ~ boot_order                           = [
          - "virtio0",
          - "net0",
        ] -> (known after apply)
      + hotplug                              = (known after apply)
      ~ id                                   = "300" -> (known after apply)
      ~ ipv4_addresses                       = [] -> (known after apply)
      ~ ipv6_addresses                       = [] -> (known after apply)
      ~ mac_addresses                        = [
          - "BC:24:11:D8:3D:89",
        ] -> (known after apply)
        name                                 = "k8s-master"
      ~ network_interface_names              = [] -> (known after apply)
        tags                                 = [
            "k8s",
            "tofu",
            "ubuntu",
        ]
        # (25 unchanged attributes hidden)

      ~ cpu {
          - flags      = [] -> null
          ~ units      = 0 -> (known after apply)
            # (6 unchanged attributes hidden)
        }

      ~ disk {
          ~ path_in_datastore = "300/vm-300-disk-0.qcow2" -> (known after apply)
          ~ size              = 60 -> 100
            # (11 unchanged attributes hidden)
        }

      ~ initialization {
          ~ file_format          = "qcow2" -> (known after apply)
          - interface            = "ide2" -> null
          + meta_data_file_id    = (known after apply)
          + network_data_file_id = (known after apply)
          + type                 = (known after apply)
          ~ user_data_file_id    = "local:snippets/ci-k8s-master.yaml" # forces replacement -> (known after apply) # forces replacement
          + vendor_data_file_id  = (known after apply)
            # (1 unchanged attribute hidden)

            # (2 unchanged blocks hidden)
        }

      ~ memory {
          ~ dedicated      = 8192 -> 16384
            # (3 unchanged attributes hidden)
        }

      ~ network_device {
          - disconnected = false -> null
          ~ mac_address  = "BC:24:11:D8:3D:89" -> (known after apply)
            # (8 unchanged attributes hidden)
        }

      ~ vga {
          + acpi                                 = (known after apply)
          + bios                                 = (known after apply)
          + boot_order                           = (known after apply)
          + delete_unreferenced_disks_on_destroy = (known after apply)
          + description                          = (known after apply)
          + hook_script_file_id                  = (known after apply)
          + hotplug                              = (known after apply)
          + id                                   = (known after apply)
          + ipv4_addresses                       = (known after apply)
          + ipv6_addresses                       = (known after apply)
          + keyboard_layout                      = (known after apply)
          + kvm_arguments                        = (known after apply)
          + mac_addresses                        = (known after apply)
          + machine                              = (known after apply)
          + migrate                              = (known after apply)
          + name                                 = (known after apply)
          + network_interface_names              = (known after apply)
          + node_name                            = (known after apply)
          + on_boot                              = (known after apply)
          + pool_id                              = (known after apply)
          + protection                           = (known after apply)
          + purge_on_destroy                     = (known after apply)
          + reboot                               = (known after apply)
          + reboot_after_update                  = (known after apply)
          + scsi_hardware                        = (known after apply)
          + started                              = (known after apply)
          + stop_on_destroy                      = (known after apply)
          + tablet_device                        = (known after apply)
          + tags                                 = (known after apply)
          + template                             = (known after apply)
          + timeout_clone                        = (known after apply)
          + timeout_create                       = (known after apply)
          + timeout_migrate                      = (known after apply)
          + timeout_move_disk                    = (known after apply)
          + timeout_reboot                       = (known after apply)
          + timeout_shutdown_vm                  = (known after apply)
          + timeout_start_vm                     = (known after apply)
          + timeout_stop_vm                      = (known after apply)
          + vm_id                                = (known after apply)
        } -> (known after apply)
    }

  # module.k8s_node["k8s-worker-01"].proxmox_virtual_environment_file.cloud_init must be replaced
-/+ resource "proxmox_virtual_environment_file" "cloud_init" {
      + file_modification_date = (known after apply)
      ~ file_name              = "ci-k8s-worker-01.yaml" -> (known after apply)
      + file_size              = (known after apply)
      + file_tag               = (known after apply)
      ~ id                     = "local:snippets/ci-k8s-worker-01.yaml" -> (known after apply)
        # (5 unchanged attributes hidden)

      ~ source_raw {
          ~ data      = <<-EOT # forces replacement
                #cloud-config
                # K8s node cloud-init — installs containerd + kubeadm + node_exporter
                # kubeadm init/join is NOT run here — done manually after boot
                
                hostname: k8s-worker-01
                manage_etc_hosts: true
                disable_root: false
                
                users:
                  - name: root
                    ssh_authorized_keys:
                      - ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIDO+Y8ns0RgUfR21POlIVsHD+Lp+x7cUBupqXsyMeVNZ claude@control-plane
                    shell: /bin/bash
                
                package_update: true
                packages:
                  - apt-transport-https
                  - ca-certificates
                  - curl
                  - gnupg
              +   - conntrack
                
                write_files:
                  # Kernel modules for K8s networking
                  - path: /etc/modules-load.d/k8s.conf
                    content: |
                      overlay
                      br_netfilter
                
                  # Sysctl for K8s networking
                  - path: /etc/sysctl.d/99-kubernetes.conf
                    content: |
                      net.bridge.bridge-nf-call-iptables = 1
                      net.bridge.bridge-nf-call-ip6tables = 1
                      net.ipv4.ip_forward = 1
                
                  # containerd config — systemd cgroup driver (required for kubeadm)
                  - path: /etc/containerd/config.toml
                    content: |
                      version = 2
                      [plugins."io.containerd.grpc.v1.cri".containerd.runtimes.runc]
                        runtime_type = "io.containerd.runc.v2"
                      [plugins."io.containerd.grpc.v1.cri".containerd.runtimes.runc.options]
                        SystemdCgroup = true
                
                  # node_exporter systemd unit
                  - path: /etc/systemd/system/node_exporter.service
                    content: |
                      [Unit]
                      Description=Prometheus Node Exporter
                      After=network.target
                
                      [Service]
                      User=node_exporter
                      ExecStart=/usr/local/bin/node_exporter
                      Restart=on-failure
                      RestartSec=5
                
                      [Install]
                      WantedBy=multi-user.target
                
                runcmd:
                  # ── Kernel modules ──
                  - modprobe overlay
                  - modprobe br_netfilter
                  - sysctl --system
                
                  # ── Disable swap (required for K8s) ──
                  - swapoff -a
                  - sed -i '/swap/d' /etc/fstab
                
                  # ── Install containerd from Docker repo ──
                  - install -m 0755 -d /etc/apt/keyrings
                  - curl -fsSL https://download.docker.com/linux/ubuntu/gpg -o /etc/apt/keyrings/docker.asc
                  - chmod a+r /etc/apt/keyrings/docker.asc
                  - echo "deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.asc] https://download.docker.com/linux/ubuntu $(. /etc/os-release && echo $VERSION_CODENAME) stable" > /etc/apt/sources.list.d/docker.list
                  - apt-get update
                  - apt-get install -y containerd.io
                  - mkdir -p /etc/containerd
                  - systemctl restart containerd
                  - systemctl enable containerd
                
                  # ── Install kubeadm, kubelet, kubectl (v1.31) ──
                  - curl -fsSL https://pkgs.k8s.io/core:/stable:/v1.31/deb/Release.key | gpg --dearmor -o /etc/apt/keyrings/kubernetes-apt-keyring.gpg
                  - echo "deb [signed-by=/etc/apt/keyrings/kubernetes-apt-keyring.gpg] https://pkgs.k8s.io/core:/stable:/v1.31/deb/ /" > /etc/apt/sources.list.d/kubernetes.list
                  - apt-get update
                  - apt-get install -y kubelet kubeadm kubectl
                  - apt-mark hold kubelet kubeadm kubectl
                
                  # ── Install node_exporter for monitoring ──
                  - useradd --no-create-home --shell /bin/false node_exporter
                  - curl -fsSL https://github.com/prometheus/node_exporter/releases/download/v1.10.2/node_exporter-1.10.2.linux-amd64.tar.gz -o /tmp/node_exporter.tar.gz
                  - tar xzf /tmp/node_exporter.tar.gz -C /tmp
                  - cp /tmp/node_exporter-1.10.2.linux-amd64/node_exporter /usr/local/bin/
                  - chown node_exporter:node_exporter /usr/local/bin/node_exporter
                  - rm -rf /tmp/node_exporter*
                  - systemctl daemon-reload
                  - systemctl enable --now node_exporter
                
                  # ── Signal cloud-init completion ──
                  - touch /var/lib/cloud/instance/k8s-ready
            EOT
            # (2 unchanged attributes hidden)
        }
    }

  # module.k8s_node["k8s-worker-01"].proxmox_virtual_environment_vm.k8s_node must be replaced
-/+ resource "proxmox_virtual_environment_vm" "k8s_node" {
      ~ boot_order                           = [
          - "virtio0",
          - "net0",
        ] -> (known after apply)
      + hotplug                              = (known after apply)
      ~ id                                   = "301" -> (known after apply)
      ~ ipv4_addresses                       = [] -> (known after apply)
      ~ ipv6_addresses                       = [] -> (known after apply)
      ~ mac_addresses                        = [
          - "BC:24:11:F1:56:29",
        ] -> (known after apply)
        name                                 = "k8s-worker-01"
      ~ network_interface_names              = [] -> (known after apply)
        tags                                 = [
            "k8s",
            "tofu",
            "ubuntu",
        ]
        # (25 unchanged attributes hidden)

      ~ cpu {
          ~ cores      = 4 -> 6
          - flags      = [] -> null
          ~ units      = 0 -> (known after apply)
            # (5 unchanged attributes hidden)
        }

      ~ disk {
          ~ path_in_datastore = "301/vm-301-disk-0.qcow2" -> (known after apply)
          ~ size              = 60 -> 450
            # (11 unchanged attributes hidden)
        }

      ~ initialization {
          ~ file_format          = "qcow2" -> (known after apply)
          - interface            = "ide2" -> null
          + meta_data_file_id    = (known after apply)
          + network_data_file_id = (known after apply)
          + type                 = (known after apply)
          ~ user_data_file_id    = "local:snippets/ci-k8s-worker-01.yaml" # forces replacement -> (known after apply) # forces replacement
          + vendor_data_file_id  = (known after apply)
            # (1 unchanged attribute hidden)

            # (2 unchanged blocks hidden)
        }

      ~ memory {
          ~ dedicated      = 8192 -> 24576
            # (3 unchanged attributes hidden)
        }

      ~ network_device {
          - disconnected = false -> null
          ~ mac_address  = "BC:24:11:F1:56:29" -> (known after apply)
            # (8 unchanged attributes hidden)
        }

      ~ vga {
          + acpi                                 = (known after apply)
          + bios                                 = (known after apply)
          + boot_order                           = (known after apply)
          + delete_unreferenced_disks_on_destroy = (known after apply)
          + description                          = (known after apply)
          + hook_script_file_id                  = (known after apply)
          + hotplug                              = (known after apply)
          + id                                   = (known after apply)
          + ipv4_addresses                       = (known after apply)
          + ipv6_addresses                       = (known after apply)
          + keyboard_layout                      = (known after apply)
          + kvm_arguments                        = (known after apply)
          + mac_addresses                        = (known after apply)
          + machine                              = (known after apply)
          + migrate                              = (known after apply)
          + name                                 = (known after apply)
          + network_interface_names              = (known after apply)
          + node_name                            = (known after apply)
          + on_boot                              = (known after apply)
          + pool_id                              = (known after apply)
          + protection                           = (known after apply)
          + purge_on_destroy                     = (known after apply)
          + reboot                               = (known after apply)
          + reboot_after_update                  = (known after apply)
          + scsi_hardware                        = (known after apply)
          + started                              = (known after apply)
          + stop_on_destroy                      = (known after apply)
          + tablet_device                        = (known after apply)
          + tags                                 = (known after apply)
          + template                             = (known after apply)
          + timeout_clone                        = (known after apply)
          + timeout_create                       = (known after apply)
          + timeout_migrate                      = (known after apply)
          + timeout_move_disk                    = (known after apply)
          + timeout_reboot                       = (known after apply)
          + timeout_shutdown_vm                  = (known after apply)
          + timeout_start_vm                     = (known after apply)
          + timeout_stop_vm                      = (known after apply)
          + vm_id                                = (known after apply)
        } -> (known after apply)
    }

  # module.k8s_node["k8s-worker-02"].proxmox_virtual_environment_file.cloud_init will be created
+ resource "proxmox_virtual_environment_file" "cloud_init" {
      + content_type           = "snippets"
      + datastore_id           = "local"
      + file_modification_date = (known after apply)
      + file_name              = (known after apply)
      + file_size              = (known after apply)
      + file_tag               = (known after apply)
      + id                     = (known after apply)
      + node_name              = "georgeops"
      + overwrite              = true
      + timeout_upload         = 1800

      + source_raw {
          + data      = <<-EOT
                #cloud-config
                # K8s node cloud-init — installs containerd + kubeadm + node_exporter
                # kubeadm init/join is NOT run here — done manually after boot
                
                hostname: k8s-worker-02
                manage_etc_hosts: true
                disable_root: false
                
                users:
                  - name: root
                    ssh_authorized_keys:
                      - ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIDO+Y8ns0RgUfR21POlIVsHD+Lp+x7cUBupqXsyMeVNZ claude@control-plane
                    shell: /bin/bash
                
                package_update: true
                packages:
                  - apt-transport-https
                  - ca-certificates
                  - curl
                  - gnupg
                  - conntrack
                
                write_files:
                  # Kernel modules for K8s networking
                  - path: /etc/modules-load.d/k8s.conf
                    content: |
                      overlay
                      br_netfilter
                
                  # Sysctl for K8s networking
                  - path: /etc/sysctl.d/99-kubernetes.conf
                    content: |
                      net.bridge.bridge-nf-call-iptables = 1
                      net.bridge.bridge-nf-call-ip6tables = 1
                      net.ipv4.ip_forward = 1
                
                  # containerd config — systemd cgroup driver (required for kubeadm)
                  - path: /etc/containerd/config.toml
                    content: |
                      version = 2
                      [plugins."io.containerd.grpc.v1.cri".containerd.runtimes.runc]
                        runtime_type = "io.containerd.runc.v2"
                      [plugins."io.containerd.grpc.v1.cri".containerd.runtimes.runc.options]
                        SystemdCgroup = true
                
                  # node_exporter systemd unit
                  - path: /etc/systemd/system/node_exporter.service
                    content: |
                      [Unit]
                      Description=Prometheus Node Exporter
                      After=network.target
                
                      [Service]
                      User=node_exporter
                      ExecStart=/usr/local/bin/node_exporter
                      Restart=on-failure
                      RestartSec=5
                
                      [Install]
                      WantedBy=multi-user.target
                
                runcmd:
                  # ── Kernel modules ──
                  - modprobe overlay
                  - modprobe br_netfilter
                  - sysctl --system
                
                  # ── Disable swap (required for K8s) ──
                  - swapoff -a
                  - sed -i '/swap/d' /etc/fstab
                
                  # ── Install containerd from Docker repo ──
                  - install -m 0755 -d /etc/apt/keyrings
                  - curl -fsSL https://download.docker.com/linux/ubuntu/gpg -o /etc/apt/keyrings/docker.asc
                  - chmod a+r /etc/apt/keyrings/docker.asc
                  - echo "deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.asc] https://download.docker.com/linux/ubuntu $(. /etc/os-release && echo $VERSION_CODENAME) stable" > /etc/apt/sources.list.d/docker.list
                  - apt-get update
                  - apt-get install -y containerd.io
                  - mkdir -p /etc/containerd
                  - systemctl restart containerd
                  - systemctl enable containerd
                
                  # ── Install kubeadm, kubelet, kubectl (v1.31) ──
                  - curl -fsSL https://pkgs.k8s.io/core:/stable:/v1.31/deb/Release.key | gpg --dearmor -o /etc/apt/keyrings/kubernetes-apt-keyring.gpg
                  - echo "deb [signed-by=/etc/apt/keyrings/kubernetes-apt-keyring.gpg] https://pkgs.k8s.io/core:/stable:/v1.31/deb/ /" > /etc/apt/sources.list.d/kubernetes.list
                  - apt-get update
                  - apt-get install -y kubelet kubeadm kubectl
                  - apt-mark hold kubelet kubeadm kubectl
                
                  # ── Install node_exporter for monitoring ──
                  - useradd --no-create-home --shell /bin/false node_exporter
                  - curl -fsSL https://github.com/prometheus/node_exporter/releases/download/v1.10.2/node_exporter-1.10.2.linux-amd64.tar.gz -o /tmp/node_exporter.tar.gz
                  - tar xzf /tmp/node_exporter.tar.gz -C /tmp
                  - cp /tmp/node_exporter-1.10.2.linux-amd64/node_exporter /usr/local/bin/
                  - chown node_exporter:node_exporter /usr/local/bin/node_exporter
                  - rm -rf /tmp/node_exporter*
                  - systemctl daemon-reload
                  - systemctl enable --now node_exporter
                
                  # ── Signal cloud-init completion ──
                  - touch /var/lib/cloud/instance/k8s-ready
            EOT
          + file_name = "ci-k8s-worker-02.yaml"
          + resize    = 0
        }
    }

  # module.k8s_node["k8s-worker-02"].proxmox_virtual_environment_vm.k8s_node will be created
+ resource "proxmox_virtual_environment_vm" "k8s_node" {
      + acpi                                 = true
      + bios                                 = "seabios"
      + boot_order                           = (known after apply)
      + delete_unreferenced_disks_on_destroy = true
      + hotplug                              = (known after apply)
      + id                                   = (known after apply)
      + ipv4_addresses                       = (known after apply)
      + ipv6_addresses                       = (known after apply)
      + keyboard_layout                      = "en-us"
      + mac_addresses                        = (known after apply)
      + migrate                              = false
      + name                                 = "k8s-worker-02"
      + network_interface_names              = (known after apply)
      + node_name                            = "georgeops"
      + on_boot                              = true
      + protection                           = false
      + purge_on_destroy                     = true
      + reboot                               = false
      + reboot_after_update                  = true
      + scsi_hardware                        = "virtio-scsi-pci"
      + started                              = true
      + stop_on_destroy                      = true
      + tablet_device                        = true
      + tags                                 = [
          + "k8s",
          + "tofu",
          + "ubuntu",
        ]
      + template                             = false
      + timeout_clone                        = 1800
      + timeout_create                       = 1800
      + timeout_migrate                      = 1800
      + timeout_move_disk                    = 1800
      + timeout_reboot                       = 1800
      + timeout_shutdown_vm                  = 1800
      + timeout_start_vm                     = 1800
      + timeout_stop_vm                      = 300
      + vm_id                                = 302

      + cpu {
          + cores      = 6
          + hotplugged = 0
          + limit      = 0
          + numa       = false
          + sockets    = 1
          + type       = "x86-64-v2-AES"
          + units      = (known after apply)
        }

      + disk {
          + aio               = "io_uring"
          + backup            = true
          + cache             = "none"
          + datastore_id      = "local"
          + discard           = "on"
          + file_format       = "qcow2"
          + file_id           = "local:iso/ubuntu-24.04-cloudimg-amd64.img"
          + interface         = "virtio0"
          + iothread          = true
          + path_in_datastore = (known after apply)
          + replicate         = true
          + size              = 450
          + ssd               = false
        }

      + initialization {
          + datastore_id         = "local"
          + file_format          = (known after apply)
          + meta_data_file_id    = (known after apply)
          + network_data_file_id = (known after apply)
          + type                 = (known after apply)
          + user_data_file_id    = (known after apply)
          + vendor_data_file_id  = (known after apply)

          + dns {
              + servers = [
                  + "8.8.8.8",
                  + "1.1.1.1",
                ]
            }

          + ip_config {
              + ipv4 {
                  + address = "10.10.10.202/24"
                  + gateway = "10.10.10.1"
                }
            }
        }

      + memory {
          + dedicated      = 24576
          + floating       = 0
          + keep_hugepages = false
          + shared         = 0
        }

      + network_device {
          + bridge      = "vmbr0"
          + enabled     = true
          + firewall    = false
          + mac_address = (known after apply)
          + model       = "virtio"
          + mtu         = 0
          + queues      = 0
          + rate_limit  = 0
          + vlan_id     = 0
        }

      + vga (known after apply)
    }

Plan: 7 to add, 0 to change, 5 to destroy.

Changes to Outputs:
~ k8s_nodes     = {
      + k8s-worker-02 = {
          + ip_address = "10.10.10.202"
          + vm_id      = 302
        }
        # (2 unchanged attributes hidden)
    }
  • ▶️ To apply this plan, comment: 
    atlantis apply -p production
  • 🚮 To delete this plan and lock, click here
  • 🔁 To plan this project again, comment: 
    atlantis plan -p production

Plan: 7 to add, 0 to change, 5 to destroy.

  • ⏩ To apply all unapplied plans from this Pull Request, comment: 
    atlantis apply
  • 🚮 To delete all plans and locks from this Pull Request, comment: 
    atlantis unlock
Ran Plan for project: `production` dir: `environments/production` workspace: `default` <details><summary>Show Output</summary> ```diff OpenTofu used the selected providers to generate the following execution plan. Resource actions are indicated with the following symbols: + create - destroy -/+ destroy and then create replacement OpenTofu will perform the following actions: # proxmox_virtual_environment_firewall_rules.k8s_api_access will be destroyed # (because proxmox_virtual_environment_firewall_rules.k8s_api_access is not in configuration) - resource "proxmox_virtual_environment_firewall_rules" "k8s_api_access" { - id = "rule-1674445101" -> null - node_name = "georgeops" -> null - rule { - action = "ACCEPT" -> null - comment = "K8s API from control plane (DNAT to k8s-master)" -> null - dport = "6443" -> null - enabled = true -> null - pos = 0 -> null - proto = "tcp" -> null - source = "78.109.17.180" -> null - type = "in" -> null } - rule { - action = "ACCEPT" -> null - comment = "ArgoCD UI from control plane (DNAT to k8s-master)" -> null - dport = "30443" -> null - enabled = true -> null - pos = 1 -> null - proto = "tcp" -> null - source = "78.109.17.180" -> null - type = "in" -> null } - rule { - action = "ACCEPT" -> null - comment = "k8s-master node_exporter (DNAT)" -> null - dport = "9200" -> null - enabled = true -> null - pos = 2 -> null - proto = "tcp" -> null - source = "78.109.17.180" -> null - type = "in" -> null } - rule { - action = "ACCEPT" -> null - comment = "k8s-worker-01 node_exporter (DNAT)" -> null - dport = "9201" -> null - enabled = true -> null - pos = 3 -> null - proto = "tcp" -> null - source = "78.109.17.180" -> null - type = "in" -> null } } # proxmox_virtual_environment_firewall_rules.k8s_monitoring_access will be created + resource "proxmox_virtual_environment_firewall_rules" "k8s_monitoring_access" { + id = (known after apply) + node_name = "georgeops" + rule { + action = "ACCEPT" + comment = "k8s-master node_exporter (DNAT)" + dport = "9200" + enabled = true + pos = (known after apply) + proto = "tcp" + source = "78.109.17.180" + type = "in" } + rule { + action = "ACCEPT" + comment = "k8s-worker-01 node_exporter (DNAT)" + dport = "9201" + enabled = true + pos = (known after apply) + proto = "tcp" + source = "78.109.17.180" + type = "in" } + rule { + action = "ACCEPT" + comment = "k8s-worker-02 node_exporter (DNAT)" + dport = "9202" + enabled = true + pos = (known after apply) + proto = "tcp" + source = "78.109.17.180" + type = "in" } } # module.k8s_node["k8s-master"].proxmox_virtual_environment_file.cloud_init must be replaced -/+ resource "proxmox_virtual_environment_file" "cloud_init" { + file_modification_date = (known after apply) ~ file_name = "ci-k8s-master.yaml" -> (known after apply) + file_size = (known after apply) + file_tag = (known after apply) ~ id = "local:snippets/ci-k8s-master.yaml" -> (known after apply) # (5 unchanged attributes hidden) ~ source_raw { ~ data = <<-EOT # forces replacement #cloud-config # K8s node cloud-init — installs containerd + kubeadm + node_exporter # kubeadm init/join is NOT run here — done manually after boot hostname: k8s-master manage_etc_hosts: true disable_root: false users: - name: root ssh_authorized_keys: - ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIDO+Y8ns0RgUfR21POlIVsHD+Lp+x7cUBupqXsyMeVNZ claude@control-plane shell: /bin/bash package_update: true packages: - apt-transport-https - ca-certificates - curl - gnupg + - conntrack write_files: # Kernel modules for K8s networking - path: /etc/modules-load.d/k8s.conf content: | overlay br_netfilter # Sysctl for K8s networking - path: /etc/sysctl.d/99-kubernetes.conf content: | net.bridge.bridge-nf-call-iptables = 1 net.bridge.bridge-nf-call-ip6tables = 1 net.ipv4.ip_forward = 1 # containerd config — systemd cgroup driver (required for kubeadm) - path: /etc/containerd/config.toml content: | version = 2 [plugins."io.containerd.grpc.v1.cri".containerd.runtimes.runc] runtime_type = "io.containerd.runc.v2" [plugins."io.containerd.grpc.v1.cri".containerd.runtimes.runc.options] SystemdCgroup = true # node_exporter systemd unit - path: /etc/systemd/system/node_exporter.service content: | [Unit] Description=Prometheus Node Exporter After=network.target [Service] User=node_exporter ExecStart=/usr/local/bin/node_exporter Restart=on-failure RestartSec=5 [Install] WantedBy=multi-user.target runcmd: # ── Kernel modules ── - modprobe overlay - modprobe br_netfilter - sysctl --system # ── Disable swap (required for K8s) ── - swapoff -a - sed -i '/swap/d' /etc/fstab # ── Install containerd from Docker repo ── - install -m 0755 -d /etc/apt/keyrings - curl -fsSL https://download.docker.com/linux/ubuntu/gpg -o /etc/apt/keyrings/docker.asc - chmod a+r /etc/apt/keyrings/docker.asc - echo "deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.asc] https://download.docker.com/linux/ubuntu $(. /etc/os-release && echo $VERSION_CODENAME) stable" > /etc/apt/sources.list.d/docker.list - apt-get update - apt-get install -y containerd.io - mkdir -p /etc/containerd - systemctl restart containerd - systemctl enable containerd # ── Install kubeadm, kubelet, kubectl (v1.31) ── - curl -fsSL https://pkgs.k8s.io/core:/stable:/v1.31/deb/Release.key | gpg --dearmor -o /etc/apt/keyrings/kubernetes-apt-keyring.gpg - echo "deb [signed-by=/etc/apt/keyrings/kubernetes-apt-keyring.gpg] https://pkgs.k8s.io/core:/stable:/v1.31/deb/ /" > /etc/apt/sources.list.d/kubernetes.list - apt-get update - apt-get install -y kubelet kubeadm kubectl - apt-mark hold kubelet kubeadm kubectl # ── Install node_exporter for monitoring ── - useradd --no-create-home --shell /bin/false node_exporter - curl -fsSL https://github.com/prometheus/node_exporter/releases/download/v1.10.2/node_exporter-1.10.2.linux-amd64.tar.gz -o /tmp/node_exporter.tar.gz - tar xzf /tmp/node_exporter.tar.gz -C /tmp - cp /tmp/node_exporter-1.10.2.linux-amd64/node_exporter /usr/local/bin/ - chown node_exporter:node_exporter /usr/local/bin/node_exporter - rm -rf /tmp/node_exporter* - systemctl daemon-reload - systemctl enable --now node_exporter # ── Signal cloud-init completion ── - touch /var/lib/cloud/instance/k8s-ready EOT # (2 unchanged attributes hidden) } } # module.k8s_node["k8s-master"].proxmox_virtual_environment_vm.k8s_node must be replaced -/+ resource "proxmox_virtual_environment_vm" "k8s_node" { ~ boot_order = [ - "virtio0", - "net0", ] -> (known after apply) + hotplug = (known after apply) ~ id = "300" -> (known after apply) ~ ipv4_addresses = [] -> (known after apply) ~ ipv6_addresses = [] -> (known after apply) ~ mac_addresses = [ - "BC:24:11:D8:3D:89", ] -> (known after apply) name = "k8s-master" ~ network_interface_names = [] -> (known after apply) tags = [ "k8s", "tofu", "ubuntu", ] # (25 unchanged attributes hidden) ~ cpu { - flags = [] -> null ~ units = 0 -> (known after apply) # (6 unchanged attributes hidden) } ~ disk { ~ path_in_datastore = "300/vm-300-disk-0.qcow2" -> (known after apply) ~ size = 60 -> 100 # (11 unchanged attributes hidden) } ~ initialization { ~ file_format = "qcow2" -> (known after apply) - interface = "ide2" -> null + meta_data_file_id = (known after apply) + network_data_file_id = (known after apply) + type = (known after apply) ~ user_data_file_id = "local:snippets/ci-k8s-master.yaml" # forces replacement -> (known after apply) # forces replacement + vendor_data_file_id = (known after apply) # (1 unchanged attribute hidden) # (2 unchanged blocks hidden) } ~ memory { ~ dedicated = 8192 -> 16384 # (3 unchanged attributes hidden) } ~ network_device { - disconnected = false -> null ~ mac_address = "BC:24:11:D8:3D:89" -> (known after apply) # (8 unchanged attributes hidden) } ~ vga { + acpi = (known after apply) + bios = (known after apply) + boot_order = (known after apply) + delete_unreferenced_disks_on_destroy = (known after apply) + description = (known after apply) + hook_script_file_id = (known after apply) + hotplug = (known after apply) + id = (known after apply) + ipv4_addresses = (known after apply) + ipv6_addresses = (known after apply) + keyboard_layout = (known after apply) + kvm_arguments = (known after apply) + mac_addresses = (known after apply) + machine = (known after apply) + migrate = (known after apply) + name = (known after apply) + network_interface_names = (known after apply) + node_name = (known after apply) + on_boot = (known after apply) + pool_id = (known after apply) + protection = (known after apply) + purge_on_destroy = (known after apply) + reboot = (known after apply) + reboot_after_update = (known after apply) + scsi_hardware = (known after apply) + started = (known after apply) + stop_on_destroy = (known after apply) + tablet_device = (known after apply) + tags = (known after apply) + template = (known after apply) + timeout_clone = (known after apply) + timeout_create = (known after apply) + timeout_migrate = (known after apply) + timeout_move_disk = (known after apply) + timeout_reboot = (known after apply) + timeout_shutdown_vm = (known after apply) + timeout_start_vm = (known after apply) + timeout_stop_vm = (known after apply) + vm_id = (known after apply) } -> (known after apply) } # module.k8s_node["k8s-worker-01"].proxmox_virtual_environment_file.cloud_init must be replaced -/+ resource "proxmox_virtual_environment_file" "cloud_init" { + file_modification_date = (known after apply) ~ file_name = "ci-k8s-worker-01.yaml" -> (known after apply) + file_size = (known after apply) + file_tag = (known after apply) ~ id = "local:snippets/ci-k8s-worker-01.yaml" -> (known after apply) # (5 unchanged attributes hidden) ~ source_raw { ~ data = <<-EOT # forces replacement #cloud-config # K8s node cloud-init — installs containerd + kubeadm + node_exporter # kubeadm init/join is NOT run here — done manually after boot hostname: k8s-worker-01 manage_etc_hosts: true disable_root: false users: - name: root ssh_authorized_keys: - ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIDO+Y8ns0RgUfR21POlIVsHD+Lp+x7cUBupqXsyMeVNZ claude@control-plane shell: /bin/bash package_update: true packages: - apt-transport-https - ca-certificates - curl - gnupg + - conntrack write_files: # Kernel modules for K8s networking - path: /etc/modules-load.d/k8s.conf content: | overlay br_netfilter # Sysctl for K8s networking - path: /etc/sysctl.d/99-kubernetes.conf content: | net.bridge.bridge-nf-call-iptables = 1 net.bridge.bridge-nf-call-ip6tables = 1 net.ipv4.ip_forward = 1 # containerd config — systemd cgroup driver (required for kubeadm) - path: /etc/containerd/config.toml content: | version = 2 [plugins."io.containerd.grpc.v1.cri".containerd.runtimes.runc] runtime_type = "io.containerd.runc.v2" [plugins."io.containerd.grpc.v1.cri".containerd.runtimes.runc.options] SystemdCgroup = true # node_exporter systemd unit - path: /etc/systemd/system/node_exporter.service content: | [Unit] Description=Prometheus Node Exporter After=network.target [Service] User=node_exporter ExecStart=/usr/local/bin/node_exporter Restart=on-failure RestartSec=5 [Install] WantedBy=multi-user.target runcmd: # ── Kernel modules ── - modprobe overlay - modprobe br_netfilter - sysctl --system # ── Disable swap (required for K8s) ── - swapoff -a - sed -i '/swap/d' /etc/fstab # ── Install containerd from Docker repo ── - install -m 0755 -d /etc/apt/keyrings - curl -fsSL https://download.docker.com/linux/ubuntu/gpg -o /etc/apt/keyrings/docker.asc - chmod a+r /etc/apt/keyrings/docker.asc - echo "deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.asc] https://download.docker.com/linux/ubuntu $(. /etc/os-release && echo $VERSION_CODENAME) stable" > /etc/apt/sources.list.d/docker.list - apt-get update - apt-get install -y containerd.io - mkdir -p /etc/containerd - systemctl restart containerd - systemctl enable containerd # ── Install kubeadm, kubelet, kubectl (v1.31) ── - curl -fsSL https://pkgs.k8s.io/core:/stable:/v1.31/deb/Release.key | gpg --dearmor -o /etc/apt/keyrings/kubernetes-apt-keyring.gpg - echo "deb [signed-by=/etc/apt/keyrings/kubernetes-apt-keyring.gpg] https://pkgs.k8s.io/core:/stable:/v1.31/deb/ /" > /etc/apt/sources.list.d/kubernetes.list - apt-get update - apt-get install -y kubelet kubeadm kubectl - apt-mark hold kubelet kubeadm kubectl # ── Install node_exporter for monitoring ── - useradd --no-create-home --shell /bin/false node_exporter - curl -fsSL https://github.com/prometheus/node_exporter/releases/download/v1.10.2/node_exporter-1.10.2.linux-amd64.tar.gz -o /tmp/node_exporter.tar.gz - tar xzf /tmp/node_exporter.tar.gz -C /tmp - cp /tmp/node_exporter-1.10.2.linux-amd64/node_exporter /usr/local/bin/ - chown node_exporter:node_exporter /usr/local/bin/node_exporter - rm -rf /tmp/node_exporter* - systemctl daemon-reload - systemctl enable --now node_exporter # ── Signal cloud-init completion ── - touch /var/lib/cloud/instance/k8s-ready EOT # (2 unchanged attributes hidden) } } # module.k8s_node["k8s-worker-01"].proxmox_virtual_environment_vm.k8s_node must be replaced -/+ resource "proxmox_virtual_environment_vm" "k8s_node" { ~ boot_order = [ - "virtio0", - "net0", ] -> (known after apply) + hotplug = (known after apply) ~ id = "301" -> (known after apply) ~ ipv4_addresses = [] -> (known after apply) ~ ipv6_addresses = [] -> (known after apply) ~ mac_addresses = [ - "BC:24:11:F1:56:29", ] -> (known after apply) name = "k8s-worker-01" ~ network_interface_names = [] -> (known after apply) tags = [ "k8s", "tofu", "ubuntu", ] # (25 unchanged attributes hidden) ~ cpu { ~ cores = 4 -> 6 - flags = [] -> null ~ units = 0 -> (known after apply) # (5 unchanged attributes hidden) } ~ disk { ~ path_in_datastore = "301/vm-301-disk-0.qcow2" -> (known after apply) ~ size = 60 -> 450 # (11 unchanged attributes hidden) } ~ initialization { ~ file_format = "qcow2" -> (known after apply) - interface = "ide2" -> null + meta_data_file_id = (known after apply) + network_data_file_id = (known after apply) + type = (known after apply) ~ user_data_file_id = "local:snippets/ci-k8s-worker-01.yaml" # forces replacement -> (known after apply) # forces replacement + vendor_data_file_id = (known after apply) # (1 unchanged attribute hidden) # (2 unchanged blocks hidden) } ~ memory { ~ dedicated = 8192 -> 24576 # (3 unchanged attributes hidden) } ~ network_device { - disconnected = false -> null ~ mac_address = "BC:24:11:F1:56:29" -> (known after apply) # (8 unchanged attributes hidden) } ~ vga { + acpi = (known after apply) + bios = (known after apply) + boot_order = (known after apply) + delete_unreferenced_disks_on_destroy = (known after apply) + description = (known after apply) + hook_script_file_id = (known after apply) + hotplug = (known after apply) + id = (known after apply) + ipv4_addresses = (known after apply) + ipv6_addresses = (known after apply) + keyboard_layout = (known after apply) + kvm_arguments = (known after apply) + mac_addresses = (known after apply) + machine = (known after apply) + migrate = (known after apply) + name = (known after apply) + network_interface_names = (known after apply) + node_name = (known after apply) + on_boot = (known after apply) + pool_id = (known after apply) + protection = (known after apply) + purge_on_destroy = (known after apply) + reboot = (known after apply) + reboot_after_update = (known after apply) + scsi_hardware = (known after apply) + started = (known after apply) + stop_on_destroy = (known after apply) + tablet_device = (known after apply) + tags = (known after apply) + template = (known after apply) + timeout_clone = (known after apply) + timeout_create = (known after apply) + timeout_migrate = (known after apply) + timeout_move_disk = (known after apply) + timeout_reboot = (known after apply) + timeout_shutdown_vm = (known after apply) + timeout_start_vm = (known after apply) + timeout_stop_vm = (known after apply) + vm_id = (known after apply) } -> (known after apply) } # module.k8s_node["k8s-worker-02"].proxmox_virtual_environment_file.cloud_init will be created + resource "proxmox_virtual_environment_file" "cloud_init" { + content_type = "snippets" + datastore_id = "local" + file_modification_date = (known after apply) + file_name = (known after apply) + file_size = (known after apply) + file_tag = (known after apply) + id = (known after apply) + node_name = "georgeops" + overwrite = true + timeout_upload = 1800 + source_raw { + data = <<-EOT #cloud-config # K8s node cloud-init — installs containerd + kubeadm + node_exporter # kubeadm init/join is NOT run here — done manually after boot hostname: k8s-worker-02 manage_etc_hosts: true disable_root: false users: - name: root ssh_authorized_keys: - ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIDO+Y8ns0RgUfR21POlIVsHD+Lp+x7cUBupqXsyMeVNZ claude@control-plane shell: /bin/bash package_update: true packages: - apt-transport-https - ca-certificates - curl - gnupg - conntrack write_files: # Kernel modules for K8s networking - path: /etc/modules-load.d/k8s.conf content: | overlay br_netfilter # Sysctl for K8s networking - path: /etc/sysctl.d/99-kubernetes.conf content: | net.bridge.bridge-nf-call-iptables = 1 net.bridge.bridge-nf-call-ip6tables = 1 net.ipv4.ip_forward = 1 # containerd config — systemd cgroup driver (required for kubeadm) - path: /etc/containerd/config.toml content: | version = 2 [plugins."io.containerd.grpc.v1.cri".containerd.runtimes.runc] runtime_type = "io.containerd.runc.v2" [plugins."io.containerd.grpc.v1.cri".containerd.runtimes.runc.options] SystemdCgroup = true # node_exporter systemd unit - path: /etc/systemd/system/node_exporter.service content: | [Unit] Description=Prometheus Node Exporter After=network.target [Service] User=node_exporter ExecStart=/usr/local/bin/node_exporter Restart=on-failure RestartSec=5 [Install] WantedBy=multi-user.target runcmd: # ── Kernel modules ── - modprobe overlay - modprobe br_netfilter - sysctl --system # ── Disable swap (required for K8s) ── - swapoff -a - sed -i '/swap/d' /etc/fstab # ── Install containerd from Docker repo ── - install -m 0755 -d /etc/apt/keyrings - curl -fsSL https://download.docker.com/linux/ubuntu/gpg -o /etc/apt/keyrings/docker.asc - chmod a+r /etc/apt/keyrings/docker.asc - echo "deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.asc] https://download.docker.com/linux/ubuntu $(. /etc/os-release && echo $VERSION_CODENAME) stable" > /etc/apt/sources.list.d/docker.list - apt-get update - apt-get install -y containerd.io - mkdir -p /etc/containerd - systemctl restart containerd - systemctl enable containerd # ── Install kubeadm, kubelet, kubectl (v1.31) ── - curl -fsSL https://pkgs.k8s.io/core:/stable:/v1.31/deb/Release.key | gpg --dearmor -o /etc/apt/keyrings/kubernetes-apt-keyring.gpg - echo "deb [signed-by=/etc/apt/keyrings/kubernetes-apt-keyring.gpg] https://pkgs.k8s.io/core:/stable:/v1.31/deb/ /" > /etc/apt/sources.list.d/kubernetes.list - apt-get update - apt-get install -y kubelet kubeadm kubectl - apt-mark hold kubelet kubeadm kubectl # ── Install node_exporter for monitoring ── - useradd --no-create-home --shell /bin/false node_exporter - curl -fsSL https://github.com/prometheus/node_exporter/releases/download/v1.10.2/node_exporter-1.10.2.linux-amd64.tar.gz -o /tmp/node_exporter.tar.gz - tar xzf /tmp/node_exporter.tar.gz -C /tmp - cp /tmp/node_exporter-1.10.2.linux-amd64/node_exporter /usr/local/bin/ - chown node_exporter:node_exporter /usr/local/bin/node_exporter - rm -rf /tmp/node_exporter* - systemctl daemon-reload - systemctl enable --now node_exporter # ── Signal cloud-init completion ── - touch /var/lib/cloud/instance/k8s-ready EOT + file_name = "ci-k8s-worker-02.yaml" + resize = 0 } } # module.k8s_node["k8s-worker-02"].proxmox_virtual_environment_vm.k8s_node will be created + resource "proxmox_virtual_environment_vm" "k8s_node" { + acpi = true + bios = "seabios" + boot_order = (known after apply) + delete_unreferenced_disks_on_destroy = true + hotplug = (known after apply) + id = (known after apply) + ipv4_addresses = (known after apply) + ipv6_addresses = (known after apply) + keyboard_layout = "en-us" + mac_addresses = (known after apply) + migrate = false + name = "k8s-worker-02" + network_interface_names = (known after apply) + node_name = "georgeops" + on_boot = true + protection = false + purge_on_destroy = true + reboot = false + reboot_after_update = true + scsi_hardware = "virtio-scsi-pci" + started = true + stop_on_destroy = true + tablet_device = true + tags = [ + "k8s", + "tofu", + "ubuntu", ] + template = false + timeout_clone = 1800 + timeout_create = 1800 + timeout_migrate = 1800 + timeout_move_disk = 1800 + timeout_reboot = 1800 + timeout_shutdown_vm = 1800 + timeout_start_vm = 1800 + timeout_stop_vm = 300 + vm_id = 302 + cpu { + cores = 6 + hotplugged = 0 + limit = 0 + numa = false + sockets = 1 + type = "x86-64-v2-AES" + units = (known after apply) } + disk { + aio = "io_uring" + backup = true + cache = "none" + datastore_id = "local" + discard = "on" + file_format = "qcow2" + file_id = "local:iso/ubuntu-24.04-cloudimg-amd64.img" + interface = "virtio0" + iothread = true + path_in_datastore = (known after apply) + replicate = true + size = 450 + ssd = false } + initialization { + datastore_id = "local" + file_format = (known after apply) + meta_data_file_id = (known after apply) + network_data_file_id = (known after apply) + type = (known after apply) + user_data_file_id = (known after apply) + vendor_data_file_id = (known after apply) + dns { + servers = [ + "8.8.8.8", + "1.1.1.1", ] } + ip_config { + ipv4 { + address = "10.10.10.202/24" + gateway = "10.10.10.1" } } } + memory { + dedicated = 24576 + floating = 0 + keep_hugepages = false + shared = 0 } + network_device { + bridge = "vmbr0" + enabled = true + firewall = false + mac_address = (known after apply) + model = "virtio" + mtu = 0 + queues = 0 + rate_limit = 0 + vlan_id = 0 } + vga (known after apply) } Plan: 7 to add, 0 to change, 5 to destroy. Changes to Outputs: ~ k8s_nodes = { + k8s-worker-02 = { + ip_address = "10.10.10.202" + vm_id = 302 } # (2 unchanged attributes hidden) } ``` </details> * :arrow_forward: To **apply** this plan, comment: ```shell atlantis apply -p production ``` * :put_litter_in_its_place: To **delete** this plan and lock, click [here](http://atlantis:4141/lock?id=claude%252Finfrastructure%252Fenvironments%252Fproduction%252Fdefault%252Fproduction) * :repeat: To **plan** this project again, comment: ```shell atlantis plan -p production ``` Plan: 7 to add, 0 to change, 5 to destroy. --- * :fast_forward: To **apply** all unapplied plans from this Pull Request, comment: ```shell atlantis apply ``` * :put_litter_in_its_place: To **delete** all plans and locks from this Pull Request, comment: ```shell atlantis unlock ```
claude added 1 commit 2026-02-14 09:35:12 +01:00
Fix: add lifecycle ignore_changes to prevent VM replacement on cloud-init updates
Some checks failed
PR Checks / tofu-checks (pull_request) Failing after 3s
Details
1/1 projects applied successfully.
a32b76033f
claude commented 2026-02-14 09:36:48 +01:00
Author
Owner
Copy Link

atlantis plan -p production

atlantis plan -p production
claude commented 2026-02-14 09:36:53 +01:00
Author
Owner
Copy Link

Ran Plan for project: production dir: environments/production workspace: default

Show Output 
OpenTofu used the selected providers to generate the following execution
plan. Resource actions are indicated with the following symbols:
+ create
~ update in-place (current -> planned)
- destroy

OpenTofu will perform the following actions:

  # proxmox_virtual_environment_firewall_rules.k8s_api_access will be destroyed
  # (because proxmox_virtual_environment_firewall_rules.k8s_api_access is not in configuration)
- resource "proxmox_virtual_environment_firewall_rules" "k8s_api_access" {
      - id        = "rule-1674445101" -> null
      - node_name = "georgeops" -> null

      - rule {
          - action  = "ACCEPT" -> null
          - comment = "K8s API from control plane (DNAT to k8s-master)" -> null
          - dport   = "6443" -> null
          - enabled = true -> null
          - pos     = 0 -> null
          - proto   = "tcp" -> null
          - source  = "78.109.17.180" -> null
          - type    = "in" -> null
        }
      - rule {
          - action  = "ACCEPT" -> null
          - comment = "ArgoCD UI from control plane (DNAT to k8s-master)" -> null
          - dport   = "30443" -> null
          - enabled = true -> null
          - pos     = 1 -> null
          - proto   = "tcp" -> null
          - source  = "78.109.17.180" -> null
          - type    = "in" -> null
        }
      - rule {
          - action  = "ACCEPT" -> null
          - comment = "k8s-master node_exporter (DNAT)" -> null
          - dport   = "9200" -> null
          - enabled = true -> null
          - pos     = 2 -> null
          - proto   = "tcp" -> null
          - source  = "78.109.17.180" -> null
          - type    = "in" -> null
        }
      - rule {
          - action  = "ACCEPT" -> null
          - comment = "k8s-worker-01 node_exporter (DNAT)" -> null
          - dport   = "9201" -> null
          - enabled = true -> null
          - pos     = 3 -> null
          - proto   = "tcp" -> null
          - source  = "78.109.17.180" -> null
          - type    = "in" -> null
        }
    }

  # proxmox_virtual_environment_firewall_rules.k8s_monitoring_access will be created
+ resource "proxmox_virtual_environment_firewall_rules" "k8s_monitoring_access" {
      + id        = (known after apply)
      + node_name = "georgeops"

      + rule {
          + action  = "ACCEPT"
          + comment = "k8s-master node_exporter (DNAT)"
          + dport   = "9200"
          + enabled = true
          + pos     = (known after apply)
          + proto   = "tcp"
          + source  = "78.109.17.180"
          + type    = "in"
        }
      + rule {
          + action  = "ACCEPT"
          + comment = "k8s-worker-01 node_exporter (DNAT)"
          + dport   = "9201"
          + enabled = true
          + pos     = (known after apply)
          + proto   = "tcp"
          + source  = "78.109.17.180"
          + type    = "in"
        }
      + rule {
          + action  = "ACCEPT"
          + comment = "k8s-worker-02 node_exporter (DNAT)"
          + dport   = "9202"
          + enabled = true
          + pos     = (known after apply)
          + proto   = "tcp"
          + source  = "78.109.17.180"
          + type    = "in"
        }
    }

  # module.k8s_node["k8s-master"].proxmox_virtual_environment_vm.k8s_node will be updated in-place
~ resource "proxmox_virtual_environment_vm" "k8s_node" {
        id                                   = "300"
        name                                 = "k8s-master"
        tags                                 = [
            "k8s",
            "tofu",
            "ubuntu",
        ]
        # (30 unchanged attributes hidden)

      ~ disk {
          ~ size              = 60 -> 100
            # (12 unchanged attributes hidden)
        }

      ~ memory {
          ~ dedicated      = 8192 -> 16384
            # (3 unchanged attributes hidden)
        }

        # (3 unchanged blocks hidden)
    }

  # module.k8s_node["k8s-worker-01"].proxmox_virtual_environment_vm.k8s_node will be updated in-place
~ resource "proxmox_virtual_environment_vm" "k8s_node" {
        id                                   = "301"
        name                                 = "k8s-worker-01"
        tags                                 = [
            "k8s",
            "tofu",
            "ubuntu",
        ]
        # (30 unchanged attributes hidden)

      ~ cpu {
          ~ cores      = 4 -> 6
            # (7 unchanged attributes hidden)
        }

      ~ disk {
          ~ size              = 60 -> 450
            # (12 unchanged attributes hidden)
        }

      ~ memory {
          ~ dedicated      = 8192 -> 24576
            # (3 unchanged attributes hidden)
        }

        # (2 unchanged blocks hidden)
    }

  # module.k8s_node["k8s-worker-02"].proxmox_virtual_environment_file.cloud_init will be created
+ resource "proxmox_virtual_environment_file" "cloud_init" {
      + content_type           = "snippets"
      + datastore_id           = "local"
      + file_modification_date = (known after apply)
      + file_name              = (known after apply)
      + file_size              = (known after apply)
      + file_tag               = (known after apply)
      + id                     = (known after apply)
      + node_name              = "georgeops"
      + overwrite              = true
      + timeout_upload         = 1800

      + source_raw {
          + data      = <<-EOT
                #cloud-config
                # K8s node cloud-init — installs containerd + kubeadm + node_exporter
                # kubeadm init/join is NOT run here — done manually after boot
                
                hostname: k8s-worker-02
                manage_etc_hosts: true
                disable_root: false
                
                users:
                  - name: root
                    ssh_authorized_keys:
                      - ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIDO+Y8ns0RgUfR21POlIVsHD+Lp+x7cUBupqXsyMeVNZ claude@control-plane
                    shell: /bin/bash
                
                package_update: true
                packages:
                  - apt-transport-https
                  - ca-certificates
                  - curl
                  - gnupg
                  - conntrack
                
                write_files:
                  # Kernel modules for K8s networking
                  - path: /etc/modules-load.d/k8s.conf
                    content: |
                      overlay
                      br_netfilter
                
                  # Sysctl for K8s networking
                  - path: /etc/sysctl.d/99-kubernetes.conf
                    content: |
                      net.bridge.bridge-nf-call-iptables = 1
                      net.bridge.bridge-nf-call-ip6tables = 1
                      net.ipv4.ip_forward = 1
                
                  # containerd config — systemd cgroup driver (required for kubeadm)
                  - path: /etc/containerd/config.toml
                    content: |
                      version = 2
                      [plugins."io.containerd.grpc.v1.cri".containerd.runtimes.runc]
                        runtime_type = "io.containerd.runc.v2"
                      [plugins."io.containerd.grpc.v1.cri".containerd.runtimes.runc.options]
                        SystemdCgroup = true
                
                  # node_exporter systemd unit
                  - path: /etc/systemd/system/node_exporter.service
                    content: |
                      [Unit]
                      Description=Prometheus Node Exporter
                      After=network.target
                
                      [Service]
                      User=node_exporter
                      ExecStart=/usr/local/bin/node_exporter
                      Restart=on-failure
                      RestartSec=5
                
                      [Install]
                      WantedBy=multi-user.target
                
                runcmd:
                  # ── Kernel modules ──
                  - modprobe overlay
                  - modprobe br_netfilter
                  - sysctl --system
                
                  # ── Disable swap (required for K8s) ──
                  - swapoff -a
                  - sed -i '/swap/d' /etc/fstab
                
                  # ── Install containerd from Docker repo ──
                  - install -m 0755 -d /etc/apt/keyrings
                  - curl -fsSL https://download.docker.com/linux/ubuntu/gpg -o /etc/apt/keyrings/docker.asc
                  - chmod a+r /etc/apt/keyrings/docker.asc
                  - echo "deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.asc] https://download.docker.com/linux/ubuntu $(. /etc/os-release && echo $VERSION_CODENAME) stable" > /etc/apt/sources.list.d/docker.list
                  - apt-get update
                  - apt-get install -y containerd.io
                  - mkdir -p /etc/containerd
                  - systemctl restart containerd
                  - systemctl enable containerd
                
                  # ── Install kubeadm, kubelet, kubectl (v1.31) ──
                  - curl -fsSL https://pkgs.k8s.io/core:/stable:/v1.31/deb/Release.key | gpg --dearmor -o /etc/apt/keyrings/kubernetes-apt-keyring.gpg
                  - echo "deb [signed-by=/etc/apt/keyrings/kubernetes-apt-keyring.gpg] https://pkgs.k8s.io/core:/stable:/v1.31/deb/ /" > /etc/apt/sources.list.d/kubernetes.list
                  - apt-get update
                  - apt-get install -y kubelet kubeadm kubectl
                  - apt-mark hold kubelet kubeadm kubectl
                
                  # ── Install node_exporter for monitoring ──
                  - useradd --no-create-home --shell /bin/false node_exporter
                  - curl -fsSL https://github.com/prometheus/node_exporter/releases/download/v1.10.2/node_exporter-1.10.2.linux-amd64.tar.gz -o /tmp/node_exporter.tar.gz
                  - tar xzf /tmp/node_exporter.tar.gz -C /tmp
                  - cp /tmp/node_exporter-1.10.2.linux-amd64/node_exporter /usr/local/bin/
                  - chown node_exporter:node_exporter /usr/local/bin/node_exporter
                  - rm -rf /tmp/node_exporter*
                  - systemctl daemon-reload
                  - systemctl enable --now node_exporter
                
                  # ── Signal cloud-init completion ──
                  - touch /var/lib/cloud/instance/k8s-ready
            EOT
          + file_name = "ci-k8s-worker-02.yaml"
          + resize    = 0
        }
    }

  # module.k8s_node["k8s-worker-02"].proxmox_virtual_environment_vm.k8s_node will be created
+ resource "proxmox_virtual_environment_vm" "k8s_node" {
      + acpi                                 = true
      + bios                                 = "seabios"
      + boot_order                           = (known after apply)
      + delete_unreferenced_disks_on_destroy = true
      + hotplug                              = (known after apply)
      + id                                   = (known after apply)
      + ipv4_addresses                       = (known after apply)
      + ipv6_addresses                       = (known after apply)
      + keyboard_layout                      = "en-us"
      + mac_addresses                        = (known after apply)
      + migrate                              = false
      + name                                 = "k8s-worker-02"
      + network_interface_names              = (known after apply)
      + node_name                            = "georgeops"
      + on_boot                              = true
      + protection                           = false
      + purge_on_destroy                     = true
      + reboot                               = false
      + reboot_after_update                  = true
      + scsi_hardware                        = "virtio-scsi-pci"
      + started                              = true
      + stop_on_destroy                      = true
      + tablet_device                        = true
      + tags                                 = [
          + "k8s",
          + "tofu",
          + "ubuntu",
        ]
      + template                             = false
      + timeout_clone                        = 1800
      + timeout_create                       = 1800
      + timeout_migrate                      = 1800
      + timeout_move_disk                    = 1800
      + timeout_reboot                       = 1800
      + timeout_shutdown_vm                  = 1800
      + timeout_start_vm                     = 1800
      + timeout_stop_vm                      = 300
      + vm_id                                = 302

      + cpu {
          + cores      = 6
          + hotplugged = 0
          + limit      = 0
          + numa       = false
          + sockets    = 1
          + type       = "x86-64-v2-AES"
          + units      = (known after apply)
        }

      + disk {
          + aio               = "io_uring"
          + backup            = true
          + cache             = "none"
          + datastore_id      = "local"
          + discard           = "on"
          + file_format       = "qcow2"
          + file_id           = "local:iso/ubuntu-24.04-cloudimg-amd64.img"
          + interface         = "virtio0"
          + iothread          = true
          + path_in_datastore = (known after apply)
          + replicate         = true
          + size              = 450
          + ssd               = false
        }

      + initialization {
          + datastore_id         = "local"
          + file_format          = (known after apply)
          + meta_data_file_id    = (known after apply)
          + network_data_file_id = (known after apply)
          + type                 = (known after apply)
          + user_data_file_id    = (known after apply)
          + vendor_data_file_id  = (known after apply)

          + dns {
              + servers = [
                  + "8.8.8.8",
                  + "1.1.1.1",
                ]
            }

          + ip_config {
              + ipv4 {
                  + address = "10.10.10.202/24"
                  + gateway = "10.10.10.1"
                }
            }
        }

      + memory {
          + dedicated      = 24576
          + floating       = 0
          + keep_hugepages = false
          + shared         = 0
        }

      + network_device {
          + bridge      = "vmbr0"
          + enabled     = true
          + firewall    = false
          + mac_address = (known after apply)
          + model       = "virtio"
          + mtu         = 0
          + queues      = 0
          + rate_limit  = 0
          + vlan_id     = 0
        }

      + vga (known after apply)
    }

Plan: 3 to add, 2 to change, 1 to destroy.

Changes to Outputs:
~ k8s_nodes     = {
      + k8s-worker-02 = {
          + ip_address = "10.10.10.202"
          + vm_id      = 302
        }
        # (2 unchanged attributes hidden)
    }
  • ▶️ To apply this plan, comment: 
    atlantis apply -p production
  • 🚮 To delete this plan and lock, click here
  • 🔁 To plan this project again, comment: 
    atlantis plan -p production

Plan: 3 to add, 2 to change, 1 to destroy.

  • ⏩ To apply all unapplied plans from this Pull Request, comment: 
    atlantis apply
  • 🚮 To delete all plans and locks from this Pull Request, comment: 
    atlantis unlock
Ran Plan for project: `production` dir: `environments/production` workspace: `default` <details><summary>Show Output</summary> ```diff OpenTofu used the selected providers to generate the following execution plan. Resource actions are indicated with the following symbols: + create ~ update in-place (current -> planned) - destroy OpenTofu will perform the following actions: # proxmox_virtual_environment_firewall_rules.k8s_api_access will be destroyed # (because proxmox_virtual_environment_firewall_rules.k8s_api_access is not in configuration) - resource "proxmox_virtual_environment_firewall_rules" "k8s_api_access" { - id = "rule-1674445101" -> null - node_name = "georgeops" -> null - rule { - action = "ACCEPT" -> null - comment = "K8s API from control plane (DNAT to k8s-master)" -> null - dport = "6443" -> null - enabled = true -> null - pos = 0 -> null - proto = "tcp" -> null - source = "78.109.17.180" -> null - type = "in" -> null } - rule { - action = "ACCEPT" -> null - comment = "ArgoCD UI from control plane (DNAT to k8s-master)" -> null - dport = "30443" -> null - enabled = true -> null - pos = 1 -> null - proto = "tcp" -> null - source = "78.109.17.180" -> null - type = "in" -> null } - rule { - action = "ACCEPT" -> null - comment = "k8s-master node_exporter (DNAT)" -> null - dport = "9200" -> null - enabled = true -> null - pos = 2 -> null - proto = "tcp" -> null - source = "78.109.17.180" -> null - type = "in" -> null } - rule { - action = "ACCEPT" -> null - comment = "k8s-worker-01 node_exporter (DNAT)" -> null - dport = "9201" -> null - enabled = true -> null - pos = 3 -> null - proto = "tcp" -> null - source = "78.109.17.180" -> null - type = "in" -> null } } # proxmox_virtual_environment_firewall_rules.k8s_monitoring_access will be created + resource "proxmox_virtual_environment_firewall_rules" "k8s_monitoring_access" { + id = (known after apply) + node_name = "georgeops" + rule { + action = "ACCEPT" + comment = "k8s-master node_exporter (DNAT)" + dport = "9200" + enabled = true + pos = (known after apply) + proto = "tcp" + source = "78.109.17.180" + type = "in" } + rule { + action = "ACCEPT" + comment = "k8s-worker-01 node_exporter (DNAT)" + dport = "9201" + enabled = true + pos = (known after apply) + proto = "tcp" + source = "78.109.17.180" + type = "in" } + rule { + action = "ACCEPT" + comment = "k8s-worker-02 node_exporter (DNAT)" + dport = "9202" + enabled = true + pos = (known after apply) + proto = "tcp" + source = "78.109.17.180" + type = "in" } } # module.k8s_node["k8s-master"].proxmox_virtual_environment_vm.k8s_node will be updated in-place ~ resource "proxmox_virtual_environment_vm" "k8s_node" { id = "300" name = "k8s-master" tags = [ "k8s", "tofu", "ubuntu", ] # (30 unchanged attributes hidden) ~ disk { ~ size = 60 -> 100 # (12 unchanged attributes hidden) } ~ memory { ~ dedicated = 8192 -> 16384 # (3 unchanged attributes hidden) } # (3 unchanged blocks hidden) } # module.k8s_node["k8s-worker-01"].proxmox_virtual_environment_vm.k8s_node will be updated in-place ~ resource "proxmox_virtual_environment_vm" "k8s_node" { id = "301" name = "k8s-worker-01" tags = [ "k8s", "tofu", "ubuntu", ] # (30 unchanged attributes hidden) ~ cpu { ~ cores = 4 -> 6 # (7 unchanged attributes hidden) } ~ disk { ~ size = 60 -> 450 # (12 unchanged attributes hidden) } ~ memory { ~ dedicated = 8192 -> 24576 # (3 unchanged attributes hidden) } # (2 unchanged blocks hidden) } # module.k8s_node["k8s-worker-02"].proxmox_virtual_environment_file.cloud_init will be created + resource "proxmox_virtual_environment_file" "cloud_init" { + content_type = "snippets" + datastore_id = "local" + file_modification_date = (known after apply) + file_name = (known after apply) + file_size = (known after apply) + file_tag = (known after apply) + id = (known after apply) + node_name = "georgeops" + overwrite = true + timeout_upload = 1800 + source_raw { + data = <<-EOT #cloud-config # K8s node cloud-init — installs containerd + kubeadm + node_exporter # kubeadm init/join is NOT run here — done manually after boot hostname: k8s-worker-02 manage_etc_hosts: true disable_root: false users: - name: root ssh_authorized_keys: - ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIDO+Y8ns0RgUfR21POlIVsHD+Lp+x7cUBupqXsyMeVNZ claude@control-plane shell: /bin/bash package_update: true packages: - apt-transport-https - ca-certificates - curl - gnupg - conntrack write_files: # Kernel modules for K8s networking - path: /etc/modules-load.d/k8s.conf content: | overlay br_netfilter # Sysctl for K8s networking - path: /etc/sysctl.d/99-kubernetes.conf content: | net.bridge.bridge-nf-call-iptables = 1 net.bridge.bridge-nf-call-ip6tables = 1 net.ipv4.ip_forward = 1 # containerd config — systemd cgroup driver (required for kubeadm) - path: /etc/containerd/config.toml content: | version = 2 [plugins."io.containerd.grpc.v1.cri".containerd.runtimes.runc] runtime_type = "io.containerd.runc.v2" [plugins."io.containerd.grpc.v1.cri".containerd.runtimes.runc.options] SystemdCgroup = true # node_exporter systemd unit - path: /etc/systemd/system/node_exporter.service content: | [Unit] Description=Prometheus Node Exporter After=network.target [Service] User=node_exporter ExecStart=/usr/local/bin/node_exporter Restart=on-failure RestartSec=5 [Install] WantedBy=multi-user.target runcmd: # ── Kernel modules ── - modprobe overlay - modprobe br_netfilter - sysctl --system # ── Disable swap (required for K8s) ── - swapoff -a - sed -i '/swap/d' /etc/fstab # ── Install containerd from Docker repo ── - install -m 0755 -d /etc/apt/keyrings - curl -fsSL https://download.docker.com/linux/ubuntu/gpg -o /etc/apt/keyrings/docker.asc - chmod a+r /etc/apt/keyrings/docker.asc - echo "deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.asc] https://download.docker.com/linux/ubuntu $(. /etc/os-release && echo $VERSION_CODENAME) stable" > /etc/apt/sources.list.d/docker.list - apt-get update - apt-get install -y containerd.io - mkdir -p /etc/containerd - systemctl restart containerd - systemctl enable containerd # ── Install kubeadm, kubelet, kubectl (v1.31) ── - curl -fsSL https://pkgs.k8s.io/core:/stable:/v1.31/deb/Release.key | gpg --dearmor -o /etc/apt/keyrings/kubernetes-apt-keyring.gpg - echo "deb [signed-by=/etc/apt/keyrings/kubernetes-apt-keyring.gpg] https://pkgs.k8s.io/core:/stable:/v1.31/deb/ /" > /etc/apt/sources.list.d/kubernetes.list - apt-get update - apt-get install -y kubelet kubeadm kubectl - apt-mark hold kubelet kubeadm kubectl # ── Install node_exporter for monitoring ── - useradd --no-create-home --shell /bin/false node_exporter - curl -fsSL https://github.com/prometheus/node_exporter/releases/download/v1.10.2/node_exporter-1.10.2.linux-amd64.tar.gz -o /tmp/node_exporter.tar.gz - tar xzf /tmp/node_exporter.tar.gz -C /tmp - cp /tmp/node_exporter-1.10.2.linux-amd64/node_exporter /usr/local/bin/ - chown node_exporter:node_exporter /usr/local/bin/node_exporter - rm -rf /tmp/node_exporter* - systemctl daemon-reload - systemctl enable --now node_exporter # ── Signal cloud-init completion ── - touch /var/lib/cloud/instance/k8s-ready EOT + file_name = "ci-k8s-worker-02.yaml" + resize = 0 } } # module.k8s_node["k8s-worker-02"].proxmox_virtual_environment_vm.k8s_node will be created + resource "proxmox_virtual_environment_vm" "k8s_node" { + acpi = true + bios = "seabios" + boot_order = (known after apply) + delete_unreferenced_disks_on_destroy = true + hotplug = (known after apply) + id = (known after apply) + ipv4_addresses = (known after apply) + ipv6_addresses = (known after apply) + keyboard_layout = "en-us" + mac_addresses = (known after apply) + migrate = false + name = "k8s-worker-02" + network_interface_names = (known after apply) + node_name = "georgeops" + on_boot = true + protection = false + purge_on_destroy = true + reboot = false + reboot_after_update = true + scsi_hardware = "virtio-scsi-pci" + started = true + stop_on_destroy = true + tablet_device = true + tags = [ + "k8s", + "tofu", + "ubuntu", ] + template = false + timeout_clone = 1800 + timeout_create = 1800 + timeout_migrate = 1800 + timeout_move_disk = 1800 + timeout_reboot = 1800 + timeout_shutdown_vm = 1800 + timeout_start_vm = 1800 + timeout_stop_vm = 300 + vm_id = 302 + cpu { + cores = 6 + hotplugged = 0 + limit = 0 + numa = false + sockets = 1 + type = "x86-64-v2-AES" + units = (known after apply) } + disk { + aio = "io_uring" + backup = true + cache = "none" + datastore_id = "local" + discard = "on" + file_format = "qcow2" + file_id = "local:iso/ubuntu-24.04-cloudimg-amd64.img" + interface = "virtio0" + iothread = true + path_in_datastore = (known after apply) + replicate = true + size = 450 + ssd = false } + initialization { + datastore_id = "local" + file_format = (known after apply) + meta_data_file_id = (known after apply) + network_data_file_id = (known after apply) + type = (known after apply) + user_data_file_id = (known after apply) + vendor_data_file_id = (known after apply) + dns { + servers = [ + "8.8.8.8", + "1.1.1.1", ] } + ip_config { + ipv4 { + address = "10.10.10.202/24" + gateway = "10.10.10.1" } } } + memory { + dedicated = 24576 + floating = 0 + keep_hugepages = false + shared = 0 } + network_device { + bridge = "vmbr0" + enabled = true + firewall = false + mac_address = (known after apply) + model = "virtio" + mtu = 0 + queues = 0 + rate_limit = 0 + vlan_id = 0 } + vga (known after apply) } Plan: 3 to add, 2 to change, 1 to destroy. Changes to Outputs: ~ k8s_nodes = { + k8s-worker-02 = { + ip_address = "10.10.10.202" + vm_id = 302 } # (2 unchanged attributes hidden) } ``` </details> * :arrow_forward: To **apply** this plan, comment: ```shell atlantis apply -p production ``` * :put_litter_in_its_place: To **delete** this plan and lock, click [here](http://atlantis:4141/lock?id=claude%252Finfrastructure%252Fenvironments%252Fproduction%252Fdefault%252Fproduction) * :repeat: To **plan** this project again, comment: ```shell atlantis plan -p production ``` Plan: 3 to add, 2 to change, 1 to destroy. --- * :fast_forward: To **apply** all unapplied plans from this Pull Request, comment: ```shell atlantis apply ``` * :put_litter_in_its_place: To **delete** all plans and locks from this Pull Request, comment: ```shell atlantis unlock ```
claude commented 2026-02-14 09:39:03 +01:00
Author
Owner
Copy Link

atlantis apply -p production

atlantis apply -p production
claude commented 2026-02-14 09:39:04 +01:00
Author
Owner
Copy Link

Ran Apply for project: production dir: environments/production workspace: default

Apply Failed: Pull request must be approved according to the project's approval rules before running apply.

Ran Apply for project: `production` dir: `environments/production` workspace: `default` **Apply Failed**: Pull request must be approved according to the project's approval rules before running apply.
admin approved these changes 2026-02-14 09:39:59 +01:00
admin left a comment
Collaborator
Copy Link

LGTM — security hardening + scaling

LGTM — security hardening + scaling
claude commented 2026-02-14 09:40:06 +01:00
Author
Owner
Copy Link

atlantis apply -p production

atlantis apply -p production
claude commented 2026-02-14 09:40:30 +01:00
Author
Owner
Copy Link

Ran Apply for project: production dir: environments/production workspace: default

Apply Error

Show Output 
running 'sh -c' '/atlantis-data/bin/tofu1.11.5 apply -input=false "/atlantis-data/repos/claude/infrastructure/65/default/environments/production/production-default.tfplan"' in '/atlantis-data/repos/claude/infrastructure/65/default/environments/production': exit status 1
proxmox_virtual_environment_firewall_rules.k8s_api_access: Destroying... [id=rule-1674445101]
proxmox_virtual_environment_firewall_rules.k8s_monitoring_access: Creating...
module.k8s_node["k8s-worker-02"].proxmox_virtual_environment_file.cloud_init: Creating...
module.k8s_node["k8s-worker-01"].proxmox_virtual_environment_vm.k8s_node: Modifying... [id=301]
module.k8s_node["k8s-master"].proxmox_virtual_environment_vm.k8s_node: Modifying... [id=300]
proxmox_virtual_environment_firewall_rules.k8s_api_access: Destruction complete after 1s
module.k8s_node["k8s-worker-02"].proxmox_virtual_environment_file.cloud_init: Creation complete after 1s [id=local:snippets/ci-k8s-worker-02.yaml]
module.k8s_node["k8s-worker-02"].proxmox_virtual_environment_vm.k8s_node: Creating...
module.k8s_node["k8s-master"].proxmox_virtual_environment_vm.k8s_node: Modifications complete after 3s [id=300]
module.k8s_node["k8s-worker-01"].proxmox_virtual_environment_vm.k8s_node: Modifications complete after 3s [id=301]
module.k8s_node["k8s-worker-02"].proxmox_virtual_environment_vm.k8s_node: Still creating... [10s elapsed]
module.k8s_node["k8s-worker-02"].proxmox_virtual_environment_vm.k8s_node: Still creating... [20s elapsed]
module.k8s_node["k8s-worker-02"].proxmox_virtual_environment_vm.k8s_node: Creation complete after 21s [id=302]
╷
│ Error: Existing rules detected. Aborting...
│ 
│   with proxmox_virtual_environment_firewall_rules.k8s_monitoring_access,
│   on k8s-cluster.tf line 51, in resource "proxmox_virtual_environment_firewall_rules" "k8s_monitoring_access":
│   51: resource "proxmox_virtual_environment_firewall_rules" "k8s_monitoring_access" {
│ 
╵
Ran Apply for project: `production` dir: `environments/production` workspace: `default` **Apply Error** <details><summary>Show Output</summary> ``` running 'sh -c' '/atlantis-data/bin/tofu1.11.5 apply -input=false "/atlantis-data/repos/claude/infrastructure/65/default/environments/production/production-default.tfplan"' in '/atlantis-data/repos/claude/infrastructure/65/default/environments/production': exit status 1 proxmox_virtual_environment_firewall_rules.k8s_api_access: Destroying... [id=rule-1674445101] proxmox_virtual_environment_firewall_rules.k8s_monitoring_access: Creating... module.k8s_node["k8s-worker-02"].proxmox_virtual_environment_file.cloud_init: Creating... module.k8s_node["k8s-worker-01"].proxmox_virtual_environment_vm.k8s_node: Modifying... [id=301] module.k8s_node["k8s-master"].proxmox_virtual_environment_vm.k8s_node: Modifying... [id=300] proxmox_virtual_environment_firewall_rules.k8s_api_access: Destruction complete after 1s module.k8s_node["k8s-worker-02"].proxmox_virtual_environment_file.cloud_init: Creation complete after 1s [id=local:snippets/ci-k8s-worker-02.yaml] module.k8s_node["k8s-worker-02"].proxmox_virtual_environment_vm.k8s_node: Creating... module.k8s_node["k8s-master"].proxmox_virtual_environment_vm.k8s_node: Modifications complete after 3s [id=300] module.k8s_node["k8s-worker-01"].proxmox_virtual_environment_vm.k8s_node: Modifications complete after 3s [id=301] module.k8s_node["k8s-worker-02"].proxmox_virtual_environment_vm.k8s_node: Still creating... [10s elapsed] module.k8s_node["k8s-worker-02"].proxmox_virtual_environment_vm.k8s_node: Still creating... [20s elapsed] module.k8s_node["k8s-worker-02"].proxmox_virtual_environment_vm.k8s_node: Creation complete after 21s [id=302] ╷ │ Error: Existing rules detected. Aborting... │ │ with proxmox_virtual_environment_firewall_rules.k8s_monitoring_access, │ on k8s-cluster.tf line 51, in resource "proxmox_virtual_environment_firewall_rules" "k8s_monitoring_access": │ 51: resource "proxmox_virtual_environment_firewall_rules" "k8s_monitoring_access" { │ ╵ ``` </details>
claude commented 2026-02-14 09:43:22 +01:00
Author
Owner
Copy Link

atlantis plan -p production

atlantis plan -p production
claude commented 2026-02-14 09:43:27 +01:00
Author
Owner
Copy Link

Ran Plan for project: production dir: environments/production workspace: default

Show Output 
OpenTofu used the selected providers to generate the following execution
plan. Resource actions are indicated with the following symbols:
+ create

OpenTofu will perform the following actions:

  # proxmox_virtual_environment_firewall_rules.k8s_monitoring_access will be created
+ resource "proxmox_virtual_environment_firewall_rules" "k8s_monitoring_access" {
      + id        = (known after apply)
      + node_name = "georgeops"

      + rule {
          + action  = "ACCEPT"
          + comment = "k8s-master node_exporter (DNAT)"
          + dport   = "9200"
          + enabled = true
          + pos     = (known after apply)
          + proto   = "tcp"
          + source  = "78.109.17.180"
          + type    = "in"
        }
      + rule {
          + action  = "ACCEPT"
          + comment = "k8s-worker-01 node_exporter (DNAT)"
          + dport   = "9201"
          + enabled = true
          + pos     = (known after apply)
          + proto   = "tcp"
          + source  = "78.109.17.180"
          + type    = "in"
        }
      + rule {
          + action  = "ACCEPT"
          + comment = "k8s-worker-02 node_exporter (DNAT)"
          + dport   = "9202"
          + enabled = true
          + pos     = (known after apply)
          + proto   = "tcp"
          + source  = "78.109.17.180"
          + type    = "in"
        }
    }

Plan: 1 to add, 0 to change, 0 to destroy.
  • ▶️ To apply this plan, comment: 
    atlantis apply -p production
  • 🚮 To delete this plan and lock, click here
  • 🔁 To plan this project again, comment: 
    atlantis plan -p production

Plan: 1 to add, 0 to change, 0 to destroy.

  • ⏩ To apply all unapplied plans from this Pull Request, comment: 
    atlantis apply
  • 🚮 To delete all plans and locks from this Pull Request, comment: 
    atlantis unlock
Ran Plan for project: `production` dir: `environments/production` workspace: `default` <details><summary>Show Output</summary> ```diff OpenTofu used the selected providers to generate the following execution plan. Resource actions are indicated with the following symbols: + create OpenTofu will perform the following actions: # proxmox_virtual_environment_firewall_rules.k8s_monitoring_access will be created + resource "proxmox_virtual_environment_firewall_rules" "k8s_monitoring_access" { + id = (known after apply) + node_name = "georgeops" + rule { + action = "ACCEPT" + comment = "k8s-master node_exporter (DNAT)" + dport = "9200" + enabled = true + pos = (known after apply) + proto = "tcp" + source = "78.109.17.180" + type = "in" } + rule { + action = "ACCEPT" + comment = "k8s-worker-01 node_exporter (DNAT)" + dport = "9201" + enabled = true + pos = (known after apply) + proto = "tcp" + source = "78.109.17.180" + type = "in" } + rule { + action = "ACCEPT" + comment = "k8s-worker-02 node_exporter (DNAT)" + dport = "9202" + enabled = true + pos = (known after apply) + proto = "tcp" + source = "78.109.17.180" + type = "in" } } Plan: 1 to add, 0 to change, 0 to destroy. ``` </details> * :arrow_forward: To **apply** this plan, comment: ```shell atlantis apply -p production ``` * :put_litter_in_its_place: To **delete** this plan and lock, click [here](http://atlantis:4141/lock?id=claude%252Finfrastructure%252Fenvironments%252Fproduction%252Fdefault%252Fproduction) * :repeat: To **plan** this project again, comment: ```shell atlantis plan -p production ``` Plan: 1 to add, 0 to change, 0 to destroy. --- * :fast_forward: To **apply** all unapplied plans from this Pull Request, comment: ```shell atlantis apply ``` * :put_litter_in_its_place: To **delete** all plans and locks from this Pull Request, comment: ```shell atlantis unlock ```
claude commented 2026-02-14 09:43:50 +01:00
Author
Owner
Copy Link

atlantis apply -p production

atlantis apply -p production
claude commented 2026-02-14 09:43:54 +01:00
Author
Owner
Copy Link

Ran Apply for project: production dir: environments/production workspace: default

Show Output 
proxmox_virtual_environment_firewall_rules.k8s_monitoring_access: Creating...
proxmox_virtual_environment_firewall_rules.k8s_monitoring_access: Creation complete after 0s [id=rule-1674445101]

Apply complete! Resources: 1 added, 0 changed, 0 destroyed.

Outputs:

k8s_nodes = {
  "k8s-master" = {
    "ip_address" = "10.10.10.200"
    "vm_id" = 300
  }
  "k8s-worker-01" = {
    "ip_address" = "10.10.10.201"
    "vm_id" = 301
  }
  "k8s-worker-02" = {
    "ip_address" = "10.10.10.202"
    "vm_id" = 302
  }
}
proxmox_nodes = tolist([
  "georgeops",
])
tenant_vms = {}
Ran Apply for project: `production` dir: `environments/production` workspace: `default` <details><summary>Show Output</summary> ```diff proxmox_virtual_environment_firewall_rules.k8s_monitoring_access: Creating... proxmox_virtual_environment_firewall_rules.k8s_monitoring_access: Creation complete after 0s [id=rule-1674445101] Apply complete! Resources: 1 added, 0 changed, 0 destroyed. Outputs: k8s_nodes = { "k8s-master" = { "ip_address" = "10.10.10.200" "vm_id" = 300 } "k8s-worker-01" = { "ip_address" = "10.10.10.201" "vm_id" = 301 } "k8s-worker-02" = { "ip_address" = "10.10.10.202" "vm_id" = 302 } } proxmox_nodes = tolist([ "georgeops", ]) tenant_vms = {} ``` </details>
claude merged commit a58f11eaf3 into main 2026-02-14 09:44:26 +01:00
claude commented 2026-02-14 09:44:28 +01:00
Author
Owner
Copy Link

Locks and plans deleted for the projects and workspaces modified in this pull request:

  • dir: environments/production workspace: default
Locks and plans deleted for the projects and workspaces modified in this pull request: - dir: `environments/production` workspace: `default`
Sign in to join this conversation.
Reviewers
No Reviewers
admin
Labels
No items
No Label
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
admin claude
No Assignees
2 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: claude/infrastructure#65
No description provided.
Delete Branch "k8s-hardening-scaling"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?