Phase 6: K8s PoC — create k8s-master + k8s-worker-01 VMs #64

claude · 2026-02-14T01:12:48+01:00

claude commented

2026-02-14 01:12:48 +01:00

Phase 6: Kubernetes PoC

Creates 2 VMs for K8s cluster on NAT bridge vmbr0:

k8s-master (VMID 300, 10.10.10.200, 4CPU/8GB/60GB)
k8s-worker-01 (VMID 301, 10.10.10.201, 4CPU/8GB/60GB)

New module: `modules/k8s-node/`

VM on vmbr0 (NAT), firewall=false
Cloud-init: containerd + kubeadm v1.31 + node_exporter
on_boot=true, SSH key-only

Proxmox FW rules

Allow 6443 (K8s API) from 78.109.17.180
Allow 30443 (ArgoCD UI) from 78.109.17.180
Allow 9200-9201 (node_exporter DNAT) from 78.109.17.180

Notes

VMIDs 300+ (no conflict with tenant VMs 201-210)
kubeadm init/join done manually after VMs boot
DNAT rules added separately (host prerequisite)

## Phase 6: Kubernetes PoC Creates 2 VMs for K8s cluster on NAT bridge vmbr0: - **k8s-master** (VMID 300, 10.10.10.200, 4CPU/8GB/60GB) - **k8s-worker-01** (VMID 301, 10.10.10.201, 4CPU/8GB/60GB) ### New module: `modules/k8s-node/` - VM on vmbr0 (NAT), firewall=false - Cloud-init: containerd + kubeadm v1.31 + node_exporter - on_boot=true, SSH key-only ### Proxmox FW rules - Allow 6443 (K8s API) from 78.109.17.180 - Allow 30443 (ArgoCD UI) from 78.109.17.180 - Allow 9200-9201 (node_exporter DNAT) from 78.109.17.180 ### Notes - VMIDs 300+ (no conflict with tenant VMs 201-210) - kubeadm init/join done manually after VMs boot - DNAT rules added separately (host prerequisite)

claude added 5 commits 2026-02-14 01:12:48 +01:00

Add modules/k8s-node/main.tf 6de60586d6

Add modules/k8s-node/variables.tf 35119274ac

Add modules/k8s-node/outputs.tf db711a28b9

Add modules/k8s-node/cloud-init.yaml.tftpl c26947696e

Add environments/production/k8s-cluster.tf

PR Checks / tofu-checks (pull_request) Failing after 2s

Details

1/1 projects applied successfully.

d78a78004e

claude commented

2026-02-14 01:12:53 +01:00

Ran Plan for project: production dir: environments/production workspace: default

Show Output

proxmox_virtual_environment_download_file.ubuntu_2404_cloud: Refreshing state... [id=local:iso/ubuntu-24.04-cloudimg-amd64.img]
data.proxmox_virtual_environment_nodes.nodes: Reading...
data.proxmox_virtual_environment_nodes.nodes: Read complete after 0s [id=nodes]

OpenTofu used the selected providers to generate the following execution
plan. Resource actions are indicated with the following symbols:
+ create

OpenTofu will perform the following actions:

  # proxmox_virtual_environment_firewall_rules.k8s_api_access will be created
+ resource "proxmox_virtual_environment_firewall_rules" "k8s_api_access" {
      + id        = (known after apply)
      + node_name = "georgeops"

      + rule {
          + action  = "ACCEPT"
          + comment = "K8s API from control plane (DNAT to k8s-master)"
          + dport   = "6443"
          + enabled = true
          + pos     = (known after apply)
          + proto   = "tcp"
          + source  = "78.109.17.180"
          + type    = "in"
        }
      + rule {
          + action  = "ACCEPT"
          + comment = "ArgoCD UI from control plane (DNAT to k8s-master)"
          + dport   = "30443"
          + enabled = true
          + pos     = (known after apply)
          + proto   = "tcp"
          + source  = "78.109.17.180"
          + type    = "in"
        }
      + rule {
          + action  = "ACCEPT"
          + comment = "k8s-master node_exporter (DNAT)"
          + dport   = "9200"
          + enabled = true
          + pos     = (known after apply)
          + proto   = "tcp"
          + source  = "78.109.17.180"
          + type    = "in"
        }
      + rule {
          + action  = "ACCEPT"
          + comment = "k8s-worker-01 node_exporter (DNAT)"
          + dport   = "9201"
          + enabled = true
          + pos     = (known after apply)
          + proto   = "tcp"
          + source  = "78.109.17.180"
          + type    = "in"
        }
    }

  # module.k8s_node["k8s-master"].proxmox_virtual_environment_file.cloud_init will be created
+ resource "proxmox_virtual_environment_file" "cloud_init" {
      + content_type           = "snippets"
      + datastore_id           = "local"
      + file_modification_date = (known after apply)
      + file_name              = (known after apply)
      + file_size              = (known after apply)
      + file_tag               = (known after apply)
      + id                     = (known after apply)
      + node_name              = "georgeops"
      + overwrite              = true
      + timeout_upload         = 1800

      + source_raw {
          + data      = <<-EOT
                #cloud-config
                # K8s node cloud-init — installs containerd + kubeadm + node_exporter
                # kubeadm init/join is NOT run here — done manually after boot
                
                hostname: k8s-master
                manage_etc_hosts: true
                disable_root: false
                
                users:
                  - name: root
                    ssh_authorized_keys:
                      - ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIDO+Y8ns0RgUfR21POlIVsHD+Lp+x7cUBupqXsyMeVNZ claude@control-plane
                    shell: /bin/bash
                
                package_update: true
                packages:
                  - apt-transport-https
                  - ca-certificates
                  - curl
                  - gnupg
                
                write_files:
                  # Kernel modules for K8s networking
                  - path: /etc/modules-load.d/k8s.conf
                    content: |
                      overlay
                      br_netfilter
                
                  # Sysctl for K8s networking
                  - path: /etc/sysctl.d/99-kubernetes.conf
                    content: |
                      net.bridge.bridge-nf-call-iptables = 1
                      net.bridge.bridge-nf-call-ip6tables = 1
                      net.ipv4.ip_forward = 1
                
                  # containerd config — systemd cgroup driver (required for kubeadm)
                  - path: /etc/containerd/config.toml
                    content: |
                      version = 2
                      [plugins."io.containerd.grpc.v1.cri".containerd.runtimes.runc]
                        runtime_type = "io.containerd.runc.v2"
                      [plugins."io.containerd.grpc.v1.cri".containerd.runtimes.runc.options]
                        SystemdCgroup = true
                
                  # node_exporter systemd unit
                  - path: /etc/systemd/system/node_exporter.service
                    content: |
                      [Unit]
                      Description=Prometheus Node Exporter
                      After=network.target
                
                      [Service]
                      User=node_exporter
                      ExecStart=/usr/local/bin/node_exporter
                      Restart=on-failure
                      RestartSec=5
                
                      [Install]
                      WantedBy=multi-user.target
                
                runcmd:
                  # ── Kernel modules ──
                  - modprobe overlay
                  - modprobe br_netfilter
                  - sysctl --system
                
                  # ── Disable swap (required for K8s) ──
                  - swapoff -a
                  - sed -i '/swap/d' /etc/fstab
                
                  # ── Install containerd from Docker repo ──
                  - install -m 0755 -d /etc/apt/keyrings
                  - curl -fsSL https://download.docker.com/linux/ubuntu/gpg -o /etc/apt/keyrings/docker.asc
                  - chmod a+r /etc/apt/keyrings/docker.asc
                  - echo "deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.asc] https://download.docker.com/linux/ubuntu $(. /etc/os-release && echo $VERSION_CODENAME) stable" > /etc/apt/sources.list.d/docker.list
                  - apt-get update
                  - apt-get install -y containerd.io
                  - mkdir -p /etc/containerd
                  - systemctl restart containerd
                  - systemctl enable containerd
                
                  # ── Install kubeadm, kubelet, kubectl (v1.31) ──
                  - curl -fsSL https://pkgs.k8s.io/core:/stable:/v1.31/deb/Release.key | gpg --dearmor -o /etc/apt/keyrings/kubernetes-apt-keyring.gpg
                  - echo "deb [signed-by=/etc/apt/keyrings/kubernetes-apt-keyring.gpg] https://pkgs.k8s.io/core:/stable:/v1.31/deb/ /" > /etc/apt/sources.list.d/kubernetes.list
                  - apt-get update
                  - apt-get install -y kubelet kubeadm kubectl
                  - apt-mark hold kubelet kubeadm kubectl
                
                  # ── Install node_exporter for monitoring ──
                  - useradd --no-create-home --shell /bin/false node_exporter
                  - curl -fsSL https://github.com/prometheus/node_exporter/releases/download/v1.10.2/node_exporter-1.10.2.linux-amd64.tar.gz -o /tmp/node_exporter.tar.gz
                  - tar xzf /tmp/node_exporter.tar.gz -C /tmp
                  - cp /tmp/node_exporter-1.10.2.linux-amd64/node_exporter /usr/local/bin/
                  - chown node_exporter:node_exporter /usr/local/bin/node_exporter
                  - rm -rf /tmp/node_exporter*
                  - systemctl daemon-reload
                  - systemctl enable --now node_exporter
                
                  # ── Signal cloud-init completion ──
                  - touch /var/lib/cloud/instance/k8s-ready
            EOT
          + file_name = "ci-k8s-master.yaml"
          + resize    = 0
        }
    }

  # module.k8s_node["k8s-master"].proxmox_virtual_environment_vm.k8s_node will be created
+ resource "proxmox_virtual_environment_vm" "k8s_node" {
      + acpi                                 = true
      + bios                                 = "seabios"
      + boot_order                           = (known after apply)
      + delete_unreferenced_disks_on_destroy = true
      + hotplug                              = (known after apply)
      + id                                   = (known after apply)
      + ipv4_addresses                       = (known after apply)
      + ipv6_addresses                       = (known after apply)
      + keyboard_layout                      = "en-us"
      + mac_addresses                        = (known after apply)
      + migrate                              = false
      + name                                 = "k8s-master"
      + network_interface_names              = (known after apply)
      + node_name                            = "georgeops"
      + on_boot                              = true
      + protection                           = false
      + purge_on_destroy                     = true
      + reboot                               = false
      + reboot_after_update                  = true
      + scsi_hardware                        = "virtio-scsi-pci"
      + started                              = true
      + stop_on_destroy                      = true
      + tablet_device                        = true
      + tags                                 = [
          + "k8s",
          + "tofu",
          + "ubuntu",
        ]
      + template                             = false
      + timeout_clone                        = 1800
      + timeout_create                       = 1800
      + timeout_migrate                      = 1800
      + timeout_move_disk                    = 1800
      + timeout_reboot                       = 1800
      + timeout_shutdown_vm                  = 1800
      + timeout_start_vm                     = 1800
      + timeout_stop_vm                      = 300
      + vm_id                                = 300

      + cpu {
          + cores      = 4
          + hotplugged = 0
          + limit      = 0
          + numa       = false
          + sockets    = 1
          + type       = "x86-64-v2-AES"
          + units      = (known after apply)
        }

      + disk {
          + aio               = "io_uring"
          + backup            = true
          + cache             = "none"
          + datastore_id      = "local"
          + discard           = "on"
          + file_format       = "qcow2"
          + file_id           = "local:iso/ubuntu-24.04-cloudimg-amd64.img"
          + interface         = "virtio0"
          + iothread          = true
          + path_in_datastore = (known after apply)
          + replicate         = true
          + size              = 60
          + ssd               = false
        }

      + initialization {
          + datastore_id         = "local"
          + file_format          = (known after apply)
          + meta_data_file_id    = (known after apply)
          + network_data_file_id = (known after apply)
          + type                 = (known after apply)
          + user_data_file_id    = (known after apply)
          + vendor_data_file_id  = (known after apply)

          + dns {
              + servers = [
                  + "8.8.8.8",
                  + "1.1.1.1",
                ]
            }

          + ip_config {
              + ipv4 {
                  + address = "10.10.10.200/24"
                  + gateway = "10.10.10.1"
                }
            }
        }

      + memory {
          + dedicated      = 8192
          + floating       = 0
          + keep_hugepages = false
          + shared         = 0
        }

      + network_device {
          + bridge      = "vmbr0"
          + enabled     = true
          + firewall    = false
          + mac_address = (known after apply)
          + model       = "virtio"
          + mtu         = 0
          + queues      = 0
          + rate_limit  = 0
          + vlan_id     = 0
        }

      + vga (known after apply)
    }

  # module.k8s_node["k8s-worker-01"].proxmox_virtual_environment_file.cloud_init will be created
+ resource "proxmox_virtual_environment_file" "cloud_init" {
      + content_type           = "snippets"
      + datastore_id           = "local"
      + file_modification_date = (known after apply)
      + file_name              = (known after apply)
      + file_size              = (known after apply)
      + file_tag               = (known after apply)
      + id                     = (known after apply)
      + node_name              = "georgeops"
      + overwrite              = true
      + timeout_upload         = 1800

      + source_raw {
          + data      = <<-EOT
                #cloud-config
                # K8s node cloud-init — installs containerd + kubeadm + node_exporter
                # kubeadm init/join is NOT run here — done manually after boot
                
                hostname: k8s-worker-01
                manage_etc_hosts: true
                disable_root: false
                
                users:
                  - name: root
                    ssh_authorized_keys:
                      - ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIDO+Y8ns0RgUfR21POlIVsHD+Lp+x7cUBupqXsyMeVNZ claude@control-plane
                    shell: /bin/bash
                
                package_update: true
                packages:
                  - apt-transport-https
                  - ca-certificates
                  - curl
                  - gnupg
                
                write_files:
                  # Kernel modules for K8s networking
                  - path: /etc/modules-load.d/k8s.conf
                    content: |
                      overlay
                      br_netfilter
                
                  # Sysctl for K8s networking
                  - path: /etc/sysctl.d/99-kubernetes.conf
                    content: |
                      net.bridge.bridge-nf-call-iptables = 1
                      net.bridge.bridge-nf-call-ip6tables = 1
                      net.ipv4.ip_forward = 1
                
                  # containerd config — systemd cgroup driver (required for kubeadm)
                  - path: /etc/containerd/config.toml
                    content: |
                      version = 2
                      [plugins."io.containerd.grpc.v1.cri".containerd.runtimes.runc]
                        runtime_type = "io.containerd.runc.v2"
                      [plugins."io.containerd.grpc.v1.cri".containerd.runtimes.runc.options]
                        SystemdCgroup = true
                
                  # node_exporter systemd unit
                  - path: /etc/systemd/system/node_exporter.service
                    content: |
                      [Unit]
                      Description=Prometheus Node Exporter
                      After=network.target
                
                      [Service]
                      User=node_exporter
                      ExecStart=/usr/local/bin/node_exporter
                      Restart=on-failure
                      RestartSec=5
                
                      [Install]
                      WantedBy=multi-user.target
                
                runcmd:
                  # ── Kernel modules ──
                  - modprobe overlay
                  - modprobe br_netfilter
                  - sysctl --system
                
                  # ── Disable swap (required for K8s) ──
                  - swapoff -a
                  - sed -i '/swap/d' /etc/fstab
                
                  # ── Install containerd from Docker repo ──
                  - install -m 0755 -d /etc/apt/keyrings
                  - curl -fsSL https://download.docker.com/linux/ubuntu/gpg -o /etc/apt/keyrings/docker.asc
                  - chmod a+r /etc/apt/keyrings/docker.asc
                  - echo "deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.asc] https://download.docker.com/linux/ubuntu $(. /etc/os-release && echo $VERSION_CODENAME) stable" > /etc/apt/sources.list.d/docker.list
                  - apt-get update
                  - apt-get install -y containerd.io
                  - mkdir -p /etc/containerd
                  - systemctl restart containerd
                  - systemctl enable containerd
                
                  # ── Install kubeadm, kubelet, kubectl (v1.31) ──
                  - curl -fsSL https://pkgs.k8s.io/core:/stable:/v1.31/deb/Release.key | gpg --dearmor -o /etc/apt/keyrings/kubernetes-apt-keyring.gpg
                  - echo "deb [signed-by=/etc/apt/keyrings/kubernetes-apt-keyring.gpg] https://pkgs.k8s.io/core:/stable:/v1.31/deb/ /" > /etc/apt/sources.list.d/kubernetes.list
                  - apt-get update
                  - apt-get install -y kubelet kubeadm kubectl
                  - apt-mark hold kubelet kubeadm kubectl
                
                  # ── Install node_exporter for monitoring ──
                  - useradd --no-create-home --shell /bin/false node_exporter
                  - curl -fsSL https://github.com/prometheus/node_exporter/releases/download/v1.10.2/node_exporter-1.10.2.linux-amd64.tar.gz -o /tmp/node_exporter.tar.gz
                  - tar xzf /tmp/node_exporter.tar.gz -C /tmp
                  - cp /tmp/node_exporter-1.10.2.linux-amd64/node_exporter /usr/local/bin/
                  - chown node_exporter:node_exporter /usr/local/bin/node_exporter
                  - rm -rf /tmp/node_exporter*
                  - systemctl daemon-reload
                  - systemctl enable --now node_exporter
                
                  # ── Signal cloud-init completion ──
                  - touch /var/lib/cloud/instance/k8s-ready
            EOT
          + file_name = "ci-k8s-worker-01.yaml"
          + resize    = 0
        }
    }

  # module.k8s_node["k8s-worker-01"].proxmox_virtual_environment_vm.k8s_node will be created
+ resource "proxmox_virtual_environment_vm" "k8s_node" {
      + acpi                                 = true
      + bios                                 = "seabios"
      + boot_order                           = (known after apply)
      + delete_unreferenced_disks_on_destroy = true
      + hotplug                              = (known after apply)
      + id                                   = (known after apply)
      + ipv4_addresses                       = (known after apply)
      + ipv6_addresses                       = (known after apply)
      + keyboard_layout                      = "en-us"
      + mac_addresses                        = (known after apply)
      + migrate                              = false
      + name                                 = "k8s-worker-01"
      + network_interface_names              = (known after apply)
      + node_name                            = "georgeops"
      + on_boot                              = true
      + protection                           = false
      + purge_on_destroy                     = true
      + reboot                               = false
      + reboot_after_update                  = true
      + scsi_hardware                        = "virtio-scsi-pci"
      + started                              = true
      + stop_on_destroy                      = true
      + tablet_device                        = true
      + tags                                 = [
          + "k8s",
          + "tofu",
          + "ubuntu",
        ]
      + template                             = false
      + timeout_clone                        = 1800
      + timeout_create                       = 1800
      + timeout_migrate                      = 1800
      + timeout_move_disk                    = 1800
      + timeout_reboot                       = 1800
      + timeout_shutdown_vm                  = 1800
      + timeout_start_vm                     = 1800
      + timeout_stop_vm                      = 300
      + vm_id                                = 301

      + cpu {
          + cores      = 4
          + hotplugged = 0
          + limit      = 0
          + numa       = false
          + sockets    = 1
          + type       = "x86-64-v2-AES"
          + units      = (known after apply)
        }

      + disk {
          + aio               = "io_uring"
          + backup            = true
          + cache             = "none"
          + datastore_id      = "local"
          + discard           = "on"
          + file_format       = "qcow2"
          + file_id           = "local:iso/ubuntu-24.04-cloudimg-amd64.img"
          + interface         = "virtio0"
          + iothread          = true
          + path_in_datastore = (known after apply)
          + replicate         = true
          + size              = 60
          + ssd               = false
        }

      + initialization {
          + datastore_id         = "local"
          + file_format          = (known after apply)
          + meta_data_file_id    = (known after apply)
          + network_data_file_id = (known after apply)
          + type                 = (known after apply)
          + user_data_file_id    = (known after apply)
          + vendor_data_file_id  = (known after apply)

          + dns {
              + servers = [
                  + "8.8.8.8",
                  + "1.1.1.1",
                ]
            }

          + ip_config {
              + ipv4 {
                  + address = "10.10.10.201/24"
                  + gateway = "10.10.10.1"
                }
            }
        }

      + memory {
          + dedicated      = 8192
          + floating       = 0
          + keep_hugepages = false
          + shared         = 0
        }

      + network_device {
          + bridge      = "vmbr0"
          + enabled     = true
          + firewall    = false
          + mac_address = (known after apply)
          + model       = "virtio"
          + mtu         = 0
          + queues      = 0
          + rate_limit  = 0
          + vlan_id     = 0
        }

      + vga (known after apply)
    }

Plan: 5 to add, 0 to change, 0 to destroy.

Changes to Outputs:
+ k8s_nodes     = {
      + k8s-master    = {
          + ip_address = "10.10.10.200"
          + vm_id      = 300
        }
      + k8s-worker-01 = {
          + ip_address = "10.10.10.201"
          + vm_id      = 301
        }
    }

▶️ To apply this plan, comment:
```
atlantis apply -p production
```
🚮 To delete this plan and lock, click here
🔁 To plan this project again, comment:
```
atlantis plan -p production
```

Plan: 5 to add, 0 to change, 0 to destroy.

⏩ To apply all unapplied plans from this Pull Request, comment:
```
atlantis apply
```
🚮 To delete all plans and locks from this Pull Request, comment:
```
atlantis unlock
```

Ran Plan for project: `production` dir: `environments/production` workspace: `default` <details><summary>Show Output</summary> ```diff proxmox_virtual_environment_download_file.ubuntu_2404_cloud: Refreshing state... [id=local:iso/ubuntu-24.04-cloudimg-amd64.img] data.proxmox_virtual_environment_nodes.nodes: Reading... data.proxmox_virtual_environment_nodes.nodes: Read complete after 0s [id=nodes] OpenTofu used the selected providers to generate the following execution plan. Resource actions are indicated with the following symbols: + create OpenTofu will perform the following actions: # proxmox_virtual_environment_firewall_rules.k8s_api_access will be created + resource "proxmox_virtual_environment_firewall_rules" "k8s_api_access" { + id = (known after apply) + node_name = "georgeops" + rule { + action = "ACCEPT" + comment = "K8s API from control plane (DNAT to k8s-master)" + dport = "6443" + enabled = true + pos = (known after apply) + proto = "tcp" + source = "78.109.17.180" + type = "in" } + rule { + action = "ACCEPT" + comment = "ArgoCD UI from control plane (DNAT to k8s-master)" + dport = "30443" + enabled = true + pos = (known after apply) + proto = "tcp" + source = "78.109.17.180" + type = "in" } + rule { + action = "ACCEPT" + comment = "k8s-master node_exporter (DNAT)" + dport = "9200" + enabled = true + pos = (known after apply) + proto = "tcp" + source = "78.109.17.180" + type = "in" } + rule { + action = "ACCEPT" + comment = "k8s-worker-01 node_exporter (DNAT)" + dport = "9201" + enabled = true + pos = (known after apply) + proto = "tcp" + source = "78.109.17.180" + type = "in" } } # module.k8s_node["k8s-master"].proxmox_virtual_environment_file.cloud_init will be created + resource "proxmox_virtual_environment_file" "cloud_init" { + content_type = "snippets" + datastore_id = "local" + file_modification_date = (known after apply) + file_name = (known after apply) + file_size = (known after apply) + file_tag = (known after apply) + id = (known after apply) + node_name = "georgeops" + overwrite = true + timeout_upload = 1800 + source_raw { + data = <<-EOT #cloud-config # K8s node cloud-init — installs containerd + kubeadm + node_exporter # kubeadm init/join is NOT run here — done manually after boot hostname: k8s-master manage_etc_hosts: true disable_root: false users: - name: root ssh_authorized_keys: - ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIDO+Y8ns0RgUfR21POlIVsHD+Lp+x7cUBupqXsyMeVNZ claude@control-plane shell: /bin/bash package_update: true packages: - apt-transport-https - ca-certificates - curl - gnupg write_files: # Kernel modules for K8s networking - path: /etc/modules-load.d/k8s.conf content: | overlay br_netfilter # Sysctl for K8s networking - path: /etc/sysctl.d/99-kubernetes.conf content: | net.bridge.bridge-nf-call-iptables = 1 net.bridge.bridge-nf-call-ip6tables = 1 net.ipv4.ip_forward = 1 # containerd config — systemd cgroup driver (required for kubeadm) - path: /etc/containerd/config.toml content: | version = 2 [plugins."io.containerd.grpc.v1.cri".containerd.runtimes.runc] runtime_type = "io.containerd.runc.v2" [plugins."io.containerd.grpc.v1.cri".containerd.runtimes.runc.options] SystemdCgroup = true # node_exporter systemd unit - path: /etc/systemd/system/node_exporter.service content: | [Unit] Description=Prometheus Node Exporter After=network.target [Service] User=node_exporter ExecStart=/usr/local/bin/node_exporter Restart=on-failure RestartSec=5 [Install] WantedBy=multi-user.target runcmd: # ── Kernel modules ── - modprobe overlay - modprobe br_netfilter - sysctl --system # ── Disable swap (required for K8s) ── - swapoff -a - sed -i '/swap/d' /etc/fstab # ── Install containerd from Docker repo ── - install -m 0755 -d /etc/apt/keyrings - curl -fsSL https://download.docker.com/linux/ubuntu/gpg -o /etc/apt/keyrings/docker.asc - chmod a+r /etc/apt/keyrings/docker.asc - echo "deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.asc] https://download.docker.com/linux/ubuntu $(. /etc/os-release && echo $VERSION_CODENAME) stable" > /etc/apt/sources.list.d/docker.list - apt-get update - apt-get install -y containerd.io - mkdir -p /etc/containerd - systemctl restart containerd - systemctl enable containerd # ── Install kubeadm, kubelet, kubectl (v1.31) ── - curl -fsSL https://pkgs.k8s.io/core:/stable:/v1.31/deb/Release.key | gpg --dearmor -o /etc/apt/keyrings/kubernetes-apt-keyring.gpg - echo "deb [signed-by=/etc/apt/keyrings/kubernetes-apt-keyring.gpg] https://pkgs.k8s.io/core:/stable:/v1.31/deb/ /" > /etc/apt/sources.list.d/kubernetes.list - apt-get update - apt-get install -y kubelet kubeadm kubectl - apt-mark hold kubelet kubeadm kubectl # ── Install node_exporter for monitoring ── - useradd --no-create-home --shell /bin/false node_exporter - curl -fsSL https://github.com/prometheus/node_exporter/releases/download/v1.10.2/node_exporter-1.10.2.linux-amd64.tar.gz -o /tmp/node_exporter.tar.gz - tar xzf /tmp/node_exporter.tar.gz -C /tmp - cp /tmp/node_exporter-1.10.2.linux-amd64/node_exporter /usr/local/bin/ - chown node_exporter:node_exporter /usr/local/bin/node_exporter - rm -rf /tmp/node_exporter* - systemctl daemon-reload - systemctl enable --now node_exporter # ── Signal cloud-init completion ── - touch /var/lib/cloud/instance/k8s-ready EOT + file_name = "ci-k8s-master.yaml" + resize = 0 } } # module.k8s_node["k8s-master"].proxmox_virtual_environment_vm.k8s_node will be created + resource "proxmox_virtual_environment_vm" "k8s_node" { + acpi = true + bios = "seabios" + boot_order = (known after apply) + delete_unreferenced_disks_on_destroy = true + hotplug = (known after apply) + id = (known after apply) + ipv4_addresses = (known after apply) + ipv6_addresses = (known after apply) + keyboard_layout = "en-us" + mac_addresses = (known after apply) + migrate = false + name = "k8s-master" + network_interface_names = (known after apply) + node_name = "georgeops" + on_boot = true + protection = false + purge_on_destroy = true + reboot = false + reboot_after_update = true + scsi_hardware = "virtio-scsi-pci" + started = true + stop_on_destroy = true + tablet_device = true + tags = [ + "k8s", + "tofu", + "ubuntu", ] + template = false + timeout_clone = 1800 + timeout_create = 1800 + timeout_migrate = 1800 + timeout_move_disk = 1800 + timeout_reboot = 1800 + timeout_shutdown_vm = 1800 + timeout_start_vm = 1800 + timeout_stop_vm = 300 + vm_id = 300 + cpu { + cores = 4 + hotplugged = 0 + limit = 0 + numa = false + sockets = 1 + type = "x86-64-v2-AES" + units = (known after apply) } + disk { + aio = "io_uring" + backup = true + cache = "none" + datastore_id = "local" + discard = "on" + file_format = "qcow2" + file_id = "local:iso/ubuntu-24.04-cloudimg-amd64.img" + interface = "virtio0" + iothread = true + path_in_datastore = (known after apply) + replicate = true + size = 60 + ssd = false } + initialization { + datastore_id = "local" + file_format = (known after apply) + meta_data_file_id = (known after apply) + network_data_file_id = (known after apply) + type = (known after apply) + user_data_file_id = (known after apply) + vendor_data_file_id = (known after apply) + dns { + servers = [ + "8.8.8.8", + "1.1.1.1", ] } + ip_config { + ipv4 { + address = "10.10.10.200/24" + gateway = "10.10.10.1" } } } + memory { + dedicated = 8192 + floating = 0 + keep_hugepages = false + shared = 0 } + network_device { + bridge = "vmbr0" + enabled = true + firewall = false + mac_address = (known after apply) + model = "virtio" + mtu = 0 + queues = 0 + rate_limit = 0 + vlan_id = 0 } + vga (known after apply) } # module.k8s_node["k8s-worker-01"].proxmox_virtual_environment_file.cloud_init will be created + resource "proxmox_virtual_environment_file" "cloud_init" { + content_type = "snippets" + datastore_id = "local" + file_modification_date = (known after apply) + file_name = (known after apply) + file_size = (known after apply) + file_tag = (known after apply) + id = (known after apply) + node_name = "georgeops" + overwrite = true + timeout_upload = 1800 + source_raw { + data = <<-EOT #cloud-config # K8s node cloud-init — installs containerd + kubeadm + node_exporter # kubeadm init/join is NOT run here — done manually after boot hostname: k8s-worker-01 manage_etc_hosts: true disable_root: false users: - name: root ssh_authorized_keys: - ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIDO+Y8ns0RgUfR21POlIVsHD+Lp+x7cUBupqXsyMeVNZ claude@control-plane shell: /bin/bash package_update: true packages: - apt-transport-https - ca-certificates - curl - gnupg write_files: # Kernel modules for K8s networking - path: /etc/modules-load.d/k8s.conf content: | overlay br_netfilter # Sysctl for K8s networking - path: /etc/sysctl.d/99-kubernetes.conf content: | net.bridge.bridge-nf-call-iptables = 1 net.bridge.bridge-nf-call-ip6tables = 1 net.ipv4.ip_forward = 1 # containerd config — systemd cgroup driver (required for kubeadm) - path: /etc/containerd/config.toml content: | version = 2 [plugins."io.containerd.grpc.v1.cri".containerd.runtimes.runc] runtime_type = "io.containerd.runc.v2" [plugins."io.containerd.grpc.v1.cri".containerd.runtimes.runc.options] SystemdCgroup = true # node_exporter systemd unit - path: /etc/systemd/system/node_exporter.service content: | [Unit] Description=Prometheus Node Exporter After=network.target [Service] User=node_exporter ExecStart=/usr/local/bin/node_exporter Restart=on-failure RestartSec=5 [Install] WantedBy=multi-user.target runcmd: # ── Kernel modules ── - modprobe overlay - modprobe br_netfilter - sysctl --system # ── Disable swap (required for K8s) ── - swapoff -a - sed -i '/swap/d' /etc/fstab # ── Install containerd from Docker repo ── - install -m 0755 -d /etc/apt/keyrings - curl -fsSL https://download.docker.com/linux/ubuntu/gpg -o /etc/apt/keyrings/docker.asc - chmod a+r /etc/apt/keyrings/docker.asc - echo "deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.asc] https://download.docker.com/linux/ubuntu $(. /etc/os-release && echo $VERSION_CODENAME) stable" > /etc/apt/sources.list.d/docker.list - apt-get update - apt-get install -y containerd.io - mkdir -p /etc/containerd - systemctl restart containerd - systemctl enable containerd # ── Install kubeadm, kubelet, kubectl (v1.31) ── - curl -fsSL https://pkgs.k8s.io/core:/stable:/v1.31/deb/Release.key | gpg --dearmor -o /etc/apt/keyrings/kubernetes-apt-keyring.gpg - echo "deb [signed-by=/etc/apt/keyrings/kubernetes-apt-keyring.gpg] https://pkgs.k8s.io/core:/stable:/v1.31/deb/ /" > /etc/apt/sources.list.d/kubernetes.list - apt-get update - apt-get install -y kubelet kubeadm kubectl - apt-mark hold kubelet kubeadm kubectl # ── Install node_exporter for monitoring ── - useradd --no-create-home --shell /bin/false node_exporter - curl -fsSL https://github.com/prometheus/node_exporter/releases/download/v1.10.2/node_exporter-1.10.2.linux-amd64.tar.gz -o /tmp/node_exporter.tar.gz - tar xzf /tmp/node_exporter.tar.gz -C /tmp - cp /tmp/node_exporter-1.10.2.linux-amd64/node_exporter /usr/local/bin/ - chown node_exporter:node_exporter /usr/local/bin/node_exporter - rm -rf /tmp/node_exporter* - systemctl daemon-reload - systemctl enable --now node_exporter # ── Signal cloud-init completion ── - touch /var/lib/cloud/instance/k8s-ready EOT + file_name = "ci-k8s-worker-01.yaml" + resize = 0 } } # module.k8s_node["k8s-worker-01"].proxmox_virtual_environment_vm.k8s_node will be created + resource "proxmox_virtual_environment_vm" "k8s_node" { + acpi = true + bios = "seabios" + boot_order = (known after apply) + delete_unreferenced_disks_on_destroy = true + hotplug = (known after apply) + id = (known after apply) + ipv4_addresses = (known after apply) + ipv6_addresses = (known after apply) + keyboard_layout = "en-us" + mac_addresses = (known after apply) + migrate = false + name = "k8s-worker-01" + network_interface_names = (known after apply) + node_name = "georgeops" + on_boot = true + protection = false + purge_on_destroy = true + reboot = false + reboot_after_update = true + scsi_hardware = "virtio-scsi-pci" + started = true + stop_on_destroy = true + tablet_device = true + tags = [ + "k8s", + "tofu", + "ubuntu", ] + template = false + timeout_clone = 1800 + timeout_create = 1800 + timeout_migrate = 1800 + timeout_move_disk = 1800 + timeout_reboot = 1800 + timeout_shutdown_vm = 1800 + timeout_start_vm = 1800 + timeout_stop_vm = 300 + vm_id = 301 + cpu { + cores = 4 + hotplugged = 0 + limit = 0 + numa = false + sockets = 1 + type = "x86-64-v2-AES" + units = (known after apply) } + disk { + aio = "io_uring" + backup = true + cache = "none" + datastore_id = "local" + discard = "on" + file_format = "qcow2" + file_id = "local:iso/ubuntu-24.04-cloudimg-amd64.img" + interface = "virtio0" + iothread = true + path_in_datastore = (known after apply) + replicate = true + size = 60 + ssd = false } + initialization { + datastore_id = "local" + file_format = (known after apply) + meta_data_file_id = (known after apply) + network_data_file_id = (known after apply) + type = (known after apply) + user_data_file_id = (known after apply) + vendor_data_file_id = (known after apply) + dns { + servers = [ + "8.8.8.8", + "1.1.1.1", ] } + ip_config { + ipv4 { + address = "10.10.10.201/24" + gateway = "10.10.10.1" } } } + memory { + dedicated = 8192 + floating = 0 + keep_hugepages = false + shared = 0 } + network_device { + bridge = "vmbr0" + enabled = true + firewall = false + mac_address = (known after apply) + model = "virtio" + mtu = 0 + queues = 0 + rate_limit = 0 + vlan_id = 0 } + vga (known after apply) } Plan: 5 to add, 0 to change, 0 to destroy. Changes to Outputs: + k8s_nodes = { + k8s-master = { + ip_address = "10.10.10.200" + vm_id = 300 } + k8s-worker-01 = { + ip_address = "10.10.10.201" + vm_id = 301 } } ``` </details> * :arrow_forward: To **apply** this plan, comment: ```shell atlantis apply -p production ``` * :put_litter_in_its_place: To **delete** this plan and lock, click [here](http://atlantis:4141/lock?id=claude%252Finfrastructure%252Fenvironments%252Fproduction%252Fdefault%252Fproduction) * :repeat: To **plan** this project again, comment: ```shell atlantis plan -p production ``` Plan: 5 to add, 0 to change, 0 to destroy. --- * :fast_forward: To **apply** all unapplied plans from this Pull Request, comment: ```shell atlantis apply ``` * :put_litter_in_its_place: To **delete** all plans and locks from this Pull Request, comment: ```shell atlantis unlock ```

admin approved these changes 2026-02-14 08:22:11 +01:00

admin left a comment

Phase 6 K8s PoC approved. Plan: 5 to add.

claude commented

2026-02-14 08:22:21 +01:00

atlantis apply -p production

claude commented

2026-02-14 08:22:35 +01:00

Ran Apply for project: production dir: environments/production workspace: default

Show Output

proxmox_virtual_environment_firewall_rules.k8s_api_access: Creating...
module.k8s_node["k8s-master"].proxmox_virtual_environment_file.cloud_init: Creating...
module.k8s_node["k8s-worker-01"].proxmox_virtual_environment_file.cloud_init: Creating...
proxmox_virtual_environment_firewall_rules.k8s_api_access: Creation complete after 0s [id=rule-1674445101]
module.k8s_node["k8s-master"].proxmox_virtual_environment_file.cloud_init: Creation complete after 1s [id=local:snippets/ci-k8s-master.yaml]
module.k8s_node["k8s-worker-01"].proxmox_virtual_environment_file.cloud_init: Creation complete after 1s [id=local:snippets/ci-k8s-worker-01.yaml]
module.k8s_node["k8s-master"].proxmox_virtual_environment_vm.k8s_node: Creating...
module.k8s_node["k8s-worker-01"].proxmox_virtual_environment_vm.k8s_node: Creating...
module.k8s_node["k8s-master"].proxmox_virtual_environment_vm.k8s_node: Still creating... [10s elapsed]
module.k8s_node["k8s-worker-01"].proxmox_virtual_environment_vm.k8s_node: Still creating... [10s elapsed]
module.k8s_node["k8s-master"].proxmox_virtual_environment_vm.k8s_node: Creation complete after 11s [id=300]
module.k8s_node["k8s-worker-01"].proxmox_virtual_environment_vm.k8s_node: Creation complete after 11s [id=301]

Apply complete! Resources: 5 added, 0 changed, 0 destroyed.

Outputs:

k8s_nodes = {
  "k8s-master" = {
    "ip_address" = "10.10.10.200"
    "vm_id" = 300
  }
  "k8s-worker-01" = {
    "ip_address" = "10.10.10.201"
    "vm_id" = 301
  }
}
proxmox_nodes = tolist([
  "georgeops",
])
tenant_vms = {}

Ran Apply for project: `production` dir: `environments/production` workspace: `default` <details><summary>Show Output</summary> ```diff proxmox_virtual_environment_firewall_rules.k8s_api_access: Creating... module.k8s_node["k8s-master"].proxmox_virtual_environment_file.cloud_init: Creating... module.k8s_node["k8s-worker-01"].proxmox_virtual_environment_file.cloud_init: Creating... proxmox_virtual_environment_firewall_rules.k8s_api_access: Creation complete after 0s [id=rule-1674445101] module.k8s_node["k8s-master"].proxmox_virtual_environment_file.cloud_init: Creation complete after 1s [id=local:snippets/ci-k8s-master.yaml] module.k8s_node["k8s-worker-01"].proxmox_virtual_environment_file.cloud_init: Creation complete after 1s [id=local:snippets/ci-k8s-worker-01.yaml] module.k8s_node["k8s-master"].proxmox_virtual_environment_vm.k8s_node: Creating... module.k8s_node["k8s-worker-01"].proxmox_virtual_environment_vm.k8s_node: Creating... module.k8s_node["k8s-master"].proxmox_virtual_environment_vm.k8s_node: Still creating... [10s elapsed] module.k8s_node["k8s-worker-01"].proxmox_virtual_environment_vm.k8s_node: Still creating... [10s elapsed] module.k8s_node["k8s-master"].proxmox_virtual_environment_vm.k8s_node: Creation complete after 11s [id=300] module.k8s_node["k8s-worker-01"].proxmox_virtual_environment_vm.k8s_node: Creation complete after 11s [id=301] Apply complete! Resources: 5 added, 0 changed, 0 destroyed. Outputs: k8s_nodes = { "k8s-master" = { "ip_address" = "10.10.10.200" "vm_id" = 300 } "k8s-worker-01" = { "ip_address" = "10.10.10.201" "vm_id" = 301 } } proxmox_nodes = tolist([ "georgeops", ]) tenant_vms = {} ``` </details>

admin merged commit e415a84bff into main

2026-02-14 08:22:57 +01:00

claude commented

2026-02-14 08:22:57 +01:00

Locks and plans deleted for the projects and workspaces modified in this pull request:

dir: environments/production workspace: default

Locks and plans deleted for the projects and workspaces modified in this pull request: - dir: `environments/production` workspace: `default`

admin referenced this issue from a commit

2026-02-14 08:22:57 +01:00

Merge pull request 'Phase 6: K8s PoC — create k8s-master + k8s-worker-01 VMs' (#64) from phase6-k8s-vms into main

Sign in to join this conversation.

2 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: claude/infrastructure#64

Phase 6: K8s PoC — create k8s-master + k8s-worker-01 VMs #64

Phase 6: Kubernetes PoC

New module: modules/k8s-node/

Proxmox FW rules

Notes

New module: `modules/k8s-node/`