proxmox-talos-opentofu

A turnkey Kubernetes cluster built with Talos Linux running on a Proxmox VE hypervisor. Provisioning is done with OpenTofu.

Kubernetes cluster features:

• no kube-proxy
• Cilium as Container Network Interface (CNI)
  • without kube-proxy
  • with L2 loadbalancer support
  • with Ingress controller support
  • with Gateway API support
  • with Egress gateway support
• Gateway API CRDs are installed

Requirements

You need to have installed on your local machine:

• OpenTofu
• kubectl (for testing and cluster interaction)

Provisioning

The project is grouped in two sections:

• proxmox: provisioning of virtual machines, operating systems and Kubernetes cluster
• kubernetes: provisioning of Kubernetes cluster resources

You will have an ArgoCD instance running in the cluster eventually. You can then install your applications using the GitOps approach.

Proxmox VE

So you want first to provision the Proxmox part: create a configuration.auto.tfvars file based on the example and edit it so it suits your needs:

cd proxmox
cope configuration.auto.tfvars.example configuration.auto.tfvars
vim configuration.auto.tfvars

Then apply the configuration using OpenTofu:

tofu init
tofu plan
tofu apply

You can then grab and move the kube config file for Kubernetes provisioning like so:

tofu output kubeconfig -raw > ~/.kube/config
chmod 600 ~/.kube/config

Test if your cluster access works by listing the nodes:

kubectl get nodes

You might need to wait a bit until the cluster comes up. Proceed with the next step when all nodes are in the ready state.

Kubernetes

Secondly, you can provision the Resources inside the Kubernetes cluster. Currently, this project just installs ArgoCD in the argocd namespace in the cluster. You can then add on top of this by adding your own resources using the GitOps approach. You need to create a configuration.auto.tfvars file as well first:

cd kubernetes
cope configuration.auto.tfvars.example configuration.auto.tfvars
vim configuration.auto.tfvars

Then do the provisiong with OpenTofu:

tofu init
tofu plan
tofu apply

The ArgoCD instance should be available under the argocd_domain you configured in your configuration.auto.tfvars file i.e., http://argocd.local.

Information Sources

• Talos Linux documentation
• Talos Linux Image Factory
• Cilium documentation
• Gateway API
• Terraform providers:
  • terraform-provider-proxmox
  • terraform-provider-talos
  • terraform-provider-helm
• Helm charts:
  • ArgoCD
  • Cilium