Merge pull request #10 from max-pfeiffer/feature/make-gitops-part-configurable

Make gitops part configurable

README.md

@@ -4,6 +4,7 @@ A turnkey Kubernetes cluster built with [Talos Linux](https://www.talos.dev/) ru
 Provisioning is done with [OpenTofu](https://opentofu.org/).
 
 Kubernetes cluster features:
+* Talos Linux v1.11.6
 * Kubernetes v1.34.2
 * no kube-proxy
 * [Cilium v1.18.3](https://cilium.io/) as Container Network Interface (CNI) 
@@ -22,70 +23,117 @@ You need to have installed on your local machine:
 * [kubectl](https://kubernetes.io/docs/reference/kubectl/) (for testing and cluster interaction)
 
 ## Provisioning
-The project is grouped in two sections:
+The project is grouped in three sections:
 * proxmox: provisioning of virtual machines, operating systems and Kubernetes cluster
 * kubernetes: provisioning of Kubernetes cluster resources
+* argocd: provisioning of Kubernetes resources using GitOps, can be installed with `install_argocd_app_of_apps` flag 
 
 This way you can choose to only provision the cluster itself or/and provision Kubernetes resources and bootstrap
 also [ArgoCD](https://argoproj.github.io/cd/).
 
 You will have an [ArgoCD](https://argoproj.github.io/cd/) instance running in the cluster eventually. You can then
-install your applications using the GitOps approach. 
+install your applications using the GitOps approach. Have a look at `install_argocd_app_of_apps` and the related
+configuration variables for further options.
+
+The main idea is to configure the Kubernetes cluster and also the [ArgoCD](https://argoproj.github.io/cd/) bootstrap with infrastructure as code
+using [OpenTofu](https://opentofu.org/). So it can be rolled out very quickly and consistently. All other Kubernetes resources are then
+provisioned using a git repository via the GitOps approach.
+
+Usually you want to keep your cluster infrastructure and [ArgoCD](https://argoproj.github.io/cd/) bootstrap separate from your Kubernetes resources.
+That way you have everything decoupled and migrate to a new cluster infrastructure more easily. I added the `argocd`
+directory mainly for demonstration purposes.
 
 ### Proxmox VE
-So you want first to provision the Proxmox part: create a `configuration.auto.tfvars` file based on the example and
+First step is to provision the Proxmox part: create a `configuration.auto.tfvars` file based on the example and
 edit it so it suits your needs:
 ```shell
-cd proxmox
+$ cd proxmox
-cope configuration.auto.tfvars.example configuration.auto.tfvars
+$ cope configuration.auto.tfvars.example configuration.auto.tfvars
-vim configuration.auto.tfvars
+$ vim configuration.auto.tfvars
 ```
 Then apply the configuration using OpenTofu:
 ```shell
-tofu init
+$ tofu init
-tofu plan
+$ tofu plan
-tofu apply
+$ tofu apply
 ```
 You can then grab and move the kube config file for Kubernetes provisioning like so:
 ```shell
-tofu output kubeconfig -raw > ~/.kube/config
+$ tofu output kubeconfig -raw > ~/.kube/config
-chmod 600 ~/.kube/config
+$ chmod 600 ~/.kube/config
 ```
 Test if your cluster access works by listing the nodes:
 ```shell
-kubectl get nodes
+$ kubectl get nodes
+NAME            STATUS   ROLES           AGE   VERSION
+your-cp-0       Ready    control-plane   5d    v1.34.2
+your-worker-0   Ready    <none>          5d    v1.34.2
 ```
-You might need to wait a bit until the cluster comes up. Proceed with the next step when all nodes are in the `ready`
+You might need to wait a bit until the nodes come up. Proceed with the next step when all nodes are in the `Ready`
 state.
 
 ### Kubernetes
-Secondly, you can provision the Resources inside the Kubernetes cluster. Currently, this project just installs
+Secondly, you can provision the resources inside the Kubernetes cluster. You have a couple of options to choose 
-ArgoCD in the `argocd` namespace in the cluster. You can then add on top of this by adding your own resources 
+from. All options can be configured using variables in `configuration.auto.tfvars`:
-using the GitOps approach.
+1. **Quick start**: installs Cilium LB config, ArgoCD, Ingress without TLS (default settings) with OpenTofu. [ArgoCD](https://argoproj.github.io/cd/) is
-You need to create a `configuration.auto.tfvars` file as well first:
+   available on http://argocd.local.
+   * install_cilium_lb_config = true
+   * argocd_domain = "argocd.local"
+   * argocd_server_insecure = true
+   * argocd_ingress_enabled = true
+   * install_argocd_app_of_apps = false
+   * install_argocd_app_of_apps_git_repo_secret = false
+2. **GitOps using your own repository**: installs ArgoCD, no Cilium LB config, no Ingress and the Kubernetes resources in
+   the repository you specify in `argocd_app_of_apps_source`. Credentials for a private repository can be configured
+   and installed with OpenTofu using `install_argocd_app_of_apps_git_repo_secret` and the related variables:
+   * install_cilium_lb_config = false
+   * argocd_domain = "yourpublicdomain.com"
+   * argocd_server_insecure = true
+   * argocd_ingress_enabled = false
+   * install_argocd_app_of_apps = true
+   * argocd_app_of_apps_source = YOUR SOURCE SETTINGS
+   * install_argocd_app_of_apps_git_repo_secret = true
+   * argocd_app_of_apps_git_repo_secret_url = "https://github.com/you/yourrepo.git"
+   * argocd_app_of_apps_git_repo_secret_password_or_token = "github_pat_OLImf09435459hfjoi9m435298524jtfjn45i8tmnmds329023jdhn"
+
+These are two use cases I envision here. Please regard them as examples. Of course, you can combine the variables to
+any other setup which suits your needs.
+
+For doing a **GitOps quick start** you can fork this repository and point the `argocd_app_of_apps_source` to the 
+`argocd` directory of your newly forked repository. This way you can make use of the example Kubernetes resources in
+`argocd` directory and edit them to match your infrastructure. 
+
+Create a `configuration.auto.tfvars` like so and edit it to your liking:
 ```shell
-cd kubernetes
+$ cd kubernetes
-cope configuration.auto.tfvars.example configuration.auto.tfvars
+$ cope configuration.auto.tfvars.example configuration.auto.tfvars
-vim configuration.auto.tfvars
+$ vim configuration.auto.tfvars
 ```
-Then do the provisiong with OpenTofu:
+Then do the provisioning with OpenTofu:
 ```shell
-tofu init
+$ tofu init
-tofu plan
+$ tofu plan
-tofu apply
+$ tofu apply
+```
+You can grab the [ArgoCD](https://argoproj.github.io/cd/) initial admin password with `kubectl` afterwards:
+```shell
+$ kubectl -n argocd get secret argocd-initial-admin-secret -o jsonpath='{.data.password}' | base64 -d
 ```
-The [ArgoCD](https://argoproj.github.io/cd/) instance should be available under the `argocd_domain` you configured
-in your `configuration.auto.tfvars` file i.e., http://argocd.local.
 
 ## Roadmap
-My todo list for the GitOps part:
+Proxmox part:
-* bootstrap a certificate authority
+* make node resources configurable (CPU, memory, etc.)
-* add storage options i. e. NFS, Ceph, local
+* make version upgrades possible for Kubernetes Nodes with OpenTofu
+
+GitOps part:
+* add storage options i.e. Ceph, local
 * add Keycloak operator and Keycloak instance for SSO
 * add Prometheus/Grafana for monitoring
 * add Alloy/Loki for logging
 * add Velero for disaster recovery
 
+I am happy to receive pull requests for any improvements.
+
 ## Information Sources
 * [Talos Linux documentation](https://www.talos.dev/v1.8/)
 * [Talos Linux Image Factory](https://factory.talos.dev/)
@@ -94,6 +142,7 @@ My todo list for the GitOps part:
 * Terraform providers:
   * [terraform-provider-proxmox](https://github.com/Telmate/terraform-provider-proxmox)
   * [terraform-provider-talos](https://github.com/siderolabs/terraform-provider-talos)
+  * [terraform-provider-kubernetes](https://github.com/hashicorp/terraform-provider-kubernetes) 
   * [terraform-provider-helm](https://github.com/hashicorp/terraform-provider-helm)
 * Helm charts:
   * [ArgoCD](https://github.com/argoproj/argo-helm/tree/main/charts/argo-cd)

argocd/cluster-resources/cilium-load-balancer-ip-pool.yaml

@@ -0,0 +1,11 @@
+apiVersion: cilium.io/v2
+kind: CiliumLoadBalancerIPPool
+metadata:
+  name: default
+  annotations:
+    argocd.argoproj.io/sync-wave: "-1000"
+spec:
+  blocks:
+    # Configure your IP pool here
+    - start: "192.168.10.95"
+      stop: "192.168.10.99"

argocd/cluster-resources/cluster-issuers.yaml

@@ -0,0 +1,23 @@
+apiVersion: cert-manager.io/v1
+kind: ClusterIssuer
+metadata:
+  name: letsencrypt-http01
+  annotations:
+    argocd.argoproj.io/sync-wave: "20"
+    argocd.argoproj.io/sync-options: SkipDryRunOnMissingResource=true
+spec:
+  acme:
+    server: https://acme-v02.api.letsencrypt.org/directory
+    # Add your email address here
+    email: you@yourdomain.com
+    privateKeySecretRef:
+      name: letsencrypt-http01-cluster-issuer-account-key
+    solvers:
+      - http01:
+          gatewayHTTPRoute:
+            parentRefs:
+              - name: acme
+                namespace: network
+                sectionName: http
+                kind: Gateway
+---

argocd/cluster-resources/storageclasses.yaml

@@ -0,0 +1,18 @@
+apiVersion: storage.k8s.io/v1
+kind: StorageClass
+metadata:
+  name: csi-nfs
+  annotations:
+    argocd.argoproj.io/sync-wave: "-700"
+    storageclass.kubernetes.io/is-default-class: "true"
+provisioner: nfs.csi.k8s.io
+parameters:
+  # Configure you NFS server here
+  server: "your-nfs-server.com"
+  share: "/mnt/big-storage-pool/nfs"
+reclaimPolicy: Delete
+volumeBindingMode: Immediate
+allowVolumeExpansion: true
+mountOptions:
+  - nfsvers=4.1
+---

argocd/namespaces/argocd/http-routes.yaml

@@ -0,0 +1,23 @@
+apiVersion: gateway.networking.k8s.io/v1
+kind: HTTPRoute
+metadata:
+  name: argocd
+  namespace: argocd
+  annotations:
+    argocd.argoproj.io/sync-wave: "30"
+spec:
+  parentRefs:
+  - name: public
+    namespace: network
+    sectionName: argocd
+  hostnames:
+  # Configure the FQDN for ArgoCD here
+  - "argocd.yourdomain.com"
+  rules:
+  - matches:
+    - path:
+        type: PathPrefix
+        value: /
+    backendRefs:
+    - name: argo-cd-argocd-server
+      port: 80

argocd/namespaces/cert/applications/cert-manager.yaml

@@ -0,0 +1,28 @@
+apiVersion: argoproj.io/v1alpha1
+kind: Application
+metadata:
+  name: cert-manager
+  namespace: argocd
+  annotations:
+    argocd.argoproj.io/sync-wave: "10"
+  finalizers:
+  - resources-finalizer.argocd.argoproj.io
+spec:
+  project: cert
+  syncPolicy:
+    automated:
+      prune: true
+      selfHeal: true
+  destination:
+    namespace: cert
+    server: https://kubernetes.default.svc
+  source:
+    chart: cert-manager
+    repoURL: https://charts.jetstack.io
+    targetRevision: 1.19.2
+    helm:
+      valuesObject:
+        crds:
+          enabled: true
+        extraArgs:
+          - "--enable-gateway-api"

argocd/namespaces/external-secrets/applications/external-secrets-operator.yaml

@@ -0,0 +1,29 @@
+apiVersion: argoproj.io/v1alpha1
+kind: Application
+metadata:
+  name: external-secrets-operator
+  namespace: argocd
+  annotations:
+    argocd.argoproj.io/sync-wave: "10"
+  finalizers:
+  - resources-finalizer.argocd.argoproj.io
+spec:
+  project: external-secrets
+  ignoreDifferences:
+  - group: apiextensions.k8s.io
+    kind: CustomResourceDefinition
+    jsonPointers:
+    - /metadata/annotations
+  syncPolicy:
+    automated:
+      prune: true
+      selfHeal: true
+    syncOptions:
+      - ServerSideApply=true
+  destination:
+    namespace: external-secrets
+    server: https://kubernetes.default.svc
+  source:
+    chart: external-secrets
+    repoURL: https://charts.external-secrets.io
+    targetRevision: 0.19.2

argocd/namespaces/external-secrets/namespace.yaml

@@ -0,0 +1,9 @@
+apiVersion: v1
+kind: Namespace
+metadata:
+  name: external-secrets
+  annotations:
+    argocd.argoproj.io/sync-wave: "-1000"
+  labels:
+    name: external-secrets
+spec: {}

argocd/namespaces/external-secrets/project.yaml

@@ -0,0 +1,25 @@
+apiVersion: argoproj.io/v1alpha1
+kind: AppProject
+metadata:
+  name: external-secrets
+  namespace: argocd
+  annotations:
+    argocd.argoproj.io/sync-wave: "-900"
+spec:
+  description: External Secrets
+  clusterResourceWhitelist:
+    - group: apiextensions.k8s.io
+      kind: CustomResourceDefinition
+    - group: rbac.authorization.k8s.io
+      kind: ClusterRole
+    - group: rbac.authorization.k8s.io
+      kind: ClusterRoleBinding
+    - group: admissionregistration.k8s.io
+      kind: ValidatingWebhookConfiguration
+    - group: external-secrets.io
+      kind: ClusterSecretStore
+  sourceRepos:
+    - '*'
+  destinations:
+    - namespace: external-secrets
+      server: '*'

argocd/namespaces/kube-system/applications/csi-driver-nfs.yaml

@@ -0,0 +1,29 @@
+apiVersion: argoproj.io/v1alpha1
+kind: Application
+metadata:
+  name: csi-driver-nfs
+  namespace: argocd
+  annotations:
+    argocd.argoproj.io/sync-wave: "-800"
+  finalizers:
+  - resources-finalizer.argocd.argoproj.io
+spec:
+  project: default
+  syncPolicy:
+    automated:
+      prune: true
+      selfHeal: true
+  destination:
+    namespace: kube-system
+    server: https://kubernetes.default.svc
+  source:
+    chart: csi-driver-nfs
+    repoURL: https://raw.githubusercontent.com/kubernetes-csi/csi-driver-nfs/master/charts
+    targetRevision: 4.12.1
+    helm:
+      valuesObject:
+        externalSnapshotter:
+          enabled: true
+        controller:
+          runOnControlPlane: true
+

argocd/namespaces/kube-system/cilium-l2-announcement-policy.yaml

@@ -0,0 +1,10 @@
+apiVersion: cilium.io/v2alpha1
+kind: CiliumL2AnnouncementPolicy
+metadata:
+  name: default
+  namespace: kube-system
+  annotations:
+    argocd.argoproj.io/sync-wave: "-1000"
+spec:
+  externalIPs: true
+  loadBalancerIPs: true

argocd/namespaces/namespace.yaml

@@ -0,0 +1,9 @@
+apiVersion: v1
+kind: Namespace
+metadata:
+  name: cert
+  annotations:
+    argocd.argoproj.io/sync-wave: "-1000"
+  labels:
+    name: cert
+spec: {}

argocd/namespaces/project.yaml

@@ -0,0 +1,27 @@
+apiVersion: argoproj.io/v1alpha1
+kind: AppProject
+metadata:
+  name: cert
+  namespace: argocd
+  annotations:
+    argocd.argoproj.io/sync-wave: "-900"
+spec:
+  description: Certs
+  clusterResourceWhitelist:
+    - group: apiextensions.k8s.io
+      kind: CustomResourceDefinition
+    - group: rbac.authorization.k8s.io
+      kind: ClusterRole
+    - group: rbac.authorization.k8s.io
+      kind: ClusterRoleBinding
+    - group: admissionregistration.k8s.io
+      kind: ValidatingWebhookConfiguration
+    - group: admissionregistration.k8s.io
+      kind: MutatingWebhookConfiguration
+  sourceRepos:
+    - '*'
+  destinations:
+    - namespace: cert
+      server: '*'
+    - namespace: kube-system
+      server: '*'

argocd/network/certificates.yaml

@@ -0,0 +1,16 @@
+apiVersion: cert-manager.io/v1
+kind: Certificate
+metadata:
+  name: argocd
+  namespace: network
+  annotations:
+    argocd.argoproj.io/sync-wave: "25"
+    argocd.argoproj.io/sync-options: SkipDryRunOnMissingResource=true
+spec:
+  secretName: argocd-tls
+  issuerRef:
+    name: letsencrypt-http01
+    kind: ClusterIssuer
+  dnsNames:
+  # Configure the FQDN for ArgoCD here
+  - "argocd.yourdomain.com"

argocd/network/gateways.yaml

@@ -0,0 +1,48 @@
+apiVersion: gateway.networking.k8s.io/v1
+kind: Gateway
+metadata:
+  name: acme
+  namespace: network
+  annotations:
+    argocd.argoproj.io/sync-wave: "30"
+spec:
+  gatewayClassName: cilium
+  addresses:
+  - type: IPAddress
+    # Configure your IP address here
+    value: 192.168.10.96
+  listeners:
+  - name: http
+    protocol: HTTP
+    port: 80
+    allowedRoutes:
+        namespaces:
+          from: All
+---
+apiVersion: gateway.networking.k8s.io/v1
+kind: Gateway
+metadata:
+  name: public
+  namespace: network
+  annotations:
+    argocd.argoproj.io/sync-wave: "30"
+spec:
+  gatewayClassName: cilium
+  addresses:
+  - type: IPAddress
+    # Configure your IP address here
+    value: 192.168.10.97
+  listeners:
+  - name: argocd
+    protocol: HTTPS
+    port: 443
+    # Configure the FQDN for ArgoCD here
+    hostname: "argocd.yourdomain.com"
+    tls:
+      mode: Terminate
+      certificateRefs:
+      - kind: Secret
+        name: argocd-tls
+    allowedRoutes:
+        namespaces:
+          from: All

argocd/network/namespace.yaml

@@ -0,0 +1,9 @@
+apiVersion: v1
+kind: Namespace
+metadata:
+  name: network
+  annotations:
+    argocd.argoproj.io/sync-wave: "-1000"
+  labels:
+    name: network
+spec: {}

kubernetes/.terraform.lock.hcl

@@ -17,3 +17,19 @@ provider "registry.opentofu.org/hashicorp/helm" {
     "zh:f6fe7ecfafc344f4e6aecacf5ae12ac73b94389b9679dcd0f04fc5ff45bdc066",
   ]
 }
+
+provider "registry.opentofu.org/hashicorp/kubernetes" {
+  version = "3.0.1"
+  hashes = [
+    "h1:e0dSpTDhKjin6KYIwLWTR+AHVC7wWlU3VfIx27n1bec=",
+    "zh:0a6aff192781cfd062efe814d87ec21c84273005a685c818fb3c771ec9fd7051",
+    "zh:129f10760e8c727f7b593111e0026aa36aeb28c98f6500c749007aabba402332",
+    "zh:4a0995010f32949b1fbe580db15e76c73ba15aa265f73a7e535addd15dfade0d",
+    "zh:8b518be59029e8f0ad0767dbbd87f169ac6c906e50636314f8a5ff3c952f0ad5",
+    "zh:a2f1c113ae07dc5da8410d7a93b7e9ad24c3f17db357f090e6d68b41ed52e616",
+    "zh:b1d3604a2f545beae0965305d7bca821076cc9127fc34a77eef01c2d0cf916d2",
+    "zh:c2f2d371018d77affce46fee8b9a9ff0d27c4d5c3c64f8bce654e7c8d3305dc1",
+    "zh:c7cf958fb9bb429086ff1d371a4b824ec601ec0913dddaf85cd2e38d73ca7ec0",
+    "zh:f7753278388598c8e27140c5700e5699a0131926df8dad362f86ad67c36585ea",
+  ]
+}

kubernetes/configuration.auto.tfvars.example

@@ -3,8 +3,32 @@ kubernetes_config_path    = "~/.kube/config"
 Kubernetes_config_context = "admin@yourclustername"
 
 # Cilium Load Balancer
+install_cilium_lb_config            = true
 cilium_load_balancer_ip_range_start = "192.168.10.95"
 cilium_load_balancer_ip_range_stop = "192.168.10.99"
 
 # ArgoCD
-argocd_domain = "argocd.local"
+argocd_domain          = "argocd.local"
+argocd_server_insecure = true
+argocd_ingress_enabled = true
+
+## App of Apps
+install_argocd_app_of_apps                 = false
+argocd_app_of_apps_source                  = <<-EOT
+repoURL: https://github.com/you/yourrepo.git
+targetRevision: main
+path: argocd
+directory:
+  recurse: true
+EOT
+argocd_app_of_apps_sync_policy             = <<-EOT
+automated:
+  prune: true
+  selfHeal: true
+syncOptions:
+- SkipDryRunOnMissingResource=true
+EOT
+install_argocd_app_of_apps_git_repo_secret = false
+argocd_app_of_apps_git_repo_secret_url     = "https://github.com/you/yourrepo.git"
+argocd_app_of_apps_git_repo_secret_password_or_token   = "github_pat_OLImf09435459hfjoi9m435298524jtfjn45i8tmnmds329023jdhn"
+

kubernetes/helm_charts/app-of-apps/Chart.yaml

@@ -0,0 +1,23 @@
+apiVersion: v2
+name: app-of-apps
+description: Helm chart for installing the ArgoCD base application
+
+# A chart can be either an 'application' or a 'library' chart.
+#
+# Application charts are a collection of templates that can be packaged into versioned archives
+# to be deployed.
+#
+# Library charts provide useful utilities or functions for the chart developer. They're included as
+# a dependency of application charts to inject those utilities and functions into the rendering
+# pipeline. Library charts do not define any templates and therefore cannot be deployed.
+type: application
+
+# This is the chart version. This version number should be incremented each time you make changes
+# to the chart and its templates, including the app version.
+# Versions are expected to follow Semantic Versioning (https://semver.org/)
+version: 0.1.0
+
+# This is the version number of the application being deployed. This version number should be
+# incremented each time you make changes to the application. Versions are not expected to
+# follow Semantic Versioning. They should reflect the version the application is using.
+appVersion: "1.0"

kubernetes/helm_charts/app-of-apps/templates/app-of-apps.yaml

@@ -0,0 +1,16 @@
+apiVersion: argoproj.io/v1alpha1
+kind: Application
+metadata:
+  name: app-of-apps
+  namespace: argocd
+  finalizers:
+  - resources-finalizer.argocd.argoproj.io
+spec:
+  project: default
+  destination:
+    server: https://kubernetes.default.svc
+    namespace: argocd
+  source:
+    {{- .Values.source | nindent 4 }}
+  syncPolicy:
+    {{- .Values.syncPolicy | nindent 4 }}

kubernetes/helm_charts/app-of-apps/values.yaml

@@ -0,0 +1,2 @@
+source: {}
+syncPolicy: {}

kubernetes/helm_releases.tf

@@ -1,36 +1,16 @@
 resource "helm_release" "argocd" {
-  name             = "argo-cd"
+  depends_on = [kubernetes_namespace_v1.argocd]
-  namespace        = "argocd"
+  name       = "argo-cd"
-  create_namespace = true
+  chart      = "argo-cd"
-  chart            = "argo-cd"
+  version    = "9.2.4"
-  version          = "9.1.0"
+  repository = "https://argoproj.github.io/argo-helm"
-  repository       = "https://argoproj.github.io/argo-helm"
+  namespace  = kubernetes_namespace_v1.argocd.id
-  timeout          = 120
+  timeout    = 120
-  set = [
+  set        = local.argocd_values
-    {
-      name  = "global.domain"
-      value = var.argocd_domain
-    },
-    {
-      name  = "configs.params.server\\.insecure"
-      value = "true"
-    },
-    {
-      name  = "server.ingress.enabled"
-      value = "true"
-    },
-    {
-      name  = "server.ingress.ingressClassName"
-      value = "cilium"
-    },
-    {
-      name  = "server.ingress.annotations.ingress\\.cilium\\.io/force-https"
-      value = "disabled"
-    },
-  ]
 }
 
 resource "helm_release" "cilium_lb_config" {
+  count      = var.install_cilium_lb_config ? 1 : 0
   depends_on = [helm_release.argocd]
   name       = "cilium-lb-config"
   chart      = "${path.module}/helm_charts/cilium-lb-config"
@@ -46,3 +26,45 @@ resource "helm_release" "cilium_lb_config" {
     },
   ]
 }
+
+resource "helm_release" "argocd_app_of_apps" {
+  count      = var.install_argocd_app_of_apps ? 1 : 0
+  depends_on = [helm_release.argocd]
+  name       = "app-of-apps"
+  chart      = "${path.module}/helm_charts/app-of-apps"
+  namespace  = kubernetes_namespace_v1.argocd.id
+  timeout    = 60
+  set = [
+    {
+      name  = "source"
+      value = var.argocd_app_of_apps_source
+    },
+    {
+      name  = "syncPolicy"
+      value = var.argocd_app_of_apps_sync_policy
+    },
+  ]
+}
+
+# data "helm_template" "argocd_app_of_apps" {
+#
+#   depends_on = [helm_release.argocd]
+#   name       = "app-of-apps"
+#   chart      = "${path.module}/helm_charts/app-of-apps"
+#   namespace  = kubernetes_namespace_v1.argocd.id
+#   timeout    = 60
+#   set = [
+#     {
+#       name  = "source"
+#       value = var.argocd_app_of_apps_source
+#     },
+#     {
+#       name  = "syncPolicy"
+#       value = var.argocd_app_of_apps_sync_policy
+#     },
+#   ]
+# }
+#
+# output "argocd_app_of_apps_rendered_yaml" {
+#   value = data.helm_template.argocd_app_of_apps.manifest
+# }

kubernetes/locals.tf

@@ -0,0 +1,26 @@
+locals {
+  argocd_values = concat(
+    [
+      {
+        name  = "global.domain"
+        value = var.argocd_domain
+      },
+    ],
+    var.argocd_server_insecure ? [
+      {
+        name  = "configs.params.server\\.insecure"
+        value = "true"
+      },
+    ] : [],
+    var.argocd_ingress_enabled ? [
+      {
+        name  = "server.ingress.enabled"
+        value = "true"
+      },
+      {
+        name  = "server.ingress.ingressClassName"
+        value = "cilium"
+      },
+    ] : []
+  )
+}

kubernetes/namespaces.tf

@@ -0,0 +1,5 @@
+resource "kubernetes_namespace_v1" "argocd" {
+  metadata {
+    name = "argocd"
+  }
+}

kubernetes/providers.tf

@@ -1,5 +1,9 @@
 terraform {
   required_providers {
+    kubernetes = {
+      source  = "hashicorp/kubernetes"
+      version = "3.0.1"
+    }
     helm = {
       source  = "hashicorp/helm"
       version = "3.1.1"
@@ -7,6 +11,11 @@ terraform {
   }
 }
 
+provider "kubernetes" {
+  config_path    = var.kubernetes_config_path
+  config_context = var.Kubernetes_config_context
+}
+
 provider "helm" {
   kubernetes = {
     config_path    = var.kubernetes_config_path

kubernetes/secrets.tf

@@ -0,0 +1,17 @@
+resource "kubernetes_secret_v1" "argocd_app_of_apps_git_repo" {
+  count      = var.install_argocd_app_of_apps_git_repo_secret ? 1 : 0
+  depends_on = [kubernetes_namespace_v1.argocd]
+  metadata {
+    namespace = kubernetes_namespace_v1.argocd.id
+    name      = "argocd-app-of-apps-git-repo"
+    labels = {
+      "argocd.argoproj.io/secret-type" = "repository"
+    }
+  }
+  data = {
+    type     = "git"
+    url      = var.argocd_app_of_apps_git_repo_secret_url
+    username = var.argocd_app_of_apps_git_repo_secret_username
+    password = var.argocd_app_of_apps_git_repo_secret_password_or_token
+  }
+}

kubernetes/variables.tf

@@ -1,22 +1,103 @@
 variable "kubernetes_config_path" {
-  type      = string
+  description = "Path to kubeconfig for this cluster"
-  sensitive = true
+  type        = string
+  sensitive   = true
 }
 
 variable "Kubernetes_config_context" {
-  type      = string
+  description = "Name of the Kubernetes context in kubeconfig"
-  sensitive = true
+  type        = string
+  sensitive   = true
+}
+
+variable "install_cilium_lb_config" {
+  description = "Flag for installing CiliumL2AnnouncementPolicy and CiliumLoadBalancerIPPool via the Helm chart with OpenTofu"
+  type        = bool
+  default     = true
 }
 
 variable "cilium_load_balancer_ip_range_start" {
-  type = string
+  description = "IP range start for CiliumLoadBalancerIPPool in Helm chart"
+  type        = string
 }
 
 variable "cilium_load_balancer_ip_range_stop" {
-  type = string
+  description = "IP range stop for CiliumLoadBalancerIPPool in Helm chart"
+  type        = string
 }
 
 variable "argocd_domain" {
-  type = string
+  description = "The FQDN for ArgoCD application"
+  type        = string
 }
 
+# See: https://argo-cd.readthedocs.io/en/stable/operator-manual/tls/#configuring-tls-for-argocd-server
+variable "argocd_server_insecure" {
+  description = "Flag for disabling internal TLS with --insecure in ArgoCD Helm chart"
+  type        = bool
+  default     = true
+}
+
+variable "argocd_ingress_enabled" {
+  description = "Flag for enabling/disabling creating an Ingress in ArgoCD Helm chart"
+  type        = bool
+  default     = true
+}
+
+# See: https://argo-cd.readthedocs.io/en/latest/operator-manual/cluster-bootstrapping/#app-of-apps-pattern
+variable "install_argocd_app_of_apps" {
+  description = "Flag for bootstrapping ArgoCD with an App of Apps"
+  type        = bool
+  default     = false
+}
+
+# See: https://argo-cd.readthedocs.io/en/latest/user-guide/application-specification/
+variable "argocd_app_of_apps_source" {
+  description = "Source section of ArgoCD Application CRD, use it to configure a git repository of your choice"
+  type        = string
+  default     = <<-EOT
+repoURL: https://github.com/max-pfeiffer/proxmox-talos-opentofu.git
+targetRevision: feature/make-gitops-part-configurable
+path: argocd
+directory:
+  recurse: true
+EOT
+}
+
+# See: https://argo-cd.readthedocs.io/en/latest/user-guide/application-specification/
+variable "argocd_app_of_apps_sync_policy" {
+  description = "syncPolicy section of ArgoCD Application CRD, use it to configure syncPolicy settings of your choice"
+  type        = string
+  default     = <<-EOT
+automated:
+  prune: true
+  selfHeal: true
+syncOptions:
+- SkipDryRunOnMissingResource=true
+EOT
+}
+
+# See: https://argo-cd.readthedocs.io/en/stable/user-guide/private-repositories/
+variable "install_argocd_app_of_apps_git_repo_secret" {
+  description = "Flag for provisioning the credentials for a private App of Apps repo in ArgoCD namespace with OpenTofu"
+  type        = bool
+  default     = false
+}
+
+variable "argocd_app_of_apps_git_repo_secret_url" {
+  description = "Repository URL for your private App of Apps repository"
+  type        = string
+  default     = "https://github.com/max-pfeiffer/proxmox-talos-opentofu.git"
+}
+
+variable "argocd_app_of_apps_git_repo_secret_username" {
+  description = "Username for your private App of Apps repository"
+  type        = string
+  default     = "git"
+}
+
+variable "argocd_app_of_apps_git_repo_secret_password_or_token" {
+  description = "Password or token for your private App of Apps repository"
+  type        = string
+  default     = "yourtoken"
+}