diff --git a/README.md b/README.md index 8952a4e..3f0930d 100644 --- a/README.md +++ b/README.md @@ -23,7 +23,7 @@ You need to have installed on your local machine: * [kubectl](https://kubernetes.io/docs/reference/kubectl/) (for testing and cluster interaction) ## Provisioning -The project is grouped in two sections: +The project is grouped in three sections: * proxmox: provisioning of virtual machines, operating systems and Kubernetes cluster * kubernetes: provisioning of Kubernetes cluster resources * argocd: provisioning of Kubernetes resources using GitOps, can be installed with `install_argocd_app_of_apps` flag @@ -73,7 +73,7 @@ You might need to wait a bit until the nodes come up. Proceed with the next step state. ### Kubernetes -Secondly, you can provision the Resources inside the Kubernetes cluster. Here you have a couple of options to choose +Secondly, you can provision the resources inside the Kubernetes cluster. You have a couple of options to choose from. All options can be configured using variables in `configuration.auto.tfvars`: 1. **Quick start**: installs Cilium LB config, ArgoCD, Ingress without TLS (default settings) with OpenTofu. [ArgoCD](https://argoproj.github.io/cd/) is available on http://argocd.local. @@ -83,16 +83,7 @@ from. All options can be configured using variables in `configuration.auto.tfvar * argocd_ingress_enabled = true * install_argocd_app_of_apps = false * install_argocd_app_of_apps_git_repo_secret = false -2. **GitOps quick start**: installs ArgoCD, no Cilium LB config, no Ingress and the Kubernetes resources - in `argocd` directory (App of Apps) with OpenTofu: cert-manager, Gateway, HTTPRoute, External Secrets Operator etc. - [ArgoCD](https://argoproj.github.io/cd/) is available on https://yourpublicdomain.com: - * install_cilium_lb_config = false - * argocd_domain = "yourpublicdomain.com" - * argocd_server_insecure = true - * argocd_ingress_enabled = false - * install_argocd_app_of_apps = true - * install_argocd_app_of_apps_git_repo_secret = false -3. **GitOps using your own repository**: installs ArgoCD, no Cilium LB config, no Ingress and the Kubernetes resources in +2. **GitOps using your own repository**: installs ArgoCD, no Cilium LB config, no Ingress and the Kubernetes resources in the repository you specify in `argocd_app_of_apps_source`. Credentials for a private repository can be configured and installed with OpenTofu using `install_argocd_app_of_apps_git_repo_secret` and the related variables: * install_cilium_lb_config = false @@ -105,7 +96,12 @@ from. All options can be configured using variables in `configuration.auto.tfvar * argocd_app_of_apps_git_repo_secret_url = "https://github.com/you/yourrepo.git" * argocd_app_of_apps_git_repo_secret_password_or_token = "github_pat_OLImf09435459hfjoi9m435298524jtfjn45i8tmnmds329023jdhn" -These are three use cases I envision here. Of course can combine the variables to any other setup which suits your needs. +These are two use cases I envision here. Please regard them as examples. Of course, you can combine the variables to +any other setup which suits your needs. + +For doing a **GitOps quick start** you can fork this repository and point the `argocd_app_of_apps_source` to the +`argocd` directory of your newly forked repository. This way you can make use of the example Kubernetes resources in +`argocd` directory and edit them to match your infrastructure. Create a `configuration.auto.tfvars` like so and edit it to your liking: ```shell @@ -113,7 +109,7 @@ $ cd kubernetes $ cope configuration.auto.tfvars.example configuration.auto.tfvars $ vim configuration.auto.tfvars ``` -Then do the provisiong with OpenTofu: +Then do the provisioning with OpenTofu: ```shell $ tofu init $ tofu plan @@ -125,13 +121,19 @@ $ kubectl -n argocd get secret argocd-initial-admin-secret -o jsonpath='{.data.p ``` ## Roadmap -My todo list for the GitOps part: -* add storage options i.e. NFS, Ceph, local +Proxmox part: +* make node resources configurable (CPU, memory, etc.) +* make version upgrades possible for Kubernetes Nodes with OpenTofu + +GitOps part: +* add storage options i.e. Ceph, local * add Keycloak operator and Keycloak instance for SSO * add Prometheus/Grafana for monitoring * add Alloy/Loki for logging * add Velero for disaster recovery +I am happy to receive pull requests for any improvements. + ## Information Sources * [Talos Linux documentation](https://www.talos.dev/v1.8/) * [Talos Linux Image Factory](https://factory.talos.dev/) diff --git a/argocd/cluster-resources/cilium-load-balancer-ip-pool.yaml b/argocd/cluster-resources/cilium-load-balancer-ip-pool.yaml new file mode 100644 index 0000000..b033c1f --- /dev/null +++ b/argocd/cluster-resources/cilium-load-balancer-ip-pool.yaml @@ -0,0 +1,11 @@ +apiVersion: cilium.io/v2 +kind: CiliumLoadBalancerIPPool +metadata: + name: default + annotations: + argocd.argoproj.io/sync-wave: "-1000" +spec: + blocks: + # Configure your IP pool here + - start: "192.168.10.95" + stop: "192.168.10.99" diff --git a/argocd/cluster-resources/cluster-issuers.yaml b/argocd/cluster-resources/cluster-issuers.yaml new file mode 100644 index 0000000..8b30ab4 --- /dev/null +++ b/argocd/cluster-resources/cluster-issuers.yaml @@ -0,0 +1,23 @@ +apiVersion: cert-manager.io/v1 +kind: ClusterIssuer +metadata: + name: letsencrypt-http01 + annotations: + argocd.argoproj.io/sync-wave: "20" + argocd.argoproj.io/sync-options: SkipDryRunOnMissingResource=true +spec: + acme: + server: https://acme-v02.api.letsencrypt.org/directory + # Add your email address here + email: you@yourdomain.com + privateKeySecretRef: + name: letsencrypt-http01-cluster-issuer-account-key + solvers: + - http01: + gatewayHTTPRoute: + parentRefs: + - name: acme + namespace: network + sectionName: http + kind: Gateway +--- \ No newline at end of file diff --git a/argocd/cluster-resources/storageclasses.yaml b/argocd/cluster-resources/storageclasses.yaml new file mode 100644 index 0000000..3dc8acf --- /dev/null +++ b/argocd/cluster-resources/storageclasses.yaml @@ -0,0 +1,18 @@ +apiVersion: storage.k8s.io/v1 +kind: StorageClass +metadata: + name: csi-nfs + annotations: + argocd.argoproj.io/sync-wave: "-700" + storageclass.kubernetes.io/is-default-class: "true" +provisioner: nfs.csi.k8s.io +parameters: + # Configure you NFS server here + server: "your-nfs-server.com" + share: "/mnt/big-storage-pool/nfs" +reclaimPolicy: Delete +volumeBindingMode: Immediate +allowVolumeExpansion: true +mountOptions: + - nfsvers=4.1 +--- diff --git a/argocd/namespaces/argocd/http-routes.yaml b/argocd/namespaces/argocd/http-routes.yaml new file mode 100644 index 0000000..9cefa8c --- /dev/null +++ b/argocd/namespaces/argocd/http-routes.yaml @@ -0,0 +1,23 @@ +apiVersion: gateway.networking.k8s.io/v1 +kind: HTTPRoute +metadata: + name: argocd + namespace: argocd + annotations: + argocd.argoproj.io/sync-wave: "30" +spec: + parentRefs: + - name: public + namespace: network + sectionName: argocd + hostnames: + # Configure the FQDN for ArgoCD here + - "argocd.yourdomain.com" + rules: + - matches: + - path: + type: PathPrefix + value: / + backendRefs: + - name: argo-cd-argocd-server + port: 80 \ No newline at end of file diff --git a/argocd/namespaces/cert/applications/cert-manager.yaml b/argocd/namespaces/cert/applications/cert-manager.yaml index af19a1e..64b07b0 100644 --- a/argocd/namespaces/cert/applications/cert-manager.yaml +++ b/argocd/namespaces/cert/applications/cert-manager.yaml @@ -4,7 +4,7 @@ metadata: name: cert-manager namespace: argocd annotations: - argocd.argoproj.io/sync-wave: "100" + argocd.argoproj.io/sync-wave: "10" finalizers: - resources-finalizer.argocd.argoproj.io spec: diff --git a/argocd/namespaces/external-secrets/applications/external-secrets-operator.yaml b/argocd/namespaces/external-secrets/applications/external-secrets-operator.yaml new file mode 100644 index 0000000..225d117 --- /dev/null +++ b/argocd/namespaces/external-secrets/applications/external-secrets-operator.yaml @@ -0,0 +1,29 @@ +apiVersion: argoproj.io/v1alpha1 +kind: Application +metadata: + name: external-secrets-operator + namespace: argocd + annotations: + argocd.argoproj.io/sync-wave: "10" + finalizers: + - resources-finalizer.argocd.argoproj.io +spec: + project: external-secrets + ignoreDifferences: + - group: apiextensions.k8s.io + kind: CustomResourceDefinition + jsonPointers: + - /metadata/annotations + syncPolicy: + automated: + prune: true + selfHeal: true + syncOptions: + - ServerSideApply=true + destination: + namespace: external-secrets + server: https://kubernetes.default.svc + source: + chart: external-secrets + repoURL: https://charts.external-secrets.io + targetRevision: 0.19.2 \ No newline at end of file diff --git a/argocd/namespaces/external-secrets/namespace.yaml b/argocd/namespaces/external-secrets/namespace.yaml new file mode 100644 index 0000000..2a03f9c --- /dev/null +++ b/argocd/namespaces/external-secrets/namespace.yaml @@ -0,0 +1,9 @@ +apiVersion: v1 +kind: Namespace +metadata: + name: external-secrets + annotations: + argocd.argoproj.io/sync-wave: "-1000" + labels: + name: external-secrets +spec: {} diff --git a/argocd/namespaces/external-secrets/project.yaml b/argocd/namespaces/external-secrets/project.yaml new file mode 100644 index 0000000..97c628c --- /dev/null +++ b/argocd/namespaces/external-secrets/project.yaml @@ -0,0 +1,25 @@ +apiVersion: argoproj.io/v1alpha1 +kind: AppProject +metadata: + name: external-secrets + namespace: argocd + annotations: + argocd.argoproj.io/sync-wave: "-900" +spec: + description: External Secrets + clusterResourceWhitelist: + - group: apiextensions.k8s.io + kind: CustomResourceDefinition + - group: rbac.authorization.k8s.io + kind: ClusterRole + - group: rbac.authorization.k8s.io + kind: ClusterRoleBinding + - group: admissionregistration.k8s.io + kind: ValidatingWebhookConfiguration + - group: external-secrets.io + kind: ClusterSecretStore + sourceRepos: + - '*' + destinations: + - namespace: external-secrets + server: '*' \ No newline at end of file diff --git a/argocd/namespaces/kube-system/applications/csi-driver-nfs.yaml b/argocd/namespaces/kube-system/applications/csi-driver-nfs.yaml new file mode 100644 index 0000000..86b7b57 --- /dev/null +++ b/argocd/namespaces/kube-system/applications/csi-driver-nfs.yaml @@ -0,0 +1,29 @@ +apiVersion: argoproj.io/v1alpha1 +kind: Application +metadata: + name: csi-driver-nfs + namespace: argocd + annotations: + argocd.argoproj.io/sync-wave: "-800" + finalizers: + - resources-finalizer.argocd.argoproj.io +spec: + project: default + syncPolicy: + automated: + prune: true + selfHeal: true + destination: + namespace: kube-system + server: https://kubernetes.default.svc + source: + chart: csi-driver-nfs + repoURL: https://raw.githubusercontent.com/kubernetes-csi/csi-driver-nfs/master/charts + targetRevision: 4.12.1 + helm: + valuesObject: + externalSnapshotter: + enabled: true + controller: + runOnControlPlane: true + diff --git a/argocd/namespaces/kube-system/cilium-l2-announcement-policy.yaml b/argocd/namespaces/kube-system/cilium-l2-announcement-policy.yaml new file mode 100644 index 0000000..233c1ed --- /dev/null +++ b/argocd/namespaces/kube-system/cilium-l2-announcement-policy.yaml @@ -0,0 +1,10 @@ +apiVersion: cilium.io/v2alpha1 +kind: CiliumL2AnnouncementPolicy +metadata: + name: default + namespace: kube-system + annotations: + argocd.argoproj.io/sync-wave: "-1000" +spec: + externalIPs: true + loadBalancerIPs: true \ No newline at end of file diff --git a/argocd/namespaces/project.yaml b/argocd/namespaces/project.yaml index b8104b3..b49465c 100644 --- a/argocd/namespaces/project.yaml +++ b/argocd/namespaces/project.yaml @@ -4,7 +4,7 @@ metadata: name: cert namespace: argocd annotations: - argocd.argoproj.io/sync-wave: "0" + argocd.argoproj.io/sync-wave: "-900" spec: description: Certs clusterResourceWhitelist: diff --git a/argocd/network/certificates.yaml b/argocd/network/certificates.yaml new file mode 100644 index 0000000..3231c53 --- /dev/null +++ b/argocd/network/certificates.yaml @@ -0,0 +1,16 @@ +apiVersion: cert-manager.io/v1 +kind: Certificate +metadata: + name: argocd + namespace: network + annotations: + argocd.argoproj.io/sync-wave: "25" + argocd.argoproj.io/sync-options: SkipDryRunOnMissingResource=true +spec: + secretName: argocd-tls + issuerRef: + name: letsencrypt-http01 + kind: ClusterIssuer + dnsNames: + # Configure the FQDN for ArgoCD here + - "argocd.yourdomain.com" diff --git a/argocd/network/gateways.yaml b/argocd/network/gateways.yaml new file mode 100644 index 0000000..a7dce81 --- /dev/null +++ b/argocd/network/gateways.yaml @@ -0,0 +1,48 @@ +apiVersion: gateway.networking.k8s.io/v1 +kind: Gateway +metadata: + name: acme + namespace: network + annotations: + argocd.argoproj.io/sync-wave: "30" +spec: + gatewayClassName: cilium + addresses: + - type: IPAddress + # Configure your IP address here + value: 192.168.10.96 + listeners: + - name: http + protocol: HTTP + port: 80 + allowedRoutes: + namespaces: + from: All +--- +apiVersion: gateway.networking.k8s.io/v1 +kind: Gateway +metadata: + name: public + namespace: network + annotations: + argocd.argoproj.io/sync-wave: "30" +spec: + gatewayClassName: cilium + addresses: + - type: IPAddress + # Configure your IP address here + value: 192.168.10.97 + listeners: + - name: argocd + protocol: HTTPS + port: 443 + # Configure the FQDN for ArgoCD here + hostname: "argocd.yourdomain.com" + tls: + mode: Terminate + certificateRefs: + - kind: Secret + name: argocd-tls + allowedRoutes: + namespaces: + from: All diff --git a/argocd/network/namespace.yaml b/argocd/network/namespace.yaml new file mode 100644 index 0000000..7b17591 --- /dev/null +++ b/argocd/network/namespace.yaml @@ -0,0 +1,9 @@ +apiVersion: v1 +kind: Namespace +metadata: + name: network + annotations: + argocd.argoproj.io/sync-wave: "-1000" + labels: + name: network +spec: {}