How to customise CoreDNS Corefile?

[Javex]

I am getting started with Kubernetes and am trying to now point DNS names at the cluster to access services running on it. For example myapp.example.com. The approach I wanted to take was leverage the k8s_external plugin to e.g. create k8s.example.com as a new subdomain, then CNAME myapp.example.com to something like myapp.k8s.example.com (or whatever the final DNS name would be).

However, I am stuck, because I don't know how to persist changes to the Corefile that I would need to edit. I found a similar issue for k3s at k3s-io/k3s#462 which ended up with a solution of creating a secondary ConfigMap that modifies the configuration. Looking at https://github.com/siderolabs/talos/blob/main/internal/app/machined/pkg/controllers/k8s/templates/core-dns-template.yaml#L53 I don't see any provision for a similar mechanism so I assume this approach wouldn't work at the moment.

I have considered using external-dns, but as I am currently experimenting I don't want to accidentally make bad changes to my production DNS server. Plus, this is a homelab environment and my DNS server is a Pi-Hole which does exist for external-dns, but is in an Alpha stage and the Pi-Hole API is not that well documented and in flux (soon Pi-Hole v6 will come out). All in all, I'd rather point domains manually via CNAME at the k8s cluster than have k8s automatically insert records.

From what I could find, the above CoreDNS solution seems like the best option. I've also considered simply changing the dnsDomain of the cluster, but I could not find much and saw a few places discourage that, though nothing definitive. Since I'm new, I don't know which way is best here, so maybe there's a "supposed" way to do this with Talos that I'm currently missing.

[smira]

You can disable Talos CoreDNS manifest via machine configuration and provide your own.

[Javex]

Thank you for your suggestion, that worked! For future travellers, here are the steps:

• Set cluster.coredns.disabled = true (see https://www.talos.dev/v1.8/reference/configuration/v1alpha1/config/#Config.cluster.coreDNS)
• Set machine.kubelet.clusterDNS = [10.96.0.10] (see https://www.talos.dev/v1.8/reference/configuration/v1alpha1/config/#Config.machine.kubelet). Note the IP here depends on what you configure on the coreDNS side, I chose to stick with the Talos default.
• Apply the config above via talosctl apply-config.
• Remove the existing CoreDNS installation. This isn't necessary if you're bootstrapping a fresh system. Probably best to start fresh if you can, it was a bit fiddly to remove all the parts. The Helm chart will give you errors if you forgot parts.
• Install CoreDNS via Helm chart, see my values.yaml below. I copied the config from what Talos had deployed so this should work.
• You can obviously tweaks things further. For example, I have set serviceType: LoadBalancer and gave it a static external IP as well. But this gets you to the point where you can do that since you control CoreDNS config.

values.yaml

isClusterService: true
serviceType: ClusterIP
service:
  clusterIP: 10.96.0.10

servers:
- zones:
  - zone: .
  port: 53
  # -- expose the service on a different port
  # servicePort: 5353
  # If serviceType is nodePort you can specify nodePort here
  # nodePort: 30053
  # hostPort: 53
  plugins:
  - name: errors

  # Serves a /health endpoint on :8080, required for livenessProbe
  - name: health
    configBlock: |-
      lameduck 5s

  # Serves a /ready endpoint on :8181, required for readinessProbe
  - name: ready

  - name: log
    parameters: .
    configBlock: |-
      class error

  # Serves a /metrics endpoint on :9153, required for serviceMonitor
  - name: prometheus
    parameters: :9153

  # Required to query kubernetes API for data
  - name: kubernetes
    parameters: cluster.local in-addr.arpa ip6.arpa
    configBlock: |-
      pods insecure
      fallthrough in-addr.arpa ip6.arpa
      ttl 30

  - name: forward
    parameters: . /etc/resolv.conf
    configBlock: |-
      max_concurrent 1000

  - name: cache
    parameters: 30
    configBlock: |-
      disable success cluster.local
      disable denial cluster.local

  - name: loop
  - name: reload
  - name: loadbalance