selfhosted.dad

/home

/posts

/the-list

Generate DNS records with Python and Traefik API

  ·  4 min read

Introduction #

I finally tackled a tedious task that’s been bothering me: creating DNS records for container services.

I’m using Traefik as the reverse proxy for my containers, and I use Adguard Home as a local DNS resolver in my home. When I first started self-hosting I only had a single virtual server running all my containers which meant that pointing DNS records for the services was easy, as I only needed a wildcard entry pointing to the virtual machine running Traefik.

Recently I decided to separate my containers so they run on different VMs depending on their purpose, each with a separate Traefik instance. This, of course, means I can no longer use the wildcard entry in Adguard Home, instead I need to create a DNS record for every service I deploy, which can become quite tedious.

Fast forward past me agonizing over this and coming up with a solution for my selfhosted inflicted problem: using python to query Traefiks API and autogenerate DNS records.

The setup #

To make this work, I needed an alternative to Adguard for handling the auto-generated records and Unbound was a good fit, so I spun up a debian LXC in Proxmox and installed it: 

sudo apt install unbound
Traefiks API also needs to be enabled. This is done, for example, by adding the following environment variables to the compose file (on every instance of Traefik): 
---
services:
  traefik:
    environment:
      - TRAEFIK_API=true
...

The script #

Next up is the python script. It’s quite short and nothing fancy. I’ll break it down for those who are interested, the full script is at the bottom of the post.

To start off we need to create a few variables, starting with one containing the IPs of all the Traefik servers. 

traefik_servers = "10.50.10.10","10.50.10.11","10.50.10.12"
A regular expression is needed to filter out our FQDNs (fully qualified domain names) from the API, I used the re module for this. 
p = re.compile("\w+\.self-hosted\.dad")
Then we need to open, read, and close the .conf file so we can use it for comparing later. This file needs to be created manually, before the script is run for the first time! 
conf = open("/etc/unbound/unbound.conf.d/traefik.conf","r")
conf_content = conf.read()
conf.close()
A dictionary variable is also initialized that will later be filled with our FQDNs and their IP. 
dns = {}
Here we loop through the Traefik servers that we set in the beginning, calling their API at /api/http/routers and storing the response as JSON. Afterwards, we iterate again, this time through the JSON response, using our regex method to extract the FQDN from the Traefik rule (this is the router rule). 
for server in traefik_servers: #Iterate through all traefik APIs
    api_url = "http://" + server + ":8080/api/http/routers"
    response = requests.get(api_url)
    response_json = response.json() #Save the response as JSON.
    for x in response_json: #Iterate through the JSON
        match = p.search(x["rule"]) #Match any Traerik "rule" that contains a FQDN.
        if match:
            name = match.group(0)
            dns.update({name: server}) #Save the domain and IP in the dict.

After we’ve filled our dictionary we compare it with our current config. If any FQDNs are new, we overwrite the config. Afterwards we loop through the dictionary, writing a new line per item, after which we reload the Unbound service. 

for hostname in dns.items(): #Iterate through the items in the dict
    if hostname[0] not in conf_content: #Compare with the Unbound conf
        with open("/etc/unbound/unbound.conf.d/traefik.conf","w") as f:
            f.write("server:\n") 
            for hostname, ip in dns.items():
                f.write(f"\tlocal-data: \"{hostname} IN A {ip}\"\n")

        subprocess.run(["systemctl", "reload", "unbound"], check=True) #Reload unbound
        break #End the loop, no need to write more than once
That’s it! Here’s the complete script. 
import requests
import re
import subprocess

traefik_servers = "10.50.10.10","10.50.10.11","10.50.10.12"
p = re.compile("\w+\.self-hosted\.dad")
conf = open("/etc/unbound/unbound.conf.d/traefik.conf","r")
conf_content = conf.read()
conf.close()
dns = {}

for server in traefik_servers: #Iterate through all traefik APIs
    api_url = "http://" + server + ":8080/api/http/routers"
    response = requests.get(api_url)
    response_json = response.json() #Save the response as JSON.
    for x in response_json: #Iterate through the JSON
        match = p.search(x["rule"]) #Match any Traerik "rule" that contains a FQDN.
        if match:
            name = match.group(0)
            dns.update({name: server}) #Save the domain and IP in the dict.

for hostname in dns.items(): #Iterate through the items in the dict
    if hostname[0] not in conf_content: #Compare with the Unbound conf
        with open("/etc/unbound/unbound.conf.d/traefik.conf","w") as f:
            f.write("server:\n") 
            for hostname, ip in dns.items():
                f.write(f"\tlocal-data: \"{hostname} IN A {ip}\"\n")

        subprocess.run(["systemctl", "reload", "unbound"], check=True) #Reload unbound
        break #End the loop, no need to write more than once

The end #

Let this script run as a cron job on the same server running Unbound to have it periodically check your Traefik instances for any new domain names.

Finally, if one is using Adguard Home (like I am), the Unbound server needs to be specified so that Adguard knows where to ask for these domain names. This is done by going to Settings > DNS Settings > Upstream DNS Servers, and adding: 

[/self-hosted.dad/]10.80.10.2

That’s it! Maybe someone else, who likes to overcomplicate their home infrastructure, will find this useful.

back to top