Build Your Own Tailscale: A Self-Hosted Headscale Mesh with Docker and Nginx
| 11 minutes read
I wanted one thing: open opengym.nas.local on my laptop, from anywhere, without exposing a single port of my NAS to the internet.
Tailscale does this beautifully, but its control server is a service somebody else runs. Headscale is an open-source implementation of that control server. You run it yourself, and the official Tailscale clients connect to it. Your devices, your keys, your rules.
This post is the setup that ended up working for me, including the things that broke on the way. By the end you will have:
- a Headscale control server on a VPS behind Nginx and HTTPS
- a NAS and a laptop talking to each other over the mesh
- services on the NAS reachable by name through Nginx Proxy Manager, mesh-only
How the pieces fit together
laptop (100.64.0.2) <==== WireGuard, end-to-end encrypted ====> NAS (100.64.0.3)
\ /
\___ direct if possible, otherwise via a DERP relay ___/
|
VPS: Headscale + Nginx
(control plane at hs.example.com)
Headscale only coordinates: it hands out IPs, keys and DNS. Your traffic flows between the devices themselves (WireGuard). When two devices cannot reach each other directly, for example behind strict NAT, they relay through a DERP server. By default those are Tailscale’s public relays; the traffic stays end-to-end encrypted.
1. Prerequisites
- a Linux VPS with Docker and Nginx installed
- a domain with an
Arecord (herehs.example.com) pointing at the VPS - Tailscale installed on every device you want in the mesh
mkdir -p /opt/headscale/config
cd /opt/headscale
2. Configure Headscale
Create /opt/headscale/config/config.yaml:
server_url: https://hs.example.com
listen_addr: 0.0.0.0:8080
metrics_listen_addr: 0.0.0.0:9090
grpc_listen_addr: 0.0.0.0:50443
grpc_allow_insecure: false
database:
type: sqlite3
sqlite:
path: /var/lib/headscale/db.sqlite
prefixes:
v4: 100.64.0.0/10
v6: fd7a:115c:a1e0::/48
noise:
private_key_path: /var/lib/headscale/noise_private.key
dns:
magic_dns: true
base_domain: mesh.example.com
nameservers:
global:
- 1.1.1.1
- 9.9.9.9
extra_records:
- name: "nas.mesh.example.com"
type: "A"
value: "100.64.0.3"
derp:
server:
enabled: false
urls:
- https://controlplane.tailscale.com/derpmap/default
paths: []
freq: 24h
log:
format: text
level: info
A few things in here are not obvious:
ip_prefixesis gone. Recent versions useprefixeswithv4andv6. Old tutorials will break your start-up.base_domainmust differ from theserver_urldomain. I usemesh.example.com. It is a virtual suffix that only MagicDNS uses on the mesh, so it does not need to exist in public DNS.- Relays.
urlsloads Tailscale’s public relay (DERP) list, so devices fall back to whichever region is closest. The catch: Headscale fetches that URL on start-up. If you want it to start even when Tailscale is down, see the troubleshooting section. - Optional own relay. Headscale can run an embedded DERP (
derp.server.enabled: trueplusregion_id,stun_listen_addr,private_key_pathand your VPS’s publicipv4). I run with it disabled, so I do not cover it here.
Docker Compose
Create /opt/headscale/docker-compose.yml:
services:
headscale:
image: headscale/headscale:0.29.7
container_name: headscale
restart: unless-stopped
command: headscale serve
volumes:
- ./config:/etc/headscale
- ./data:/var/lib/headscale
ports:
- "127.0.0.1:8080:8080"
environment:
- TZ=UTC
Pin the image version. The config format changes between releases, and latest will eventually break you on a restart.
Only the HTTP port is bound to localhost, because Nginx sits in front of it:
docker compose up -d
Port 50443 (remote gRPC) stays unpublished. The CLI talks to Headscale over a local socket through docker exec.
3. Nginx and HTTPS
Headscale listens locally. Nginx terminates TLS and proxies to it. The WebSocket-style upgrade headers matter, because the control connection upgrades to a long-lived stream.
Create /etc/nginx/sites-available/headscale:
map $http_upgrade $connection_upgrade {
default upgrade;
'' close;
}
server {
server_name hs.example.com;
location / {
proxy_pass http://127.0.0.1:8080;
proxy_set_header Host $http_host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection $connection_upgrade;
proxy_buffering off;
}
}
Enable it and get a certificate:
sudo ln -s /etc/nginx/sites-available/headscale /etc/nginx/sites-enabled/
sudo certbot --nginx -d hs.example.com
sudo systemctl reload nginx
Check that it works: docker logs headscale --tail 15 should show listening and serving HTTP on: 0.0.0.0:8080, and https://hs.example.com/health should answer.
4. Users and pre-auth keys
Headscale groups devices by user. Create one and look up its numeric ID:
docker exec -it headscale headscale users create --name admin
docker exec -it headscale headscale users list
A pre-auth key lets a device register from the command line without a browser login. Keep the lifetime short:
docker exec -it headscale headscale preauthkeys create --user 1 --expiration 1h
The key starts with hskey-auth-. Treat it like a password.
5. Connect your devices
On each device:
sudo tailscale up --login-server=https://hs.example.com --authkey=<YOUR_KEY> --accept-routes
A macOS tip that cost me time: use the normal Tailscale app, or a daemon with a real tun device. tailscaled --tun=userspace-networking creates no network interface, so curl and your browser cannot reach 100.64.x.x at all. Everything just hangs. You can check with route -n get 100.64.0.3; you want to see a utun interface.
Manage nodes from the VPS:
docker exec -it headscale headscale nodes list
docker exec -it headscale headscale nodes delete --identifier <NODE_ID>
Test the mesh from a client:
tailscale ping --tsmp -c 3 100.64.0.3
curl -sI http://100.64.0.3:8082
The --tsmp ping works even when no direct connection exists yet. A plain tailscale ping can time out while the mesh is perfectly usable through a relay, so trust a real curl over it.
6. Reach NAS services by name
Now the part I actually cared about. My NAS is a UGREEN DXP4800 (UGOS) at mesh IP 100.64.0.3, running a pile of Docker containers, including a small web app called OpenGym.
Names via MagicDNS
Add one more extra_records entry per name to the dns: block from section 2, then restart Headscale:
extra_records:
- name: "opengym.nas.local"
type: "A"
value: "100.64.0.3"
docker compose restart headscale
Two lessons here:
- Wildcards do not work.
extra_recordsis an exact-match list answered by the client’s built-in resolver (100.100.100.100), so*.nas...resolves nothing and every name needs its own line. If that annoys you, either add one wildcard record at your public DNS provider (*.nas.example.com A 100.64.0.3), or use split DNS: run a small resolver such as dnsmasq on the NAS (address=/nas.example.com/100.64.0.3) and point Headscale at it withdns.nameservers.split: {nas.example.com: [100.64.0.3]}. - Prefer a real domain over
.local. It is reserved for mDNS and can bypass MagicDNS on some systems, and you can never get a Let’s Encrypt certificate for it. Use a real domain you own.
Verify what a client currently knows:
dig +short opengym.nas.local @100.100.100.100
Nginx Proxy Manager, bound to the mesh only
Bind Nginx Proxy Manager (NPM) to the NAS’s mesh IP. It then does not exist on your LAN or the internet:
nginx:
image: jc21/nginx-proxy-manager:latest
ports:
- "100.64.0.3:80:80"
- "100.64.0.3:443:443"
- "100.64.0.3:8082:81" # admin UI
Binding to the mesh IP means ports 80 and 443 are yours, so URLs need no port: http://opengym.nas.local. But on my NAS those ports were taken at first, and the culprits were not where I expected. Find the owner with:
sudo ss -tlnp | grep -E ':(80|443)\b'
Two things held them:
- A custom nginx server block. UGOS runs its own nginx, and I had once added
/etc/nginx/server.d/nas-auryn-dev.confthat listened on 80 and 443. The UGOS portal setting to “release” the ports did nothing about it. Find such files withsudo nginx -T | grep -E "configuration file|listen", move them away, thensudo nginx -t && sudo systemctl reload nginx. tailscale serve. An old config still published services on100.64.0.3:443throughtailscaled.tailscale serve statusshows it andsudo tailscale serve resetremoves it. Docker then reportsaddress already in useuntil you do.
Also check that your other ports are free, since another container already owned 8080 on mine.
When you create a proxy host in NPM, forward to the NAS LAN IP and the port the service publishes, for example 192.168.178.66:4321 for OpenGym. Not localhost, and not the mesh IP: from inside the NPM container, localhost is the container itself, and that gives you a 502.
HTTPS for internal names
Use a real domain and a Let’s Encrypt certificate with the DNS challenge in NPM (SSL Certificates, Let’s Encrypt via DNS). That needs no inbound access, so the services stay mesh-only. Point a name in a domain you own (for example opengym.nas.example.com) at 100.64.0.3 with an extra_records entry, as above. The browser then shows a proper certificate at https://opengym.nas.example.com. A .local name can only be plain HTTP, which is acceptable here because WireGuard already encrypts the traffic.
Wildcard names with Netcup DNS
To send every *.nas.example.com to the NAS, add one wildcard A record at your DNS provider that points at the mesh IP. A private address in public DNS is fine: only devices on the mesh can reach it.
*.nas.example.com A 100.64.0.3
nas.example.com A 100.64.0.3
Then create one wildcard certificate in NPM (SSL Certificates, Add, Let’s Encrypt via DNS) for *.nas.example.com and nas.example.com, and select it on each proxy host. A new service is then just another proxy host.
My domain is at Netcup, which needs a few extra steps:
- Credentials. In the Customer Control Panel under Master Data, API, create an API key. NPM’s Netcup plugin uses the DNS API credentials: your customer number, the API key and an API password. Paste them without quotes in the format NPM shows (
dns_netcup_customer_id,dns_netcup_api_key,dns_netcup_api_password). The newer REST API key is a different thing and does not work here. - Propagation. Set Propagation Seconds in NPM to about 120. Netcup’s nameserver had the TXT records within seconds for me (check with
dig TXT _acme-challenge.nas.example.com @root-dns.netcup.net). If validation fails with only “Some challenges have failed”, raise it. - One attempt at a time. While NPM waits for propagation, certbot holds a lock. Click Save again during that time and the second attempt fails at once with “Internal Error” and
Another instance of Certbot is already runningin the log. Wait for the first run to finish. - Ignore “Test Server Reachability”. That button checks the HTTP challenge from the internet, and it times out for a mesh-only address. The DNS challenge never contacts your server.
- Debugging. The NPM UI only says “Internal Error”. The reason is in
docker logs nginx-proxy-manageranddocker exec nginx-proxy-manager tail -n 80 /data/logs/letsencrypt.log. .devneeds HTTPS. The.devTLD is on the browser HSTS preload list, so browsers refuse plain HTTP for any*.devname, whilecurlworks. Without a certificate you getERR_SSL_UNRECOGNIZED_NAME_ALERT, because NPM has no SSL host for that name yet. Attach the certificate to the proxy host (SSL tab, Force SSL).- Key type. ECDSA is fine. Only choose RSA for very old clients.
- Router. Some routers block public names that resolve to private IPs (DNS rebind protection). If the name fails only at home, allow-list the domain.
Treat API keys like passwords. If one leaks, delete it in the panel and create a new one.
Troubleshooting: what actually broke
Roughly in the order I hit them.
Headscale crash-loops with getting DERPMap ... context deadline exceeded.
By default Headscale downloads Tailscale’s public DERP map on every start and exits if it cannot. During an incident on Tailscale’s coordination service, controlplane.tailscale.com/derpmap/default accepted the TLS connection and then never answered, from every network I tried. Headscale restarted in a loop, never loaded my config changes, and new DNS records never reached the clients. The start-up fetch is a hard dependency on Tailscale. If you want Headscale to start even when that endpoint is down, save the map once as a local file (curl -sS https://controlplane.tailscale.com/derpmap/default -o config/derp.json) and list it under paths: [/etc/headscale/derp.json]. Check status.tailscale.com before you debug your own network.
What is DERP anyway? The encrypted relay Tailscale falls back to when two devices cannot connect directly, for example because of NAT or firewalls. The traffic stays end-to-end encrypted; the relay only forwards packets.
Only some names resolve after editing extra_records.
The clients still hold the old network map. Confirm Headscale is really running (docker logs headscale), then restart the Tailscale client and recheck with dig against 100.100.100.100.
tailscale ping times out, but real traffic works.
Plain ping needs a direct path. Use tailscale ping --tsmp and a curl.
client version != tailscaled server version on macOS.
The app updated, but the daemon was not restarted. Quit Tailscale completely or reboot.
tailscale up complains about non-default flags.
Re-run it with the flags it prints, or use --reset. Be careful with --advertise-routes and --advertise-exit-node if you do not need them.
Container will not start after following an old guide.
ip_prefixes was replaced by prefixes, and noise.private_key_path is required. docker logs headscale says which.
NPM returns 502.
Wrong forward host or port. On the NAS: curl -sI http://<NAS-LAN-IP>:<port>.
Is it DNS or NPM?
Bypass DNS: curl -H "Host: opengym.nas.local" http://100.64.0.3/. A response means NPM and the proxy host are fine, and the problem is name resolution.
What I would do differently
- Decide early whether Headscale should depend on Tailscale’s public DERP map at start-up, or keep a local copy.
- Skip
.localentirely, and use a real domain so HTTPS works from the start. - Pin every image version, Headscale especially.
The mesh itself now runs quietly. Everything on the NAS is a name away, and nothing is open to the internet. If you run into something I did not cover, I would like to hear about it.