Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

cannot connect to localhost using tailscale IP -- breaks MagicDNS(?) on the global nameserver's host #14052

Closed
rrix opened this issue Nov 9, 2024 · 2 comments
Closed

cannot connect to localhost using tailscale IP -- breaks MagicDNS(?) on the global nameserver's host #14052

rrix opened this issue Nov 9, 2024 · 2 comments
Labels
bug Bug

Comments

@rrix
Copy link

rrix commented Nov 9, 2024 •
edited
Loading

What is the issue?

my tailnet (catla-catla.ts.net) had for a long period of time a pihole running on one host exposed and listening on all interfaces. I had to consolidate hardware recently and moved from pihole on one host to blocky dns on another existing host (name of last-bank). This has wreaked havoc on the network. I am in a position where I have MagicDNS global nameservers set to my last-bank tailnet IP and this works everywhere but on that host.

Steps to reproduce

I had to set up split DNS with systemd-resolved because tailscale and dhclient were fighting over the /etc/hosts file and the host i migrated from had a static network configuration ... okay okay, fine.

Get that all working, find I also need to get it working on my laptops, do this... have to run the DNS server not on all interfaces but now explicitly bind it to the LAN IPs and the host's tailnet IP because otherwise it complainsa bout systemd-resolved listening on a :53, okay okay, fine...

now it listens on the interface IPs directly:

[nix-shell:~]$ sudo ss -ulnp | grep :53
[...]
UNCONN 0      0         127.0.0.54:53         0.0.0.0:*    users:(("systemd-resolve",pid=12189,fd=23))                                       
UNCONN 0      0      127.0.0.53%lo:53         0.0.0.0:*    users:(("systemd-resolve",pid=12189,fd=21))                                       
UNCONN 0      0      100.123.74.28:53         0.0.0.0:*    users:(("blocky",pid=5628,fd=16))                                                 
UNCONN 0      0      192.168.69.69:53         0.0.0.0:*    users:(("blocky",pid=5628,fd=15))                                                 
UNCONN 0      0          127.0.0.1:53         0.0.0.0:*    users:(("blocky",pid=5628,fd=12))                                                 
[...]

I direct MagicDNS to use the tailnet IP for last-bank and after more wailing and gnashing of resolvectl this works on all my endpoints except last-bank, the host running the global nameserver:

[last-bank:~]$ dig @100.123.74.28 infowars.com # known to be blocked in blocky as canary domain
;; communications error to 100.123.74.28#53: timed out
  C-c C-c
[last-bank:~]$ dig @100.100.100.100 infowars.com 
;; communications error to 100.100.100.100#53: timed out
;; communications error to 100.100.100.100#53: timed out
[last-bank:~]$ resolvectl
Global
           Protocols: +LLMNR +mDNS -DNSOverTLS DNSSEC=no/unsupported
    resolv.conf mode: stub
Fallback DNS Servers: 9.9.9.9 8.8.8.8

Link 2 (eno1)
    Current Scopes: none
         Protocols: -DefaultRoute +LLMNR +mDNS -DNSOverTLS DNSSEC=no/unsupported

Link 3 (eno2)
    Current Scopes: DNS LLMNR/IPv4 LLMNR/IPv6 mDNS/IPv4 mDNS/IPv6
         Protocols: -DefaultRoute +LLMNR +mDNS -DNSOverTLS DNSSEC=no/unsupported
Current DNS Server: 192.168.69.1
       DNS Servers: 192.168.69.1
        DNS Domain: home.arpa

Link 4 (eno3)
    Current Scopes: none
         Protocols: -DefaultRoute +LLMNR +mDNS -DNSOverTLS DNSSEC=no/unsupported

Link 5 (eno4)
    Current Scopes: none
         Protocols: -DefaultRoute +LLMNR +mDNS -DNSOverTLS DNSSEC=no/unsupported

Link 7 (tailscale0)
    Current Scopes: DNS
         Protocols: +DefaultRoute -LLMNR -mDNS -DNSOverTLS DNSSEC=no/unsupported
Current DNS Server: 100.100.100.100
       DNS Servers: 100.100.100.100
        DNS Domain: catla-catla.ts.net ~.

This lead me to this discovery, and i'm not sure how my local DNS ever worked on the first pihole host:

rrix@last-bank:~ :( $ ping last-bank.catla-catla.ts.net
PING last-bank.catla-catla.ts.net (100.123.74.28) 56(84) bytes of data.
  C-c C-c
--- last-bank.catla-catla.ts.net ping statistics ---
3 packets transmitted, 0 received, 100% packet loss, time 2062ms

[rrix@virtuous-cassette:~]$ ping virtuous-cassette.catla-catla.ts.net
PING virtuous-cassette.catla-catla.ts.net (100.116.176.133) 56(84) bytes of data.
  C-c C-c
--- virtuous-cassette.catla-catla.ts.net ping statistics ---
2 packets transmitted, 0 received, 100% packet loss, time 1024ms

Looking at table 52, I see it's created entries for all hosts in my tailnet EXCEPT the local node, there's no way for any of my TS nodes to loop traffic back to itself using the tailnet IP address?

How can I get a box running my MagicDNS global nameservers to talk to those same nameservers using MagicDNS so that I can ALSO resolve my tailnet short names on that host?

Are there any recent changes that introduced the issue?

No response

OS

Linux

OS version

nixos 24.05

Tailscale version

1.76.0 go version: go1.23.1

Other software

man idk:

Linux last-bank 6.6.57 #1-NixOS SMP PREEMPT_DYNAMIC Thu Oct 17 13:24:38 UTC 2024 x86_64 GNU/Linux

systemd 255 (255.9)
+PAM +AUDIT -SELINUX +APPARMOR +IMA +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 -IDN +IPTC +KMOD +LIBCRYPTSETUP +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -XKBCOMMON +UTMP -SYSVINIT default-hierarchy=unified

Bug report

BUG-c1e0b1cbbf04f3409b12833046ba71b7c66802388fcbd358b7d44185dbd2f63b-20241109204930Z-36b96443167c2d10
@rrix rrix changed the title cannot connect to localhost using tailscale IP -- breaks MagicDNS(?) cannot connect to localhost using tailscale IP -- breaks MagicDNS(?) on the global nameserver's host Nov 9, 2024
@Kyesarri
Copy link

#13863

Same issue?

Kernel 6.6.57 shows this behaviour. Please try ''tailscale status'' on your host machine and see / post the output.

MARK: bad value for option "--set-mark", or out of range (0-4294967295).
Try `ip6tables -h' or 'ip6tables --help' for more information.

@rrix
Copy link
Author

rrix commented Nov 28, 2024

aha, thank you. I did notice that warning in the output of tailscale status in the past. And, well, my server's UPS tripped this week and it came up running 6.6.61 and no longer has that warning, and it can ping the local tailscale IP again!

@rrix rrix closed this as completed Nov 28, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug Bug
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@rrix @Kyesarri @kelivel