9

I'm running DNSMasq as a local DNS server, so I can resolve *.local.pcfdev.io (as discussed here Using PCF Dev Offline with Mac OS X). Everything worked when I first set things up.

A couple of days later, after a few restarts of my MacBook, whilst offline I can no longer resolve things like api.local.pcfdev.io using curl or ping. However, dig does the right thing.

$ dig api.local.pcfdev.io

; <<>> DiG 9.8.3-P1 <<>> api.local.pcfdev.io
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 46877
;; flags: qr aa rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 0

;; QUESTION SECTION:
;api.local.pcfdev.io.       IN      A

;; ANSWER SECTION:
api.local.pcfdev.io.    0       IN      A       192.168.11.11

;; Query time: 1 msec
;; SERVER: 127.0.0.1#53(127.0.0.1)
;; WHEN: Tue Sep  6 10:17:44 2016
;; MSG SIZE  rcvd: 53

$ curl api.local.pcfdev.io
curl: (6) Could not resolve host: api.local.pcfdev.io

I've tried adding -AlwaysAppendSearchDomains as an argument to /usr/sbin/mDNSResponder in /System/Library/LaunchDaemons/com.apple.mDNSResponder.plist and restarted the mDNSResponder with launchctl, but to no avail.


UPDATE 1

There is definitely something listening on the right local IP:

$ nslookup api.local.pcfdev.io
Server:     127.0.0.1
Address:        127.0.0.1#53

Name:   api.local.pcfdev.io
Address: 192.168.11.11

$ ping api.local.pcfdev.io
ping: cannot resolve api.local.pcfdev.io: Unknown host

$ telnet 192.168.11.11 80
Trying 192.168.11.11...
Connected to 192.168.11.11.
Escape character is '^]'.

HTTP/1.1 400 Bad Request

Connection closed by foreign host.

UPDATE 2

After trying the suggestion below of removing all DNS servers from Network Preferences except 127.0.0.1, I can't resolve anything. I managed to get some debug logging out of mDNSResponder:

mDNSResponder[91]:  74: DNSServiceCreateConnection START PID[32612](ping)
mDNSResponder[91]:  74: Error socket 75 created 00000000 00000001
mDNSResponder[91]:  74: DNSServiceQueryRecord(15000, 0, api.local.pcfdev.io., Addr) START PID[32612]()
mDNSResponder[91]:  74: Error socket 75 closed  00000000 00000001 (0)
mDNSResponder[91]:  74: DNSServiceQueryRecord(api.local.pcfdev.io., Addr) ADD    0 api.local.pcfdev.io. Addr
mDNSResponder[91]:  74: Cancel 00000000 00000001
mDNSResponder[91]:  74: DNSServiceQueryRecord(api.local.pcfdev.io., Addr) STOP PID[32612]()
mDNSResponder[91]:  74: DNSServiceCreateConnection STOP PID[32612](ping)

I did also observe that as explained in the proposed answer, nslookup and dig don't cause anything to be logged by mDNSResponder, but other tools (ping, curl) do.

So it seems like for whatever reason either dnsmasq isn't working (I can establish a TCP connection to 127.0.0.1:53) or mDNSResponder isn't using it.


UPDATE 3

etc/resolve.conf ceases to exist when my wifi adapter is active, but I'm not connected to a network. Could this be why CLI tools don't use the local dnsmasq server?

  • Is your network adapter down by chance? If you go to 'Network' in System Preferences, is there a green dot next to the adapter that dnsmasq is configured for use in? – mango Sep 29 '16 at 15:32
  • Well, I'm on a train with no wi-fi, so presumably. – EngineerBetter_DJ Sep 29 '16 at 15:33
  • 1
    Specifically, is the wifi adapter turned off? If so, please try again with the wifi adapter turned on (even though it might not actually be connected to the internet). For the setup to work, dnsmasq needs to be a DNS server on the network interface in use. – mango Sep 29 '16 at 15:36
  • Thanks for trying to track this down. I'm also struggling with this, don't understand why "curl foo:8989" can't find host but "dig foo" can. Yes, "curl 172.20.0.17:8989" works fine. Like you I have Wi-Fi network DNS set to 127.0.0.1 (a dnsmasq running in a docker container). FWIW in my current situation, the problem is specific to the wifi network I connect to -- works fine on my personal hotspot, problem is on a coffeeshop wifi. – jamshid Sep 14 '17 at 3:35
  • I haven't reverse engineered the programs in question, but my expectation is they are calling entirely different DNS resolution code bases and that's why you are seeing breakage - some are pointing locally, others not. I'd probably dig into curl or wget or get them in instruments / profiler / debugger and see what's really happening to cause the could not resolve error. – bmike May 21 at 12:22
5

dig on the one hand and curl/ping on the other hand are retrieving data from different hosts:

dig queries a DNS server – in your case your localhost (127.0.0.1) – for a database entry: the IP address related to the FQDN api.local.pcfdev.io. The host itself doesn't have to run or even exist at all.

curl/ping try to resolve an IP-address with mDNSResponder or by other means and finally operate on/interact with the remote host. If the host 192.168.11.11 doesn't run or doesn't exist at all, both will fail.

Now, either the DNS entry is wrong (api.local.pcfdev.io has another IP than 192.168.11.11) or the DNS entry is correct but the host 192.168.11.11 is not running.


Adding -AlwaysAppendSearchDomains as an argument to /usr/sbin/mDNSResponder in /System/Library/LaunchDaemons/com.apple.mDNSResponder.plist is not recommended. Instead you should add it to /Library/Preferences/com.apple.mDNSResponder.plist (source: man mDNSResponder):

To cause mDNSResponder to run with these optional arguments when it launches on OS X 10.11 (El Capitan) and later, set the AlwaysAppendSearchDomains or NoMulticastAdvertisements boolean keys to true in /Library/Preferences/com.apple.mDNSResponder.plist and reboot.

In your case it's not necessary at all to set this key, because it's not the cause of your problem.


After digging into VirtualBox, PCF Dev (failing repeatedly with some "wrong credentials" trying to log into the VM) and dnsmasq I recommend to devolve DNS queries to dnsmasq only:

  • In System Preferences > Network > Interface > DNS server remove all DNS servers except 127.0.0.1 and apply the changes. You may also configure a second Location with a 127.0.0.1 only setup and keep your current DNS server in the other configuration.
  • add a file /usr/local/etc/resolv.dnsmasq.conf with the content

    #use your preferred DNS servers here. In the example I use some Google name servers
    nameserver 8.8.8.8
    nameserver 8.8.4.4
    
  • add resolv-file=/usr/local/etc/resolv.dnsmasq.conf at line ~46 of /usr/local/etc/dnsmasq.conf
  • add or move address=/.local.pcfdev.io/192.168.11.11 at/to line ~80 of /usr/local/etc/dnsmasq.conf
  • restart dnsmasq with:

    sudo launchctl stop homebrew.mxcl.dnsmasq
    sudo launchctl start homebrew.mxcl.dnsmasq
    
  • Thanks for taking the time to answer. There is definitely something listening on 192.168.11.11; the actual public DNS entry for *.local.pcfdev.io always points back to the same local IP, so as soon as I connect to the inforwebs curl must be getting an answer from that DNS server and is able to figure out what IP address to use. – EngineerBetter_DJ Sep 6 '16 at 19:25
  • 1
    It seems like curl, ping, and the other binaries I want to hit this thing are using one means of looking up DNS entries (which isn't using the dnsmasq server on localhost), and nslookup and dig are using another means. I guess I need to learn more about mDNSResponder! – EngineerBetter_DJ Sep 6 '16 at 19:29
  • @EngineerBetter Do you have any other entries in System Preferences > Network > Interface > DNS than 127.0.0.1? - I'm gonna install the whole suite (VBox, PCF Dev etc.) and check this... Any special config? – klanomath Sep 6 '16 at 19:34
  • Thanks again for taking the time to help me with this. Question has been updated, still having no luck. – EngineerBetter_DJ Sep 7 '16 at 9:18
4

Had this same issue. I think the local DNS cache had bad data from my previous testing. It was quickly fixed by:

sudo killall -HUP mDNSResponder
0

It took me much longer for me to solve this than it should have. After restarting mDNSResolver dozens of times as recommended on other threads:

sudo killall -HUP mDNSResponder

I finally tried something else. I disabled Wi-Fi and deleted all of my preferred networks. Then I reestablished the Wi-Fi connection and everything worked okay:

  1. Apple menu -> System Preferences -> Wi-Fi (on the left)
  2. 'Turn Wi-Fi Off' then select 'Advanced'
  3. Delete the Wi-Fi connection you're having trouble with (or all of them if you like). Do this by selecting the Wi-Fi network you want to delete and pressing "-"
  4. Click 'Apply' and 'OK'
  5. Turn Wi-Fi back on.
  6. Select your Wi-Fi network and log in again.

YMMV, but this is what finally worked for me. It probably should have been the first thing I tried.

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .