6

When using Wikipedia in Dictionary.app on macOS Big Sur 11.0.1, Dictionary.app sends HTTPS requests to *.b.akamaiedge.net and is not utilizing system proxy (HTTP or socks5). How do I fix this?

20
  • 3
    Whoa, another person actually ran into this weird edge case! I have confirmed that Dictionary also blatantly ignores the system proxy for Wikipedia access on 10.6 (Snow Leopard), 10.9 (Mavericks), and 10.14 (Mojave), and presumably does so on every OS in between. Notably, Hopper makes it seem like Dictionary is reading a Boolean key suspiciously named DCSDisableProxyForWikipediaAccess from somewhere, but nothing I set via defaults write seems to have any effect! Nov 15 '20 at 15:07
  • @Wowfunhappy You should be able to set it via defaults write -g, but either way that key does not actually control proxy behavior. Instead it controls whether the requests are sent to "lookup-api.apple.com/en.wikipedia.org..." or just to en.wikipedia.org directly.
    – 1110101001
    Dec 1 '20 at 23:53
  • @Wowfunhappy The actual connection is done via NSURLConnection and does not really seem out of the ordinary. NSUrlConnection should honor the system proxy. The only thing I can think of is that because it accesses http://... which 307 redirects to the HTTPS version, somehow it doesn't honor the https proxy in that case.
    – 1110101001
    Dec 1 '20 at 23:57
  • 1
    Oh apparently that networking code in the main dictionary app is never used. The real thing is in DictionaryServices.framework. It uses CFHTTP which bypasses system proxy: developer.apple.com/forums/thread/69452
    – 1110101001
    Dec 2 '20 at 0:23
  • 1
    Also for an authoritative source see: opensource.apple.com/source/CFNetwork/CFNetwork-128.2/Headers/… "HTTP proxies are not applied automatically." I guess if you wanted to patch it yourself you could use dyld injection and overwrite "CFReadStreamCreateForHTTPRequest" to always set system proxy before returning the read stream.
    – 1110101001
    Dec 2 '20 at 0:33
3

This phenomenon is not unique to Big Sur—it likely dates back to when the Dictionary application was first introduced in Tiger. @1110101001 did some reverse engineering to figure out what's going on.

DictionaryServices.framework makes network connections via the now-deprecated CFHTTPStream. According to an Apple engineer and Apple's own code, any software which uses CFHTTPStream will ignore the system's proxy settings, unless the developer goes out of their way to add a few extra lines of code.

To fix this problem, we'll need to inject some code which does what the original developers did not—tell the app to apply the system's proxy settings before running CFHTTPStream.

First, compile the following code (e.g. clang -framework AppKit -framework Foundation -o ProxyFix.dylib -dynamiclib /path/to/code.m) to create a library we can inject. This was also mostly written by 1110101001; I tweaked it to work with apps that use two-level namespaces.

#include <stdio.h>
#include <objc/runtime.h>
#include <Foundation/Foundation.h>
#include <dlfcn.h>
#include <AppKit/AppKit.h>

#define DYLD_INTERPOSE(_replacement,_replacee) \
    __attribute__((used)) static struct{ const void* replacement; const void* replacee; } _interpose_##_replacee \
                __attribute__ ((section ("__DATA,__interpose"))) = { (const void*)(unsigned long)&_replacement, (const void*)(unsigned long)&_replacee };

CFReadStreamRef myCFReadStreamCreateForHTTPRequest(CFAllocatorRef alloc, CFHTTPMessageRef request) {
    printf("Injected ProxyFix!\n");
    CFReadStreamRef ref = CFReadStreamCreateForHTTPRequest(alloc, request);
    CFDictionaryRef systemProxyDict = CFNetworkCopySystemProxySettings();
    CFReadStreamSetProperty(ref, kCFStreamPropertyHTTPProxy, systemProxyDict);
    return ref;
}

DYLD_INTERPOSE(myCFReadStreamCreateForHTTPRequest, CFReadStreamCreateForHTTPRequest);

We now need to insert this library into the Dictionary application. Luckily, macOS comes with a built-in mechanism for injecting code in the form of DYLD_INSERT_LIBRARIES. If you, like me, are running an ancient and lovably-hackable version of macOS such as 10.9, all you need to do is run your app after setting this environmental variable. For example, run in Terminal:

DYLD_INSERT_LIBRARIES=/path/to/ProxyFix.dylib /Applications/Dictionary.app/Contents/MacOS/Dictionary

If you're running macOS 10.6 or below, or if the application isn't code signed (possibly because you removed the code signature via optool or similar), you can also add this environment variable to the app's Info.plist, so that the library is injected automatically.

defaults write /Applications/Dictionary.app/Contents/Info LSEnvironment -dict DYLD_INSERT_LIBRARIES @executable_path/../Frameworks/ProxyFix.dylib

Unfortunately, newer versions of macOS have additional security features to prevent code injection. Starting with 10.11, you will need to disable System Integrity Protection in order to use DYLD_INSERT_LIBRARIES. On the newest OS's such as Big Sur, you may (or may not) need to take further steps as well, such as disabling AMFI—I'm not entirely familiar with all of the new security checks Apple has layered on in recent years.


P.S. If you're running 10.6 – 10.9, I wrapped this up into a little installer which helps you set up a proxy, and then applies the above patch to the Dictionary app. Without the aid of a proxy, the Dictionary app's Wikipedia functionality no longer works at all on these systems. https://jonathanalland.com/downloads/wowfunhappy-https-proxy.dmg

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .