Take the 2-minute tour ×
Ask Different is a question and answer site for power users of Apple hardware and software. It's 100% free, no registration required.

I want to setup a VPN on my mac but I can't make it work. If I start my Windows 7 VM, the VPN connects fine (from Windows) but if I try to connect from my Mac it fails…

First of all, here is the system log:

Oct  7 15:18:45 Leonardo-Ferreiras-iMac.local configd[18]: SCNC: start, triggered by System Preferen, type L2TP, status 0  
Oct  7 15:18:45 Leonardo-Ferreiras-iMac.local pppd[3385]: pppd 2.4.2 (Apple version 596.15.2) started by leo, uid 501  
Oct  7 15:18:45 Leonardo-Ferreiras-iMac.local pppd[3385]: L2TP connecting to server 'vpn.aec.com.br' (186.249.4.10)...  
Oct  7 15:18:45 Leonardo-Ferreiras-iMac.local pppd[3385]: IPSec connection started  
Oct  7 15:18:46 Leonardo-Ferreiras-iMac.local racoon[192]: Connecting.  
Oct  7 15:18:46 Leonardo-Ferreiras-iMac.local racoon[192]: IPSec Phase1 started (Initiated by me).  
Oct  7 15:18:46 Leonardo-Ferreiras-iMac.local racoon[192]: IKE Packet: transmit success. (Initiator, Aggressive-Mode message 1).  
Oct  7 15:18:47 Leonardo-Ferreiras-iMac.local com.apple.quicklook.satellite[3358]: [QL] No sandbox token for thumbnail request file://localhost/private/var/log/racoon.log, it will probably fail  
Oct  7 15:18:47 Leonardo-Ferreiras-iMac kernel[0]: Sandbox: sandboxd(3386) deny mach-lookup com.apple.coresymbolicationd  
Oct  7 15:18:48 Leonardo-Ferreiras-iMac.local sandboxd[3386] ([3358]): QuickLookSatelli(3358) deny file-read-data /private/var/log/racoon.log  
Oct  7 15:18:49 --- last message repeated 2 times ---  
Oct  7 15:18:49 Leonardo-Ferreiras-iMac.local racoon[192]: IKE Packet: transmit success. (Phase1 Retransmit).  
Oct  7 15:18:50 Leonardo-Ferreiras-iMac.local com.apple.quicklook.satellite[3358]: [QL] No sandbox token for thumbnail request file://localhost/private/var/log/racoon.log, it will probably fail  
Oct  7 15:18:50 Leonardo-Ferreiras-iMac.local sandboxd[3386] ([3358]): QuickLookSatelli(3358) deny file-read-data /private/var/log/racoon.log  
Oct  7 15:18:51 --- last message repeated 2 times ---  
Oct  7 15:18:51 Leonardo-Ferreiras-iMac.local com.apple.quicklook.satellite[3358]: [QL] No sandbox token for thumbnail request file://localhost/private/var/log/racoon.log, it will probably fail  
Oct  7 15:18:52 Leonardo-Ferreiras-iMac.local sandboxd[3386] ([3358]): QuickLookSatelli(3358) deny file-read-data /private/var/log/racoon.log  
Oct  7 15:18:52 --- last message repeated 1 time ---  
Oct  7 15:18:52 Leonardo-Ferreiras-iMac.local racoon[192]: IKE Packet: transmit success. (Phase1 Retransmit).  
Oct  7 15:18:52 Leonardo-Ferreiras-iMac.local sandboxd[3386] ([3358]): QuickLookSatelli(3358) deny file-read-data /private/var/log/racoon.log  
Oct  7 15:18:52 Leonardo-Ferreiras-iMac.local com.apple.quicklook.satellite[3358]: [QL] No sandbox token for thumbnail request file://localhost/private/var/log/racoon.log, it will probably fail  
Oct  7 15:18:53 Leonardo-Ferreiras-iMac.local sandboxd[3386] ([3358]): QuickLookSatelli(3358) deny file-read-data /private/var/log/racoon.log  
Oct  7 15:18:54 --- last message repeated 2 times ---  
Oct  7 15:18:54 Leonardo-Ferreiras-iMac.local com.apple.quicklook.satellite[3358]: [QL] No sandbox token for thumbnail request file://localhost/private/var/log/racoon.log, it will probably fail  
Oct  7 15:18:54 Leonardo-Ferreiras-iMac.local sandboxd[3386] ([3358]): QuickLookSatelli(3358) deny file-read-data /private/var/log/racoon.log  
Oct  7 15:18:55 --- last message repeated 2 times ---  
Oct  7 15:18:55 Leonardo-Ferreiras-iMac.local racoon[192]: IKE Packet: transmit success. (Phase1 Retransmit).  
Oct  7 15:18:55 Leonardo-Ferreiras-iMac.local com.apple.quicklook.satellite[3358]: [QL] No sandbox token for thumbnail request file://localhost/private/var/log/racoon.log, it will probably fail  
Oct  7 15:18:56 Leonardo-Ferreiras-iMac.local pppd[3385]: IPSec connection failed  
Oct  7 15:18:56 Leonardo-Ferreiras-iMac.local pppd[3385]: L2TP connecting to alternate server 'vpn.aec.com.br' (200.251.240.177)...  
Oct  7 15:18:56 Leonardo-Ferreiras-iMac.local pppd[3385]: IPSec connection started  
Oct  7 15:18:56 Leonardo-Ferreiras-iMac.local racoon[192]: Connecting.  
Oct  7 15:18:56 Leonardo-Ferreiras-iMac.local racoon[192]: IPSec Phase1 started (Initiated by me).  
Oct  7 15:18:56 Leonardo-Ferreiras-iMac.local racoon[192]: IKE Packet: transmit success. (Initiator, Aggressive-Mode message 1).  
Oct  7 15:18:56 Leonardo-Ferreiras-iMac.local sandboxd[3386] ([3358]): QuickLookSatelli(3358) deny file-read-data /private/var/log/racoon.log  
Oct  7 15:18:56 --- last message repeated 2 times ---  
Oct  7 15:18:56 Leonardo-Ferreiras-iMac.local com.apple.quicklook.satellite[3358]: [QL] No sandbox token for thumbnail request file://localhost/private/var/log/racoon.log, it will probably fail  
Oct  7 15:18:57 Leonardo-Ferreiras-iMac.local sandboxd[3386] ([3358]): QuickLookSatelli(3358) deny file-read-data /private/var/log/racoon.log  
Oct  7 15:18:59 --- last message repeated 2 times ---  
Oct  7 15:18:59 Leonardo-Ferreiras-iMac.local racoon[192]: IKE Packet: transmit success. (Phase1 Retransmit).  
Oct  7 15:19:00 Leonardo-Ferreiras-iMac.local com.apple.quicklook.satellite[3358]: [QL] No sandbox token for thumbnail request file://localhost/private/var/log/racoon.log, it will probably fail  
Oct  7 15:19:00 Leonardo-Ferreiras-iMac.local sandboxd[3386] ([3358]): QuickLookSatelli(3358) deny file-read-data /private/var/log/racoon.log  
Oct  7 15:19:01 --- last message repeated 2 times ---  
Oct  7 15:19:01 Leonardo-Ferreiras-iMac.local com.apple.quicklook.satellite[3358]: [QL] No sandbox token for thumbnail request file://localhost/private/var/log/racoon.log, it will probably fail  
Oct  7 15:19:02 Leonardo-Ferreiras-iMac.local racoon[192]: IKE Packet: transmit success. (Phase1 Retransmit).  
Oct  7 15:19:03 Leonardo-Ferreiras-iMac.local com.apple.quicklook.satellite[3358]: [QL] No sandbox token for thumbnail request file://localhost/private/var/log/racoon.log, it will probably fail  
Oct  7 15:19:03 Leonardo-Ferreiras-iMac.local sandboxd[3386] ([3358]): QuickLookSatelli(3358) deny file-read-data /private/var/log/racoon.log  
Oct  7 15:19:05 --- last message repeated 2 times ---  
Oct  7 15:19:05 Leonardo-Ferreiras-iMac.local com.apple.quicklook.satellite[3358]: [QL] No sandbox token for thumbnail request file://localhost/private/var/log/racoon.log, it will probably fail  
Oct  7 15:19:05 Leonardo-Ferreiras-iMac.local racoon[192]: IKE Packet: transmit success. (Phase1 Retransmit).  
Oct  7 15:19:05 Leonardo-Ferreiras-iMac.local sandboxd[3386] ([3358]): QuickLookSatelli(3358) deny file-read-data /private/var/log/racoon.log  
Oct  7 15:19:06 --- last message repeated 2 times ---  
Oct  7 15:19:06 Leonardo-Ferreiras-iMac.local pppd[3385]: IPSec connection failed  
Oct  7 15:19:06 Leonardo-Ferreiras-iMac.local racoon[192]: IPSec disconnecting from server 200.251.240.177  
Oct  7 15:19:06 Leonardo-Ferreiras-iMac.local racoon[192]: IPSec disconnecting from server 186.249.4.10  

Second: The Windows 7 VPN manual (the only one available) states that I should check the box "CHAP Protocol" & "CHAP v2" and I must select the option where encryption is required.

Third: I must also enter the pre-shared-key, which I deduce is the shared-secret option of mountain lion.

Still no luck at all… can anyone help?

EDIT 1: The dump file is here

share|improve this question
    
Are you using the built-in Windows VPN client to connect (i.e., did the instructions resemble those on this page:helpdesk.princeton.edu/kb/display.plx?ID=9987 ?) –  Eddie Kelley Oct 15 '13 at 19:51
    
@EddieKelley yes... looks a lot like that... I only have to set the shared key... –  Leonardo Oct 15 '13 at 20:11
    
A packet capture of the traffic going between your machine and the VPN gateway may help us to diagnose the issues. You can use a Terminal command to capture that traffic into a file in your home directory named "dump.pcap": sudo tcdump -w ~/dump.pcap host vpn.aec.com.br. Start that command in Terminal, then attempt the VPN connection (and allow it to timeout). You can signal the tcpdump process with an interrupt (Control+C) to kill the packet trace. After the connection fails, you can post the capture file here for evaluation. –  Eddie Kelley Oct 15 '13 at 21:12
    
@EddieKelley ok! i edited the question... i dont know if i captured the right dump though... your command was invalid... so i used "sudo tcpdump -i en0 -s 0 -B 524288 -w ~/Desktop/DumpFile01.pcap" –  Leonardo Oct 15 '13 at 21:49
    
My apologies - that should have been sudo tcpdump -w ~/dump.pcap host vpn.aec.com.br...The dump you performed definitely worked, but isn't providing any additional information. The packet trace shows your machine initiating and retrying the Phase 1 IPSec connection several times for each of the hosts, but is not receiving any response from the machines on the other side. –  Eddie Kelley Oct 15 '13 at 22:31

2 Answers 2

It seems odd that your machine is attempting to use aggressive-mode in phase 1 of the IPSec connection. From what I can see, the default in 10.8.5 is to use main-mode. I'm wondering if you've modified the file at /etc/racoon/racoon.conf in trying to get this working? Can you post the contents of the file at /etc/racoon/racoon.conf?

Alternately, it may be possible to duplicate the configuration file that the L2TP/IPSec connection in System Preferences is using, and modify it slightly (configure to use main-mode) to see if that helps. Unfortunately, that configuration file is created at connect time, and is copied into /var/run/racoon/fqdn.of.host.conf. This file is also readable only by root (to protect the PSK that's stored inside of it). If you're quick, you can prepare a Terminal command that will copy that config file, initiate the VPN connection from System Preferences or the menu bar, and perform the copy command while the VPN is attempting to connect (you may have 5-10 seconds to perform the copy). Here's the command that you'd run right after initiating the VPN connection:

sudo cp -Rp /var/run/racoon/vpn.aec.com.br ~/Desktop

That file has permissions that disallow users from reading it. Use the command to make it readable:

sudo chmod 777 ~/Desktop/vpn.aec.com.br

NOTE: the file that you've copied onto the Desktop contains your PSK. You should not post this file (unmodified) to this website. If you would like to post this configuration file for troubleshooting, obfuscate the value of shared_secret before posting.

You can configure the VPN connection to use main-mode by modifying the configuration file that you've copied onto your Desktop. Open that file in any text editor, and locate the line that says exchange_mode. You may find that it says something like:

exchange_mode aggressive;

If so, see if you can change it so that it uses main-mode:

exchange_mode main;

Next, disable the built-in launchd script that launches racoon normally:

sudo launchctl unload /System/Library/LaunchDaemons/com.apple.racoon.plist

Then, you can temporarily launch racoon, and instruct it to use the configuration file that you exported:

sudo racoon -f ~/Desktop/vpn.aec.com.br -l /var/log/racoon.log

The racoon process will run in the background using the configuration file that you copied onto the Desktop. The process will also output to /var/log/racoon.log Next, you can initiate a VPN connection using racoonctl:

sudo racoonctl vpn-connect vpn.aec.com.br

The VPN menubar item will not update while this process is connecting. However, you should see output in /var/log/system.log and in /var/log/racoon.log that reflects the fact that the IPSec Phase 1 connection is moving along (i.e., Main Mode Message 2, 3, 4, etc.). If you see that the machine is able to establish the IPSec tunnel (look for "IPSec Phase1 established" in system.log), you're likely having problems with the exchange_mode that's configured.

How to undo this configuration

To put things back to the way that they were, you can disconnect (if the connection was successful) and then terminate the racoon process that's running:

sudo racoonctl vpn-disconnect

sudo killall racoon

Then, re-load the launchd config for racoon:

sudo launchctl load -w /System/Library/LaunchDaemons/com.apple.racoon.plist

How to make this configuration permanent

If the racoon configuration file that you used in the tests above seems to be allowing phase 1 to complete, you can configure racoon to use that config file in the future:

Copy the config file that is on your Desktop into the folder /etc/racoon:

sudo cp ~/Desktop/vpn.aec.com.br.conf /etc/racoon/vpn.aec.com.br.conf

Make sure to secure the permissions on that file:

sudo chmod 600 /etc/racoon/vpn.aec.com.br.conf

Comment out the last line of the file at /etc/racoon/racoon.conf, and add a line that will load the configuration that you copied to /etc/racoon in the step above:

include "/var/run/racoon/*.conf" ;

Should be changed to:

#include "/var/run/racoon/*.conf" ;
include "/etc/racoon/vpn.aec.com.br.conf" ;

After you've made this change, you can save the file and attempt the VPN connection again from the menu bar.

share|improve this answer
    
well... "Oct 15 22:25:18 Leonardo-Ferreiras-iMac.local racoon[27389]: IPSec Phase1 established (Initiated by me)."... is that a good thing? –  Leonardo Oct 16 '13 at 1:33
    
It seems to be...This indicates that your machine should be using main mode when connecting to the VPN (or at least you're getting further than you were with aggressive mode). As far as how to enable main mode in the configuration - it should be generating a config file with that information...I"m assuming that the file you copied from /var/run/racoon had aggressive mode set? Is the L2TP/IPSec VPN interface that you have created in System Preferences one that was carried over from an old machine? It might help to remove and re-add that VPN interface or create a new network location? –  Eddie Kelley Oct 16 '13 at 2:27
    
yes, aggressive mode was set. I already re-created the VPN from scratch a couple of times... every-time aggressive mode comes back on... when using the command line: How can i know that the VPN is up? will i get a "connected" or something? –  Leonardo Oct 16 '13 at 3:27
    
i add the ",base" to the exchange and i got a "VPN connexion established" but i dont think that it means what i think it means... or maybe some config is missing... because, for example, i cant ping my machine based on its name... –  Leonardo Oct 16 '13 at 3:33
    
Can you clarify what you mean by "i add the ",base" to the exchange"? Note: the steps using racoon above are only for troubleshooting the phase 1 connection. You should undo the setup as described at the end of the article to restore the normal function of racoon. Once you've restored the configuration, it will still be necessary to figure out how to get OS X to create the correct configuration files. It's possible that removing the file at /Library/Preferences/SystemConfiguration/preferences.plist may help, but I have not confirmed where that configuration is being generated from. –  Eddie Kelley Oct 16 '13 at 4:13
  1. Make sure that Back to My Mac is disabled under System PreferencesiCloud.

  2. There are certain characters that OS X does not like in shared secrets. Make sure that your shared secret…

    • is shorter than 64 characters in length
    • does not contain ' (apostrophe)
    • does not contain , or (prime, double prime or triple prime)
  3. Allow access to your key stored in Keychain Access:

    1. Open Keychain Access (/Applications/Utilities/)

    2. Select the System keychain on the top-left list of keychains.

    3. Find the certificate relating to your VPN server and click the disclosure triangle to the left of it to show the Private Key (should have a key icon).

    4. Double-click it to open the Private Key properties, and select the Access Control tab.

    5. Select Allow all applications to access this item.

    6. Click Save Changes and enter your administrator password.

share|improve this answer
    
no luck... "Back to My Mac" was activated, but clearing it did not solved it... the secret does contains specials chars but they are '*','$' and '+'... the logs remain the same... –  Leonardo Oct 15 '13 at 18:03
    
@Leonardo Try my edit?… –  grgarside Oct 15 '13 at 18:37
    
well, there are no certificates that match my vpn... all I have is "IPSec Shared Secret"... no certificates... anyway, I allowed every one to access it but it did not get me through... logs remain the same –  Leonardo Oct 15 '13 at 19:20

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.