Mac OS X Server has included VPN support for some time.  And, in true Apple fashion, it brings simplicity to a very complicated and technical server function.  Virtual Private Networks, or VPNs, are used to securely connect two networks over the internet.  This is done by creating an encrypted tunnel between the two networks.  The tunnel wraps around all data that is passed in either direction.  This keeps the information safe from prying eyes as it crosses the insecure internet.  The tunnel endpoints take care of all the encryption and decryption so that, once the tunnel is established, the network communication is seamless to users.

In many cases, VPNs connect two routers and effectively bridges two networks.  In the case of a telecommuter, the home router might establish a tunnel with a corporate router in order to allow the home user access to services on the company network.  In this scenario, the two routers are the endpoints for the VPN.  Router to router based VPNs are often very difficult to configure, especially when one of the endpoints is a high powered enterprise class device like those provided by companies like Cisco.  Router to router VPNs are often hardware based because the routers on either end have hardware built into them that is dedicated to processing VPN traffic.

Mac OS X Server has the ability to create software based VPN tunnels.  Combine that with the VPN client software built into the client version of Mac OS X and you have a very powerful and easy to configure VPN solution.

Consider this scenario.  A corporation runs Mac OS X server on their network.  A number of mobile users need to connect to the corporate network in order to access internal systems.  Once the Mac server is properly configured, the remote users can establish a secure VPN tunnel between their desktop machine and the corporate network using nothing more that software already built into their operating system.  And, once the VPN tunnel is established, all of the information exchanged between the remote user and the office network is fully encrypted and secure.

In this article, we will set up Mac X 10.4 Server to function as a VPN server.  We will also look at the client configurations needed to connect to that server from Mac OS X 10.4 client (the non-server version of the OS) as well as from Windows XP.

Server:
The server must be running Mac OS X Server (10.4.3 as of this writing).  The VPN server capabilities are not built-in to the client version of the OS.

Open Server Admin, located here on your drive: /Applications/Server.  Connect to the server using its IP address and the proper username and password.  Once you do, you will see a list of services available on that machine.  Click on VPN and the VPN settings will appear on the right.

Select the L2TP tab and use the image above as an example.  Note that the IP addresses used in the image are for example only.

When a remote user connects to the internet, they receive an IP address from their service provider.  When the VPN tunnel is negotiated with the VPN server, the server assigns the client an IP address from the corporate network.  When the client accepts that address as part of the VPN negotiation, it adds it to the network interface in addition to the IP address from the internet service provider (ISP).  This means that the VPN client actually has 2 addresses bound to it.  One from the ISP, and one from the corporate network.

The VPN server needs to dynamically assign clients IP addresses from a pool of possible addresses.  That is what we are specifying in this screen.  You must specify both the starting and ending addresses of the IP pool that the VPN server is allowed to hand out to connect clients.  Note that when a client disconnects from the VPN, his IP address is freed up and put back in the pool to be used by future clients.  It is also essential to be sure that the addresses that are used in this pool are not used by any other computers on the corporate network.  If they are, conflicts will occur and neither user will be able to access the network.

Set PPP Authentication to MS-CHAPv2 and specify a Shared Secret.  This Shared Secret should be the strongest possible password you can come up with.  Make sure it is not a dictionary word.  And, the more digits in the Shared Secret, the better.  The 3 weakest parts of the VPN are the username and password the user uses to connect, and the Shared Secret.  If you use weak passwords or secrets, a tunnel could be established by anyone who might be able to guess that information.

Next, select the PPTP tab.  Just as before, you must specify a pool of addresses that can be used by VPN users who connect using PPTP.

Under Mac OS X Server, Mac clients generally connect to the VPN server using L2TP.  Windows XP users connect using PPTP.  L2TP is considered more cryptographically sound, but since Microsoft did not conform to IPSec based standards when they wrote XP’s VPN client, Windows users are forced to use PPTP.

Finally, select the Client Information tab.

Here we specify the DNS servers the client should use once they have connected to the VPN.  Since many corporations use internal DNS servers, the servers specified here will be used on any traffic that is traveling through the VPN.

Under Network Routing Definition we set the rules for the VPN routing.  In my example, the corporate network is a Class C or addresses ranging from 66.62.25.1 – 66.62.25.255.  In this example, the Network Address is entered as 66.62.25.22, but it might more appropriately be entered as 66.62.25.0 since the Network Mask of 255.255.255.0 details the assignment of the entire Class C.  The final key value here is the Network Type.  It is set to Private.  This means that any traffic to or from the client that is destined for the 66.62.25.x network is considered internal and should remain on the secure VPN.  Any addresses not listed as private here are not secure and the VPN client will route that traffic over the normal internet connection rather than sending it down the VPN tunnel to the corporate network.  This is why the VPN client maintains a connection to the ISP assigned IP address in addition to the address that is assigned to it by the VPN server.

Lastly, a user account must be created on the server.  This is done through the Workgroup Manager, and application located in the same directory as the Server Admin.  When you create the account, be sure to set a strong password for the account.  The username and password created here will be the credentials that the remote user will use when they log into the VPN.

Mac OS X VPN Client Configuration:
The Mac VPN client is much easier to configure than the Window XP based equivalent.

Select New VPN Connection from the file menu, then choose L2TP over IPSec and continue.

A new profile will open.  Don’t fill in the information in this screen.  If you do, you will miss one vital piece of information.  There is no place to specify the Shared Secret for the connection.  Without it, the tunnel will never establish.  Select Edit Configurations from the Configuration menu.

Fill in the fields with the appropriate information.  The description can be anything you want it to be.  The Server Address is the IP address of the Mac VPN server.  The Account Name and Password is the login that you created for the user in the Workgroup Manager.  Be sure to enter the same Shared Secret that you used when setup L2TP on the VPN server.

VPN On Demand is a new feature in 10.4.  When you enable this feature, you are required to list domains that will trigger activation of the VPN tunnel when you try to access them.

When you click OK, your client is all set.

It is worth looking at some of the advanced options available under the Connect menu and then Options.  There is an option to send all traffic over the VPN.  This can be a powerful option.  Normally you would not want to do this as it will increase traffic on the corporate end of the network.  But, if you are a user on the road and using a hotspot or public wireless network, it might be a good idea to enable this option.  In doing that, all of the traffic becomes protected from other users who might be sniffing traffic on the wireless network.

Windows XP VPN Client Configuration:
Windows XP also has a built-in VPN client, but it has some disadvantages.  First and foremost, it does not fully comply with standards based VPN servers.  Once again, Microsoft has decided that it knows better and went in it’s own direction.  On the upside, if you enabled PPTP on your Mac VPN server, XP users can still access the network.

First of all, right click on My Network Places a choose Properties.  You will see a list of your network adapters.  Click Create a New Connection on the left.

Select Connect to the Network At My Workplace.  Its an odd name for it, but this allows you to create a VPN.

Select Virtual Private Network Connection and click Next.

Give your VPN connection a logical name.  Anything that works for you is fine here.

Here you specify the IP address of the Mac VPN server.

Click finish here.  You’re not really done yet.  We need to make some changes to the VPN adapters configuration before you can connect to the Mac server.

Now go back to the Network Connections window.  A new adapter should have been added to the screen.  It will have the name that you gave the VPN connection when you ran the wizard.

Right click on the VPN adapter and select Properties.

Under the General tab, you should see the IP address of the Mac VPN server.

Under Security, select Advanced and then click Settings.

Select the Allow These Protocols radio button and then uncheck all of the boxes except for Microsoft CHAP Version 2.

Now select the Networking tab and set the Type of VPN menu to PPTP VPN.  Click OK and you are done configuring the client.  In order to connect the VPN, double click on the VPN adapter in My Network Places.  You will be prompted for your login information.  Once you click connect, your computer should negotiate the connection with the Mac sever.

Firewalling:
Most corporate VPN servers are behind a firewall.  In order for people outside of the firewall to gain access to the VPN server, certain Access Controls need to be added to the firewall.  In my example, the Mac VPN server is behind a Cisco 2600 series router with its firewall enabled.  This ACL shows the ports that were opened to allow both L2TP and PPTP access to the Mac server:

remark SOFTWARE VPN ACCESS RULES:
 permit udp any 66.62.25.0 0.0.0.255 eq isakmp
 permit udp any 66.62.25.0 0.0.0.255 eq non500-isakmp
 permit esp any 66.62.25.0 0.0.0.255
 permit gre any host 66.62.25.22
 permit tcp any host 66.62.25.22 eq 1723

In some cases, you may need to also enable 1701 for both TCP and UDP.  Some users have reported their configurations would not work until these rules were added.

The specifics of these rules are beyond the scope of this article.

In my testing of the Mac VPN server, I had some other issues.  I attempted to set up the VPN server on my home network so that I could access my files from remote locations.  Given the limitations of my Linksys broadband router, I was unable to make the configuration work.  I could not establish any rules on the Linksys to allow GRE or ESP traffic as consumer based routers only allow port mapping of TCP and UDP (layer 4 protocols).

Additionally, there may be an issue setting up a Mac VPN server on a corporate network if the address on the VPN server is a NAT’d virtual address.  I was able to get a Mac remote client connected through the NAT some time ago, but never had luck connecting with a PC.  If you setup a server in this configuration, I am interested in hearing about your experiences.  Please leave your comments in the field below.

Closing:
Apple has really made VPN easy with the latest release of its OS’s.  With only a little knowledge of the subject, it is easy to get a remote secure connection up and running.  In addition to the VPN service, Mac OS X Server includes a powerful Apache based web server, a DHCP server, Mail server, DNS server, Jabber chat server, Print server, and fileserver support for both Mac and Windows clients.  All in all, a wide range of services and support for a single operating system.  I strongly suggest trying it out for yourself!

Read more about Mac OS X Server.

Update: 12/30/05 10:30am
Here's an Apple Tech Note that lists 'TCP and UDP Ports Used By Apple Software Products.'  The list includes UDP port 1701 (L2TP) and UDP port 4500 (IKE NAT Traversal).  I did not need those in my firewall rules, but several readers have emailed explaining that their VPNs work until they activate 10.4's firewall service on the server.  Allowing these ports may resolve the issue.  Please post your feedback below.

This is done for a number of reasons, but mainly for security.  But it is important to consider this when you specify the DNS servers in the VPN settings.  If your VPN server is sitting on your corporate network, be sure to specify the DNS servers that other clients on the corporate network would use.  Similarly, if your VPN server is on your home network, specify the DNS servers you would use when you are surfing the web at home (Example: Comcast DNS servers if Comcast provides your internet connection at home).  Remember that when clients connect to the VPN server, they receive an IP address from the pool of addresses you specified when you set up the server.  Effectively, this makes a VPN user a client of that remote network, and their DNS requests will be made accordingly.

If you connect to the VPN server but find that you cannot connect to any other services once you are there, you can easily determine the problem.  If you are entering the name of the remote service but cannot connect, open up the Terminal and try to ping that address via its name.  Also try to ping the address via its IP.  If you can ping it via the IP and not via the name, odds are the DNS servers you specified are your problem.

When you do this, you should note the setting of the "Send All Traffic Over VPN Connection" checkbox located in the Internet Connect application under the Connect menu, then Options.  If the box is not checked, the pings you send must be located on the VPN servers network.  If the box is checked, you should be able to ping any address that would normally be ping able.

Network to Network VPN Connections:
This should not be a factor when making a client to server VPN connection as we do with the Mac's VPN server, but this point is worth making.  Should you work with router to router, or network to network VPN connections in the future, be mindful of the virtual IP addresses distributed on either side of the VPN.  By default, most routers use 192.168.1.x as the internal addresses.  In router to router VPN connections, it is essential to have unique subnets if virtual addresses are used on both sides of the VPN tunnel.  For example, if your home router connects to your corporate router and your business uses an internal NAT subnet of 192.168.1.x, your home network must use a different set of internal NAT address.  Try something like 192.168.2.x for your home network.

The point should not apply to client to server VPN connections, but I have heard of some users trying to connect to OS X's VPN Server from routers rather than clients.  I am not sure how well that works, but this rule will be something to keep in mind.

Update: 1/3/06 1:50pm
One of the cool new features in 10.4's VPN client is the ability to send all traffic over the VPN.  As Joe noted in the comments below, this is great for people using public access, like a wireless network at the upcoming MacWorld show.  This comment was right on the money, and I thought the idea warranted a little further detail.

Setting the VPN client to send all traffic over the VPN has several advantages, and two possible disadvantages.  First, the down side.

Disadvantages:
Consider the bandwidth available to your VPN server.  If you are on a corporate network, odds are you have a synchronous internet connection, meaning that the internet connections upstream bandwidth is equal to its down stream.  This is the case with the T1 at my office.  If your VPN server is using a consumer level broadband provider, odds are your connection is asynchronous.  This is often the case with DSL or cable modem connections.  The downstream might be a high as 8Mb, while the upstream is limited to 384Kb.  That is the case with my cable modem at home.

The problem occurs when you route all of your traffic through an asynchronous connection.  If the downstream is 8MB and the upstream is 384Kb and I am running my VPN Server from that network, the fastest my VPN client will be able to either send or receive data will be at 384Kb.  This is because all traffic is essentially being funneled through the asynchronous network connection before it arrives at the VPN client.  Even if your clients access point might offer higher speed access, this performance bottleneck will keep you from surfing at the speeds you might expect.  Also, keep in mind that several VPN users in this situation can use up the available bandwidth much quicker than you expect.

Advantages:
As for the advantages, there are many worth considering.  For example, say you are accessing a wireless hotspot from the MacWorld show floor.  If you understand how wireless networks function, you realize that everyone on that same wireless node has the ability to sniff your data, unless it's encrypted.  That means that your mail servers POP3 login information is sent in the clear for anyone to literally grab out of thin air.  So are the contents of your email messages for that matter.  The same goes for your FTP login, or any telnet access.

When you route all traffic through the VPN tunnel, you effectively protect all of that data.  Since the data is passing through the tunnel (both incoming and outgoing), it is unreadable to anyone between you and your VPN server.  Once the traffic reaches the VPN Server, it is no longer encrypted and it flows out onto the internet as needed to reach its intended destination.  By then, your data is clear of the danger zone.  The VPN connection makes you data as safe as it would be if you were sitting right beside the VPN server.

This concept is important to consider when you realize that once someone has access to your email login, they have full control over your email.  And if you plan on blogging from the show floor, this may be the only way to stay truly secure.  If consider any of your internet based traffic confidential, this really is the best way to go.

--
Steve

Thanks! nice tutorial, I could never figure out why I couldn't get the OS X Client to connect until now.

It works when my firewall in Server Admin is disabled, and I have it set to allow

(continued from abouve)...VPN connections, but whenever I enable the firewall, it can't establish a connection. What do I need to change?

I just posted an update to the story above. It links to an Apple Tech Note that details the ports used in Mac software.

Let me know how this works for you.

Very cool. Thank you for this timely tutorial. It will be nice to have secure access to files stored at my office while attending Macworld in just over a week. I was able to get PPTP to work, but attempting to connect via L2TP says the server is not responding. I set up port forwarding on my router to ping the server, I enabled both VPN protocols, and I do not have the firewall turned on. Not sure what I missed configuring L2TP. Anyway, I'm happy PPTP works, and I was even able to test it using my mobile phone modem connected to my PowerBook via Bluetooth. Thanks again.

Interesting to see how clunky the windows client configuration is, compared to the Mac. Not surprising, just interesting.

Thanks for the timely tutorial, we've been trying to get this running on and off for the past few months. Oddly this tutorial doesn't work for me. The mac to mac side is all fine, but XP just tries to verify the username and password and then fails.

The log file on the server looks roughly like this:

Sat Dec 31 12:18:12 2005 : PPTP incoming call in progress from '151.203.158.130'... Sat Dec 31 12:18:12 2005 : PPTP connection established. Sat Dec 31 12:18:12 2005 : using link 0 Sat Dec 31 12:18:12 2005 : Using interface ppp0 Sat Dec 31 12:18:12 2005 : Connect: ppp0 <--> socket[34:17] Sat Dec 31 12:18:12 2005 : sent [LCP ConfReq id=0x1 ] Sat Dec 31 12:18:15 2005 : sent [LCP ConfReq id=0x1 ] Sat Dec 31 12:18:18 2005 : sent [LCP ConfReq id=0x1 ] Sat Dec 31 12:18:21 2005 : sent [LCP ConfReq id=0x1 ] Sat Dec 31 12:18:24 2005 : sent [LCP ConfReq id=0x1 ] Sat Dec 31 12:18:27 2005 : sent [LCP ConfReq id=0x1 ] Sat Dec 31 12:18:30 2005 : sent [LCP ConfReq id=0x1 ] Sat Dec 31 12:18:33 2005 : sent [LCP ConfReq id=0x1 ] Sat Dec 31 12:18:36 2005 : sent [LCP ConfReq id=0x1 ] Sat Dec 31 12:18:39 2005 : sent [LCP ConfReq id=0x1 ] Sat Dec 31 12:18:42 2005 : LCP: timeout sending Config-Requests Sat Dec 31 12:18:42 2005 : Connection terminated. Sat Dec 31 12:18:42 2005 : PPTP disconnecting... Sat Dec 31 12:18:42 2005 : PPTP disconnected 2005-12-31 12:18:42 EST --> Client with address = 10.0.100.125 has hungup

if anyone has any clue what's going on, we'd really like to know as this has stumped us for months. Thanks again.

thanks smanke, enabling ESP and GRE in the firewall was the trick.

Elliot, From what I gather from the debug info you posted, it looks like your VPN server is behind a NAT of some sort (is 10.0.100.125 the client or the server?).

I was not able to get PPTP working when my VPN server was behind a NAT (only L2TP worked through the NAT). I am not sure why this fails, but from what I have read, it has something to do with the changes made to packets when they pass through the NAT. It may be a limitation of PPTP.

smanke, thanks, the machine has an external address static address, but that is NATted as you guessed. I'll have to look into that more closely.

Hi,

I got my VPN connection to work, I have a OS X 10.4 server and a 10.3 client. But I can't access the network where the 10.4 server is located, I can only access that server. I can't surf the internet either with my client, even though I set the DNS servers and routed the traffic in "Client information".

I routed it like this: 192.168.1.0 255.255.255.0 Private 213.xxx.xx.0 255.255.255.0 Private

Really strange, I'm using PPTP btw.

Hi, thanks for your time. I've been using L2TP succesfully for a year with my server (10.3) behind a USR 8000A-02 broadband router with 500 1701 & 4500 open. A few weeks ago the router was broken. Since then I was using my Airport Base Station as router but I was not able to get L2TP working, 500 1701 or 4500 are forwarded to the server, but if I scan from wan side seems to be all closed. Then I've try with PPTP ( forwarding 1723) and the vpn connection betwen server & client was established but I can't afp or ftp the Server.

Urme,

I'm not sure the problem is in your routing rule. I just posted an update to the end of the story above (under todays date). I think you will want to check your DNS server settings. I think the problem might be recursion as i detail above.

Still, once you connect to the VPN server, you should be able to ping other machines on that same network. If you can't, you might want to remove the rule for the 213.xxx.xxx.0 network and see what happens. Depending on your router config, that could be a problem.

Andy,

I'm not sure if this will help, but Apple has added a new feature to the Airport if you are running the latest firmware and running 10.4 on your machines.

Open the Airport Admin Utility and Base Station Options. There is a checkbox to "Enable NAT Port Mapping Protocol." I'm not very familiar with this new feature, but it seems to be Apple's version of UPnP.

It would be interesting to see if this has any effect on your problem. It seems that the cause is unique to the Apple hardware.

If you restrict access based on MAC address, you might want to remove those rules to help troubleshoot the problem.

Please let me know if you resolve the problem. I can see this sort of thing being an issue for others as well.

Smanke, thanks for your answer. I have tried to "Enable NAT Port Mapping Protocol" a few days ago but nothing changed. Yes, my Airport Base Station restrict acces based on MAC Adress but only for Wi-Fi. I think that I will buy a D-Link router....

I have updated the story to further explain the advantages and disadvantages of routing all internet traffic over the VPN connection.

If you travel a lot, or you plan on attending the upcoming MacWorld Expo, please checkout the notes I just added to the end of the article.

I was having the same problem as Elliot outlines (VPN through NAT to an internal VPN server) and when I port forwarded UDP 500 (ISAKMP/IKE for Mac OS X Server VPN service) to my VPN server L2TP works just fine! PPTP still doesn't work but this is a big step forward. Hope that helps, Elliot. And thanks for this really well-written article, Steve!

Finally Ibought a Linksys BEFX41. Now L2TP its working again. I'm quiet sure the Airport Base Station its not compatible with vpn something is wrong with port forwarding

Hello. I'm running a Netgear FVS318 prosafe firewall router with OSX 10.3.9 server VPN. I've tried opening all the ports for the VPN server, and I still can't get the thing to connect. If I try from within my network I'm golden, but once I go on the other side of my firewall it doesn't work. Could it just be a crappy firewall, and should I look into getting a better one? Or am I missing the wrong ports? Here's how I have it so far: 1023: TCP 1701: TCP/UDP 1723: TCP/UDP 500: TCP/UDP 4500: TCP/UDP 10000: UDP

Thanks for all the help!

This is truly a great tutorial. Thank you so much for posting this.

I've go almost everything working just fine on my set-up except for windows XP clients. They are able see see the LAN, but unable to connect to the internet. I believe that I have everything set up correctly. The one thing that I'm a little soft on is the Network Routing Definition and I'm hoping this is what's causing my problem.

I'm using all Private IPs maps thru my router, so my Router IP is 192.168.0.254 with a subnet mask of 255.255.255.0.` All of the network client have IP address from 192.168.0.1 to 192.168.0.100. For the L2TP client, I've set up 192.168.0.230 to 240 and the PPTP clients, 192.168.0.241 to 250. So in the Network Routing Definition, what IP address would I want to place in there?? And, is this what's causing my problem with XP clients?? If not, what should I look at next???

Thanks

Jamie

Tadd,

It looks like you have everything set. Are you trying L2TP or PPTP? As long as you are using L2TP from a Mac, you should be able to connect. PPTP does not seem to work because NAT routers normally don’t allow us to map protocols other than TCP and UDP. PPTP needs GTE and ESP.

Jamie,

You should only have to setup 192.168.0.0 with a 255.255.255.0 subnet and mark it as private. After that, your secure traffic should go through the VPN and the rest should go over your normal internet connection.

I did have someone tell me that they made one other change to the PPTP setup. Go into the Properties on the VPN network connection and click on the Networking tab. Then double click on TCP/IP. Next click Advanced and uncheck the box for Use Default Gateway On Remote Network. From what I understand, this prevents all of the traffic from flowing through the VPN connection.

Let me know if that works for you.

Andy,

Thanks for the update. That's good to know. Hopefully Apple will correct the problem in a future update of the firmware.

Hi Steve, thanks for the quick reply. I did test it using L2TP off of a Mac running 10.4.2 and I still got a timeout error. I do have a couple of windows machines, but that's not critical since I can still SSH into the server with any of the windows boxes if necessary. This netgear firewall has a VPN "wizard" which may be a problem too. Thanks again.

HEy Steve,

Thanks for the reply. I've tried as you suggested. I've got the Network Routing definition set up with 192.168.0.0, mask as 255.255.255.0 and Private. And no go. Still can only seen the LAN, but not the Internet.

I then tried turning off the "Use Default Gateway in Remote Network" setting on the XP. With this done, I can then see the internet, but then I can't see the LAN.

So at this point, using Windows XP I can have the LAN or the WAN, but not both at the same time. On the Mac, I have both, at the same time, no problems.

Any other suggestions?? Or just tell this Windows user to switch, which would be the right thing to do for more reasons then just this.

Thanks again for your reply and any additional help you can give.

Jamie

Jamie,

When in doubt, dump Windows. It's always the safest means to an end. :-)

It's odd. I think that somewhere in all of this, the PC just isn't finding the gateway when the VPN is connected. I think there is a way to force this by setting a route at the command line, but I'm not sure of the command.

I have a friend that can do magic with this stuff. I will ask him to take a look at your notes and see if he has any ideas.

Oh dear God, I wish I could get rid of every windows machine in this place. Life would be much easier.

THanks for asking your friend to look at thins for me. Any advice they could give would be much appreciated.

And thanks again for your help.

Jamie

Jamie,

Couple of questions:

1) Do you have DNS servers set up in your "Client Information" section? 2) What method are you using to test access to either the LAN or the Internet?

By default Window XP will have that "Use default gateway on remote network" checked. What this does is make the default route of the Windows machine the VPN tunnel, sending all traffic to the Mac VPN server. So in this configuration you should be able to access the LAN and the Internet over the VPN tunnel. When you uncheck that box on the Windows VPN client setup that allows for split tunneling (only traffic destined for the remote LAN will go over the tunnel), which should also allow your XP box to access the LAN (via the tunnel) and the Internet via the default gateway of the XP machine.

It might be helpful to take a look at the routing table on the XP machine before and after connecting to the VPN server to see what changes. To do this just open a command prompt and type "route print", this should display all routing information. In a default configuration you should see a couple things happen, once connected to the VPN server you should see a route entry for the IP address assigned to XP from the VPN server pointing to the IP address of the server. You should also see an entry for the remote LAN pointing to the assigned IP address in XP. Then you should also see a new entry for the default route (0.0.0.0) also pointing to the assigned IP. You should notice that the new default route has a metric of 1, which should be lower thant the default gateway already in XP (usually 20).

If that all works fine you might want to take a look at the routing info on the VPN server to make sure that's all ok as well.

Hope this helps.

Matt

Hi

I am trying to set up vpn from a remote desktop running OSX10.4 to our office network on 10.4 server. I have set most of it up as described above, but I do not understand what I should enter in the client info part of the server vpn settings.

I have an outside fixed IP address (don't know if that is relevant), broadband line coming through a router, going straight to the server with IP address of 192.168.1.100

The internal network, fed off the server with IP addresses 192.168.2.2 - 192.168.2.127. The allocated addresses for the vpn are 192.168.2.128 - 192.168.2.254 The internal address of the server is 192.168.2.1

Can you please tell me what I put in the three client info fields - 1. DNS servers. 2. Search domains. 3. Network Routing Definition.

I would be most grateful for any help you can give.

Thanks

Richard

Richard,

1. DNS Servers: Put in the DNS server IP addresses that you use on the workstations that are already located on the LAN.

2. Search Domains: I leave this empty.

3. Set that to 192.168.1.0 255.255.255.xxx Private. Where .xxx, replace with the correct subnet for your LAN. I don't recall what it would be if you are only using 192.168.1.1-.127. You should be able to check one of your other workstations to find that, or check your DHCP server for the info.

Then, in the clients VPN clients you want to point them to the outside routable IP that NAT converts the internal 192.168.1.100 address to.

I hope that helps.

Richard,

Correction. I see what your subnet is. You want to use a subnet of 255.255.255.0 where I indicated 255.255.255.xxx. I read through your info too quickly and missed the note that the rest were address for your VPN pool.

Thanks for your help. Unfortunately we still cannot get the connection to work.

Why on earth does this have to be so complicated?

Regards

Another thought - would the router at either end be the problem? Does the firewall need to be disabled on the router, or any other settings changed? I read somewhere about port forwarding. What is that? Thanks for taking the time to read this and any help is appreciated.

Richard,

Assuming you are using a NAT router, you must set up the proper port forwarding in order to get this working.

In the example posted in the story, I explained that these rules were added to my firewall in order to allow access: permit udp any 66.62.25.0 0.0.0.255 eq isakmp permit udp any 66.62.25.0 0.0.0.255 eq non500-isakmp permit esp any 66.62.25.0 0.0.0.255 permit gre any host 66.62.25.22 permit tcp any host 66.62.25.22 eq 1723

If you are using a NAT router, the only ports you can map are for TCP and UDP. You won't be able to do the ESP or GRE, so you won't be able to use PPTP. But if you map the TCP and UDP ports for the OS X VPN server, you will be able to use L2TP Over IPSec.

Note that the example lists permit udp any 66.62.25.0 0.0.0.255 eq isakmp. ISAKMP is really UDP port 500.

I am betting this is what has been keeping your from getting the VPN server functions. Without the port maps, all of the traffic will just bounce off the firewall.

I hsve sll this stuff setup and working, at two sites, now I need to connect both sites using Site-to-site VPN in Tiger Server 10.4.4. When I run the s2svpnadmin and have te firewall rules setup, like the apple document says to do, nothing connects, and services fail on eithr private network.

In Hopes of not having you repeat yourself in some steps, would you be so kind as to run through a successful Site-to-Site Setup using s2svpnadmin like discribed in this document. http://images.apple.com/server/pdfs/Network_Servic...

the steps are on pages 112-116 I believe.

Thanks for any help Lawrence

Lawrence (Twintails),

That was a very interesting question. I had never looked into a site to site VPN using OSX Server. I took a look at the PDF in your link, and the documentation looks good. I am tempted to try, but there are other alternatives that might be easier.

If both sites are using small office broadband gear, I suggest linking the VPN using routers that can act as VPN endpoints. I have had a lot of luck setting things up that way. Plus, when you use the routers for the VPN, you don't need to worry about the port mapping as you do with OSX Server.

As for the software VPN solution, I am guessing there is a problem with the portmap. Even if you are using clients to connect to the VPN server, it is possible that a site to site link will not be possible using two broadband or small office routers. I suspect that the site to site link is less tolerant of the fact the home routers will not let you map protocols other than TCP and UDP. The docs in that PDF instruct you to open up firewall ports for ESP. That can't be done on any of the home routers I have seen. As I say, this is only a guess. I have seen similar issues in the past.

I am planning to do a review of the Linksys RV082 in the near future. It's more expensive (just under $300), but it does all kinds of cool VPN stuff including site to site as well as having its own built in PPTP server.

Something like that might be better for your needs.

I know this advice won't solve your problem, but i hope it helps.

I found a nifty program that helps configure port forwading on OS X Server. Its Called The NATural. It makes configuring port forwading a breeze...well for me anway. I was able to get VNC & VPN through the NAT.

WEBSITE http://www.jamiegriffin.com/gdog/thenatural/index....

Using NAT with VPN is complicated when OSX is doing DHCP, DNS, FIREWALL, NAT & VPN services. The NAT service is lacking functions. Hence you need a 3rd party product...or just edit /etc/nat/natd.plist

Basically I had to convert the Cisco NAT & Firewall to OS X NAT & Firewall. I did it...but I it was no cakewalk.

Hello,

Brilliant tutorial, thanks very much for this.

I had the same problem as Urme - I could connect to my Server, but not to other servers on the LAN. I got the solution from the Apple discussions forum (thanks Leif): You need to turn on IP forwarding on the server. At least, it solved the problem for me.

Eric

Lucky you Eric I have been trying to get this working for three months, with 10.4 server and 10.4 client and getting absolutely nowhere. I have followed all instructions to the letter. Have done the port forwarding from the nat router, but that changed nothing!!!!

Any further suggestions please

Richard

I have posted an update to the end of the story to include the NAT fix that Urme and Eric have described.

Thank you all for your feedback.

hi there, i have this situation. i am (mac tiger)behind a nat router and want a vpn connection to a os x server (10.3.?) which also is behind a nat router. i followed your instructions but tried it over pptp because easier to set up. i made a port redirect on the router (server side) to 192.168.1.99 port:1723 tcp. i have a dynamic ip at no-ip.com which works correctly on the router. client information>dns servers: 192.168.1.99 (server ip) routing definition: 192.168.1.0 255.255.255.0 i cannot connect to the vpn server. do you have an idea or tip for me. thank you very much

uwe,

From what I can tell, PPTP will not work through home NAT routers because there is no way to port map the ESP and GRE protocols. If you switch to L2TP, you should be all set. Its really no more difficult if you are using the Mac client.

ok, good to know. but i did read that l2tp will not work with a connection between 2 routers. is this right, or does it only not work with l2tp and ipsec.

uwe,

I don't think that's the case. I would give it a shot. Good luck!

Hi Smanke, cool topic! Unfortunately I still can't get this thing working =( I've read through and checked all that's on offer as solutions with other people but still no luck......

New to OSX Server (Tiger), not new to OSX but have previous Windows Server 2003 experience and just finding my way in OSX Server with tutorials on the web. Wanted to get remote access working first so that I can jump in from anywhere to toy around with the G4.

I have configured DNS for the server so that when I ping by name, the correct IP comes back so I guess that I have this bit configured properly:

DNS SETTINGS Zone name: foo.co.uk Server Name : spongebob

Server IP address 192.168.1.2 (static - In network prefs I have manual setting: IP Address: 192.168.1.2, Subnet Mask: 255.255.255.0, Router: 192.168.1.1, DSN Servers ???.???.???.??? (ISP DNS), Search Domains: foo.co.uk (Tiger server's fully qualified DNS Name))

Name Servers: spongebob.foo.co.uk

NAT SETTINGS: NAT switched on

DHCP SETTINGS: DHCP switched off (since VPN has own subnet range?)

VPN SETTINGS: VPN switched on

L2TP: Enabled, Starting IP Address: 192.168.1.20, Ending IP Address: 192.168.1.30, PPP Authentication: MS-CHAPv2, IPSec Authentication: Shared secred (yeah I did put one in!), Certificate: No Certificate.

PPTP: Enabled, no 40-bit encryption keys, Starting IP address: 192.168.1.40, Ending IP address: 192.168.1.49

Client Information Settings: DNS Servers: 192.168.1.2, Search Domains: foo.co.uk Network Routing Definition: Network Address: 192.168.1.0, Network Mask: 255.255.255.0, Network Type: Private (to not channel everything through the VPN)

I've got a D-Link Router where I have forwarded PPTP (1723) and L2TP (1701) to 192.168.1.2 on the respective ports.

I then created a test user (VPNUser) and even added them to the dialup group (this would be dialin access with Windows - not too sure about this bit) and created a VPN from the server (I don't know whether it's possible to establish a VPN from inside the network but this does work on Windows sometimes......btw I also tried from a friend's OSX Mac from outside and still did not work!). I also tried PPTP config too.....still does not work.

Now..2 questions arise!

Am I going nuts because all the settings are correct and the router is shagged?

or......have I missed out on something vital?????

Oh yeah....did the update to 10.4.3 which killed my internet connection but recreating the manual settings seemed to fix it?!??!! Got lucky with that one....

You help would be mostly appreciated......I'm the only one in a Windows support team championing the Mac flag!

~Groovy~

Hi sorry, forgot to add, it's a D-Link DG604-T Wireless ADSL Router and here's the ports that are open:

1023: TCP 1701: TCP/UDP 1723: TCP/UDP 500: TCP/UDP 4500: TCP/UDP 10000: UDP

I think it could be the router that's to blame. Tried port scanning and only Telnet and Web ports say they're open....

This is really wierd though, because Windows servers don't have a problem with PPTP behind NAT as far as I know.

Alixir,

I can't see anything wrong in your config. After your first post, I was going to suggest that you open up UDP port 500, but thanks to your second post I see you have done that.

I know that another user was using an Apple Airport as a router and for whatever reason, he was unable to get his VPN configuration working. When he replaced the Airport with another router the problem was resolved. It's possible that your D-Link is the problem. I can't really say. And I would hate to suggest that you replace hardware when I can't be sure that will fix it.

One other idea comes to mind. Your router should allow you to specify one internal IP address as a DMZ. If you make your VPN server the DMZ address, you should be able to bypass all of the port mapping as the DMZ should fully expose all of that servers ports. It would be interesting to see if that helps.

Good luck! And let us know how it works out for you. It's difficult to tell if the router is the culprit some times.

My L2TP VPN wouldn't work until I opened port 1701 (L2TP) on both TCP and UDP, on my Linksys WRT54GS router's firewall. The Apple tech note ("Well Known" TCP and UDP Ports Used By Apple Software Products) mentioned in the article has port 1701 only under UDP.

Just wondering if the starting and ending IP addresses for L2TP are meant to be internal IP addresses or are they external? Do I have to own those IPs? The machines on the LAN all have manually assigned addresses in the 192.168.0.xxx range. Can I use 192.168.1.xxx for the starting and ending IPs?

Cheers, Ben

We have OS X server. We want to use VPN. What would your suggestion be. Buy a Linksys VPN router for BOTH ends of our connection? Since we haven't bought anything yet, this is the PERFECT time to ask. Thanks in advance for your help. Graham

Graham,

As much as I love the OS X VPN, I would suggest the Linksys VPN router. Its a hardware VPN solution, so it should be more stable. That is what I use most of the time.

I like to have th OS X VPN as a backup. I keep it ready for action at all times. It works well for me when I am on the road and need secure access to my network.

If you go with the Linksys RV082, you will have the best of both worlds. It will do a network to network VPN, and has a PPTP server built in so you can access your network via a software tunnel from your Mac or XP laptop.

I need to get my review of the RV082 finished, but I have been very impressed with it. It is a more expensive VPN router, but it really is worth the money. If you want a cheaper VPN router, Linksys offers those as well.

Good luck!

Ben,

You want to assign internal addresses, so you are on the right track. Just use a range from your 192.168.0.xxx pool.

Keep in mind that you will need to make sure the subnet on the other end of the VPN tunnel does not use 192.168.0.xxx IP addresses as well. Anything other than those will work, but the same subnet cannot be on both sides of the tunnel.

francois,

Thanks for the tip. I have added the note to the main article.

Thanks SO much for the advice smanke, would the Linksys RV042 work for our purposes. Noticed there is NO internal PPTP server. Would we need this? Sorry if the answer is obvious, and thanks again for your advice! Graham

Graham,

I haven't had the chance to play with that model. From what I have read, I believe its the same router as the RV082 except that it has less ethernet ports and a slightly slower processor. It may actually have a PPTP built-in. That part is not clear.

Since your main goal is to connect 2 office between routers, the RV042 should be perfect. If you do get it, let me know what you think of it (and whether or not it has a PPTP server built in).

Will do. I'm actually looking into using an SSH tunnel to securely connect to a machine. I'm sure it's not AS secure as VPN but it's easier (only slightly mind you) than setting up a VPN. What is your opinion on this?

And I WILL indeed let you know if we DO end up getting an RV042.

Thanks Gp

Graham,

Thanks!

I have only played with SSL tunneling a little bit. There is a really cool new point to point VPN tool coming out soon from this site: http://hamachi.cc/

The XP client is really kick ass. The Mac client is still pre release, but looks encouraging. They have yet to finish the GUI for it. Once we have the GUI, it will be a really nice alternative. I haven’t heard of a solution that works like Hamachi. Its beyond my ability to explain here, but I encourage everyone to check it out!

First off, thanks a ton for this article. Can't begin to tell you how helpful it was in setting this up.

A comment and a question:

Question -- when setting up the VPN client on XP I never actually have to put in the shared secret at any point. Security-wise, how much should that scare the s**t out of me?

Comment: I had a problem initially seeing the whole LAN when connecting through a wireless router from home (straight through the cable modem was no problem). No amount of IP forwarding helped (server, router, etc.), but simply changing the home router to a 192.168.2.x ip scheme knocked it out no problem (both scheme's were 192.168.1.x -- when VPN'd in could only see the VPN server, nothing else). Hope that helps anyone experiencing that.

Thanks again.

jrose,

I'm glad this has helped!

The shared secret is only necessary when you are connecting using L2TP. Windows is connecting using PPTP so it doesn't need a shared secret.

Secondly, I think I see why you couldn't see the other machines on the network when you initially connected to the VPN. You can't have the same virtual set of addresses on both sides of the VPN tunnel. If the IPs on the VPN network are 192.168.1.x, then the IPs on the remote client must be anything other than 192.168.1.x (in your case 192.168.2.x). That should explain why changing the subnet solved the routing issue.

You are right, that could solve some problems for anyone seeing a similar issue.

Great article. I was able to set up VPN access to a server behind a NAT firewall, using L2TP. I did have to open ports 500, 1701, and 4500 (all UDP). I haven't been able to access anything else on the remote network, though (can't even ping). I have NAT turned on on the server, and have tried sending all traffic through the VPN, but that doesn't make any difference. Both networks have different IP address ranges.

Brian,

In one case, I needed to open up another port with this rule in my Cisco's ACL: permit udp any 66.62.25.0 0.0.0.255 eq non500-isakmp.

The problem is that will only work on a Cisco. If you are using a broadband router, I'm not sure what port number would take the place of "non500-isakmp."

I only have one other idea. Have you checked to make sure your firewall is disabled on the Mac server?

That's 500/UDP, which I have open. Something must be missing somewhere. time to bug an Apple rep, I guess...

http://www.networksorcery.com/enp/protocol/isakmp....

(Sorry, I left this out: the firewall is disabled.)

i need to setup a vpn to connect 5 databses/servers, where several clients connect to. they use mac's, what kind of hardware would you advise me to use for the vpn router as the software is already in the mac server an mac clients.

Hi - sorry to bother all but am new to Mac let alone OSX 10.4 Server.

I have gone through all this and got L2TP running fine, but really need PPTP.

I ave set everything up, however in the overview section on VPN it says PPTP is "Enabled but not running".

I have tried everything but to no avail - is there some other step to kick PPTP into gear?

mijkel71,

If you are trying to connect one server to another from different sides of the VPN, I would suggest a router to router VPN. And , if you go that far, you could just use the Mac's VPN client to access the VPN via the router, if you get the right VPN router.

I suggest the VPN router over the Mac VPN server because, while I have read that the Mac VPN server can do network to network VPN connections, it is difficult. I have never attempted it.

I think you can do what you need with a Linksys RV082 on either end. You can also do it with a full blown Cisco router, but that is well beyond my scope.

From the sound of what you are trying to do, I would suggest you contact a consultant for help. It will be the simplest solution and it will get you up and running quickly. I can provide you the contact info for someone that I recommend, if you like. He can do amazing things with these routers and he has never found a VPN issue he could not solve.

anikan,

Is it possible that you checked the box to activate PPTP but didn't specify a pool of IP addresses to use for logged in clients? That is the only reason I can think of for that message.

You should also check the log for more information. Odds are that it will give you some sort of explanation when the server is first activated. Keep in mind that you might need to refresh the log view unless you are using the Console application.

It might be a good idea to take a look at the system log file using the Console app. If you have never used it before, it allows you to read and search all of the log files on the machine. Just choose System.log from the logs list, or even the VPN log listed under /var/log/ppp/vpnd.log.

That should provide further information.

hi Smanke thx for your info, last question do these linksys rv082 work fine with macintosh, as far as you know

mijkel71,

The RV082's built-in PPTP server works great with the Mac's built-in VPN server. The only down side is that it is limited to 5 accounts. I am guessing they did that so they could up-sell another model to anyone needing more accounts.

Those 5 accounts are separate from the accounts used to create router to router tunnels.

I ran across a cool new trick in the Mac VPN client the other day. I did a post about it yesterday.

If anyone has an interest in connecting to multiple VPN servers at the same time, check out this post: http://maclive.net/sid/186

smanke,

these 5 accoutns are 5 accoutn we can use on 1 vpn tunnel as i saw this model can support up to 50 vpns, , so this means 50 vpn's with 5 accoutns each or max 5 account on the complete router

mijkel71,

I believe that 45 of the accounts can be router to router. If you set up 5 computer to router accounts (in this case Mac client to router) that would take up 5 of the accounts. If all 5 are logged in at the same time, you will have 5 separate tunnels. If no one is logged in, no tunnels are active as the tunnels are initiated by the client computer in this situation.

So, based on your question, it is 5 accounts for the entire router. But keep in mind this is only for the routers built in PPTP server. You still have a lot of other VPN options in this router. They include router to router VPNs as well as Linksys QuickVPN clients from Windows users. None of those count against you 5 users max on the PPTP server. There are just a lot of VPN options in this router!

Hope that helps.

Smanke, fantastic article. Thank you. I am using the VPN server in OSX Tiger Server - but I have the same problem as may folks here. I can see the server - but cannot see any of the other machines on the network. Strangely enough I see other machines on the network in ARD - so my guess is that there is something wrong with my Firewall setup. On February 6, 2006 at 10:52 AM you replied to a post from Richard about firewall rules. Would it be possible for you to put this explanation in a notation or help file that relates to how one might set up these sorts of rules using the Tiger Server firewall setup? I didn't really understand the shorthand you were using. eg. "eq". I assume that this means equal? but I may be wrong and I can't realate your instructions to how I would set rules in the software firewall. Would one need to set up a new address group, and then apply settings to just those? I feel I am tantalisingly close to solving my issue... thanks in anticipation.

Great tutorial! Now I have my Macbook AND my XP Tablet connecting the way it should! Now if only you could do a tutorial that would fix my mail server issues, I would be all set! =)

The instructions for Windows client are longer than for setting up the SERVER. hahahahahaaaaaa....

Mark,

I just added an update to the story explaining the firewall rules in more clear English.

The update is interleaved in the post, just below the firewall rules you mentioned. If you are having trouble with the VPN server, i recommend disabling your firewall temporarily in order to narrow down the cause of the issue. If you disable the firewall (assuming its the Apple firewall), and the problem is gone, then you know where you issue lies for sure. I the problem persists with the firewall disabled, then there must be some other issue.

I hope that helps!

I've used your instructions above on my OS X Serve behind a netrgear RP614v1 Router and it does'nt work for Mac or PC clients.

I setup exactly the same on my friends OS X Server behind a Netgear WGT834 and all works fine.

QED the type of Router is very, very important. Eeven if I fporward ports 1701, 1723, 50, 500 and 4500 to my server it still does not wotk on the Netgar RP614v4 even though it has VPN pass-through.

On this basis, can you recommend a usable Netgear or D-Link router please, and what specifically we should look for in the spec's of a Router to make this as seemless as possible.

Even then, I assume we should 'always' port forward 1701, 1723, 500, 4500?

redleader,

I don't have much experience with Netgear routers. Most of the broadband routers i work with are from Linksys.

I have heard other reports of some models having spotty support for VPN. The only question I have is whether or not you are using the latest firmware for your router. It's been my experience that most up to date routers work alright.

I don't have a specific list of routers that work. Maybe some of the other readers can post the models they have gotten working.

If you have port mapped 1701, 1723, 500, 4500, you should be in good shape. And, since you said you have your friends machine working correctly, it sounds like you have your config right. Its the router that is giving you the issue. As long as your router does NAT, can portmap, and supports VPN pass through, you should be good to go.

Sorry I can't be of more help.

Thanks smacke for all the advice you are giving everyone.

I have a problem connecting my 10.4 laptop at home to my os x server 10.4 at work. I am connected via airport at home which I read could be part of the problem. With L2TP I get nothing (cant connect) and i checked all the logs on the server and it is not even receiving the request.

With PPTP the connection is made but gets terminated...
Here is the server log:

2006-06-17 12:48:50 PDT Incoming call... Address given to client = 192.168.2.128
Sat Jun 17 12:48:50 2006 : Directory Services Authentication plugin initialized
Sat Jun 17 12:48:50 2006 : Directory Services Authorization plugin initialized
Sat Jun 17 12:48:50 2006 : PPTP incoming call in progress from '67.49.116.245'...
Sat Jun 17 12:48:50 2006 : PPTP connection established.
Sat Jun 17 12:48:50 2006 : using link 0
Sat Jun 17 12:48:50 2006 : Using interface ppp0
Sat Jun 17 12:48:50 2006 : Connect: ppp0 <--> socket[34:17]
Sat Jun 17 12:48:50 2006 : sent [LCP ConfReq id=0x1 ]
Sat Jun 17 12:48:53 2006 : sent [LCP ConfReq id=0x1 ]
Sat Jun 17 12:48:56 2006 : sent [LCP ConfReq id=0x1 ]
Sat Jun 17 12:48:59 2006 : sent [LCP ConfReq id=0x1 ]
Sat Jun 17 12:49:02 2006 : sent [LCP ConfReq id=0x1 ]
Sat Jun 17 12:49:05 2006 : sent [LCP ConfReq id=0x1 ]
Sat Jun 17 12:49:08 2006 : sent [LCP ConfReq id=0x1 ]
Sat Jun 17 12:49:11 2006 : sent [LCP ConfReq id=0x1 ]
Sat Jun 17 12:49:14 2006 : sent [LCP ConfReq id=0x1 ]
Sat Jun 17 12:49:17 2006 : sent [LCP ConfReq id=0x1 ]
Sat Jun 17 12:49:20 2006 : LCP: timeout sending Config-Requests
Sat Jun 17 12:49:20 2006 : Connection terminated.
Sat Jun 17 12:49:20 2006 : PPTP disconnecting...
Sat Jun 17 12:49:20 2006 : PPTP disconnected
2006-06-17 12:49:20 PDT --> Client with address = 192.168.2.128 has hungup

---- END LOG ----

So it looks like the client (me) does a bunch of ConfReq (whatever that is) that get ingored then the client "hans up"...

I have tried all the advice so far...

Thanks for any help!
Nick

nick,

I'm getting the impression that there might be a firewall issue. It looks like the negotiation is never completing, so the connection times out and shuts down. I would check for any router level firewall rules that might be getting in the way, and check the firewall on OS X server.

It looks to me that your home subnet is 192.168.2.x. You should also be sure that the subnet the server sits on it not the same. That has been the cause of problems for most of the people i have heard from.

Sorry I can't be of more help. I am betting there is a firewall issue here, or something in the NAT is giving you trouble.

I think you are right about the firewall issue...

I enabled GRE and now I get the following in the firewall log:

Jun 17 18:58:44 lymabean ipfw: 65534 Deny TCP 69.45.170.37:4118 69.45.147.226:445 in via en0
Jun 17 18:58:47 lymabean ipfw: 65534 Deny TCP 69.45.170.37:4118 69.45.147.226:445 in via en0

It almost works!!!
If I could figure out how to tweak the firwall rule that is blocking the final connection it think it would work!

nick,

Cool! You're almost there. Did you enable ESP also?

Does enabling port 445 get you any further? I don't know that its necessary, but looking at your log, it seems that's where it is now being blocked.

Hello everyone,

I have a problem. We have a RV042 and have been able to set uo gategay to gateway, Quick VPN and the Built in PPtP (but only for windows not mac). I was olso able both VPN sevices on the Xserve running version 10.4.6.

The problem we have is that we experiance internet connection slowdon and failure when the L2TP over Ipsec is active. I have these ports forwad to the Xserve TCP 1723, UDC 1701, UDC 500 abd UDC 4500 on the RVO42. I aslo have the NAT service enable in the Xserve

Our Service provider gave us a Netopia moem/router but we disable the router functionand use it as a stright router. They said that they detected a problem with the IP configuration on the Modem or Router and suggesting using a diferent modem.

Does anyone have any ideas ?

FILI

Hi,

On the previous post I ment to say that we disable the Netopia's router funtion and use it just as a modem.

I also just found out that we do not need to have the NAT service running on the server in order to establish a tunnel. Ther is a funtion on the route of One to one NAT but it is disabled.

FILI

FILI,

You seems to have a unique reconfiguration in that you are doing PPTP at the router and L2TP on the Mac VPN server.

Are you saying that you have a slowdown when you have L2TP is active on the Mac? I am wondering if that is because the router is trying take deal with the PPTP traffic itself and has also been port mapped to send PPTP to the Mac server. If you have port mapped 1723 to the Mac, this could be a problem.

If the Mac is not doing PPTP and the router is, you should not point GRE (protocol 47) or PPTP 1723 to the Mac. Let the router deal with them.

BTW, I really hate those Netopia routers. Some providers foist those on customers and they really aren't capable of the configurations people often need.

I have a new Netgear WGT624v3 router. By 'NEW" I mean a recent piece of hardware and firmware, as opposed to my old Netgear RP614v1.

I'm a happy chap now, all is working fine.
Thank you for this page, it's brilliant!

Steve,

The PPtP on the router is working only for windows so I disable it and are using the PPtP on the Xserve for both windows and macs. I do have the quick VPN active on the router and router to router tunnels active. We want to give the remote users as many options as possible. This configuration works fine until I activete the the port fowarding applications to make the L2TP on the Xserve work. The reason that we want to use L2TP is for security purposes. But it seams that we would have to stick to PPtP only and get creative with the username and paswords.

Do you think if I disable the quick VPN on the router it would make a diference ? Does anyone know if it uses the L2TP protocal ? I'll like to give it a shot but I hate to slow down the office workflow.

Thanks,

FILI

Fili,

I had a hard time finding docs detailing how the QuickVPN client connects, but i found 2 pages that mentioned that it does use IPSec.

Based on that, I am betting that your QuickVPN is interfering with your L2TP traffic, and that could be causing your issues.

I think the only way to be sure is to give it a shot. This does sound like the most logical cause.

Let us know what you find!

Can anyone explain why the Network Globe does not function when connected to a remote network via a VPN connection? I can connect to my office from home but can only see the server and its share points. Am I missing something?!

Mark,

Network globe? Are you referring to the network icon in the finder window?

If so, and you are seeing only the network shares on the server and not the rest of the machines on the LAN, i am betting you have an issue like others have further up in the thread. Experiment with the NAT service on the server and check your routing rules in the server configuration. Your problem should be somewhere in there.

And checkout the rest of this thread. Some good solutions have been found along the way.

Steve, Yes I am referring to the Network Icon. Are you able to use this method to browse your network?

I have already posted to this thread once or twice but it seems that everyone's issues although related are all ever so slightly different enough to mean that stitching together the various solutions offered up does not result in a solve... if you now what i mean! I am using the OS X Server firewall... and alot of the problems discussed here centre around issues with external routers...

Maybe some enterprising soul can set up a VPN help website. There is definitely enough material.

I had read elsewhere (apple discussion list on VPNs dominated by a poster called Leif Carlsson who catagorically denied the ability to browse Networks using the finder over VPN) See thread here - comments welcome:

http://discussions.apple.com/message.jspa?messageI...

Mark,

Thanks for the link. I will take a look. To be honest, I have never tried using the icon to browse the network. I have gotten in the habit of using the connection window (Apple+K in the Finder) to connect directly to the IP addresses. Once i had all of the IPs I use added to the list, I guess I was just set in my ways.

You're right, there is enough material to devote an entire site to.

For what its worth, I am about to post a story about Hamachi. Its a really interesting VPN solution that has recently been released for the Mac. It won't be the right tool for everyone, but it does simplify some things greatly.

I should have that post up in a day or two. Its a fairly complicated piece to write and I'm on the 3rd draft of it.

Do you think it would be easier, if using an Xserve, to just configure the Xserve as the DHCP, firewall, and VPN? I'm still having some issues getting this thing to work correctly.

Tadd,

I don't think that will help you at all. Using an Xserve as a firewall for the entire network is only possible is you are using the same server as a router, and I don't think you want to try that.

If your router is already acting as a firewall, or the router is using NAT to turn one IP address into an entire network of internal addresses, then i would just turn off the firewall service on the Mac. You won't need it in that situation and it only stands to get in your way as you work with the VPN.

I'm wondering if the problem is your Netgear FVS318. redelader had a Netgear RP614v1 that simply would not work for him. He replaced it with a Netgear WGT624v3 and it solved his issues. I'm starting to think you're in the same boat.

Hi,
I have gone through all the same issues as most people on this forum (see my previous postings). However I fixed the problem with a phone call to the helpline of the router manufacturer. I have a DrayTek Vigor 2600Gi (which I highly recommend) at both ends of the tunnel, going into mac osx server. The guy at the helpline, went in remotely to both routers, configured them, checked them and left the vpn up and running within less than half an hour. At 75p per minute, I think it was £20 well spent.
I know everyone won't be able to do this, but it might be worth seeing if it is available for your router.

I just posted a review of an alternative VPN solution for the Mac called Hamachi.

It could be a good fix for some of you who have had not been able to get OS X's VPN server to work on your networks. Hamachi is a unique alternative, and it exceptionally adept at dealing with NAT traversal issues.

http://maclive.net/sid/202

Please feel free to leave feedback in that thread as well!

I appreciate your tutorial. I have not started to set this up, but am trying to plan. My concern is that my ISP is providing a static address that is associated with either a router or the server itself and uses the devices mac address to associate the two. My question is when setting up the VPN connection do you point to the static IP or to something else. I will admit up front that Networking is not my strong point. Bear with me please.

madneb,

You're on the right track. If your VPN server is on a network that is behind a router and only has one IP address, the VPN clients on the outside would need to point to the real IP address of your router. From there, you would set up NAT rules to send the traffic to the internal virtual IP address of the VPN router on all of the necessary ports.

You should be able to use a static or dynamic IP on the outside of your router if you use a service like DynDNS.org to keep track of a changing router IP.

Hi,

Mac os x server 10.3

When i have access with a vpn client,i can't ping and haven't acces to the others servers of the local network.Even if i set the Nat, it doesn't function. Until now the only thing which function is the vpn connection.


Any suggestions?

thx

TigerMac,

It sounds like there might be an issue in your Client Information settings. They define how the internal and external routing are handled.

I think that the configurations on my client are ok, but when i have a vpn connection, i can't even ping the vpn server. But when i look the mask on the client, it's different than my local mask.
Is that normal?

And i try with the pptp because the l2tp doesn't start on the server.

TigerMac,

Make sure the OSX firewall is disabled and try again.

What are the subnet masks of the client, and the local networks?

firewall is down already; i will start it when it works. lol

ip distributed by the vpn server: between 10.99.99.251 and 10.99.99.254

subnet received by the client: 255.255.255.255

subnet and ip of the local network: 255.255.225.0 ip between 10.99.99.0 and 10.99.99.250

TigerMac,

I always try to start with the stupid question. :-)

That subnet does seem like it would be a an issue (255.255.255.255). I tried to test it on my network, but oddly my system isn't showing me the subnet mask on the client when its connected. How are you getting your clients VPN subnet mask to show? I will compare your results to mine and see what i get.

Another stupid question... the subnets on your remote and local networks aren't the same, are they. For example, your local network is not in the same range as the remote network? If it where, that would kill the routing as well.

to see the mask you put in the console "ifconfig" (Mac) and "ipconfig" for xp.

and the two networks are different.
very strange that i can't even ping my server.
don't understand why.

thx for help

thnx Smanke for a very helpful vpn guide.

I've got a VPN from my home to my office Xserve working well, but I can't seem to get the "Send all traffic over VPN connection" feature to work on a mac.

When I connect and authenticate the client tries to change the L2PT device to the Default Route for about a second, then fails, and the network falls back to my original default route. The VPN otherwise works well. It connects and can access the remote network, but I can't force all traffic over the VPN with the mac client.

What makes me blame the mac [or the mac client machine] is everything works the way it should under the Windows PPTP client. (I can browse the Internet from the remote location's public IP address with no problem.)

Here's the errors I'm getting in the Mac client's /var/log/system.log:

Jul 18 15:49:45 ocam2 pppd[591]: L2TP connection established.
Jul 18 15:49:45 ocam2 pppd[591]: Connect: ppp0 <--> socket[34:18]
Jul 18 15:49:49 ocam2 pppd[591]: local IP address 172.22.2.123
Jul 18 15:49:49 ocam2 pppd[591]: remote IP address 208.177.xxx.xxx
Jul 18 15:49:49 ocam2 pppd[591]: primary DNS address 65.106.xxx.xxx
Jul 18 15:49:49 ocam2 pppd[591]: secondary DNS address 65.106.xxx.xxx
Jul 18 15:30:11 ocam2 launchd: Server 0 in bootstrap 1103 uid 0: "/usr/sbin/lookupd"[570]: exited abnormally: Hangup
Jul 18 15:30:11 ocam2 configd[37]: posting notification com.apple.system.config.network_change
Jul 18 15:30:11 ocam2 lookupd[576]: lookupd (version 369.5) starting - Tue Jul 18 15:30:11 2006
Jul 18 15:30:12 ocam2 launchd: Server 490b in bootstrap 1103 uid 0: "/usr/sbin/lookupd"[576]: exited abnormally: Hangup
Jul 18 15:30:12 ocam2 configd[37]: posting notification com.apple.system.config.network_change
Jul 18 15:30:12 ocam2 lookupd[577]: lookupd (version 369.5) starting - Tue Jul 18 15:30:12 2006

Here's the ppp log (omitting public IP addreses) on the Xserve:

Tue Jul 18 15:49:49 2006 : ipcp: up
Tue Jul 18 15:49:49 2006 : local IP address 172.22.2.123
Tue Jul 18 15:49:49 2006 : remote IP address 208.177.xxx.xxx
Tue Jul 18 15:49:49 2006 : primary DNS address 65.106.xxx.xxx
Tue Jul 18 15:49:49 2006 : secondary DNS address 65.106.xxx.xxx
Tue Jul 18 15:49:49 2006 : rcvd [ACSCP] 02 02 00 0a 01 06 00 00 00 01
Tue Jul 18 15:49:49 2006 : rcvd [ACSP data]
01 00 00 14 00 0b 00 00 ac 16 02 00 ff ff ff 00 '................'
00 01 00 00 '....'
Tue Jul 18 15:49:50 2006 : sent [ACSP data]
01 00 00 08 00 04 00 00 '........'

Client:
Mac G4 Powerbook OS X 10.4.7
Internet Connect 1.4.2
Server:
Mac Xserve G5 OS X Server 10.4.7


Any thoughts on why lookupd is "exited abnormally"?

Many thanks to anyone that may have some insight on this issue. --jk

Tigerhart,

I had tried ifconfig, but was surprised not to see a netmask listed. All I see is a mask that shows as 0xffffff00, which I believe is the hex version. I haven't found a converter that will tell me what the resolves to. Not sure if that helps you at all.

I can't seem to get my MacBook to show me the subnet mask in a standard format.

selsyn,

This is the first I have heard of an issue like this. It leaves me wondering if there is an issue with 10.4.7. Did you get the same error with 10.4.6?

FWIW, I have 10.4.7 on both my clients and server and can't seem to reproduce your error.

For the moment, I am at a loss. Maybe one of our other readers can make a recommendation.

Steve:

Interesting tidbit I finally figured out.. maybe Maclive worthy since there's many posts (unanswered) of similar problems......

Problem: OSX Server 10.3.9-10.4.7 VPN server fails to connect Windows 2k or XP VPN clients (error 732 or other 73X errors) after certain crashes OR upgrades. This has plagued me with every update and a few crashes.

Solution which worked today (after hours of trying different solutions (ie new user etc..):
-In Server Admin:
Shut down NAT
Shut down VPN
Note your configuration of NAT and VPN settings (imperitive) screenshot or whatever

In Finder:
Locate the com.apple.RemoteAccessServers.plist and delete and empty trash (/Library/Preferences/SystemConfig/ generally)

Go back to Server Admin:
Verify NAT is set to Forwarding and Translation
Then go to VPN and re enter your settings.
Start NAT service
Start VPN service
-Then shout out explitives :)

Its almost like a corruption occurs during a crash or in the case of upgrade the settings fields are changed and it doesn't pass the right PPP attributes (it's always a PPP settings error in windows or similar).....

This came up because I did an osx update over the weekend and no one except Macs could connect.....

Jon.

Hi. So this thread was very helpful, but I continue to have problems. Heres the story:

I have an xserve g5, hooked up to a static ip dsl from verizon. The server is acting as the gateway to everything. I mean that the dsl line goes from the modem into ethernet port 1, and then the rest of the network is in ethernet port 2. All of the computers below the server are working fine DHCP and NAT wise. They can access the network, internet, etc. VPN is turned on, and set exactly as described in this tutorial, using L2tp. In order to insure it wasnt the firewall, under firewall I turned on any to all connections, just to test, which should open everything up. However when I connect from outside it just hangs on the connecting and never connects. The server log under VPN shows no connections at all.

Given this I assume that the problem is relating to DNS and NAT. I think probably just NAT. Because my server is acting as DHCP, NAT and VPN, I am not sure how the server knows where to route VPN requests coming into the one static ip. Is there something I need to configure in the NAT to change that? The fact that the VPN says that no one is trying to connect, but I seperately can connect to that server via ARD3 and via server admin and monitor is confusing. Any thoughts?

jpf,

Allowing all traffic through the firewall with a rule is a good idea, but I'm not sure that rule (or rules) will apply to GRE or ESP. Just to be on the safe side, can you disable the firewall all together? I'm wondering if that will make a difference.

Aside from that, I think you're right. It seems to be a NAT issue. Luckily (or not) the NAT options in 10.4 Server are few so trial and error is the only way to really get through it. I think that if you set the option for IP Forward and NAT, and then specify the port that your modem is hooked to, you will be set.

To be on the safe side, don't be afraid to reboot the box after making the changes. It shouldn't be necessary, but sometimes a good reboot can clean caches no one knew existed.

This raises a very interesting question. I admit, I have never tried a setup like this, so I'm in the dark. Broadband routers allow external access through NAT because they let you set portmap rules. I'm not sure how that works with OSX's routing capabilities. I can't see the firewall being used for such a service. There must be another place to set rules.

Hmm... anyone else have an idea?

Great article, helped me - to a point. Maybe I'm just misunderstanding how VPN works? I set this up as above for a few IPs that we have available through our T1 provider. I can successfully connect to the VPN server, BUT ALL of my traffic looks like it comes from my ISP's IP! I do have "Send all traffic over VPN" active.

In other words, in my head, when I'm connected via VPN to the server, my traffic should look like it is coming from an IP on the remote network - 216.220.x.x. But when I get my IP from sites like http://whatismyipaddress.com, it shows up as 24.60.x.x.

Ergo, the access rules that I setup with the OS X Firewall, don't work.

Ergo, despite being CONNECTED through VPN, nothing is... actually happening.

Honestly I'm just having a hard time determining what is going wrong, where. When you are connected via VPN, shouldn't your originating IP be reported as whatever the IP is that VPN assigned you?

Argh! Any help would be SO appreciated - once I have an answer to that, I can ask another one (namely, WHY is this happening?! =)

Thanks in advance!

hi there

i've found out the non500-iskmp is actually udp 4500 for those that need to know. I'm still experimenting with this so i'll check back later to let you know the results. Almost completed the set up.

Re: JustinD

You are correct as far as I know, not sure why it doesn't work for you though, did you set the dns and route info correctly?

Hi!

I'm new to network administration and have just been asked to enable the VPN service on our Mac OS X server. I've done a good bit of research on this topic and now my mind is swimming with questions that I'm hoping someone would answer.

Here's the situation: we are a small, 20 person maximum non-profit that runs 10.4.7 on our Xserve. We have both Windows and Mac in our organization, so both clients would utilize the VPN. Currently, the only services that have been enabled (as shown in Server Admin) are: DNS, Firewall, Open Directory, iChat, Windows Services. After reading the Apple Network Services documentation, I've been leary of enabling the VPN service because it indicated that VPN and DHCP should work closely together, and right now, DHCP is disabled on the server. Internally, we follow the 192.168.x.xxx convention for our IP addresses and I've noticed that most LAN clients go into the 100 range for the final octet. In the Apple docs, it says to allot the addresses 192.168.x.128 thru 192.168.x.254 for VPN clients. However, we currently have LAN clients that are in this range! What to do in this case? Enable DHCP and create a subnet group so that LAN clients get new, distributed IP addresses, *then* enable VPN with the specified IP address range?

Also, I contacted our T1 provider and requested our external IP addresses. Do any of these addresses need to be specified in any of the Mac services that would be required to enable the VPN?

Thank you very much for any advice you can offer. And thanks to the author of this document for creating such a forum for discussion.

Christine

Re: Christine

I'll try to help with what I can. Regarding your DHCP situation, just limit the DHCP server on the other machine to make sure it only give out from 192.168.0.10-192.168.0.100 then on the Xserve you can limit it to give out from any range you want. I myself have the DHCP on another machine and have it setup so that it only gives out from the range of 100-199, and the Xserve will get the range of 200-220. DHCP doesn't have to be enable on the Xserve, this will be taken care of by the VPN service. I think you should keep that as simple as you can unless you have to do other crazy stuffs otherwise it would be tough to troubleshoot anything later.

And if you know how to enable to firewall and such already you shouldn't have to have any other external IP address unless you want to create a new name for it to match to.

JustinD,

Sorry it has taken me so long to reply. I have been swamped with work and had no time for the site at all.

Given the fact that the Route all Traffic over VPN option is currently broken, external sites that show you your IP address would be showing you the broadband providers IP address. If the option worked correctly, you are correct... you should be seeing an address from your remote secure network.

No word as to when Apple will fix the router all traffic option. To be honest, I haven't had time to check and see if they are even aware there is an issue.

The only way you will know if your traffic is going through the VPN is by running a few trace routes. Run one to Yahoo.com. You should see your traffic running through your ISP's network address on its way to Yahoo. When you do a trace to an address on your remote secure network, the trace should be very short and it should not include any references to your ISP's network.

Christine,

mac appreciator is correct, you don't need to enable DHCP serving on your Xserve. If you already have a DHCP server on the secure network, you don't want to enable 2 DHCP servers.

Even without having the DHCP service enabled on the Mac server, the VPN server will allow you to specify any range of IP address on the virtual secure LAN that you want to allocate to people who are connecting via VPN. You just specify a pool of addresses to pull from, and the VPN server doles them out as needed to people as they connect.

As for your Firewall on the Xserve, you might want to disable that while you are getting the VPN working. Most people waste tons of time trying to get VPN working before they ever realize that the Firewall is what has been preventing them.

Post back if you are still having problems. I have been away from the site for some time, but I should have more time to keep up with it.

Good luck!

I have managed to get our client computer connecting to our server via VPN but the client machine is not showing up on our network nor does it have access to any network devices, ie. printers. Just wondering if there is any additional calibration to do to allow the user access to all network services? If anyone has any ideas please respond to my email webmaster@outeraspect.com

if the client machine doesn't show up on the connections tab then it means there aren't any connections at all. You'll need to recheck that and test it first from within the network to make sure it works then you can go out and enable firewall and all that stuffs.

Hi. I have an OS X server 10.4.7. I've set it up as a VPN server using l2tp with a shared IPSec secret. The server is behind a D-Link DI-808HV router. The router has IPSec passthrough enabled, and I have UDP ports 500, 1701, and 4500 open.

When I try to connect with an OS X Tiger client, I get a "Connecting to VPN server" message for a while, then "server did not respond." In the VPN server log, there is no sign that anything occurred - no log entries at all for the attempted connection.

Where else should I look to troubleshoot this?

I've tried PPTP, which at least makes a connection but the fails at the negotiation with the error "Wed Sep 13 13:50:28 2006 : sent [LCP ConfReq id=0x1 ]" in the log.

Also - I am using the local server's user database, not Open Directory.

Thanks
David

David,

Try opening 1723 and see if that lets you in via PPTP.

As for L2TP, see if you can open up ESP (protocol #50) and GRE (protocol #47). If you have OS X's firewall on while you are setting the box up, disable it until you know everything else is up.

Hi smanke, thanks for your reply. Yes I opened 1723 for PPTP and I still got the PPTP errors. I have the OS X Server firewall service turned off. Unfortunately I can't manipulate ESP or GRE on the Dlink router.

I spent a couple hours yesterday researching VPN routers that allow multiple L2TP passthrough - I might have to bite the bullet and just purchase VPN Tracker software instead.