MTU & VPN Tunnels

[HCHTech]

In troubleshooting a problem for a client recently, I've uncovered a gap in my understanding (I know, "What, only one?")

So, I know how to find the most efficient MTU for a particular WAN connection, that's networking 101 stuff:

ping -f -l xxxx www.somewebsite.com
Adjust 'xxxx' until you find the highest number that does not produce the result that "Packet needs to be fragmented."
Then, add 28 bytes to that number for the headers to get your answer. Plug that into your router settings and go about your merry way.

When setting up site-to-site VPNs, though, things get more complicated. The amount of space needed for the headers varies depending on the type of encryption used, among other things. The list I found in a Sonicwall technote is:

• Any time you add another outer IPv4 header adds 20 bytes.
• IPSec encryption performed by the DMVPN adds 73 bytes for ESP-AES-256 and ESP-SHA-HMAC overhead (overhead depends on transport or tunnel mode and the encryption/authentication algorithm and HMAC).
• MPLS adds 4 bytes for each label in the stack.
• IEEE 802.1Q tag adds 4 bytes (Q-in-Q would add 8 bytes).

Additionally, to make room for the extra header size, you have to reduce your MTU setting by the size of those headers now. So if your ping test reveals that the most efficient MTU is 1500, and your VPN traffic header is expected to be 73 bytes, then your optimal MTU setting for that VPN is 1500-73 = 1427. So far so good. Also, don't forget that your ping test has to use a device on the other end of the tunnel instead of a web site if you have setup a split tunnel.

I don't know about other firewall makes, but with Sonicwalls, you cannot set a separate MTU for VPN traffic, there is only one setting for the WAN connection. So if you configure that 1427 MTU to get the most efficient VPN, you are also reducing the MTU for all non-VPN traffic. I honestly don't know if this unavoidable inefficiency for the non-VPN traffic is worth worrying about, and I also don't know what kinds of problems happen when the MTU is lower than necessary.

Further - what about the connection on the other end of the VPN Tunnel? If the most efficient VPN calculated for that end is, say 1385, does that mean that you should reduce the MTU setting for BOTH firewalls to the lowest value determined on either end of the tunnel? This would seem obvious since the traffic has to go back and forth through both firewalls, but I couldn't find anything addressing this fact.

Still further - if you have more than one site-to-site VPN setup (my client has one for a remote backup server and another one for a satellite office, for example, giving me 3 firewalls in the dance), do you test everything, then set the MTU on all of the firewalls to the lowest value determined on any of them? This is making my head swim.

 

[YeOldeStonecat]

In the old days of....
*Dial Up
*Early DSL which was using a weird dial up adapter called PPPoE
*Windows 95, and to some extent....up to Win2K
.....adjusting MTU was quite important.

The earliest of broadband modems did not allow this adjustment, back when Linksys was rolling the first couple of hundred BEFSR41 broadband routers off the assembly line, I was setting some up with the old SNET.NET DSL, which usually used "EnterNET" PPPoE dial up adapter software installed directly on the computer. Which was connected to the DSL bridge....and the computer itself initiated the connection and obtained the public IP address. The first broadband routers had a really hard time with PPPoE DSL, and they could not adapt to the MTU...and sometimes you'd get weird issues on the computers such as...SMTP errors when sending email with file attachments.

Working with Linksys support, they admitted their firmware for the BEFSR41 was not matured yet, such a new product, and directed me to a certain "tech" website that had a few patches for Windows 95 to make it more adaptable to broadband. They sent me to www.speedguide.net ..the guy there, Philip, wrote some patches to apply to Windows to help its network stack handle broadband. Back then, Windows 95 had a network stack designed for dial up, and 100 meg LAN. But nothing in between...which is why it ran poorly on broadband (which was from 1-9 megs in speed mostly in the early days). There was Philips TCP Optimizer, and an MTU patch, and another vtc386 patch or something. I got to hang out in their forums, got to chatting with Philip and Ken a lot..and started flying down to Tampa to hang with them for the Gasparilla pirate festivals a lot, became an admin on those forums.

Anyways, when setting up VPNs back in those days for clients, site to site VPN tunnels, the Sonicwall SOHO3 units we used didn't handle MTU automatically very well. Not did any broadband routers for that matter, so we always manually set them. For PPPoE DSL, we'd set to 1492 (when Columbus sailed the ocean blue!). And for PPPoA DSL, as well as cable, and T's...would be 1500. But since DSL has mostly ditched PPPoE these days, it's mostly PPPoA (ATM) now.....we just leave at 1500. I haven't adjusted MTU in...jeeze...seems like nearly 2 decades.

 

[Sky-Knight]

I don't muck with MTU unless something is broken, as in RDP doesn't work. Because these settings get forgotten, and screw things up later HARD.

If you want to improve VPN performance, deploy a protocol that was designed this century. IPSec is unacceptably, terribly, and infuriatingly slow.

Wireguard all the things, OpenVPN what you can't, IPSec only what you absolutely have to.

 

[HCHTech]

If you want to improve VPN performance, deploy a protocol that was designed this century. IPSec is unacceptably, terribly, and infuriatingly slow.

I don't think there is a non-IPsec option for site-to-site VPNs with Sonicwall. I've got probably 50 or 60 setups using their system and for the most part they are bulletproof, despite your visceral reaction. Once in place, they just work - we almost never have to tinker with them. As WFH escalated this past year, we're seeing more and more non-specific problems with remote access, so we are trying to tweak settings to get the most efficient setup possible with the equipment already in place.

I have found that reducing the MTU setting has helped with RDP performance for a couple of the satellite office setups we support. My post is about just trying to understand how multiple tunnels interplay with each other. I'll try opening a ticket with Sonicwall and seeing if I can get anything useful from the nice gentleman in India - although past experience has taught me that they aren't well-suited to these types of questions.

 

[Sky-Knight]

Or... you can build a virtual network based on wireguard, get twice the performance in half the time and never have to mess with 3rd party support ever again because the product is just that brain dead easy.

You can even put the clients out on all the endpoints via your RMM via script!