What IPsec mode does not protect the IP header?

IPsec Authentication Header (AH) is a security protocol used to protect data sent over a network. Its core focus is around data integrity and authentication. It is also responsible for authenticating IP packets and helps protect against network attacks. 

The IPsec Encapsulating Security Payload (ESP) protocol protects data confidentiality and data origin authentication. Both IPsec AH and IPsec ESP focus on encryption with the difference coming in the use of both protocols in the IPsec modes which we will discuss below. IPsec Tunnel mode and IPsec Transport mode. 

In order to get a better understanding of the differences in each IPsec transport mode, let’s first discuss the use cases for them.

The IPsec AH tunnel mode sets up a secure connection between two communication endpoints on the internet. This is the most common mode to use when connecting to a VPN server. While the AH protocol establishes a VPN tunnel without encrypting data, it instead provides integrity of the data packets.

The IPsec ESP tunnel mode encrypts and encapsulates IP packets while also providing authentication and integrity. This protocol is used by VPN tunnels to see if data packets have been tampered with while in transit. This allows VPN connections to be routed through untrusted networks while maintaining encrypted data packets.

In order to configure IPsec tunnel mode, you will first need to use a different protocol such as IKE (Internet Exchange Key) to negotiate the parameters that will be used in order to secure the communication between peers. The IKE setup process is broken down into 2 phases: 

  • IKE Phase 1: The initial IKE phase establishes a secure tunnel between channels. The main purpose in the first step is to authenticate IPsec peers and to negotiate security associations (SA). 
  • IKE Phase 2: Once a security association has been established, the next step is to negotiate authentication and encryption, thus encrypting the entire packet which is then included in the payload or the transmission of data from the intended message.

IPsec AH transport mode is a security protocol used to protect data through your network, but it doesn’t make a secure connection. It encrypts the data being sent without checking for integrity or authentication, which makes it faster than IPSec AH Tunnel Mode. However, it is far less secure.

IPsec ESP transport mode secures data sent over a network, providing privacy by encrypting it, and this protocol provides authentication and integrity. It is used by VPN tunnels to ensure that data is secured while in transit without having to establish a secure connection between two points on the internet. 

The final destination in ESP transport mode is typically the host. The other point to take into consideration is that ESP transport mode encrypts the data only and not the original headers.

IPsec transport mode secures traffic from one system to another. There is also a two-step configuration process similar to tunnel mode.

  • IKE Phase 1: The key focus here is on the negotiation of the secure channel between two systems using an ISAKMP security association or Internet Security Association and Key Management Protocol. 
  • IKE Phase 2: In this step, the IKE peers dynamically negotiate the authentication and encryption algorithms to secure the payload.

Transport mode is seen as less secure than tunnel mode because the IP header is not encrypted.

IPsec tunnel mode sets up a secure connection, while IPsec Transport Mode only encrypts the data being sent without establishing a secure connection.

In transport mode, the sending and receiving hosts establish a connection before exchanging data. In tunnel mode, a second IP packet is sent in a completely different protocol. This protects data packets from being inspected or modified in transit. 

The advantages of tunnel mode over transport mode are that it can work through Network Address Translation (NAT) and that the entire original IP packet is hidden. NAT maps a private IP address to a public IP address by modifying network address information in the IP header of packets across a traffic routing device while in transit.  

The major disadvantages of tunnel mode are additional overhead from encapsulation, an inability to defend against attacks on weak integrity protocols, and that transport mode may be more compatible with some firewalls. 

Examples of such attacks include SYN floods which is a type of distributed denial-of-service (DDoS) attack. SYN floods send massive requests to overwhelm a server, rendering the system unavailable to receive legit traffic. It also prevents the completion of the TCP three-way handshake between client and server needed for a secure connection. 

In general, tunnel mode is better when both endpoints are behind a NAT device, and transport mode is preferable when there is no NAT or if the network uses pre-NAT devices with address translation only at the IP packet level. In most cases, transport mode will provide better security with less overhead.

In order to know when to use either tunnel mode or transport mode, you should consider where each endpoint is located in relation to the internet. If you are both behind the NAT device, then tunnel mode is better because it establishes a connection while transport mode simply encrypts packets.

However, if only one of the endpoints is behind the NAT device, you’ll need to use transport mode so both hosts can communicate securely with each other.

Both tunneling and transport mode encrypts data, but when implementing one over the other, you should consider whether there are NAT devices between two connected networks. If no NAT device exists, use transport mode. If a NAT or pre-NAT firewall exists, use tunnel mode.

For example, if you’re using a pre-NAT firewall and your endpoint is located in the same private network as the server, use transport mode. If your endpoint is located behind a NAT device or on a different network than the server you’ll be connecting to through IKEv2, then use tunnel mode.

Transport mode works best for firewalls that do not translate IP addresses in the packet header and for cases where transports mode is more compatible with certain firewalls.

The main advantage of IPsec transport mode is that it is more compatible with certain firewalls and it offers higher levels of security. In addition, transport mode does not require a secure connection to be established between two endpoints and has less overhead because it does not encapsulate packets.

The main disadvantage of IPsec transport mode is the difficulties it has with NAT traversal or UDP encapsulation. The User Datagram Protocol (UDP) is a technique of adding network headers to the packets and helps with load balancing to better distribute network traffic.

The main advantage of IPsec tunnel mode is that it creates a secure connection between two endpoints by encapsulating packets in an additional IP header. Tunnel mode also provides better security over transport mode because the entire original packet is encrypted.

The main disadvantage of the IPsec tunnel mode is that it requires a secure connection to be established between two endpoints and tends to create more overhead because the entire original packet must be encapsulated. In addition, transport mode may perform better than tunnel mode on some types of networks and with certain firewalls.

In order to know which mode is best for you, consider your network environment. You might also want to consider an IPsec VPN to create encrypted tunnels and secure remote access to an entire network, whether on-premises or from corporate headquarters.

Perimeter 81’s IPsec VPN enables organizations to work safely from anywhere in the world by establishing a secure connection between devices.

Perimeter 81’s IPsec VPN leverages the principles of Zero Trust to provide a stronger level of security across the network. This allows admins to create policies based on authentication factors such as Multi-Factor Authentication (MFA) and 256-bit encryption.

Instantly deploy your entire network with Perimeter 81’s IPsec VPN. See how radically simple it is for yourself. Request a demo today.

What is tunnel mode in IPsec?

IPsec tunnel mode encrypts data and the original IP packet.

What is IPsec transport mode?

IPsec transport mode only encrypts the data being sent without establishing a secure connection and leaves the original IP address unencrypted.

Is tunnel mode more secure than transport mode?

Yes, tunnel mode is more secure than transport mode because it can work through Network Address Translation (NAT) and the entire original IP packet is hidden.

What is the difference between transport mode and tunnel mode in IPsec?

The main difference between tunnel and transport mode is that transport mode retains the original IP header.

Which header is unencrypted in IPsec transport mode?

IPSec operates in two modes: Transport mode and Tunnel mode. You use transport mode for host-to-host communications. In transport mode, the data portion of the IP packet is encrypted, but the IP header is not. The security header is placed between the IP header and the IP payload.

In which of the following modes IP security protects the entire IP packet?

IPSec tunnel mode is the default mode. With tunnel mode, the entire original IP packet is protected by IPSec. This means IPSec wraps the original packet, encrypts it, adds a new IP header and sends it to the other side of the VPN tunnel (IPSec peer).

What is Phase 1 and Phase 2 in VPN?

VPN negotiations happen in two distinct phases: Phase 1 and Phase 2. The main purpose of Phase 1 is to set up a secure encrypted channel through which the two peers can negotiate Phase 2. When Phase 1 finishes successfully, the peers quickly move on to Phase 2 negotiations.

Does IPsec encrypt header?

Encryption: IPsec encrypts the payloads within each packet and each packet's IP header (unless transport mode is used instead of tunnel mode — see below). This keeps data sent over IPsec secure and private.