Layer 4, the transport layer of the OSI model, sits between the network and application layers. It breaks application data into segments, controls flow, checks for errors, and uses port numbers to route traffic to specific services. Because it manages session state, it is a frequent target for denial‑of‑service and reconnaissance attacks.
Core responsibilities of the transport layer
Transport protocols slice large application messages into manageable pieces, reassemble them on the other side, and ensure that data arrives correctly. They also regulate how fast a sender can push data to a receiver and keep a record of which packets belong to which session.
TCP and UDP: the two main protocols
TCP is connection‑oriented. It uses a three‑way handshake—SYN, SYN‑ACK, ACK—to open a session, guarantees ordered delivery, and retransmits lost packets. Because the server must keep state for every half‑open or open connection, TCP is susceptible to resource exhaustion attacks.
UDP is connectionless. It sends datagrams without any handshake, making it fast but also lacking delivery guarantees. Its stateless nature exposes it to reflection and amplification attacks, especially when misconfigured services respond to spoofed requests.
Common Layer 4 attacks
- SYN flood – attackers send many forged SYN packets, filling the server’s connection table and blocking legitimate traffic.
- Port scanning – tools map open, closed, or filtered ports to discover active services and potential vulnerabilities.
- Session hijacking – by predicting TCP sequence numbers, an attacker can inject data into an existing session.
Defending the transport layer
Hardening Layer 4 requires both network controls and proactive monitoring. Stateful firewalls track the full context of TCP sessions, dropping packets that do not fit an established state. Implementing SYN cookies allows a server to respond to connection requests without allocating resources until the handshake completes.
Port minimization is essential: close any unused ports on public servers and regularly scan for rogue services. Where possible, wrap TCP traffic in TLS to protect data confidentiality and integrity, even though TLS operates above Layer 4.
When to use TCP vs. UDP
Choose TCP when reliability, ordering, and error recovery are critical, web browsing, file transfer, and database access. Opt for UDP when low latency outweighs the need for guaranteed delivery, VoIP, live streaming, and online gaming.
Key takeaways
- Layer 4 handles segmentation, flow control, error checking, and port addressing.
- TCP guarantees delivery but requires state; UDP is fast but stateless.
- Stateful inspection and SYN cookies are effective against SYN flood attacks.
- Regular port scans and minimal exposure reduce the attack surface.