Comprehensive Port Scanning Methodologies

The Semantics of TCP SYN Scanning (-sS)

The TCP SYN scan remains Nmap's default and most fundamental technique, exploiting a deliberate violation of RFC 793's connection establishment protocol. Under normal operation, a three-way handshake (SYN → SYN-ACK → ACK) completes a connection. Nmap sends only the initial SYN, receives the SYN-ACK if the port is open, and immediately tears down the embryonic connection with RST rather than completing with ACK.

Attacker          Target
  |    SYN        |
  | ------------> |
  |   SYN-ACK     |  <- Port open: handshake never completes
  | <------------ |
  |    RST        |
  | ------------> |

RFC behavior exploited: RFC 793 §3.4 mandates that a received SYN in the CLOSED state must elicit RST if no listener exists, or SYN-ACK if one does. The protocol has no requirement to log incomplete handshakes at the application layer. This creates an asymmetry: the operating system's TCP stack sees the connection attempt, but many application logs do not.

IDS implications: The "stealth" reputation is largely historical. Modern stateful firewalls and IDS/IPS systems maintain embryonic connection tables (SYN_RECV states) and flag rapid SYN sequences without completing handshakes. The incomplete handshake itself becomes a signature. Against stateless packet filters, however, the technique remains effective—these devices examine individual packets without connection state context and may pass SYN packets that would block a full connect().

Privilege requirement: Raw socket construction demands root/capabilities, as the technique requires crafting packets with arbitrary TCP flags rather than using the OS TCP stack.

TCP Connect Scan (-sT): The Fallback Mechanism

When raw sockets are unavailable—unprivileged execution, certain containerized environments, or IPv6 configurations without kernel support—Nmap falls back to the connect() system call. This completes the full three-way handshake through the operating system's standard socket API.

Audit logging consequences: The full handshake triggers application-level logging. /var/log/secure, Windows Security Event Log, and application connection logs record the completed connection. Connection table entries appear in netstat. The scan becomes fully visible to host-based intrusion detection and forensic timeline analysis.

Application-level proxy traversal: Connect scanning's genuine TCP connection allows traversal of SOCKS4/5 and HTTP CONNECT proxies. The proxy establishes the outbound TCP connection on the scanner's behalf, obfuscating the origin. This is not evasion but functional necessity in proxied environments:

# Force connect scan through proxychains
proxychains nmap -sT -Pn -n --proxies socks4://192.168.1.10:1080 10.0.0.0/24

Performance penalty: The kernel manages retransmissions, connection state, and teardown; timeout granularity degrades from microsecond-level packet control to kernel timer resolution.

UDP Scanning (-sU): The Stateless Challenge

UDP scanning confronts the fundamental absence of connection state. RFC 768 offers no acknowledgment mechanism—datagrams may be silently dropped by firewalls, filtered by ACLs, or accepted by listening services without any mandatory response.

ICMP port unreachable interpretation: Per RFC 1122 §3.2.2.1, hosts must generate ICMP Type 3 Code 3 (Destination Unreachable, Port Unreachable) when a UDP datagram arrives at a closed port with no listener. Nmap interprets this as closed. The absence of any response, after retransmission, indicates open|filtered—the port may be accepting datagrams, or a firewall may drop silently.

Timeout amplification: UDP scanning requires significantly more time than TCP techniques. The ICMP unreachable response may be rate-limited (Linux defaults to 1/second), suppressed by firewalls, or lost in transit. Nmap's algorithm escalates timeout values aggressively. A full UDP scan of 65,535 ports against a firewalled host may require hours.

Optimization imperative: Always constrain UDP scans with -p specifications or --top-ports:

# Scan only DNS, SNMP, and NTP—services where UDP is primary transport
nmap -sU -p 53,161,123 --max-rtt-timeout 500ms 192.168.1.1

The nmap-services database ranks ports by frequency; --top-ports 20 for UDP scans domain(53), ntp(123), snmp(161), isakmp(500), and syslog(514), covering the majority of discoverable UDP services in typical environments.

Null, FIN, Xmas, and ACK Scans: Flag Manipulation Theory

These techniques exploit discrepancies between RFC 793 state machine implementations and firewall rule processing logic.

The underlying RFC behavior: RFC 793 §3.9 specifies that packets with no flags set (Null), FIN-only, or FIN-PSH-URG ("Xmas," named for the lit-up flags like tree lights) arriving at a closed port must elicit RST. For open ports, the RFC is ambiguous—implementations typically drop these illegal packets silently.

Null scan (-sN):    [ 0 0 0 0 0 0 ]  No flags set
FIN scan (-sF):     [ 0 0 0 0 0 1 ]  FIN only
Xmas scan (-sX):    [ 0 1 1 0 0 1 ]  FIN, PSH, URG
ACK scan (-sA):     [ 0 0 1 0 0 0 ]  ACK only (no SYN)

Response interpretation: | Response | Null/FIN/Xmas | ACK scan | |----------|---------------|----------| | RST | Closed | Alive (firewall/state table exists) | | No response | Open|Filtered | Filtered (stateful firewall dropped) | | ICMP unreachable | Filtered | Filtered |

Stateful inspection evasion: Null, FIN, and Xmas scans bypass stateless packet filters that examine only SYN flags for "new connection" decisions. However, stateful inspection firewalls (Netfilter conntrack, Cisco ASA, Palo Alto) track connection state and recognize these packets as invalid outside established connections, dropping them and logging anomalies. The "stealth" succeeds against poorly configured stateless ACLs; it fails against modern stateful infrastructure, often generating more alerts than standard scans due to the anomalous flag combinations.

ACK scan's unique role: Since ACK packets are valid only within established connections, ACK scanning cannot determine open/closed state. Instead, it maps firewall rule sets—distinguishing stateless (RST response) from stateful/filtered (drop/silent) configurations.

Window, Maimon, and Custom TCP Scans for Specialized Evasion

TCP Window scan (-sW): Exploits idiosyncratic TCP window field behavior in certain TCP stack implementations (notably older Cisco IOS, BSD derivatives). When a RST is returned to a probe, the window field value differs between open and closed ports in these specific implementations. This yields open/closed differentiation where standard scans return ambiguous results. The technique is highly target-dependent—modern Linux and Windows stacks have standardized window=0 on RST, rendering -sW ineffective.

Maimon scan (-sM): Named for Uriel Maimon who documented the technique in Phrack 49. Probes with FIN-ACK flags. Theoretical basis: RFC 793 implies RST for closed ports to any unexpected packet, but certain TCP implementations (again, historically BSD) treated FIN-ACK differently from plain FIN. Contemporary utility is minimal; included primarily for completeness against legacy infrastructure.

Custom TCP scans (--scanflags): Nmap permits arbitrary flag construction for research and specialized evasion:

# Construct an ECE-CWR probe to test ECN-aware filtering
nmap --scanflags ECE,CWR,ACK -p 80,443 10.0.0.1

# Fragment the probe across multiple packets to evade simple pattern matching
nmap -f --scanflags SYN,ACK -p 22 10.0.0.1

These require deep understanding of target stack behavior. Random flag combinations against unknown targets produce meaningless results.

SCTP INIT and COOKIE-ECHO Scans: Telecommunication Environments

Stream Control Transmission Protocol (RFC 4960) underpins SIGTRAN, the signaling transport architecture for SS7-over-IP in cellular networks (3G/4G/5G core, IMS, HLR/HSS interfaces).

SCTP INIT scan (-sY): Analogous to TCP SYN scan. Sends SCTP INIT chunk; expects INIT-ACK for open ports, ABORT for closed, or silence for filtered. Requires raw socket privileges.

SCTP COOKIE-ECHO scan (-sZ): Sends COOKIE-ECHO chunk without preceding INIT handshake. Open SCTP stacks should respond with COOKIE-ACK; closed ports with ABORT. This advanced technique tests for SCTP implementations with non-standard state machine behavior.

Operational context: These scans target specific telecommunications infrastructure—Diameter (RFC 6733), S1-MME interfaces, SIGTRAN M2UA/M3UA associations. Standard enterprise networks rarely expose SCTP; encountering it suggests either misconfiguration or deliberate telco/VoIP infrastructure deployment.

Port Specification, Optimization, and the nmap-services Database

Intelligent scanning requires precise port targeting. The -p syntax supports multiple specifications:

nmap -p 22,80,443            # Discrete ports
nmap -p 1-1024               # Range
nmap -p ssh,http,https       # Named services (resolved via nmap-services)
nmap -p U:53,T:22,25,80      # Protocol-specific
nmap -p-                     # All 65535 ports (equivalent to -p 1-65535)
nmap --top-ports 1000        # Highest-frequency 1000 per nmap-services rankings

The nmap-services database derives frequency rankings from empirical Internet-wide scan data. These are not arbitrary assignments—http(80) ranks highest globally, but internal networks may diverge significantly. The database supports custom entries in ~/.nmap/nmap-services for organization-specific port frequency tuning.

When "Stealth" Becomes Myth

The historical distinction between "stealth" and "noisy" scans has eroded. Modern network security infrastructure detects scanning through behavioral signatures: timing analysis (sweeps at N packets/second), source distribution patterns, and payload entropy. A SYN scan at -T5 generates more actionable intelligence for defenders than a Connect scan at -T1 with source rotation.

Intelligent scan selection prioritizes functional accuracy under target constraints over cargo-cult stealth. SYN scans succeed where raw sockets work and firewalls lack state tracking. Connect scans succeed in restricted environments. UDP scans require patience and precision. Flag manipulation scans reveal specific firewall implementation details. The sophisticated practitioner matches technique to network architecture, not to deprecated notions of undetectability.