Service and Version Detection Deep Dive

The Mechanics of Probe Construction and Matching

Version detection in Nmap operates through a sophisticated probe-prototype matching system defined in the nmap-service-probes file. Understanding this file's structure reveals how Nmap actually identifies services rather than merely treating it as a black box.

Each probe entry follows a precise syntax. The Probe directive initiates with a protocol (TCP or UDP), a unique name, and the literal string to send:

Probe TCP GetRequest q|GET / HTTP/1.0\r\n\r\n|

The q| delimiter uses | as a wrapping character; any character can serve this purpose. Nmap supports escape sequences including \r, \n, \t, and octal/hex representations (\x0D, \0D). This matters because probes often require exact byte sequences to elicit identifiable responses from picky services.

Match directives form the analytical core. Each match contains a service name, regular expression, optional version extraction, and confidence scoring:

match http m|^HTTP/1\.[01] \d\d\d.*?\r\nServer: Apache/([\w._-]+)|s p/Apache/ v/$1/

The m|...| pattern uses | as its delimiter. The trailing s flag enables . to match newlines—critical for multi-line banners. Parenthetical captures feed into version extraction variables: $1 through $9 populate the v/ (version), i/ (info), h/ (hostname), o/ (operating system), d/ (device type), and cpe:/ fields.

Softmatches (softmatch) differ crucially: they identify a service family without extracting version details, allowing Nmap to skip incompatible probes and proceed to more specific ones. This hierarchical classification—generic softmatch followed by precise hard match—reduces probe traffic while maintaining accuracy.

The nmap-service-probes file organizes probes by rarity. Common probes (rarity 1-4) run during normal -sV scans; higher rarity probes trigger only with --version-intensity elevation or when lower-rarity probes fail to match.

SSL/TLS Wrapper Detection and Encrypted Service Probing

Modern services increasingly hide behind TLS encryption, which historically forced analysts to chain external tools (openssl s_client, ncat --ssl) before version detection. Nmap's built-in SSL negotiation eliminates this friction through transparent wrapper detection.

When Nmap connects to a service, it analyzes the initial response for TLS handshake indicators—specifically the content type byte 0x16 (handshake) and version bytes (0x0301 for TLS 1.0, 0x0303 for TLS 1.2, etc.) in the record layer. Upon detecting this pattern, Nmap initiates a complete TLS handshake using its embedded OpenSSL integration, then tunnels subsequent probes through the encrypted channel.

This enables direct probing of HTTPS, SMTPS, IMAPS, POP3S, and XMPPS without manual intervention:

nmap -sV --version-intensity 7 -p 443,465,993,995 target.example.com

The probe sequence automatically adapts: Nmap first sends a generic probe, detects TLS wrapping, completes handshake with appropriate cipher suite negotiation (downgrading only when necessary for compatibility), then retransmits application-layer probes through the encrypted tunnel. Version extraction proceeds normally, with regex matches applied to decrypted responses.

For services on non-standard ports with TLS, the ssl service softmatch triggers probe tunneling regardless of port number. A web server on port 8443 receives identical treatment to one on 443, provided the TLS handshake succeeds. Nmap also handles STARTTLS upgrades for protocols like SMTP, FTP, and XMPP through dedicated probes that issue STARTTLS commands before initiating TLS.

Critically, certificate parsing occurs independently: even when application-layer probing fails, Nmap extracts subject names, validity periods, and issuer information from the TLS handshake itself, contributing to service identification.

RPC Grinding and Sequential Probing Strategy

SunRPC (ONC RPC) services present unique challenges because they typically register with rpcbind (portmapper) rather than exposing version information directly. Nmap's RPC grinding technique addresses this through systematic program number enumeration.

The standard probe sequence for suspected RPC services:

Probe dispatch: Send a NULL procedure call to the target port with varying program numbers
Response analysis: Interpret PROG_UNAVAIL (program unavailable) versus PROC_UNAVAIL (program known, procedure unavailable) versus successful NULL response
Version enumeration: Upon identifying a valid program number, probe version ranges to determine supported protocol versions

Nmap maintains a database of known RPC program numbers (rpc-names file), but grinding can discover unregistered or custom RPC services through brute-force enumeration. The --version-intensity setting directly controls grinding thoroughness—higher intensities expand the program number search space and version range probes.

The sequential strategy proves necessary because RPC services lack the banner-based identification common to other protocols. A match on port 2049 requires confirming whether it serves nfs, mountd, nlockmgr, or status programs, each with distinct program numbers. Nmap's probe file contains specialized RPC probes with rpc rarity classifications that trigger only during this grinding phase.

Scan Intensity Levels: The Time/Accuracy Spectrum

The --version-intensity parameter (0-9) controls a multi-dimensional tradeoff space:

| Intensity | Behavior | Typical Duration Impact | |-----------|----------|------------------------| | 0 (-sV --version-intensity 0) | Softmatches only; no version extraction | Minimal; ~1-2 probes per port | | 1-2 | Most common probes; covers ~60% of services | 2-4 probes per port | | 3-5 (default with -sV) | Standard rarity probes (≤5) | 5-15 probes per port | | 6-7 | Expanded probe set; RPC grinding enabled | 15-40 probes per port | | 8-9 (--version-all) | All probes regardless of rarity; exhaustive RPC grinding | 50+ probes per port; significant time increase |

Intensity selection demands contextual judgment. A perimeter scan of thousands of hosts warrants intensity 3-5; a targeted assessment of a critical database server justifies intensity 8-9. The --version-light flag (intensity 2) and --version-all (intensity 9) provide convenient shortcuts.

Probe timeouts compound this calculus. Nmap applies --version-intensity to both probe selection and timeout patience—higher intensities wait longer for slow services. Adjust --version-timeout independently when you need probe breadth without extended waiting.

Application Parsing, Version Extraction, and Confidence Scoring

Nmap's output reflects nuanced confidence rather than binary certainty. The confidence scoring system operates at multiple levels:

Match confidence derives from regex specificity. A pattern matching Apache/2.4.41 verbatim scores higher than one matching merely Apache. Parenthetical captures with strict character classes ([\w._-]+) elevate confidence over permissive patterns (.*).

CPE (Common Platform Enumeration) assignment adds validation: successful CPE extraction (cpe:/a:apache:http_server:2.4.41) confirms version parseability against known databases. Missing CPE suggests non-standard version strings or custom builds.

Interpreting output critically:

PORT   STATE SERVICE VERSION
80/tcp open  http    Apache httpd 2.4.41 ((Ubuntu))
|_http-server-header: Apache/2.4.41 (Ubuntu)

The parenthetical (Ubuntu) derives from i/ (info) capture, not v/ (version). The pipe-prefixed http-server-header line shows the raw header for manual verification. Discrepancies between VERSION field and script output warrant investigation—Nmap may have refined its match post-initial detection.

When confidence falls below threshold, Nmap appends ? to the service name or emits service unrecognized despite returning data. This occurs with modified banners, patched builds, or proprietary forks that alter identification strings without changing protocol behavior.

Limitations and Critical Verification

Deliberate deception pervades modern infrastructure. Services misreporting versions—common in security-conscious environments—defeat pattern matching without protocol-level behavioral analysis. A patched Apache server returning nginx/1.18.0 in its Server header will misidentify unless Nmap's additional probes (ETag analysis, header ordering, error page inspection) reveal behavioral discrepancies.

Load-balanced backends create apparent version inconsistency: sequential scans hit different servers with divergent software versions. The http-methods or ssl-cert scripts sometimes expose backend diversity through certificate chain differences or allowed method variations.

Honeypot fingerprint randomization actively exploits Nmap's database. Tools like honeyd and commercial deception platforms cycle through service banners, occasionally hitting real version strings but lacking corresponding protocol implementations. Behavioral verification—actually exercising protocol features implied by the version—exposes these: a claimed OpenSSH 8.2 that accepts invalid KEX algorithm negotiations reveals itself.

Manual verification techniques when confidence is suspect:

# Direct banner grab with explicit timeout
nc -w 3 target 22 | head -5

# Protocol-specific behavioral test
ssh -v -o BatchMode=yes -o ConnectTimeout=3 target 2>&1 | grep "Remote protocol"

# TLS certificate examination independent of version detection
openssl s_client -connect target:443 -servername target </dev/null 2>/dev/null | openssl x509 -noout -text | grep -A2 "Subject Alternative Name"

Cross-reference Nmap's output against CPE feeds and vulnerability databases. A service identified with 90% confidence as vsftpd 3.0.3 demands CVE correlation; the same identification at 50% confidence with behavioral anomalies suggests scanning the vendor's actual release hash to confirm build provenance.