The Nmap Scripting Engine: Architecture and Core Libraries

From User to Extender: Why NSE Matters

The Nmap Scripting Engine (NSE) transforms Nmap from a port scanner into a fully programmable reconnaissance platform. Where command-line flags limit you to built-in behaviors, NSE exposes Nmap's scanning infrastructure—host discovery, port state, version detection, and network I/O—to Lua scripts that run concurrently across thousands of targets. This section transitions you from executing others' scripts to understanding how NSE scripts function, evaluate targets, and leverage core libraries. Mastering these foundations prepares you to write your own scripts and critically assess those you inherit.

NSE Architecture: Categories, Rule Phases, and Execution Model

NSE scripts follow a rigid structure with four execution phases defined by rules: prerule, hostrule, portrule, and postrule. These determine when a script executes relative to Nmap's scanning pipeline.

| Rule Type | Trigger Condition | Typical Use | |-----------|-------------------|-------------| | prerule | Once before any host scanning begins | Global setup, reading wordlists, initializing databases | | hostrule | Per host, after host discovery completes | Host-based checks (firewall detection, traceroute analysis) | | portrule | Per open port matching the rule | Service-specific probes, vulnerability checks | | postrule | Once after all host scanning completes | Reporting, statistics, cross-host correlation |

Rules return boolean values; only true triggers the script's action function. The portrule is most common and leverages Nmap's service detection:

portrule = shortport.version_port_or_service({80, 443}, {"http", "https"})

This matches ports 80 or 443 or services detected as HTTP/HTTPS.

Categories organize scripts by purpose: auth, broadcast, brute, default, discovery, dos, exploit, external, fuzzer, intrusive, malware, safe, version, and vuln. The default category runs with -sC or --script=default; safe scripts avoid crashing services; intrusive may trigger IDS alerts.

Concurrency model: NSE achieves parallelism through Lua coroutines, not OS threads. Each script execution is a coroutine that Nmap's engine schedules across its host- and port-parallelism model. When a script yields on network I/O (socket read/write), Nmap suspends that coroutine and executes others, maximizing throughput without blocking. This cooperative multitasking scales efficiently: a single Nmap process may run thousands of concurrent script instances.

Script Selection Mechanisms

The --script argument provides granular control beyond categories:

# By script name (file basename, no .nse extension)
nmap --script http-title target

# By category
nmap --script "discovery and safe" target

# By directory path
nmap --script /custom/scripts/ target

# Boolean expressions
nmap --script "(default or discovery) and not intrusive" target

# Specific scripts with arguments
nmap --script ssh-brute --script-args userdb=users.txt,passdb=passes.txt target

Expressions support and, or, not, and parentheses. Script names can use wildcards: http-* matches all HTTP-related scripts.

Lua Essentials for NSE: Coroutines, Tables, and Standard Libraries

NSE scripts are Lua 5.3 programs. Three language features dominate NSE usage:

Tables serve as NSE's sole data structure—arrays, dictionaries, objects, and namespaces all use tables. Table iteration with pairs (any keys) and ipairs (integer indices) is ubiquitous:

local hosts = {"192.168.1.1", "192.168.1.2"}
for i, host in ipairs(hosts) do
  stdnse.debug1("Scanning %s at index %d", host, i)
end

Coroutines enable the non-blocking I/O model. Scripts rarely manipulate coroutines directly, but understanding coroutine.yield() explains how nmap.new_socket():receive() suspends execution without blocking other scripts.

NSE extends Lua with standard libraries replacing or supplementing Lua's io and os modules for security (no arbitrary file access) and sandboxing.

Core Libraries: nmap, stdnse, shortport, and table

nmap library: Provides host and port objects, socket creation, and registry access.

local socket = nmap.new_socket()
socket:connect(host, port)
socket:send("HEAD / HTTP/1.0\r\n\r\n")
local status, response = socket:receive_lines(1)
socket:close()

Key functions: nmap.fetchfile() locates data files in Nmap's search path; nmap.registry shares data between scripts targeting the same host; nmap.set_port_version() updates service fingerprint confidence.

stdnse library: Utility functions every script uses.

| Function | Purpose | |----------|---------| | stdnse.debug(level, fmt, ...) | Leveled debug output (use --script-trace) | | stdnse.format_output(success, table) | Standardized output formatting | | stdnse.get_script_args(name) | Retrieve --script-args values | | stdnse.make_retry_socket(times) | Socket with automatic retry logic | | stdnse.output_table() | Create properly formatted result table |

shortport library: Simplifies portrule construction with common patterns.

-- Match HTTP on any port, or any service on port 80/443
portrule = shortport.http

-- Match specific port/service combinations
portrule = shortport.port_or_service({21,22,23}, {"ftp","ssh","telnet"})

-- Exclude from default scans unless service matches strongly
portrule = shortport.version_port_or_service(nil, {"mysql"}, "tcp", "open")

table library: Extensions to Lua's standard table operations. table.contains(tab, val) checks membership; table.sort(tab, func) with custom comparators; table.concat() for string assembly.

Network I/O: Sockets, SSL, and Protocol Negotiation

Raw nmap.new_socket() provides TCP and UDP connectivity. For SSL/TLS:

local socket = nmap.new_socket()
socket:connect(host, port)
local status, err = socket:reconnect_ssl()
-- Now read/write encrypted data transparently

The comm library abstracts common protocol negotiation patterns—send a probe, receive response, match against expected patterns:

local comm = require "comm"
local status, response = comm.exchange(host, port, probe, {proto="tcp", timeout=5000})
if status and response:match("^HTTP/1%.[01]") then
  -- HTTP service confirmed
end

comm.tryssl() automatically attempts SSL upgrade for services that may run plaintext or encrypted (common for SMTP, IMAP, POP3).

Credential Testing: brute, creds, and unpwdb

These libraries standardize authentication testing across protocols.

brute library: Framework for credential-guessing scripts. You provide Driver class implementing login() and check() methods; brute handles threading, resume capability, and credential iteration.

local brute = require "brute"
local creds = require "creds"
local unpwdb = require "unpwdb"

-- Initialize username/password iterators
local usernames, pwds = unpwdb.usernames(), unpwdb.passwords()

local Engine = {
  new = function(self, host, port)
    local o = { host = host, port = port }
    setmetatable(o, self)
    self.__index = self
    return o
  end,
  
  login = function(self, username, password)
    local socket = nmap.new_socket()
    -- Protocol-specific authentication attempt
    -- Return true, creds.Account on success; false on failure
  end,
  
  check = function(self)
    -- Verify service is suitable for brute-forcing
    return true
  end
}

local status, result = brute.new(Engine, host, port, { usernames = usernames, passwords = pwds }):start()

creds library: Represents and stores discovered credentials with creds.Account:new(username, password, state), where state is creds.State.VALID, INVALID, or LOCKED. Accounts populate Nmap's credential database, accessible across scripts via nmap.registry.creds.

unpwdb library: Provides iterators over built-in username and password lists, user-specified files via --script-args userdb=..., or single values.

Documentation Standards and Argument Parsing

Every NSE script requires structured documentation comments parsed by nmap --script-help:

---
-- @usage nmap --script http-example --script-args http-example.path=/admin
-- @output
-- PORT   STATE SERVICE
-- 80/tcp open  http
-- | http-example:
-- |   Title: Administration Panel
-- |   Server: nginx/1.18.0
-- @xmloutput
-- <elem key="title">Administration Panel</elem>
-- <elem key="server">nginx/1.18.0</elem>

| Element | Purpose | |---------|---------| | @usage | Example command lines | | @output | Expected human-readable output (pipe-delimited) | | @xmloutput | Corresponding XML structure for -oX output | | @args | Documented --script-args parameters |

Argument parsing in action():

local path = stdnse.get_script_args(SCRIPT_NAME .. ".path") or "/"
local timeout = tonumber(stdnse.get_script_args(SCRIPT_NAME .. ".timeout")) or 10000

The SCRIPT_NAME variable is automatically set to the script's basename.

Complete Script Walkthrough

Here's a minimal script demonstrating structure from rule through action:

local http = require "http"
local shortport = require "shortport"
local stdnse = require "stdnse"

-- Rule: execute against HTTP services
portrule = shortport.http

-- Action: fetch and report server header
action = function(host, port)
  local path = stdnse.get_script_args(SCRIPT_NAME .. ".path") or "/"
  local response = http.get(host, port, path)
  
  if not response then
    return nil
  end
  
  local server = response.header["server"] or "unknown"
  return string.format("Server: %s", server)
end

Execution flow: Nmap scans target → portrule evaluates true for open HTTP port → coroutine created with action(host, port) → http.get() yields on socket I/O → other scripts execute → response received → coroutine resumes → return value formatted for output.

Understanding this pipeline—from rule matching through coroutine scheduling to library utilization—provides the foundation for evaluating existing scripts and eventually authoring your own.