Internals
Architecture
This document describes the architecture of the subenum tool, a Go-based command-line utility for subdomain enumeration. It describes responsibilities and names the identifiers involved; for exact signatures see the package docs on pkg.go.dev (for example internal/scan and internal/dns) or the source.
1. Overview
The subenum tool operates through a sequence of steps to discover valid subdomains for a given target domain:
- Initialization: Parses command-line arguments (flags may appear before or after the domain), normalizes the target domain (or reads a list of targets with
-dL), and validates the scan options. - Wordlist Ingestion: Reads and normalizes the wordlist once, then takes each target’s entries from it (only the name-length limit differs per target).
- Preflight: Resolves the target apex once; a resolver that times out, refuses or errors stops the scan before any candidate is sent.
- Wildcard Detection: Probes random subdomains of the target and keeps their answers as a fingerprint. The scan stops unless
-forceis set; with-force, results matching the fingerprint are filtered. Recursive scans also probe each new parent before expanding it. - Concurrent Resolution: A dispatcher generates candidate names (
prefix.target) lazily and feeds a pool of worker goroutines, which resolve each candidate over DNS. - Output: Resolved subdomains are printed to stdout (pipe-friendly); all progress, verbose, and diagnostic output goes to stderr.
- Completion: The tool waits for all lookups to finish, prints a per-outcome breakdown, finalizes structured output and the
-ofile, and exits with a documented exit code.
This architecture is designed to be efficient by performing multiple DNS lookups concurrently, while also providing control over concurrency, timeouts, and the query rate.
Package Structure
Every package under internal/ and pkg/, summarized from its // Package comment:
| Package | Purpose |
|---|---|
internal/scan |
Runs a scan: validates options, dispatches candidates to a worker pool, filters wildcard answers, grades the run |
internal/dns |
Resolves candidate names, classifies lookup outcomes, detects wildcards; transports, resolver pool, rate limiting, takeover hints, simulation |
internal/output |
Renders results (text, JSON, JSONL, CSV) to stdout and the -o file; run-quality summary, SARIF, ETA |
internal/report |
Builds the run-quality report from finished scans, shared by the CLI and the TUI |
internal/wordlist |
Reads wordlist files and normalizes their entries into candidate prefixes |
internal/validate |
Input validators shared by the CLI and the TUI (domain, resolver address, punycode) |
internal/permute |
Generates second-pass candidates from names a scan has already found |
internal/candidates |
A target’s candidate set: the wordlist plus the names sources and zone transfers propose, which source proposed each result, and the -permute seeds and candidates (shared by the CLI and the TUI) |
internal/diff |
Compares two runs’ results: new and missing names, changed records (shared by -diff and pkg/subenum’s Diff) |
internal/cliargs |
Parses command lines whose flags may follow positional arguments, shared by subenum and wordlist-gen |
internal/ct |
Looks up subdomain names in Certificate Transparency logs through crt.sh |
internal/passive |
The Source interface behind -sources, the source registry, the fan-out that asks every source at once and the merge of source names into candidates (shared by the CLI, the TUI and pkg/subenum), and the Wayback Machine source |
internal/retryhttp |
Sends the passive sources’ HTTP requests, retrying 429, 502, 503, 504 and transport errors |
internal/axfr |
Zone transfer attempts and the DNSSEC denial probe within fixed bounds (shared by -axfr and pkg/subenum’s ProbeZone) |
internal/labzone |
Parses -simulate-zone scenario files for lab mode |
internal/mcp |
Minimal Model Context Protocol server (JSON-RPC over stdio, tools only) behind subenum mcp |
internal/libhook |
Lets subenum mcp run a pkg/subenum scan with engine settings the library’s Config does not expose; pkg/subenum registers its scan in an init |
internal/dnsserver |
Scriptable local DNS server backing lab mode and the hermetic tests |
internal/dnstest |
Runs internal/dnsserver for tests: stopped on cleanup, listen failures fail or skip the test |
internal/cli |
The subenum command: flag parsing, the per-run session and its subsystems (config, diff, resume, permute, passive sources, labs, MCP); its files are listed below |
internal/tui |
Bubble Tea terminal UI |
pkg/subenum |
Public API that embeds the same engine in other Go programs |
Files, by role:
main.go - Entry point: passes the injected Version to cli.Main
internal/cli/cli.go - CLI entry point: run, flag parsing and validation, scanTarget, exit codes
internal/cli/session.go - runWith and the session: per-run state, the -dL target loop, -tui dispatch
internal/cli/completion.go - completion and man subcommands; dispatches diff
internal/cli/config.go - config file and SUBENUM_* environment defaults, -print-config
internal/cli/ct.go - -ct / -sources: passive source names as extra candidates
internal/cli/axfr.go - -axfr: zone transfer names as extra candidates, with resume and messages
internal/cli/prefetch.go - a target's source lookups and zone transfer in one goroutine; -dL fetches the next target's while one scans
internal/axfr/axfr.go - Try: AXFR from each name server and the denial probe (-axfr, subenum.ProbeZone)
internal/cli/diff.go - -diff / -diff-records: compare against a previous results file; runDiff: the diff subcommand; compareBaselines, shared with the MCP diff tool
internal/cli/dryrun.go - -dry-run: print the scan plan and its cost without sending queries
internal/cli/lab.go - -simulate-zone: serve a lab scenario on a loopback DNS server
internal/cli/mcp.go - subenum mcp: its flags, the server, and the lab scenarios lab_scan serves
internal/cli/mcp_tools.go - the lab_scan, scan, explain_stats and diff tools: input schemas, argument checks, the allow-list and caps
internal/cli/mcp_report.go - the scan and diff reports the tools return, the scan verdict and their outputSchemas
internal/cli/permute.go - -permute / -seeds: the permutation pass
internal/cli/ptr.go - -ptr: PTR lookups of the public addresses found, and the pass over the names they give
internal/cli/resume.go - -resume / -state: save and replay an interrupted run
internal/cli/results.go - resultLog: a target's results, kept for -permute seeds and -diff
internal/cli/summary.go - assemble the run-quality report (-stats, JSONL summary line)
internal/cli/explain.go - parseSummary, explainSummary: a run-quality report in plain language (explain_stats)
internal/cli/wordlist.go - prepareWordlist: the -w file, or the bundled list when -w is not given
internal/scan/options.go - Options: Config plus the settings only checked, shared by CLI, TUI and library (Validate, Fields)
internal/scan/runner.go - Scan engine: Config, Run, preflight, the workers and job processing
internal/scan/events.go - Event, EventKind, NoticeKind, Result: what Run sends
internal/scan/stats.go - Stats, Repeat: the run's counters (a -resume file format)
internal/scan/pending.go - The inconclusive list and the -retry-inconclusive pass
internal/scan/delegation.go - parentZone: ask the zone's name servers about a SERVFAIL name's delegation (dangling-ns)
internal/scan/wildcard.go - Wildcard cache, fingerprint and near-miss matching
internal/scan/dispatcher.go - dispatcher: lazy candidate generation in chunks, budgets, resume watermark
internal/scan/autorate.go - autoRate: the -auto-rate back-off controller
internal/scan/scope.go - -exclude patterns (ValidateExcludes, IsExcluded)
internal/scan/settings.go - Setting tokens and the one table naming each setting as flag, Config field and form label (Render, Names)
internal/scan/verdict.go - Run-quality verdicts (Stats.Verdict, VerdictRank)
internal/scan/verify.go - Verify: re-check names against the trusted resolver (used by -diff)
internal/dns/resolver.go - NewResolver, Lookup (ResolveTypes, Resolve, CheckWildcard,
FingerprintWildcard), Classify, ParseTypes
internal/dns/cname.go - lookupCNAME: a single CNAME query, without implicit A/AAAA lookups
internal/dns/transport.go - UDP (TCP fallback), DNS over TLS and DNS over HTTPS dialers
internal/dns/axfr.go - Transfer: an AXFR zone transfer over TCP (-axfr)
internal/dns/nsec.go - ProbeDenial: NSEC, NSEC3 or no DNSSEC proof of absence (-axfr, -nsec-walk)
internal/dns/nsecwalk.go - WalkNSEC: list a plain-NSEC zone by following its chain (-nsec-walk)
internal/dns/referral.go - AskDelegation: a non-recursive NS query for a name's delegation (dangling-ns)
internal/dns/pool.go - Pool: the -r resolver pool (health benching, canaries, second opinions)
internal/dns/ratelimit.go - RateLimiter, WithLimiter (per-wire-query pacing)
internal/dns/takeover.go - TakeoverProvider, TakeoverHint, HintsTakeover (dangling or takeover-prone CNAMEs, dangling NS delegations)
internal/dns/scope.go - AddressScope, InternalScope: non-public address scopes (private, loopback, cgnat, ...)
internal/dns/simulate.go - Simulation.Resolve (seeded synthetic DNS)
internal/output/writer.go - Thread-safe Writer (results to stdout, diagnostics to stderr)
internal/output/file.go - File: atomically replaced -o results file; WriteFile for the side files
internal/output/summary.go - Summary / TargetSummary, WriteSummaryFile
internal/output/sarif.go - WriteSARIF: takeover candidates as SARIF 2.1.0
internal/output/eta.go - ETA: progress estimate over a sliding window, FormatETA
internal/output/log.go - -log-format: text or JSON diagnostics on stderr (ParseLogFormat, SetLogFormat)
internal/report/report.go - New, Target, Sources, Finish, Build: the run-quality report; Status, a target's status
internal/report/permutation.go - Permutation: a target's -permute pass in the report (CLI and TUI)
internal/permute/permute.go - Generate: permutations of found names
internal/candidates/candidates.go - Set (New, Add, SourcesOf), RelativeSeeds, Permutations: candidates for the CLI and the TUI
internal/diff/diff.go - Name, RecordsChanged: the comparison behind -diff-records and subenum.Diff
internal/ct/ct.go - Client.Names: crt.sh lookup
internal/passive/passive.go - Source, Known, ParseList, New: the -sources registry; Merge: source names into candidates
internal/passive/fetch.go - Gather, FetchAll, Lookup: the source fan-out the CLI, TUI and pkg/subenum share
internal/passive/wayback.go - Wayback.Names: hostnames from the Wayback Machine's CDX index
internal/mcp/mcp.go - Server.Serve: MCP over stdio (initialize, ping, tools/list, tools/call, cancellation)
pkg/subenum/sources.go - Source, CertificateTransparency, WaybackMachine: Config.Sources
pkg/subenum/zone.go - ProbeZone, ZoneTransfer: zone transfers and the denial probe
internal/labzone/labzone.go - Load, Parse: lab scenario files
internal/dnsserver/server.go - Listen: scriptable loopback DNS server
internal/dnstest/dnstest.go - Start, StartWith, StartTLS: test helpers around dnsserver
internal/validate/validate.go - DNSServer, Domain, NormalizeDomain, DefaultDNSServer
internal/validate/punycode.go - IDN to punycode (A-label) conversion
internal/cliargs/cliargs.go - Parse: flags anywhere on the command line, "--" terminator
internal/wordlist/reader.go - ReadLines, ReadLinesFrom, Normalize, Build, Prepare / Prepared.For, NotIn, LoadWordlist
internal/tui/model.go - Root Bubble Tea model (form -> scan state machine)
internal/tui/form.go - Config form screen (textinput fields + toggles)
internal/tui/scan_view.go - Live results screen (viewport + progress bar)
internal/tui/passes.go - Sources, Zone Transfer, NSEC Walk and Permute: source and name server lookups before the scan, the permutation pass after it
internal/tui/logo.go - Styled wordmark
internal/tui/config.go - Session persistence (load/save <user config dir>/subenum/last.json)
pkg/subenum/subenum.go - Scan, Run, DefaultWords, Permutations: the embeddable API
pkg/subenum/diff.go - Diff, Changes: compare two sets of results
tools/wordlist-gen.go - Standalone wordlist generator
tools/licenses/main.go - THIRD_PARTY_LICENSES for release archives and the image
2. Key Components / Modules
2.1. Argument Parsing (internal/cli/cli.go, internal/validate)
- Purpose: This component is responsible for processing the command-line arguments provided by the user when
subenumis executed. It extracts the target domain (or a-dLdomain list), the wordlist path, and every scan, output, and tuning option. - Implementation:
run()only callsrunWith(runArgs), which takes the arguments, stdin, stdout and stderr as injected values so tests can drive the whole CLI in-process. Per-run state lives in asession(internal/cli/session.go) whose methods are the run’s phases:parse,configure,loadInputs,openOutputs,scanAll(which callsscanOneTargetper target; once a target’s passive sources and zone transfer are in, the next-dLtarget’s are fetched while it scans, their messages held until that target starts) andfinish.cliFlagsholds only parsed flag values; what the run derives from them (attempts, record types, excludes, the resolver pool, the-max-durationdeadline) is inruntimeInputs, and each pass over a target is ascanPass.runWithfirst handles thecompletion,diff,manandmcpsubcommands and-resume, then builds aflag.FlagSetwithparseFlagsWith, which binds every flag into acliFlagsstruct and applies config-file and environment defaults (see section 5). The standardflagpackage stops at the first non-flag argument, socliargs.Parse(internal/cliargs, shared withtools/wordlist-gen.go) re-parses around each positional; flags written after the domain are honored, and everything after a--terminator is positional.runWiththen:- handles
-tui(launchestui.Startwith the form pre-filled from the domain, the flags it has fields for and the-exclude-filepatterns, warning about the rest on stderr and on the form),-print-configand-versionfirst; - parses
-format(output.ParseFormat),-type(dns.ParseTypes), and merges-attemptswith the deprecated-retries(effectiveAttempts); - calls
validateFlags, which checks that a target is present, that a positional domain and-dLare not both given, and that-wand-dLdo not both read stdin (-wmay repeat, but only one entry may be-). Numeric ranges and the resolver address are checked byscan.Options.Validate, the same validator the TUI uses. The positional domain is canonicalized withvalidate.NormalizeDomain.
- handles
- Domain normalization:
validate.NormalizeDomaintrims and lowercases the input, strips a URL scheme and path, a:port, a leading*.and a trailing dot (returning a note for each change), converts IDNs to punycode, and checks the result withvalidate.Domain.validate.DNSServerchecks the resolver:ip:port,tls://host[:port]orhttps://host/path. Both are shared with the TUI. - Target list (
-dL):loadTargetsreturns either the single positional domain or the entries of the-dLfile (-for stdin). Blank lines,#comments and duplicates are skipped; invalid domains are reported and skipped. An empty result is an error. - Interactions:
scanOptionsmaps the parsed flags onto ascan.Options, which embeds thescan.Configpassed to the scan engine.
2.2. Wordlist Processing (internal/wordlist)
- Purpose: This component is responsible for reading, sanitizing, validating, and deduplicating the subdomain prefixes from the user-specified wordlist.
- Implementation:
wordlist.ReadLinesreads every line of a file, or of standard input when the path is-(wordlist.Stdin);wordlist.ReadLinesFromdoes the same for an open reader, such as the wordlist embedded in the binary. It strips a leading UTF-8 byte order mark and truncates (and drains) absurdly long lines instead of failing.wordlist.Normalizeturns one raw line into the prefix that will be queried: trimmed, lowercased, trailing dot removed. Blank lines,#comments, and entries that are not valid DNS labels are rejected. Multi-label prefixes such asdev.apiand service labels such as_dmarcare kept.wordlist.Buildnormalizes all lines for one target domain, preserving first-occurrence order. It reports the number of case-insensitive duplicates and of skipped entries (invalid, or whose full name under the target would exceed 253 characters).wordlist.Preparenormalizes the lines once into aPreparedlist, andPrepared.For(domain)returns one target’s entries with the same countsBuildgives, so a-dLrun does not normalize the wordlist again per target.wordlist.LoadWordlistcombinesReadLinesandBuild; the TUI uses it.
- Interactions: The CLI reads the wordlist once with
ReadLines(stdin can only be read once) before creating the-ofile, so a bad-wpath never truncates an existing results file. It callsPrepareonce, thenPrepared.Forper target, since the name-length limit depends on the domain. The resulting slice becomesscan.Config.Entries. Skipped counts are always reported; duplicate counts in verbose mode.
2.3. DNS Resolution Engine (internal/dns)
- Purpose: This is the core component responsible for performing the DNS lookups for each candidate name and classifying what happened. It also provides wildcard detection and wire-level rate limiting.
- Implementation:
- Resolver and transports:
dns.NewResolverbuilds anet.ResolverwithPreferGo: truewhoseDialhook always connects to the configured server through a transport chosen from the-dns-servervalue (dns.Transport,internal/dns/transport.go):ip:port- UDP, with UDP sockets reused between queries (at most 512 idle, a budget an-rpool shares across its members); a truncated answer (TC=1) falls back to TCP, which dials per query.tls://host[:853]- DNS over TLS (RFC 7858), over a small pool of reused TLS connections.https://host/path- DNS over HTTPS (RFC 8484), each query sent as an HTTP POST ofapplication/dns-message.
scan.Runbuilds one resolver per scan (unlessConfig.Resolveris set) and reuses it for every lookup;dns.CloseIdleConnectionsreleases its pooled connections when the scan ends. - Resolver pool (
-r):dns.NewPoolFromLinesparses the-rfile (iporip:port, port 53 by default) and builds adns.Poolover it with the-rate-per-resolvercap; the TUI andpkg/subenumbuild their pools with it too. Candidate lookups go round-robin over the healthy members; the-dns-serverresolver stays the trusted resolver and runs the preflight and wildcard probes. Safeguards:- Health benching: over each window of 20 lookups, a member that failed (timeout, refused, other) more than half of them is benched for 30 seconds, then tried again.
- Per-resolver rate cap (
-rate-per-resolver):Pool.SetRatePerResolverspaces the queries sent to each member like-ratedoes globally. A lookup prefers a member with room, waits for the one that frees up soonest when every cap is spent (before its timeout starts), and books one slot per record type; canaries and second opinions book slots too. Each spent cap a lookup runs into counts asthrottled. The global-ratelimiter still applies, so the stricter of the two wins. - Hit re-validation: every pool hit is resolved again on the trusted resolver, and the trusted answer is the one reported (
resolveViaPoolininternal/scan/runner.go), so a poisoned member cannot inject names. - Canaries and second opinions: names the trusted resolver confirmed (the apex, then each confirmed hit) become canaries. A member is asked a canary every 25 lookups (before each lookup while it is untrusted); one that has not answered its last canary correctly has its NXDOMAIN answers checked by a second member.
- Flagging: every contradiction counts against the member; three take it out of rotation for the rest of the run. If every member is flagged, the trusted resolver answers alone.
Per-member counts appear in the run-quality report.
- Lookups: a
dns.Lookupvalue names the resolver, per-type timeout, attempt count, record types and verbose logger, and the lookup helpers are its methods, so a swapped argument cannot compile.Lookup.ResolveTypeslooks up the requested record types concurrently, with one query per type (A and AAAA viaLookupIP, CNAME via the package’s ownlookupCNAME, NS, MX and TXT viaLookupNS,LookupMXandLookupTXT, an MX value being<preference> <host>; defaultdns.DefaultTypes, A and AAAA) and returns typeddns.Record{Type, Value}results, the time spent, and the most severe error. Each type gets its own timeout, so a slow A answer cannot starve AAAA or CNAME. Names are queried as absolute (trailing dot) so resolv.conf search suffixes never multiply queries.Lookup.Logfreceives verbose per-lookup lines (the package never writes to stderr itself;dns.StderrLogfis the unsynchronized fallback). - Retries and classification:
Lookup.Resolvelooks the name upLookup.Attemptstimes at most (at least once) with a short linear backoff, and returns the records plus adns.Outcome:OutcomeFound,OutcomeNXDomain,OutcomeTimeout,OutcomeRefused,OutcomeOther, orOutcomeCanceled.dns.Classifymaps a resolver error onto an outcome. NXDOMAIN (including NODATA, a name with no records of the requested types) is definitive and not retried; any infrastructure failure on one type outranks a not-found answer on another, so a SERVFAIL is never mistaken for NXDOMAIN. - Rate limiting:
dns.RateLimiterhands out evenly spaced reservations;dns.WithLimiterattaches it to a context.Lookup.ResolveTypesreserves one slot per query a lookup is expected to send (one per type: CNAME is sent as its own query rather than through Go’sLookupCNAME, which would add A and AAAA queries, #249) before that lookup’s timeout starts. A per-lookupwireBudgettravels on the context into theDialhook: Go’s resolver dials once per query, so any dial beyond the reserved slots (Go’s own internal retries) waits for another slot.-ratetherefore bounds queries on the wire, including retries, wildcard probes and the preflight. The same hook notices REFUSED responses, which Go reports only as “server misbehaving”, and names the server actually dialed in errors. - Wildcard detection: probes resolve random 32-character hex labels (
dns.RandomLabel) under a domain for the scan’s record types, each with the scan’s-attempts, on the trusted resolver. If any probe resolves, the domain is a wildcard and the union of the answers is its fingerprint.Lookup.FingerprintWildcardchecks the scan root, whose fingerprint filters every result. It starts with five probes. If the fingerprint grew after the first probe (a wildcard served from a rotating CDN or load-balancer pool), it keeps probing until the fingerprint has not grown for max(8, 4 x its size) consecutive probes, up to 128 probes in all, so the whole address pool is learned rather than the few addresses the first probes happened to see.Lookup.CheckWildcardis the cheap variant used for each new parent in a recursive scan: two probes, no saturation.- Inconclusive: a check is conclusive only when every probe ended found or NXDOMAIN/NODATA. Any timeout, refusal or other failure ends the check with an error wrapping
dns.ErrWildcardInconclusive, which callers must never treat as “no wildcard”: the scan stops unless-forceis set, and then it runs without a fingerprint to filter with. - The worker’s near-miss revalidation (see 2.4) keeps adding to the fingerprint during the scan when a rotating pool shows addresses the probes missed.
- Simulation:
dns.Simulation.Resolvederives a deterministic result from the name and the-seed, with-hit-ratepercent of names resolving to synthetic records, so the same seed reproduces the same scan. No packets are sent.
- Resolver and transports:
- Interactions: Workers,
scan.Verifyand the near-miss probes resolve a name through one entry point,Config.resolve(the test hook, elsedns.Simulation.Resolveunder-simulate, elseLookup.Resolveon the trusted resolver); workers layer the-rpool on top. They use the returned records and outcome; the records become anEventResult, the outcome feeds theStatscounters and the reliability guard.
2.4. Concurrency Management (internal/scan)
- Purpose: To efficiently perform DNS lookups for a large number of potential subdomains,
subenumemploys a worker pool pattern. This allows multiple DNS queries to be in flight concurrently, significantly speeding up the enumeration process compared to sequential lookups. - Implementation: The scan engine lives in
internal/scan/runner.goasscan.Run, which takes a context, ascan.Config, and an events channel. Both the CLI (scanTargetininternal/cli/cli.go) and the TUI call it.scan.Options/scan.Config:Optionsholds the user-facing settings: it embeds thescan.Configthe engine runs on, plusMaxDurationandRatePerResolver, which are only checked there (#331). ItsValidatemethod checks ranges and the resolver address, and itsConfigfield is whatscan.Runtakes. The CLI, the TUI andpkg/subenumeach fill in anOptions, so they cannot drift apart in what they accept, and each has a test that walksOptions.Fields()and fails when a setting is left unset without a listed reason.scan.DefaultConcurrency,DefaultTimeoutMsandDefaultHitRateare the defaults all three use.scan.Event/scan.EventKind: Typed events emitted on the channel:EventResult(ascan.Result: subdomain, records, simulated flag, takeover hint and internal address scope),EventProgress,EventNotice,EventError, andEventDone. AnEventNoticecarries ascan.NoticeKind(NoticeWildcard,NoticeCap,NoticeCeiling,NoticeSkip,NoticeRate) so consumers can style or filter notices without parsing the message.EventDonecarries ascan.Statssnapshot with per-outcome counters (found, nxdomain, timeout, refused, other, wildcard-filtered).- Preflight and wildcard check: Before any candidate is dispatched,
scan.Runresolves the apex (with retries) and runsLookup.FingerprintWildcard; see the Data Flow section. - Dispatcher and lazy candidate generation: A dispatcher goroutine owns the internal
jobschannel, which carries chunks of jobs rather than single jobs:nextChunkhands each send up to one worker’s share of what is left, at mostmaxChunk(8), so a large scan pays one channel operation per chunk and the tail still spreads over every worker.jobsandcompletedare buffered (handoffPerWorker= 4 sends per worker), so workers and the dispatcher rarely wait on each other. The dispatcher also keeps a queue of expansions (a parent name plus a cursor intoConfig.Entries), the set of parents already queued (recursive scans only), and a pending-work counter. A set of generated candidates is kept only for a recursive scan whose entries include a multi-label one, the only case where two parents can reach the same name, and it holds only the names a multi-label entry begins (a.b.cfor the entrya.b), since no other name can be reached twice. Candidate names are built only as they are dispatched, so a found parent costs one queue entry rather than one per wordlist entry, and the queue is compacted as it is consumed.-max-queriesstops admission once the cap is reached; the remaining candidates are counted, never built, and reported in aNoticeCapnotice. - Recursion ceiling: When
-recursiveis set,scan.RecursionCeilingcomputes the theoretical job count (sum n^d). Above 1e7 the scan refuses to start unless-max-queriesor-forceis set; otherwise aNoticeCeilingwarning is emitted. - Worker goroutines: The pool size is
Config.Concurrency, capped at the most jobs the scan can produce. Each worker takes a chunk and, for every job in it (which already holds the full domain), waits on the rate limiter if the scan is simulated, resolves it, classifies the outcome, and checks the result against the wildcard fingerprint. - Near-miss revalidation: A result whose records are all in the fingerprint is a wildcard answer and is counted as wildcard-filtered. A result that shares some records with it is a near miss, typical of a rotating wildcard pool: two fresh random labels under the same parent are probed, their answers join the fingerprint, and the result is checked again. Results sharing nothing with the fingerprint cost no extra queries.
- Reliability guard: Once 200 jobs have completed, if more than 20% failed as timeout/refused/other, the scan emits
EventErrorand cancels unless-no-abortis set (then it only warns). Lookups cut short by cancellation are not counted, so an interrupt cannot trip the guard. - Adaptive rate (
-auto-rate):autoRate(internal/scan/autorate.go) judges the last 200 lookups. When more than 20% failed it halves the limiter’s rate (starting from the rate achieved so far when-rateis unlimited), emits aNoticeRatenotice and starts a new window; a fully healthy window (at most 5% failures) raises the rate by a quarter, up to where it first backed off. While it can still slow down the reliability guard stands aside; after six back-offs, or at the 5 queries-per-second floor, it gives up and the guard applies as usual.Stats.RateBackoffsandStats.FinalRatereport the result. - Recursive enumeration (optional): when
Config.Recursiveis set and a job below the depth cap resolves, the worker first checks the new parent withLookup.CheckWildcard(cached per parent; inconclusive results are not cached). A wildcard or inconclusive parent is not expanded and aNoticeSkipnotice is emitted. Otherwise the parent is handed to the dispatcher, which expands each parent once and, when the entries include multi-label ones, deduplicates the children (loop and duplicate protection); the progress total grows as new work is admitted. - Progress ticker: A separate goroutine emits
EventProgressonce per second. The total is read atomically since recursion can expand it mid-scan. - Rate limiter (optional): one
dns.RateLimiterpaces the whole scan. Live scans attach it to the context withdns.WithLimiter, so every wire query takes a slot (see 2.3). Simulated scans have no wire traffic and take one slot per job.0means unlimited. - Retry pass (
-retry-inconclusive): workers record each name whose lookup timed out, was refused or failed. Once every worker has exited,retryPendinglooks each of them up once more against the trusted resolver, with up toConcurrencygoroutines. A name that now resolves is reported like any other result but not expanded recursively. The pass is skipped after a guard abort, an interrupt or a spent-max-duration.Stats.RetriedandStats.Recoveredcount it, and the names still without an answer are listed inStats.Inconclusivewhether or not the pass ran, whenConfig.ListInconclusiveis set: the CLI sets it for-inconclusiveandpkg/subenumalways does. Without it and without the pass the names are only counted, and the pass alone drops them once it has run, so failed lookups do not hold memory for a list nobody reads. A wordlist entry is kept as its index (pendingLookup); only a recursive child keeps its name. - Completion: a
sync.WaitGroupwaits for all workers (after the dispatcher closesjobs), the optional retry pass runs, then the progress ticker is stopped andEventDoneis emitted.
- Interactions:
scan.Runis the single entry point for scanning used by both the CLI output pipeline and the Bubble Tea TUI. It decouples the scan engine from any specific display layer.
2.5. Output Formatting (internal/output)
- Purpose: Thread-safe output that keeps stdout pipe-clean. Resolved subdomains go to stdout; everything else (progress, verbose diagnostics, notices, errors) goes to stderr.
- Implementation:
output.Writerwith mutex-protected methods, created withoutput.New(stdout plus an optional file) oroutput.NewFile(file only; used by the TUI so the alt-screen viewport keeps the terminal):Emit- writes oneoutput.Result(the engine’sscan.Resultplus the CLI’sChange,PreviousRecordsandPermutation; one result type flows from engine to writers). Intextformat it printsFound: <domain>on a terminal, or the bare name when stdout is piped (SetPlain), and the bare name to the output file;-show-records(SetShowRecords) appendsTYPE=valuepairs. Injsonformat results are streamed as array elements as they arrive, andFinishcloses the array;jsonlstreams one object per line;csvstreamssubdomain,type,valuerows with a header, plus asimulatedcolumn under-simulate, atakeover_candidatecolumn when-typeincludesCNAMEorNS, achangecolumn with-diff, and, last, asourcescolumn (each result’s sources joined with;). The format is selected with-format text|json|jsonl|csv(output.ParseFormat, defaulttext).Progress/ProgressDone- a carriage-return progress line on stderr, or whole lines whenSetProgressLinesis on.Info/Error- informational and error lines on stderr, cleared around the progress line.Finish- closes the streamed JSON array, flushes CSV, and returns the first error writing the output file.
- Simulated output marking: under
-simulateevery result is marked so invented names never pass for real recon output: text stdout printsFound (SIMULATED): ..., a text results file starts with a# SIMULATEDheader naming the seed (SetSeed), JSON and JSONL objects carry"simulated": true(scan.Result.Simulated, embedded inoutput.Result), and CSV gets asimulatedcolumn. - Atomic results file:
output.CreateFilereturns anoutput.Filethat writes to a temporary file next to the target.Close(true)renames it over the target;Close(false), or a failed flush or close, discards it. The CLI commits only if at least one scan finished, so a run that fails before scanning (preflight, wildcard abort, Ctrl+C) leaves an existing results file unchanged. Targets that are not regular files (such as/dev/stdout) are written directly. - Verbose Output (
-v): configuration summary, per-lookup resolution lines (throughInfo, serialized with the progress line), and final statistics. - Progress Reporting (
-progress, default on): drawn on stderr only when stderr is a terminal, unless-progresswas given explicitly (then printed as whole lines).
- Interactions: All CLI output goes through the
Writer. Since results are the only thing on stdout, and text results are bare names when piped,subenum ... | sort -uworks without-progress=falseor any post-processing.
2.6. Progress Monitoring
- Purpose: This component tracks the progress of the subdomain enumeration process and provides real-time feedback to the user via stderr.
- Implementation:
- Total Count: Starts at the number of wordlist entries for the target. The dispatcher republishes it as recursive expansions are admitted, capped at
-max-queries. - Counters:
scan.Runkeeps atomic processed and found counts plus per-outcome counters, which are snapshotted intoscan.StatsforEventDone. - Progress Display: The ticker goroutine in
scan.RunemitsEventProgress(processed, total, found) every second; the CLI turns it intoWriter.Progresson stderr and the TUI into its progress bar.
- Total Count: Starts at the number of wordlist entries for the target. The dispatcher republishes it as recursive expansions are admitted, capped at
- Interactions: The counters are updated by the worker goroutines with atomic operations, so they are safe to read from the ticker. Writing to stderr keeps stdout pipe-clean.
2.7. Session Persistence (internal/tui/config.go)
- Purpose: Remember the last-used TUI form values across sessions so users don’t have to re-type domain, wordlist path, and scan parameters every time.
- Implementation:
savedConfigstruct mirrors the form values with JSON tags. The form’svalidatefills one by looping overformFields, each row storing its value through itssavedpointer, so no field is copied by hand.configPath()- returnsos.UserConfigDir()/subenum/last.json(e.g.~/.config/subenum/last.jsonon Linux,%AppData%\subenum\last.jsonon Windows).saveConfig- marshals the form values to JSON, writes them to a temporary file in the same directory, and renames it into place, so a second instance reading at the same moment never sees half-written JSON. It is called when a scan starts; errors are discarded so a write failure never blocks the scan.loadSavedConfig- reads and unmarshals the file. It reportsfalseif the file doesn’t exist or is unreadable, causingnewFormModelto fall back to hardcoded defaults.
- Interactions:
tui.New()callsloadSavedConfig()on startup and passes the result tonewFormModel. Therkeybind (new scan) also callsloadSavedConfig()so the form is pre-filled with the values from the scan that just completed. The form validates throughscan.Options.Validateand normalizes the domain withvalidate.NormalizeDomain, exactly like the CLI.
3. Data Flow
The flow of data through the subenum application can be summarized as follows:
- Input: The user provides command-line arguments: the target domain or a
-dLdomain list, the wordlist (-w,-for stdin), concurrency (-t), timeout (-timeout), DNS server (-dns-server), attempts (-attempts), output options (-o,-format,-show-records), and scan tuning (-rate,-type,-recursive,-depth,-max-queries,-no-abort,-force), plus-v,-progress, and-simulate(-hit-rate,-seed). The TUI (-tui) gathers the equivalent values from its form instead. - Configuration: These arguments are parsed and validated by the Argument Parsing component; the targets are normalized and deduplicated. The wordlist is read once.
- Per-target scans: Each target is an independent scan with its own wordlist build, preflight, wildcard check,
-max-queriesbudget, rate limiter, and reliability guard. All targets share oneWriter, so-format jsonis still a single array and CSV has a single header. With several targets, each is announced on stderr and a status table (ok,failed,skipped,not run) is printed at the end. - Preflight: Inside
scan.Runthe apex is resolved once (with retries, skipped in simulation mode). Any outcome other than Found or NXDOMAIN emitsEventErrornaming the resolver and returns. - Wildcard detection:
Lookup.FingerprintWildcardprobes random labels under the target (skipped in simulation mode). If the check is inconclusive the scan stops, unless-forceis set, in which case it scans without filtering and says so in aNoticeWildcardnotice. If a wildcard is detected, aNoticeWildcardnotice is emitted and the scan stops unless-forceis set; with-force, later results matching the fingerprint are filtered. - Dispatch: The dispatcher generates
prefix.targetjobs lazily from the root expansion (a recursive scan expands each found parent once) and feeds them to the bufferedjobschannel in chunks of up to eight. It does not closejobsuntil no candidates remain and the pending counter drains to zero, or the context is cancelled. - Resolution: Worker goroutines read jobs and call
Lookup.Resolve(ordns.Simulation.Resolve) for the requested record types; every wire query waits on the rate limiter. - Result Emission: A found result that is not a wildcard answer increments the found counter and is emitted as an
EventResultwith its typed records. Every completed lookup, success or failure, increments the matchingStatscounter. The CLI routes results to its stdoutWriterin the selected-formatand to one file-onlyWriterper-ofile, each in its own format (parseOutputs, #334); the TUI renders them in its viewport. AfterEventDonethe CLI prints the per-outcome breakdown to stderr and the TUI summary line shows the same counters under the run-quality verdict and its reason. - Recursive Expansion (optional): when
-recursiveis set and a resolved, non-wildcard job is below the-depthcap, the worker hands it back to the dispatcher over theenqueuechannel, which adds an expansion for its children and grows the progress total. - Progress Tracking: Workers update atomic counters; the ticker goroutine emits
EventProgressonce per second. - Termination: After finishing a chunk, a worker reports the completion of every job in it in one send on the buffered
completedchannel; the dispatcher uses these reports for its pending count and the resume watermark, so a resumed run repeats at most one chunk per worker. When no work remains the dispatcher closesjobs, the workers exit, andscan.Runwaits on thesync.WaitGroup, runs the-retry-inconclusivepass if set, stops the progress ticker, emitsEventDone, and closes the events channel. OnSIGINT/SIGTERMthe context is cancelled, the dispatcher closesjobsearly, and the same drain-and-finish path runs with partial counts. - Finalization: If at least one scan finished, the CLI calls
Writer.Finishand commits the-ofiles, then exits with the code described in 4.5.
Visually, this can be seen as:
User Input -> Argument Parser -> scan.Options -> scan.Config -> scan.Run() [Preflight -> Wildcard fingerprint -> Dispatcher -> jobs -> Worker Pool -> DNS Resolver] -> Event Channel -> Output (if resolved)
4. Error Handling Strategy
subenum handles different types of errors at various stages of its operation:
4.1. User Input Errors
- Missing Required Arguments: When a target (domain or
-dL) is missing, the tool says what is missing, prints the usage message and flag list, and exits with status 2. - Validation (status 2 on failure): the tool validates:
- Concurrency, timeout, attempts and depth must be at least 1;
-rate,-rate-per-resolver,-max-queriesand-max-durationmust be 0 (unlimited) or positive. These checks live inscan.Options.Validate. - The DNS server must be
ip:portwith an IP host and a port in 1-65535,tls://host[:port](port 853 by default), orhttps://host/path. Encrypted transports also accept hostname hosts (validate.DNSServer; not checked in simulation mode). - The target domain must conform to DNS naming rules after normalization (
validate.NormalizeDomain,validate.Domain). In a-dLlist, invalid domains are skipped with a message instead. - Hit rate (simulation mode) must be 1-100.
-formatand-typemust name supported values;-attemptsand-retriescannot both be set.- Only one positional domain is accepted, not together with
-dL, and-wand-dLcannot both read stdin.
- Concurrency, timeout, attempts and depth must be at least 1;
4.2. File Operation Errors
- Wordlist or domain list cannot be read: the tool prints an error (
reading wordlist file: ...orreading domain list: ...) and exits with status 1. A wordlist with no valid entries, or a domain list with no valid domains, is also status 1. - Output file: if the
-ofile cannot be created the tool exits with status 1 before scanning. A failed write, flush, close or rename at the end is reported and turns a successful exit into status 1; the partial temporary file is discarded and an existing results file is left unchanged.
4.3. DNS Resolution Errors
- Lookup Failure: When a DNS lookup fails, the error is classified (
NXDomain,Timeout,Refused,Other,Canceled) and counted.Refusedmeans the resolver answered with RCODE REFUSED (or a DoH server answered HTTP 429/503); a socket error such as “connection refused” from a closed port isOther. NXDOMAIN is a definitive negative and is not treated as an infrastructure failure; this bucket also covers NODATA (the name exists but has no records of the requested types, such as-type CNAMEon a name with only A records). Lookups cut short by cancellation (Canceled) are not counted at all, so an interrupt cannot inflate the failure rate. Timeouts, refusals, and other errors feed the reliability guard. The CLI always prints the per-outcome breakdown afterEventDone. - Timeout Handling: The user-specified timeout (
-timeoutflag) limits each per-type lookup, starting only after its rate-limit slots are granted, so queueing behind-ratenever turns into a timeout. A lookup that exceeds it is classified as a timeout and retried if attempts remain. - Resolver failures before scanning: a failed preflight or an inconclusive wildcard check (without
-force) emitsEventError, followed by exactly oneEventDonewithStoppedset; the target counts as failed.
4.4. Concurrency-Related Issues
- Channel Operations: The scan engine uses three internal channels (
jobs,enqueue,completed) plus the outboundeventschannel. To avoid the classic send-on-closed and double-close panics, only the dispatcher closesjobs, and it does so exactly once (when no work remains or the context is cancelled);enqueueandcompletedare never closed. Theeventschannel is closed byscan.Runonly after the workerWaitGroupreturns and the progress ticker has confirmed its exit, so no in-flight send can race the close. Callers must draineventsuntil it is closed: the result and done sends are not individually guarded against a consumer that stops reading early, so a consumer that wants to stop on cancellation should keep draining until close rather than abandoning the channel. - Worker Goroutine Errors: Each worker goroutine processes DNS lookups independently. If an error occurs within a worker (outside of the expected DNS resolution failures), it can cause the entire goroutine to terminate. The current implementation doesn’t have specific handling for such scenarios.
4.5. Graceful Shutdown and Exit Codes
The tool listens for SIGINT and SIGTERM. On the first signal it cancels the work context, drains in-flight workers, finalizes the output written so far, and exits with the shell convention 128+signum; a second Ctrl+C force-quits a run stuck draining. Exit codes (also listed in -h):
0- success.1- a scan, the wordlist, the domain list or the output file failed.2- invalid flags or arguments.3--dL: some targets failed while others completed.4--diff: names were added or removed (or, with-diff-records, changed) since the previous run;subenum diff: the two files differ.130- interrupted (SIGINT, Ctrl+C); partial results are kept, in the resume state when the-ofile is also the-diffbaseline, which is left unchanged (#297).143- terminated (SIGTERM); partial results are kept as for130.
With -dL, a reliability-guard abort means the resolver is overloaded, so the remaining targets are skipped (and counted as failed) rather than sent to it too; -no-abort keeps going.
4.6. Output File Support
When the -o flag is provided, results are written to the specified file in addition to stdout, in the selected -format or in the format a format: prefix names. -o is repeatable (outputSpecs, #334): each file gets its own output.NewFile writer and the full current results, also under -diff, where stdout carries only the changes. Each file is replaced atomically (see 2.5), and a mutex in each Writer protects its concurrent writes.
4.7. Retry Mechanism
The -attempts flag (default: 1) controls the total number of DNS resolution attempts per subdomain. A value of 1 means no retries. A short linear backoff delay is applied between attempts to handle transient DNS failures (timeout, refused, other). NXDOMAIN is a definitive negative and is not retried. The same attempt count applies to the preflight and to wildcard probes. The deprecated -retries flag is still accepted as an alias but prints a warning to stderr.
5. Other Subsystems
Each subsystem below is small and documented in its source; this section says what it does and where to look.
| Subsystem | Where | Summary |
|---|---|---|
| Run-quality verdict | internal/scan/verdict.go, internal/report, internal/cli/summary.go |
Grades each finished scan, and the run (the worst target), as complete, degraded or unreliable with a reason (5.1). |
Scope (-exclude) |
internal/scan/scope.go |
Out-of-scope names are never queried or expanded (5.2). |
Resume state (-resume, -state) |
internal/cli/resume.go |
A run that is interrupted, runs out of -max-duration or is stopped by the reliability guard saves its state; -resume replays it and continues (5.3). |
Permutation pass (-permute, -seeds) |
internal/permute, internal/cli/permute.go |
A second scan over variations of the names found (5.4). |
CT seeding (-ct) |
internal/ct, internal/cli/ct.go |
Adds names from Certificate Transparency logs as candidates (5.5). |
Lab mode (-simulate-zone) |
internal/labzone, internal/dnsserver, internal/cli/lab.go |
Serves a scenario file on a loopback DNS server (5.6). |
SARIF (-sarif) |
internal/output/sarif.go, internal/dns/takeover.go |
Writes takeover candidates as SARIF 2.1.0 (5.7). |
| Config and environment defaults | internal/cli/config.go |
Flag > environment > config file > built-in default (5.8). |
| Go API | pkg/subenum |
The engine as a library (5.9). |
5.1. Run-Quality Verdict
scan.Stats.Verdict grades a scan that reached EventDone: unreliable when the reliability guard aborted it or more than 20% of lookups failed; degraded when it was interrupted, at least 1% of lookups failed, or -max-queries or -max-duration left candidates untested; otherwise complete. The reason names settings as {tokens} (internal/scan/settings.go) that each front end renders in its own terms (-max-queries in the CLI). internal/report builds the run summary for the CLI and the TUI; the CLI writes it as the final line of jsonl output and to the -stats file (the TUI to its Stats File), including per-resolver counts when -r is used.
5.2. Scope and Exclusions
-exclude takes exact names and *.parent patterns. An exact pattern excludes that name; *.parent excludes every name below parent but not parent itself. Patterns are normalized like targets (lowercase, no trailing dot, punycode) and checked with scan.ValidateExcludes. The dispatcher never queries or recursively expands an excluded name, and counts it as excluded in the report.
5.3. Resume State
On SIGINT/SIGTERM, and when -max-duration or the reliability guard stops a run, the CLI writes a state file (-state, default subenum-resume.json) holding the original command line, the effective settings (including those from the environment and config file), a sha256 of each input file and of each target’s normalized wordlist, and per-target progress: whether it finished, how many wordlist entries were settled, its stats, the names found so far, the lookups left for -retry-inconclusive, the -permute pass’s progress, and the names the -ct/-sources passive sources and an -axfr zone transfer returned. A lookup fetched ahead for a target the run never started is not saved. subenum -resume <file> replays the command line, refuses to continue if an input changed, re-emits the saved results and continues each target where it stopped. Runs that read stdin cannot be resumed.
5.4. Permutation Pass
With -permute (or -seeds, which implies it), each target that finished gets a second scan. permute.Generate permutes the leftmost label of every name found by the main pass (and of names in the -seeds file): api suggests api-dev, dev-api, dev.api, api2 and so on. Candidates already in the wordlist or among the seeds are skipped, so nothing is queried twice. The pass is a full, guarded scan with its own preflight, wildcard filtering, -rate, -max-queries and -exclude. It is skipped after an interrupt or a reliability-guard abort, and its hits are marked as permutation results.
5.5. Certificate Transparency Seeding
With -ct, ct.Client.Names makes one HTTPS request per target to crt.sh (no API key) and returns the names its certificates cover, relative to the target. passiveNames gathers the names through passive.FetchAll, and passive.Merge picks those not already in the wordlist. candidates.Set keeps them in a list of their own, which the scan takes as scan.Config.Extra and tries after Entries as if appended to them, so wordlist resume positions stay put and the wordlist, which every -dL target shares, is not copied per target. They are only candidates and are resolved like any other; those that resolve also seed -permute. A failed lookup is a warning, not a failed scan. A resumed run reuses the saved CT names so its candidate list is the same.
5.6. Lab Mode
-simulate-zone <file> loads a scenario with labzone.Load, or, for a name such as lab1 that is not a file, the scenario compiled in from examples/labs with its wordlist as the -w default (builtinLab) (exact names, records, *.name wildcards, and TIMEOUT, SERVFAIL, REFUSED or NXDOMAIN entries, plus $DELAY and $REFUSE-ABOVE directives), serves it with dnsserver.Listen on a loopback port and points -dns-server at it (startLabZone). Unlike -simulate, the scan then runs the real resolver, wildcard detection, -rate pacing and reliability guard, and no traffic leaves the machine. The same internal/dnsserver backs the hermetic tests through internal/dnstest.
5.7. SARIF Output
-sarif <file> writes the run’s takeover candidates (results whose CNAME dns.TakeoverHint marked as dangling or pointing at a takeover-prone provider, and names delegated to a name server that does not resolve, dangling-ns) as a SARIF 2.1.0 log with output.WriteSARIF, atomically, for GitHub code scanning. It needs CNAME or NS in -type. A run with no candidates writes a valid document with no results.
5.8. Configuration Precedence
internal/cli/config.go fills flag defaults from a JSON config file ($SUBENUM_CONFIG, else <user config dir>/subenum/config.json) and from SUBENUM_* environment variables named after the flag (-dns-server is SUBENUM_DNS_SERVER). Precedence, highest first: command-line flag > environment > config file > built-in default. Mode and action flags (-tui, -version, -print-config, -dry-run, -resume, -h) are never taken from defaults. -print-config prints every setting’s effective value and its source. An invalid config value is a usage error, except under -version, -print-config and -tui, which only warn so the value can be found and fixed. The TUI pre-fills its form from the same environment and config values.
5.9. Go API (pkg/subenum)
pkg/subenum exposes the same engine to other Go programs. subenum.Scan runs a scan to completion and returns its results and stats (with the partial results when it fails); subenum.Run streams results, notices and progress as events; subenum.DefaultWords returns the bundled wordlist. Its Simulate option produces synthetic results without sending queries, which is what examples and tests should use. The package follows semantic versioning from v1.0.0; until then it may change between minor releases.