Detect Wordlist Scanning in Microsoft Sentinel
Adversaries may iteratively probe infrastructure using brute-forcing and crawling techniques with wordlists to identify content and infrastructure rather than valid credentials. Web content discovery tools such as Dirb, DirBuster, GoBuster, ffuf, and feroxbuster enumerate websites' pages, directories, and hidden administrative portals using generic or target-specific wordlists. Cloud-targeted tools such as s3recon and GCPBucketBrute enumerate public and private cloud storage buckets using globally unique naming patterns. Discovery of exposed content can enable follow-on operations such as exploiting vulnerable pages, accessing sensitive data in cloud storage, or identifying attack surfaces for credential brute-forcing. APT41 and Volatile Cedar (Lebanese Cedar) are known to use directory brute-forcing tools as part of their initial reconnaissance phase.
MITRE ATT&CK
- Tactic
- Reconnaissance
- Technique
- T1595 Active Scanning
- Sub-technique
- T1595.003 Wordlist Scanning
- Canonical reference
- https://attack.mitre.org/techniques/T1595/003/
KQL Detection Query
let ScannerUserAgents = dynamic([
"gobuster", "DirBuster", "dirbuster", "feroxbuster", "ffuf",
"wfuzz", "nikto", "dirsearch", "nuclei", "wapiti", "skipfish",
"s3recon", "GCPBucketBrute", "dirfuzz", "dirb/", "sqlmap",
"w3af", "whatweb", "commix", "nmap scripting"
]);
// Branch 1: Known scanning tool User-Agent strings in IIS web server logs
let UADetection = W3CIISLog
| where TimeGenerated > ago(24h)
| where csUserAgent has_any (ScannerUserAgents)
| summarize
RequestCount = count(),
UniqueUris = dcount(csUriStem),
Errors404 = countif(scStatus == 404),
Errors403 = countif(scStatus == 403),
SamplePaths = make_set(csUriStem, 15),
FirstRequest = min(TimeGenerated),
LastRequest = max(TimeGenerated)
by cIP, csHost, csUserAgent, sSiteName
| extend ScanDurationMinutes = datetime_diff('minute', LastRequest, FirstRequest)
| extend DetectionType = "KnownScannerUserAgent", SuspicionScore = 3
| project FirstRequest, LastRequest, cIP, csHost, csUserAgent, RequestCount, UniqueUris,
Errors404, Errors403, SamplePaths, ScanDurationMinutes, DetectionType, SuspicionScore;
// Branch 2: High-volume 404/403 responses from single source IP (wordlist enumeration pattern)
let VolumeDetection = W3CIISLog
| where TimeGenerated > ago(24h)
| where scStatus in (404, 403)
| summarize
RequestCount = count(),
UniqueUris = dcount(csUriStem),
Errors404 = countif(scStatus == 404),
Errors403 = countif(scStatus == 403),
SamplePaths = make_set(csUriStem, 15),
FirstRequest = min(TimeGenerated),
LastRequest = max(TimeGenerated)
by cIP, csHost, sSiteName, HourBin = bin(TimeGenerated, 1h)
| where RequestCount > 200 and UniqueUris > 100
| extend ScanDurationMinutes = 60, csUserAgent = "Unknown/Multiple"
| extend DetectionType = "HighVolume404Pattern", SuspicionScore = 2
| project FirstRequest, LastRequest, cIP, csHost, csUserAgent, RequestCount, UniqueUris,
Errors404, Errors403, SamplePaths, ScanDurationMinutes, DetectionType, SuspicionScore;
union UADetection, VolumeDetection
| sort by FirstRequest descDetects web wordlist scanning activity using IIS access logs (W3CIISLog). Branch 1 identifies requests carrying known scanning tool User-Agent strings (gobuster, DirBuster, feroxbuster, ffuf, wfuzz, nikto, dirsearch, nuclei, etc.). Branch 2 detects high-volume HTTP 404/403 error patterns from a single IP within a rolling 1-hour window, identifying automated directory enumeration regardless of User-Agent spoofing. Both branches are unioned and sorted by first occurrence for triage prioritization.
Data Sources
Required Tables
False Positives & Tuning
- Internal vulnerability scanners (Nessus, Qualys, Tenable, Rapid7) running authorized scans from known IPs with recognizable User-Agent strings
- Authorized penetration testing engagements by third-party firms using gobuster, ffuf, DirBuster, or similar tools
- Legitimate SEO crawlers, web archival bots (archive.org), or monitoring services that probe large numbers of URLs and generate 404s
- CI/CD pipeline integration tests or automated health checks that probe non-existent endpoints at high frequency
- CDN or load balancer health probes that request synthetic paths and accumulate 404 errors at the origin
Other platforms for T1595.003
Testing Methodology
Validate this detection against 4 adversary techniques from Atomic Red Team. Each test below lists the behaviour to exercise and the telemetry you should expect to see. Executable commands and cleanup steps are available with Pro.
- Test 1GoBuster Directory Enumeration Against Localhost
Expected signal: Sysmon Event ID 1 (Linux): Process Create with Image=/usr/bin/gobuster or similar, CommandLine containing 'dir -u http://127.0.0.1 -w'. Web server access logs: rapid high-volume requests from 127.0.0.1 with User-Agent containing 'gobuster/2.x' or similar gobuster version string, majority returning HTTP 404. Sysmon Event ID 3: network connections from gobuster process to 127.0.0.1:80.
- Test 2ffuf Web Content Discovery with Wordlist
Expected signal: Process creation event with FileName=ffuf, CommandLine containing '-u http://127.0.0.1/FUZZ -w raft-small-words.txt'. Web server access logs: rapid requests from 127.0.0.1 with User-Agent 'Fuzz Faster U Fool v2.x.x'. Sysmon Event ID 3: outbound TCP connections from ffuf process to 127.0.0.1:80.
- Test 3Custom Python Wordlist Scanner (No Tool User-Agent)
Expected signal: Web server access logs: 35 requests from 127.0.0.1 with generic browser User-Agent across 35 unique URI paths, majority returning HTTP 404. Sysmon Event ID 1: python3 process creation with inline script containing 'requests.get'. KQL Branch 2 requires RequestCount > 200 — extend wordlist or reduce threshold during tuning. SPL HighVolume404Pattern requires error_requests > 200.
- Test 4Nikto Web Server Vulnerability and Content Scanner
Expected signal: Process creation event: FileName=nikto or nikto.pl, CommandLine containing '-host http://127.0.0.1 -maxtime 30s'. Web server access logs: high-volume requests from 127.0.0.1 with User-Agent containing 'Nikto' (e.g., 'Mozilla/5.00 (Nikto/2.1.6) (Evasions:None) (Test:Port Check)'). Hundreds of varied HTTP requests including probes for /cgi-bin/, /.git/, /backup/, /.env, /phpinfo.php within 30 seconds.
References (11)
- https://attack.mitre.org/techniques/T1595/003/
- https://www.clearskysec.com/wp-content/uploads/2021/01/Lebanese-Cedar-APT.pdf
- https://rhinosecuritylabs.com/gcp/google-cloud-platform-gcp-bucket-enumeration/
- https://github.com/clarketm/s3recon
- https://github.com/OJ/gobuster
- https://github.com/ffuf/ffuf
- https://github.com/epi052/feroxbuster
- https://github.com/sullo/nikto
- https://github.com/xmendez/wfuzz
- https://github.com/maurosoria/dirsearch
- https://github.com/danielmiessler/SecLists
Unlock Pro Content
Get the full detection package for T1595.003 including response playbook, investigation guide, and atomic red team tests.