Detect Search Open Websites/Domains in IBM QRadar
This detection identifies automated reconnaissance activity against your organization's public-facing web assets, which may indicate an adversary conducting pre-attack intelligence gathering via T1593. Since T1593 occurs externally (adversaries querying social media, search engines, and public websites), direct network-level detection from within the victim environment is impossible. This detection instead focuses on second-order observable indicators: anomalous automated scraping patterns against your web infrastructure (IIS, Apache, Nginx, Azure WAF), known OSINT/reconnaissance tool user agents in web access logs, high-velocity enumeration from single source IPs, and probing of sensitive disclosure paths such as /.git/, /robots.txt, sitemap.xml, and /admin. These patterns correlate with adversary pre-compromise reconnaissance workflows used by groups including Volt Typhoon, Mustang Panda, and Kimsuky prior to phishing or initial access operations.
MITRE ATT&CK
- Tactic
- Reconnaissance
- Technique
- T1593 Search Open Websites/Domains
- Canonical reference
- https://attack.mitre.org/techniques/T1593/
QRadar Detection Query
SELECT
DATEFORMAT(devicetime, 'yyyy-MM-dd HH:mm:ss') AS "EventTime",
LOGSOURCENAME(logsourceid) AS "LogSource",
LOGSOURCETYPENAME(devicetype) AS "LogSourceType",
"username", "sourceip", "destinationip",
"eventid", "deviceaction", "message",
CASE
WHEN LOWER("useragent") ILIKE '%python-requests%' OR LOWER("useragent") ILIKE '%nuclei%' OR LOWER("requesturl") ILIKE '%.git%' OR LOWER("requesturl") ILIKE '%.env%' THEN 8
ELSE 4
END AS "RiskScore"
FROM events
WHERE (LOWER("useragent") ILIKE '%python-requests%' OR LOWER("useragent") ILIKE '%nuclei%' OR LOWER("requesturl") ILIKE '%.git%' OR LOWER("requesturl") ILIKE '%.env%')
AND LOGSOURCETYPENAME(devicetype) NOT IN ('SIM Audit', 'Custom Rule Engine')
ORDER BY "RiskScore" DESC, "EventTime" DESC
LAST 24 HOURSQRadar AQL detection for Search Open Websites/Domains (T1593). SQL-like syntax queries the QRadar events store, correlating log source telemetry with risk scoring to surface reconnaissance and attack patterns. Filters out noise from internal SIM and rule engine log sources.
Data Sources
Required Tables
False Positives & Tuning
- Legitimate commercial web crawlers and search engine bots (Googlebot, Bingbot, DuckDuckGo) may match known user agent patterns — whitelist verified crawler IP ranges from respective ASNs
- Security vendors running authorized external attack surface scans (Qualys, Tenable, Rapid7) will produce reconnaissance-like patterns — maintain an allowlist of authorized scanner IPs
- Developers or internal teams using curl, Python requests, or httpx for legitimate API testing or load testing against production endpoints
- Content delivery networks and uptime monitoring services (Pingdom, UptimeRobot, StatusCake) making frequent automated HEAD/GET requests
Other platforms for T1593
Testing Methodology
Validate this detection against 3 adversary techniques from Atomic Red Team. Each test below lists the behaviour to exercise and the telemetry you should expect to see. Executable commands and cleanup steps are available with Pro.
- Test 1Automated Web Reconnaissance with Python Requests
Expected signal: Web server access logs will show 25+ requests from 127.0.0.1 with user agent 'python-requests/2.x.x' hitting sensitive paths including /.git/config, /.env, /wp-admin, and /wp-config.php. IIS W3CIISLog or Apache access_combined logs will capture all requests.
- Test 2Directory Enumeration with Gobuster (DNS/HTTP Mode)
Expected signal: Web server logs will show rapid sequential requests from 127.0.0.1 with user agent 'gobuster/3.x'. Each wordlist entry appears as a separate GET request. Requests arrive at ~5 concurrent requests/second. Response codes 200, 301, 302, 403, and 404 visible depending on what exists on the target.
- Test 3OSINT Reconnaissance with theHarvester Against Your Own Domain
Expected signal: DNS resolver logs and network flow logs will show multiple DNS queries for subdomains of the target domain originating from the test host. If your DNS logging infrastructure captures queries, these appear as sequential lookups for www.example.com, mail.example.com, api.example.com, etc. theHarvester queries are external to the target and logged by Bing/search infrastructure, not the victim — this validates the external nature of T1593.
References (8)
- https://attack.mitre.org/techniques/T1593/
- https://attack.mitre.org/techniques/T1593/001/
- https://attack.mitre.org/techniques/T1593/002/
- https://attack.mitre.org/techniques/T1593/003/
- https://www.cisa.gov/news-events/cybersecurity-advisories/aa24-038a
- https://www.microsoft.com/en-us/security/blog/2023/05/24/volt-typhoon-targets-us-critical-infrastructure-with-living-off-the-land-techniques/
- https://www.greynoise.io/blog/understanding-mass-internet-scanners
- https://securitytrails.com/blog/google-hacking-techniques
Unlock Pro Content
Get the full detection package for T1593 including response playbook, investigation guide, and atomic red team tests.