T1596

Search Open Technical Databases

Reconnaissance Last updated:

This detection identifies potential adversary reconnaissance activity involving queries to open technical databases such as Shodan, Censys, Certificate Transparency logs, passive DNS repositories, WHOIS services, and CDN enumeration tools. Because T1596 is a PRE-ATT&CK technique executed outside the victim network, direct detection is not possible. Instead, this rule focuses on second-order indicators: internal endpoints or service accounts making scripted bulk queries to known OSINT aggregator domains, automated certificate transparency monitoring tools being abused, and unusual DNS resolution patterns targeting passive DNS APIs. Detection of these behaviors may indicate an insider threat, a compromised endpoint conducting pre-pivot reconnaissance, or red team activity. Confidence is set to low due to high false positive potential from legitimate security research and threat intelligence workflows.

What is T1596 Search Open Technical Databases?

Search Open Technical Databases (T1596) maps to the Reconnaissance tactic — the adversary is trying to gather information they can use to plan future operations in MITRE ATT&CK.

This page provides production-ready detection logic for Search Open Technical Databases, covering the data sources and telemetry it touches: Microsoft Defender for Endpoint. The queries below are rated medium severity at low confidence, and ship for 7 SIEM platforms — KQL, SPL, Elastic, QRadar, Sumo, YARA-L, LogScale.

MITRE ATT&CK

Tactic
Reconnaissance
Technique
T1596 Search Open Technical Databases
Canonical reference
https://attack.mitre.org/techniques/T1596/
Microsoft Sentinel / Defender
kusto 
let OSINTDomains = dynamic([
    "shodan.io", "api.shodan.io",
    "censys.io", "search.censys.io",
    "dnsdumpster.com",
    "securitytrails.com", "api.securitytrails.com",
    "spyse.com",
    "zoomeye.org",
    "fofa.so", "fofa.info",
    "binaryedge.io", "api.binaryedge.io",
    "onyphe.io",
    "hunter.io",
    "intelx.io",
    "passivedns.circl.lu",
    "riddler.io",
    "robtex.com",
    "hackertarget.com",
    "whoisxmlapi.com",
    "domaintools.com",
    "crt.sh",
    "certspotter.com",
    "urlscan.io",
    "viewdns.info",
    "threatcrowd.org"
]);
let SuspiciousProcesses = dynamic([
    "curl.exe", "wget.exe",
    "python.exe", "python3.exe", "py.exe",
    "powershell.exe", "pwsh.exe",
    "cmd.exe", "wscript.exe", "cscript.exe"
]);
DeviceNetworkEvents
| where TimeGenerated > ago(1h)
| where ActionType == "ConnectionSuccess" or ActionType == "HttpConnectionInspected"
| where RemoteUrl has_any (OSINTDomains)
    or RemoteIPType != "Private" and RemotePort in (443, 80)
        and (RemoteUrl contains "shodan" or RemoteUrl contains "censys" or RemoteUrl contains "crt.sh")
| where InitiatingProcessFileName in~ (SuspiciousProcesses)
    or InitiatingProcessCommandLine has_any ("/api/", "apikey", "api_key", "--key", "-H 'API")
| summarize
    RequestCount = count(),
    UniqueOSINTDomains = dcount(RemoteUrl),
    QueriedDomains = make_set(RemoteUrl, 20),
    ProcessesUsed = make_set(InitiatingProcessFileName, 5),
    CommandLines = make_set(InitiatingProcessCommandLine, 5),
    FirstSeen = min(TimeGenerated),
    LastSeen = max(TimeGenerated)
    by DeviceName, InitiatingProcessAccountName, InitiatingProcessAccountDomain
| where RequestCount >= 5 or UniqueOSINTDomains >= 2
| extend
    RiskScore = case(
        UniqueOSINTDomains >= 5, 90,
        UniqueOSINTDomains >= 3, 70,
        RequestCount >= 20, 60,
        40
    ),
    Alert = "Possible open technical database reconnaissance from corporate endpoint"
| project
    FirstSeen, LastSeen, DeviceName,
    InitiatingProcessAccountName, InitiatingProcessAccountDomain,
    RequestCount, UniqueOSINTDomains, QueriedDomains,
    ProcessesUsed, CommandLines, RiskScore, Alert
| order by RiskScore desc

Detects scripted or bulk queries from corporate endpoints to known OSINT and open technical database services (Shodan, Censys, SecurityTrails, crt.sh, etc.), particularly via command-line tools like curl, Python, or PowerShell. Aggregates by user and device to surface bulk enumeration patterns that suggest automated reconnaissance against the organization's own infrastructure.

medium severity low confidence

Data Sources

Microsoft Defender for Endpoint

Required Tables

DeviceNetworkEvents

False Positives

  • Security analysts and threat intelligence teams performing legitimate asset discovery or exposure monitoring using these same OSINT tools
  • Red team or penetration testing engagements querying Shodan/Censys to validate external attack surface
  • Automated vulnerability management platforms (e.g., Tenable.io, Rapid7) that query third-party databases as part of external exposure scanning
  • DevSecOps pipelines using crt.sh or SecurityTrails APIs for certificate transparency monitoring in CI/CD
  • IT asset management tools that periodically validate domain and certificate configurations via WHOIS or passive DNS APIs

Sigma rule & cross-platform mapping

The detection logic for Search Open Technical Databases (T1596) above is provided in a vendor-neutral form so you can deploy it on any SIEM. The same logic is shipped here as native KQL (Microsoft Sentinel / Defender), SPL (Splunk), Elastic (Elastic Security (EQL)), QRadar (IBM QRadar (AQL)), Sumo (Sumo Logic CSE), YARA-L (Google Chronicle / SecOps), LogScale (CrowdStrike LogScale (CQL)) queries. In Sigma terms, this detection targets the following logsource: 

logsource:
  category: network_connection
  product: windows

Browse the community-maintained Sigma rules for this technique:

SigmaHQ rules for T1596 Sigma rules on detection.fyi

Platform-specific guides for T1596

Detect in Microsoft Sentinel (KQL) Detect in Splunk (SPL) Detect in Elastic Security (Elastic) Detect in IBM QRadar (QRadar) Detect in Sumo Logic CSE (Sumo) Detect in Google Chronicle (YARA-L) Detect in CrowdStrike LogScale (LogScale)
Last updated: 2026-03-19 Research depth: deep
References (11)

Testing Methodology

Validate this detection against 3 adversary techniques from Atomic Red Team. Each test below lists the behaviour to exercise and the telemetry you should expect to see. Executable commands and cleanup steps are available with Pro.

  1. Test 1Shodan CLI Reconnaissance of Organization IP Range

    Expected signal: Sysmon EventCode 3 (Network Connection) to api.shodan.io:443 from python process; Sysmon EventCode 22 (DNS Query) for api.shodan.io; DeviceNetworkEvents showing ConnectionSuccess to shodan.io from python.exe or python3.exe

  2. Test 2Certificate Transparency Bulk Query via crt.sh API

    Expected signal: Sysmon EventCode 1 (Process Create) for powershell.exe with Invoke-RestMethod in CommandLine; Sysmon EventCode 3 (Network Connection) to crt.sh:443; Windows Security EventCode 4688 (Process Creation) if Sysmon not available; DeviceNetworkEvents showing connection to crt.sh from powershell.exe

  3. Test 3Multi-Source OSINT Enumeration Script

    Expected signal: Sysmon EventCode 3 (Network Connection) entries for hackertarget.com and viewdns.info from bash/curl; Sysmon EventCode 22 (DNS Query) for api.hackertarget.com, api.viewdns.info; proxy logs showing sequential requests to multiple OSINT domains within short time window

Response Playbook

Triage

  1. Step 1: Identify the querying endpoint and user account. Check whether the user is a member of a security team, threat intelligence team, or red team. Cross-reference with an authorized OSINT tool inventory or approved software list.
  2. Step 2: Examine the specific OSINT domains queried. A single query to crt.sh is low risk (developer checking their cert). Automated queries to 4+ distinct databases (Shodan + Censys + SecurityTrails + WHOIS) in the same session strongly suggests scripted enumeration.
  3. Step 3: Review the initiating process and command line arguments. Scripted access (curl with -H 'Authorization: Shodan <key>' or Python scripts with requests.get to api.shodan.io) is higher risk than browser-based lookups. Extract any API keys referenced in command lines and check if they belong to the organization.
  4. Step 4: Determine what was queried. If possible, inspect HTTP request payloads via proxy logs. Look for query parameters containing your organization's IP ranges, domain names, or certificate fingerprints — this indicates targeted reconnaissance against your own assets.
  5. Step 5: Check for associated activity within 24 hours: Was there a new user account created, a VPN connection from an unusual country, a phishing email received, or any lateral movement alerts? Correlated activity raises severity significantly.
  6. Step 6: Review the endpoint's DeviceFileEvents and DeviceProcessEvents for any script files written to disk (e.g., osint_scan.py, recon.sh, enum_certs.ps1) or output files collecting OSINT results.

Containment

  1. If the activity is confirmed unauthorized or from a compromised account: disable the user account immediately and revoke active sessions via Azure AD or Okta.
  2. Isolate the endpoint using Microsoft Defender for Endpoint's 'Isolate device' action if malware or a compromised session is suspected.
  3. Rotate any API keys for OSINT services (Shodan, Censys, SecurityTrails) that were referenced in command lines. Treat them as potentially exposed.
  4. Block outbound access to the OSINT domains at the web proxy or firewall layer for the affected endpoint while investigation is ongoing.
  5. If an insider threat is suspected, engage HR and Legal before taking containment actions to preserve chain of custody and avoid tipping off the individual.

Evidence Collection

  1. Export DeviceNetworkEvents and DeviceProcessEvents from Microsoft Defender for Endpoint for the affected device covering 48 hours before and after the alert.
  2. Collect web proxy logs (e.g., Zscaler, Bluecoat, Squid) showing full HTTP request URLs, user-agents, and response codes for all requests to OSINT domains.
  3. Preserve any script files found on the endpoint: collect from temp directories (%TEMP%, /tmp), user home directories, and Downloads. Hash all collected files.
  4. Export PowerShell ScriptBlock logs (Event ID 4104) and command history ($env:APPDATA\PowerShell\PSReadLine\ConsoleHost_history.txt) from the affected host.
  5. Capture a memory image of the endpoint if malware is suspected (use WinPmem or Velociraptor).
  6. Collect DNS query logs from your internal resolver (Windows DNS debug log, BIND query log) for the affected host to identify any additional domains queried that may not have reached proxy inspection.

Escalation Criteria

  • ! Escalate to Tier 2 / Incident Response if queries were specifically targeting your organization's IP ranges, domain names, or SSL certificate fingerprints — this confirms directed adversary pre-attack reconnaissance.
  • ! Escalate immediately if the querying account is a service account, privileged admin, or CI/CD pipeline identity, as this may indicate credential compromise.
  • ! Escalate if the activity correlates with other reconnaissance alerts (T1592, T1593, T1594) within the same time window, suggesting a systematic campaign.
  • ! Escalate if the endpoint has no legitimate business reason to access OSINT tools (e.g., finance, HR, or operations staff) and no authorized security tooling is installed.
  • ! Escalate if OSINT queries are followed within hours by spearphishing attempts, credential stuffing alerts, or external vulnerability scanning of your infrastructure.

Investigation Guide

Forensic Artifacts

  • > Web proxy logs with full URLs to shodan.io, censys.io, crt.sh, securitytrails.com, and similar OSINT aggregators
  • > PowerShell command history at $env:APPDATA\Microsoft\Windows\PowerShell\PSReadLine\ConsoleHost_history.txt
  • > Bash history at ~/.bash_history or ~/.zsh_history on Linux/macOS systems
  • > Python script files or downloaded recon toolkits in user temp/home directories
  • > Windows Prefetch files (C:\Windows\Prefetch\) for recon tool executables (SHODAN.PY, CENSYS_SCAN.EXE)
  • > API keys or auth tokens stored in environment variables, .env files, or script headers on the endpoint
  • > DNS resolver cache (ipconfig /displaydns) showing resolution of OSINT service domains
  • > Browser history (if queries were made via browser rather than script) at user profile SQLite databases

Tuning Guidance

This detection has inherently high false positive rates because legitimate security teams, threat intelligence analysts, and DevSecOps pipelines use these exact same OSINT services. To reduce noise: (1) Build an allowlist of approved workstations and service accounts authorized to query OSINT services and exclude them from alerting; (2) Raise the RequestCount threshold from 5 to 20+ for endpoints in the security team subnet; (3) Tune severity down to 'informational' for browser-based (non-scripted) access patterns; (4) Focus high-confidence alerting on cases where queries contain your organization's specific IP ranges, domain names, or certificate fingerprints as query parameters — this is the clearest indicator of targeted reconnaissance; (5) Consider suppressing crt.sh queries entirely if your team uses certificate transparency monitoring as a standard practice.

Hunting Queries

Hunts for DNS resolution events (Sysmon EventID 22) targeting known OSINT aggregator domains. DNS-level detection captures queries that may not appear in network connection logs if the process exits before establishing a TCP connection, and catches both browser and scripted access patterns.

Hunting — KQL
kql 
// Hunt: Detect DNS queries to OSINT aggregators via Sysmon EventID 22
let OSINTDomains = dynamic([
    "shodan.io", "censys.io", "crt.sh", "securitytrails.com",
    "dnsdumpster.com", "binaryedge.io", "zoomeye.org",
    "onyphe.io", "spyse.com", "fofa.so", "hackertarget.com",
    "whoisxmlapi.com", "domaintools.com", "viewdns.info"
]);
DeviceEvents
| where ActionType == "DnsQueryResponse" or ActionType == "DnsQuery"
| where AdditionalFields has_any (OSINTDomains)
| extend ParsedFields = parse_json(AdditionalFields)
| extend DnsQuery = tostring(ParsedFields.DnsQuery)
| where DnsQuery has_any (OSINTDomains)
| summarize
    QueryCount = count(),
    UniqueDomains = dcount(DnsQuery),
    Domains = make_set(DnsQuery, 20),
    FirstQuery = min(TimeGenerated),
    LastQuery = max(TimeGenerated)
    by DeviceName, InitiatingProcessAccountName, InitiatingProcessFileName
| where QueryCount >= 3 or UniqueDomains >= 2
| order by UniqueDomains desc
Hunting — SPL
spl 
index=* sourcetype="XmlWinEventLog:Microsoft-Windows-Sysmon/Operational" EventCode=22
| rex field=QueryName "(?P<osint_hit>shodan\.io|censys\.io|crt\.sh|securitytrails\.com|dnsdumpster\.com|binaryedge\.io|zoomeye\.org|hackertarget\.com|domaintools\.com)"
| where isnotnull(osint_hit)
| stats
    count as dns_queries,
    dc(QueryName) as unique_domains,
    values(QueryName) as queried_domains,
    values(process) as processes,
    min(_time) as first_seen,
    max(_time) as last_seen
    by host, user, osint_hit
| where dns_queries >= 3 OR unique_domains >= 2
| sort - unique_domains

Hunts for process execution patterns consistent with certificate transparency monitoring tools, crt.sh API clients, or direct OSINT CLI tools (shodan CLI, censys CLI, securitytrails CLI). These tools are commonly used in automated reconnaissance scripts and their presence on non-security endpoints is anomalous.

Hunting — KQL
kql 
// Hunt: Detect certificate transparency log monitoring tools or crt.sh bulk queries
DeviceProcessEvents
| where TimeGenerated > ago(7d)
| where ProcessCommandLine has_any (
    "crt.sh", "certspotter", "certstream",
    "certificate transparency", "ct-log",
    "--ct", "crtsh", "cert_transparency"
)
    or FileName in~ ("ct-monitor.py", "certwatch.sh", "cert-monitor", "crtsh.py")
    or ProcessCommandLine has_all ("curl", "crt.sh")
    or ProcessCommandLine has_all ("python", "crt")
| project
    TimeGenerated, DeviceName, AccountName, AccountDomain,
    FileName, ProcessCommandLine,
    InitiatingProcessFileName, InitiatingProcessCommandLine
| order by TimeGenerated desc
Hunting — SPL
spl 
index=* sourcetype="XmlWinEventLog:Microsoft-Windows-Sysmon/Operational" EventCode=1
| where match(CommandLine, "crt\.sh|certspotter|certstream|ct-log|crtsh")
    OR match(CommandLine, "shodan\s+(search|host|count)")
    OR match(CommandLine, "censys\s+(search|view)")
    OR match(CommandLine, "securitytrails.*domain")
| table _time, host, user, process, CommandLine, ParentCommandLine
| sort - _time

Hunts for obfuscated or base64-encoded invocations of OSINT API calls, which may indicate an adversary attempting to evade detection while scripting reconnaissance against technical databases. Encoded PowerShell commands referencing OSINT services are a strong indicator of malicious intent.

Hunting — KQL
kql 
// Hunt: Identify encoded or obfuscated API calls to OSINT services (evasion attempt)
DeviceProcessEvents
| where TimeGenerated > ago(7d)
| where ProcessCommandLine has_any ("shodan", "censys", "securitytrails", "binaryedge", "onyphe")
    or (
        ProcessCommandLine has_any ("Invoke-WebRequest", "Invoke-RestMethod", "WebClient", "HttpClient", "curl", "wget")
        and ProcessCommandLine has_any ("api.", "/api/v", "apikey", "api_key", "Authorization")
    )
| extend IsEncoded = iff(
    ProcessCommandLine has "FromBase64String"
    or ProcessCommandLine has "-enc"
    or ProcessCommandLine has "-EncodedCommand",
    true, false
)
| where IsEncoded == true
    or ProcessCommandLine matches regex @"[A-Za-z0-9+/]{40,}={0,2}"
| project
    TimeGenerated, DeviceName, AccountName,
    FileName, ProcessCommandLine, IsEncoded,
    InitiatingProcessFileName
| order by TimeGenerated desc
Hunting — SPL
spl 
index=* sourcetype="XmlWinEventLog:Microsoft-Windows-Sysmon/Operational" EventCode=1
| where match(CommandLine, "shodan|censys|securitytrails|binaryedge")
    OR (match(CommandLine, "Invoke-WebRequest|Invoke-RestMethod|curl|wget") AND match(CommandLine, "api\.|apikey|api_key"))
| eval is_encoded=if(match(CommandLine, "FromBase64String|-enc\s|-EncodedCommand|[A-Za-z0-9+/]{40,}={0,2}"), 1, 0)
| where is_encoded=1
| table _time, host, user, process, CommandLine, is_encoded
| sort - _time

Atomic Red Team Tests

Test 1 Shodan CLI Reconnaissance of Organization IP Range
linux

Simulates an adversary or insider using the Shodan CLI tool to query Shodan's scan database for hosts in a target IP range, replicating T1596.005 (Scan Databases) behavior.

Command

bash 
# Install shodan CLI if not present
pip install shodan

# Initialize with a free API key (register at shodan.io)
shodan init YOUR_API_KEY_HERE

# Query for hosts in target IP range (use your own test range)
shodan search --fields ip_str,port,org,hostnames net:203.0.113.0/24

# Count results for target domain
shodan count 'hostname:example.com'

# Get host details
shodan host 203.0.113.1

Cleanup

bash 
pip uninstall -y shodan
rm -f ~/.shodan/api_key

Expected Telemetry

Sysmon EventCode 3 (Network Connection) to api.shodan.io:443 from python process; Sysmon EventCode 22 (DNS Query) for api.shodan.io; DeviceNetworkEvents showing ConnectionSuccess to shodan.io from python.exe or python3.exe

Expected Detection

Alert: 'Possible open technical database reconnaissance from corporate endpoint' — python.exe connecting to api.shodan.io with API key in command line arguments

Test 2 Certificate Transparency Bulk Query via crt.sh API
windows

Simulates adversary querying crt.sh (Certificate Transparency search engine) to enumerate subdomains and certificates issued for a target organization — replicating T1596.003 (Digital Certificates).

Command

powershell 
# Using PowerShell to query crt.sh for certificate transparency data
$TargetDomain = "example.com"
$Results = Invoke-RestMethod -Uri "https://crt.sh/?q=%25.$TargetDomain&output=json" -Method GET
$Results | Select-Object -Property name_value, issuer_name, not_before, not_after | Format-Table

# Also query via curl (generates different process telemetry)
curl -s "https://crt.sh/?q=%25.example.com&output=json" | python3 -c "import sys,json; data=json.load(sys.stdin); [print(d.get('name_value','')) for d in data[:20]]"

Cleanup

powershell 
Remove-Item -Path $env:TEMP\crt_results.json -ErrorAction SilentlyContinue

Expected Telemetry

Sysmon EventCode 1 (Process Create) for powershell.exe with Invoke-RestMethod in CommandLine; Sysmon EventCode 3 (Network Connection) to crt.sh:443; Windows Security EventCode 4688 (Process Creation) if Sysmon not available; DeviceNetworkEvents showing connection to crt.sh from powershell.exe

Expected Detection

Alert fires when powershell.exe or curl.exe connects to crt.sh — classified as certificate transparency reconnaissance. If multiple OSINT domains queried in same session, RiskScore elevates to 70+.

Test 3 Multi-Source OSINT Enumeration Script
linux

Simulates a comprehensive reconnaissance script that queries multiple open technical databases in sequence (Censys, SecurityTrails, HackerTarget, ViewDNS) — the combined pattern is a strong indicator of systematic T1596 activity.

Command

bash 
#!/bin/bash
# Multi-source OSINT enumeration (simulates adversary recon workflow)
TARGET="example.com"

# 1. Query HackerTarget for DNS records (free, no key needed)
echo "[*] DNS Records from HackerTarget:"
curl -s "https://api.hackertarget.com/hostsearch/?q=$TARGET"

# 2. Query ViewDNS for reverse IP lookup
echo "[*] Reverse IP from ViewDNS:"
curl -s "https://api.viewdns.info/reverseip/?host=$TARGET&apikey=YOUR_KEY&output=json"

# 3. Query Censys for certificates (requires free account)
# pip install censys
# censys search "parsed.names: example.com" --index certificates --fields parsed.subject_dn,parsed.names

# 4. Check SSL certificate info
curl -s "https://api.hackertarget.com/zonetransfer/?q=$TARGET"

echo "[*] Enumeration complete"

Cleanup

bash 
# No persistent artifacts to clean — network-only activity
# Remove bash history entry:
history -d $(history | grep -n 'hackertarget\|viewdns\|censys' | tail -1 | awk '{print $1}')

Expected Telemetry

Sysmon EventCode 3 (Network Connection) entries for hackertarget.com and viewdns.info from bash/curl; Sysmon EventCode 22 (DNS Query) for api.hackertarget.com, api.viewdns.info; proxy logs showing sequential requests to multiple OSINT domains within short time window

Expected Detection

Alert fires when curl connects to 2+ distinct OSINT domains (UniqueOSINTDomains >= 2). RiskScore = 70. Sequential multi-source pattern triggers higher-confidence hunting queries.

Related Detections

Tactic Hub

TA0043Reconnaissance

Detection Variants (1)

Different telemetry and tradecraft for the same technique — pick the one that matches the data you collect.

Sub-techniques (5)

Related Techniques

T1592Gather Victim Host InformationT1593Search Open Websites/DomainsT1594Search Victim-Owned WebsitesT1596.001DNS/Passive DNST1596.002WHOIST1596.003Digital CertificatesT1596.004CDNsT1596.005Scan DatabasesT1583Acquire InfrastructureT1598Phishing for InformationT1133External Remote ServicesT1590.002DNS

Same Tactic: Reconnaissance

Popular Detections