Detect Non-Standard Encoding in Microsoft Sentinel
Adversaries may encode data with a non-standard data encoding system to make the content of command and control traffic more difficult to detect. Non-standard encoding schemes diverge from existing protocol specifications — for example, modified Base64 using a custom alphabet, XOR encoding with a static or rolling key, character substitution (replacing '/' with '-s', '+' with '-p'), or custom binary serialization. Real-world examples include OceanSalt (NOT operation on bytes), Small Sieve (hex byte swapping), TONESHELL (XOR with 32/256-byte key), NightClub (modified Base64 in DNS subdomains), RDAT (Base64 with character substitutions in DNS), InvisiMole (modified Base32 in DNS subdomains), and Uroburos (custom Base62/Base32). Detection focuses on anomalous DNS subdomain lengths and entropy, unusual encoded patterns in network traffic, and scripting processes generating high-entropy outbound data.
MITRE ATT&CK
- Tactic
- Command and Control
- Technique
- T1132 Data Encoding
- Sub-technique
- T1132.002 Non-Standard Encoding
- Canonical reference
- https://attack.mitre.org/techniques/T1132/002/
KQL Detection Query
// Branch 1: DNS tunneling — long or high-cardinality subdomains suggesting encoded data
let DnsTunnelingIndicators =
DeviceNetworkEvents
| where Timestamp > ago(24h)
| where ActionType == "DnsConnectionInspected" or ActionType == "NetworkSignatureInspected"
| where isnotempty(RemoteUrl)
| extend SubdomainParts = split(RemoteUrl, ".")
| extend SubdomainLabel = tostring(SubdomainParts[0])
| where strlen(SubdomainLabel) > 50
| extend HasBase64Chars = SubdomainLabel matches regex @"^[A-Za-z0-9+/=-]{40,}$"
| extend HasModifiedBase64 = SubdomainLabel matches regex @"[A-Za-z0-9_\-]{40,}"
| extend HasHexPattern = SubdomainLabel matches regex @"^[0-9a-fA-F]{40,}$"
| where HasBase64Chars or HasModifiedBase64 or HasHexPattern
| extend EncodingType = case(
HasHexPattern, "HexEncoded",
HasBase64Chars, "Base64Like",
HasModifiedBase64, "ModifiedBase64",
"Unknown"
)
| project Timestamp, DeviceName, AccountName, RemoteUrl, SubdomainLabel,
EncodingType, InitiatingProcessFileName, InitiatingProcessCommandLine;
// Branch 2: DNS queries with anomalous subdomain lengths via Sysmon-style telemetry
let DnsQueryAnomaly =
DeviceNetworkEvents
| where Timestamp > ago(24h)
| where ActionType == "DnsConnectionInspected"
| where isnotempty(RemoteUrl)
| summarize QueryCount=count(), UniqueLabels=dcount(RemoteUrl), MaxLabelLen=max(strlen(RemoteUrl))
by DeviceName, InitiatingProcessFileName, bin(Timestamp, 10m)
| where QueryCount > 20 and UniqueLabels > 15
| extend Indicator = "HighVolumeDNS_LikelyTunneling";
// Branch 3: HTTP/S connections with encoded path or query strings suggesting custom encoding
let EncodedHttpTraffic =
DeviceNetworkEvents
| where Timestamp > ago(24h)
| where ActionType in ("HttpConnectionInspected", "ConnectionSuccess")
| where isnotempty(RemoteUrl)
| where RemoteUrl matches regex @"/[A-Za-z0-9_\-]{60,}(\?|/|$)"
or RemoteUrl matches regex @"[?&][a-z]{1,4}=[A-Za-z0-9+/_%\-]{60,}"
| extend UrlLength = strlen(RemoteUrl)
| extend SuspiciousPath = RemoteUrl matches regex @"/[A-Za-z0-9_\-]{60,}"
| extend SuspiciousParam = RemoteUrl matches regex @"[?&][a-z]{1,4}=[A-Za-z0-9+/_%\-]{60,}"
| project Timestamp, DeviceName, AccountName, RemoteUrl, RemoteIP, RemotePort,
UrlLength, SuspiciousPath, SuspiciousParam,
InitiatingProcessFileName, InitiatingProcessCommandLine;
// Branch 4: Scripting engines or uncommon processes making repetitive beaconing connections
let BeaconingWithEncoding =
DeviceNetworkEvents
| where Timestamp > ago(24h)
| where InitiatingProcessFileName in~ ("python.exe", "python3", "perl.exe", "ruby.exe",
"wscript.exe", "cscript.exe", "mshta.exe", "powershell.exe", "pwsh.exe")
| where RemoteIPType == "Public"
| summarize ConnectionCount=count(), UniqueDestinations=dcount(RemoteIP),
Ports=make_set(RemotePort), BytesSent=sum(SentBytes)
by DeviceName, InitiatingProcessFileName, InitiatingProcessCommandLine, bin(Timestamp, 1h)
| where ConnectionCount > 10 and UniqueDestinations < 3
| extend Indicator = "RegularBeaconing_PotentialCustomEncoding";
union DnsTunnelingIndicators, EncodedHttpTraffic
| sort by Timestamp descDetects non-standard encoding used in C2 communications via four complementary approaches: (1) DNS subdomain labels exceeding 50 characters matching Base64-like, modified Base64, or hex patterns — a strong indicator of DNS tunneling used by NightClub, RDAT, and InvisiMole; (2) high-volume DNS query anomalies suggesting automated data exfiltration over DNS; (3) HTTP/S requests with suspiciously long encoded path components or query parameters consistent with modified Base64 C2; (4) scripting engine beaconing patterns to a small number of public IPs at high frequency. Uses DeviceNetworkEvents from Microsoft Defender for Endpoint.
Data Sources
Required Tables
False Positives & Tuning
- CDN and cloud services that use long, base64-encoded tokens in URLs (AWS S3 presigned URLs, Azure SAS tokens, CloudFront signed URLs)
- Legitimate DNS-over-HTTPS or DNS security products that may generate high-volume DNS query patterns
- Monitoring and telemetry agents (Datadog, Dynatrace, New Relic) that POST encoded metrics to collection endpoints using long encoded query strings
- Single-page applications and web APIs that encode state or session data in URL path components (JWT tokens, serialized objects)
- Certificate transparency logs and OCSP responders that use base64-encoded certificate data in URLs
Other platforms for T1132.002
Testing Methodology
Validate this detection against 4 adversary techniques from Atomic Red Team. Each test below lists the behaviour to exercise and the telemetry you should expect to see. Executable commands and cleanup steps are available with Pro.
- Test 1Simulate DNS Tunneling with Modified Base64 Subdomain Encoding
Expected signal: Sysmon Event ID 22 (DNS Query): QueryName will contain a long alphanumeric subdomain label (length > 30 characters) matching the pattern [a-z0-9ps]{30,}\.df00tech-test\.local. Sysmon Event ID 1 (Process Create): powershell.exe with command line containing Base64, Replace, and Resolve-DnsName. PowerShell ScriptBlock Log Event ID 4104 capturing the encoding logic.
- Test 2XOR-Encoded C2 Data Transmission Simulation (TONESHELL Pattern)
Expected signal: Sysmon Event ID 1: powershell.exe with CommandLine containing -bxor, New-Object System.Net.WebClient, and UploadString. Sysmon Event ID 3: Network connection attempt to 127.0.0.1:8080 (connection will be refused but event fires). PowerShell ScriptBlock Log Event ID 4104 capturing the full XOR encoding loop and WebClient upload code.
- Test 3High-Volume DNS Query Burst Simulating DNS Tunneling Data Transfer
Expected signal: 25x Sysmon Event ID 22 (DNS Query) events within ~5 seconds, each with a unique QueryName containing a long base64-like subdomain label (length 40-70 characters) under df00tech-dnstest.local. All queries initiated by powershell.exe. The burst pattern with unique subdomains matches DNS tunneling telemetry.
- Test 4HTTP C2 with Custom Base64 Alphabet Encoding (Neo-reGeorg Pattern)
Expected signal: Sysmon Event ID 1: powershell.exe with CommandLine containing IndexOf, ToCharArray, WebClient, and UploadString — all indicators of custom encoding implementation. Sysmon Event ID 3: Network connection to 127.0.0.1:8080. PowerShell ScriptBlock Log Event ID 4104 capturing the full custom alphabet encoding logic. If stream:http is available, the POST body will contain d=<60+ char custom-alphabet string>.
References (11)
- https://attack.mitre.org/techniques/T1132/002/
- https://en.wikipedia.org/wiki/Binary-to-text_encoding
- https://en.wikipedia.org/wiki/Character_encoding
- https://www.welivesecurity.com/2023/08/10/moustachedbouncer-espionage-targeted-isp-level-adversary-in-the-middle-attacks-against-belarus/
- https://unit42.paloaltonetworks.com/rdat-used-to-target-middle-eastern-energy-company/
- https://www.welivesecurity.com/2020/06/18/digging-up-invismole-hidden-arsenal/
- https://media.kasperskycontenthub.com/wp-content/uploads/sites/43/2017/08/07172148/ShadowPad_technical_description_PDF.pdf
- https://www.mandiant.com/sites/default/files/2022-02/rt-apt41-dual-operation.pdf
- https://github.com/L-codes/Neo-reGeorg
- https://www.ncsc.gov.uk/files/NCSC-GCHQ-Small-Sieve-Malware-Analysis-Report.pdf
- https://www.cisa.gov/sites/default/files/2022-02/aa22-055a-iranian-government-sponsored-actors-conduct-cyber-operations.pdf
Unlock Pro Content
Get the full detection package for T1132.002 including response playbook, investigation guide, and atomic red team tests.