Detect Automated Collection in IBM QRadar
Once established within a system or network, an adversary may use automated techniques for collecting internal data. Methods for performing this technique could include use of a Command and Scripting Interpreter to search for and copy information fitting set criteria such as file type, location, or name at specific time intervals. In cloud-based environments, adversaries may also use cloud APIs, data pipelines, command line interfaces, or ETL services to automatically collect data. This functionality could also be built into remote access tools. This technique may incorporate use of other techniques such as File and Directory Discovery and Lateral Tool Transfer to identify and move files, as well as Cloud Service Dashboard and Cloud Storage Object Discovery to identify resources in cloud environments.
MITRE ATT&CK
- Tactic
- Collection
- Technique
- T1119 Automated Collection
- Canonical reference
- https://attack.mitre.org/techniques/T1119/
QRadar Detection Query
SELECT
DATEFORMAT(starttime, 'yyyy-MM-dd HH:mm:ss') AS event_time,
logsourcename(logsourceid) AS log_source,
sourceip,
username,
"QIDNAME"(qid) AS event_name,
"Process Name" AS process_name,
"Command" AS command_line,
"Parent Process Name" AS parent_process
FROM events
WHERE
LOGSOURCETYPEID(logsourceid) IN (12, 13, 40) -- Windows Security, Sysmon, WinCollect
AND (CATEGORYNAME(category) LIKE '%Process%' OR qid IN (5001, 5002)) -- Process creation events
AND starttime > (CURRENT_TIMESTAMP - 86400000) -- Last 24 hours
AND (
-- PowerShell recursive collection
(LOWER("Process Name") LIKE '%powershell.exe%' OR LOWER("Process Name") LIKE '%pwsh.exe%')
AND (
LOWER("Command") LIKE '%-recurse%' OR LOWER("Command") LIKE '%get-childitem%' OR LOWER("Command") LIKE '%gci %'
)
AND (
LOWER("Command") LIKE '%.doc%' OR LOWER("Command") LIKE '%.pdf%' OR LOWER("Command") LIKE '%.pst%'
OR LOWER("Command") LIKE '%.kdbx%' OR LOWER("Command") LIKE '%.pfx%' OR LOWER("Command") LIKE '%.pem%'
OR LOWER("Command") LIKE '%.xls%' OR LOWER("Command") LIKE '%.csv%'
)
)
OR (
-- CMD dir /s with sensitive extensions
LOWER("Process Name") LIKE '%cmd.exe%'
AND LOWER("Command") LIKE '%dir%'
AND LOWER("Command") LIKE '%/s%'
AND (
LOWER("Command") LIKE '%.doc%' OR LOWER("Command") LIKE '%.pdf%'
OR LOWER("Command") LIKE '%.pst%' OR LOWER("Command") LIKE '%.xls%'
)
)
OR (
-- forfiles processing
LOWER("Command") LIKE '%forfiles%'
AND (
LOWER("Command") LIKE '%.doc%' OR LOWER("Command") LIKE '%.pdf%'
OR LOWER("Command") LIKE '%.pst%' OR LOWER("Command") LIKE '%.kdbx%'
)
)
OR (
-- robocopy mass copy
LOWER("Process Name") LIKE '%robocopy.exe%'
AND (
LOWER("Command") LIKE '%/s%' OR LOWER("Command") LIKE '%/e%'
OR LOWER("Command") LIKE '%/mir%'
)
)
OR (
-- Archive tools targeting documents
(LOWER("Process Name") LIKE '%rar.exe%' OR LOWER("Process Name") LIKE '%winrar.exe%' OR LOWER("Process Name") LIKE '%7z.exe%')
AND (
LOWER("Command") LIKE '% a %' OR LOWER("Command") LIKE '%-a %'
)
AND (
LOWER("Command") LIKE '%.doc%' OR LOWER("Command") LIKE '%.pdf%'
OR LOWER("Command") LIKE '%.pst%' OR LOWER("Command") LIKE '%.kdbx%'
)
)
OR (
-- Python file traversal
(LOWER("Process Name") LIKE '%python.exe%' OR LOWER("Process Name") LIKE '%python3.exe%')
AND (
LOWER("Command") LIKE '%os.walk%' OR LOWER("Command") LIKE '%glob.glob%'
OR LOWER("Command") LIKE '%shutil.copy%' OR LOWER("Command") LIKE '%scandir%'
)
)
ORDER BY starttime DESC Detects automated data collection (T1119) in IBM QRadar by querying process creation events from Windows Security, Sysmon, and WinCollect log sources. Identifies recursive file enumeration via PowerShell/CMD, mass copy operations with robocopy, document archiving via RAR/7z, and Python-based file traversal scripts.
Data Sources
Required Tables
False Positives & Tuning
- Enterprise backup agents (Veeam, Commvault, NetBackup) performing scheduled recursive file copies with archive compression
- Security scanning tools and DLP agents that enumerate file types across directories during scheduled policy checks
- System administrators running legitimate robocopy or xcopy scripts for server migrations or file server restructuring
Other platforms for T1119
Testing Methodology
Validate this detection against 4 adversary techniques from Atomic Red Team. Each test below lists the behaviour to exercise and the telemetry you should expect to see. Executable commands and cleanup steps are available with Pro.
- Test 1PowerShell Recursive Document Collection to Staging Directory
Expected signal: Sysmon Event ID 1: Process Create with Image=powershell.exe, CommandLine containing 'Get-ChildItem', '-Recurse', '.docx', '.xlsx', '.pdf'. Sysmon Event ID 11: Multiple file creation events in %TEMP%\df00tech-stage for each copied file. PowerShell ScriptBlock Log Event ID 4104 with the full collection script. Security Event ID 4663 (if SACL auditing enabled) for each source document read.
- Test 2CMD Recursive File Enumeration with dir /s
Expected signal: Sysmon Event ID 1: Process Create with Image=cmd.exe, CommandLine containing 'dir /s /b' and '.docx', '.xlsx', '.pdf'. Sysmon Event ID 11: File creation event for %TEMP%\df00tech-filelist.txt. Security Event ID 4688 (if process creation auditing and command line logging are enabled) with full command line including extension targets.
- Test 3forfiles Automated Document Enumeration
Expected signal: Sysmon Event ID 1: Process Create for the shell executing forfiles with CommandLine containing 'forfiles', '/S', and '.docx'. Child cmd.exe process creation events as forfiles spawns a cmd.exe instance per matching file. Sysmon Event ID 11: File creation for %TEMP%\df00tech-forfiles.txt. The child cmd.exe processes with 'echo @PATH' are also logged individually.
- Test 47-Zip Archive Collection — Document Staging Pre-Exfiltration
Expected signal: Sysmon Event ID 1: Process Create with Image=7z.exe, CommandLine containing 'a' (add to archive), target paths with '.docx', '.xlsx', '.pdf', '-r' (recursive), and '-p' (password). Sysmon Event ID 11: File creation event for %TEMP%\df00tech-archive.7z. Security Event ID 4663 (if auditing enabled) for each source document file opened by 7z.exe during archiving.
References (12)
- https://attack.mitre.org/techniques/T1119/
- https://www.mandiant.com/resources/blog/unc3944-sms-phishing-sim-swapping-ransomware
- https://unit42.paloaltonetworks.com/unit42-t9000-advanced-modular-backdoor-uses-complex-anti-analysis-techniques/
- https://www.radware.com/security/ddos-threats-center/threat-advisories-and-attack-reports/micropsia-apt-malware/
- https://www.welivesecurity.com/2020/06/11/gamaredon-group-grows-its-game/
- https://dirkjanm.io/introducing-roadtools/
- https://github.com/dirkjanm/ROADtools
- https://github.com/redcanaryco/atomic-red-team/blob/master/atomics/T1119/T1119.md
- https://learn.microsoft.com/en-us/defender-endpoint/advanced-hunting-devicefileevents-table
- https://learn.microsoft.com/en-us/windows-server/administration/windows-commands/forfiles
- https://docs.microsoft.com/en-us/windows-server/administration/windows-commands/robocopy
- https://www.group-ib.com/resources/research/redcurl/
Unlock Pro Content
Get the full detection package for T1119 including response playbook, investigation guide, and atomic red team tests.