Detect Systemctl in Splunk
Adversaries may abuse systemctl to execute commands or programs as systemd services on Linux systems. Systemctl is the primary interface for systemd, the Linux init system and service manager. By crafting malicious service unit files and using systemctl start, enable, and daemon-reload, adversaries can execute arbitrary code immediately and establish persistent execution across reboots. Real-world abuse patterns include TeamTNT deploying cryptocurrency mining services, threat actors writing reverse shell service units pointing to payloads in /dev/shm or /tmp, and web shell compromise chains where an attacker-controlled web process creates a privileged service for lateral movement or persistence. Common subcommands used in attacks include: systemctl start, systemctl enable, systemctl daemon-reload, and systemctl link.
MITRE ATT&CK
- Tactic
- Execution
- Technique
- T1569 System Services
- Sub-technique
- T1569.003 Systemctl
- Canonical reference
- https://attack.mitre.org/techniques/T1569/003/
SPL Detection Query
index=linux_audit sourcetype="linux_audit" type=EXECVE
| search (a0="systemctl" OR a0="/bin/systemctl" OR a0="/usr/bin/systemctl")
| eval systemctl_action=coalesce(a1, "")
| eval service_arg=coalesce(a2, a3, "")
| where systemctl_action IN ("start", "enable", "daemon-reload", "link", "mask")
| eval SuspiciousServiceName=if(
match(service_arg, "(\.sh\.service|\.py\.service|\.pl\.service|\/tmp\/|\/dev\/shm\/)") OR
match(service_arg, "(xmrig|minerd|cpuminer|kworker[^d]|cryptominer|backdoor|revshell|netcat|ncat)"),
1, 0
)
| eval HighRiskAction=if(systemctl_action IN ("enable", "link") AND SuspiciousServiceName=1, 1, 0)
| eval RiskScore=SuspiciousServiceName + HighRiskAction
| join type=left pid [search index=linux_audit sourcetype="linux_audit" type=SYSCALL
| eval pid=tostring(pid)
| fields host, pid, ppid, comm, exe, uid, auid
| rename ppid as parent_pid
]
| eval SuspiciousParentProcess=if(
match(coalesce(comm, ""), "(apache2|nginx|httpd|php|php-fpm|node|java|python|python3|ruby|perl|gunicorn|uwsgi)"),
1, 0
)
| eval RiskScore=RiskScore + SuspiciousParentProcess
| table _time, host, user, uid, auid, systemctl_action, service_arg, comm, exe,
SuspiciousServiceName, HighRiskAction, SuspiciousParentProcess, RiskScore, _raw
| sort - _time
| union [
search index=syslog (sourcetype="syslog" OR sourcetype="linux_syslog") process="systemd"
| search "Created symlink" "/etc/systemd/system/"
| eval RiskScore=1
| eval systemctl_action="enable"
| eval service_arg=replace(_raw, ".*Created symlink.*?([\w\-\.]+\.service).*", "\1")
| eval SuspiciousServiceName=if(
match(service_arg, "(xmrig|miner|backdoor|revshell|netcat)"), 1, 0
)
| table _time, host, sourcetype, systemctl_action, service_arg, SuspiciousServiceName, RiskScore, _raw
]Detects malicious systemctl usage on Linux via auditd EXECVE records and syslog systemd messages. The primary branch parses auditd EXECVE events for systemctl invocations with start/enable/daemon-reload/link subcommands, scoring risk based on suspicious service names (cryptomining tools, paths in /dev/shm or /tmp, reverse shell patterns) and suspicious parent process names (web servers, interpreters). Joins with SYSCALL records to correlate the calling process context. A secondary branch monitors syslog for systemd 'Created symlink' messages indicating service enablement. Requires the Splunk Add-on for Unix and Linux (SA-nix) with auditd configured to log execve syscalls.
Data Sources
Required Sourcetypes
False Positives & Tuning
- Legitimate service management by system administrators via SSH — auditd will log these with auid matching the admin's UID
- Automated configuration management runs by Ansible/Puppet/Chef that use systemctl to enable services — typically have recognizable process chains
- Software installation via apt/yum/dnf that registers systemd services as post-install hooks — comm field will show package manager or its sub-processes
- Monitoring and deployment agents (Datadog, New Relic, Elastic) that manage their own systemd service units during installation and updates
- Developer workstations where users routinely create and manage local systemd --user services for development services
Other platforms for T1569.003
Testing Methodology
Validate this detection against 4 adversary techniques from Atomic Red Team. Each test below lists the behaviour to exercise and the telemetry you should expect to see. Executable commands and cleanup steps are available with Pro.
- Test 1Create and Start Persistence Service with Benign Payload
Expected signal: Auditd PATH record: write to /etc/systemd/system/df00tech-persist.service. Auditd EXECVE records: (1) systemctl daemon-reload, (2) systemctl enable df00tech-persist.service, (3) systemctl start df00tech-persist.service. Syslog/journal: 'Created symlink /etc/systemd/system/multi-user.target.wants/df00tech-persist.service', 'Starting System Performance Monitor...', 'Started System Performance Monitor'. MDE DeviceFileEvents: file create in /etc/systemd/system/. MDE DeviceProcessEvents: systemctl invocations with daemon-reload and enable subcommands.
- Test 2TeamTNT-Style Cryptomining Service Registration
Expected signal: Auditd PATH: write to /etc/systemd/system/kworker-d.service. Auditd EXECVE: systemctl daemon-reload, systemctl enable kworker-d.service. Syslog: 'Created symlink...kworker-d.service'. MDE DeviceFileEvents: new .service file in /etc/systemd/system/ with InitiatingProcessCommandLine containing 'cp /tmp/df00tech-kworker.service'. SPL SuspiciousServiceName=1 fires on 'pool.example.com:4444' pattern and /tmp/ path reference in ExecStart.
- Test 3Reverse Shell Service Unit with /dev/shm Payload Path
Expected signal: Auditd PATH: write to /etc/systemd/system/netconfig.service. Auditd EXECVE: systemctl daemon-reload, systemctl enable netconfig.service. Syslog: 'Created symlink...netconfig.service'. MDE DeviceFileEvents: new service file in /etc/systemd/system/ with /dev/shm pattern detectable if file content is captured.
- Test 4Web Process Spawning Systemctl (Web Shell Simulation)
Expected signal: Auditd EXECVE: systemctl daemon-reload with uid=33 (www-data on Debian/Ubuntu) or current test UID. Auditd SYSCALL: uid/auid fields identifying the web process account. MDE DeviceProcessEvents: systemctl with AccountName=www-data, InitiatingProcessFileName=bash, InitiatingProcessParentFileName may show sudo. Syslog: service enable/start events. The key telemetry is systemctl running under a web service account UID.
References (10)
- https://attack.mitre.org/techniques/T1569/003/
- https://www.redhat.com/en/blog/linux-systemctl-manage-services
- https://blog.talosintelligence.com/teamtnt-with-new-campaign-against/
- https://www.freedesktop.org/software/systemd/man/systemd.service.html
- https://github.com/redcanaryco/atomic-red-team/blob/master/atomics/T1543.002/T1543.002.md
- https://attack.mitre.org/groups/G0139/
- https://www.crowdstrike.com/blog/carbon-spider-morphs-to-target-esxi-servers/
- https://github.com/SigmaHQ/sigma/tree/master/rules/linux
- https://www.microsoft.com/en-us/security/blog/2022/05/19/rise-in-xorddos-a-deeper-look-at-the-stealthy-ddos-malware-targeting-linux/
- https://learn.microsoft.com/en-us/azure/sentinel/connect-syslog
Unlock Pro Content
Get the full detection package for T1569.003 including response playbook, investigation guide, and atomic red team tests.