Detect Fileless Malware Execution with Falco Syscall Monitoring (2026)

Falco’s syscall monitoring lets you detect fileless malware by observing process behavior that deviates from the norm, even when no malicious files are ever written to disk.

Let’s see Falco in action. Imagine a legitimate Python script that’s supposed to download a file, execute it in memory, and then clean up.

# Legitimate Python script (example)
import requests
import subprocess
import os

url = "http://example.com/legitimate_payload.bin"
filename = "temp_payload.bin"

# Download
response = requests.get(url)
with open(filename, "wb") as f:
    f.write(response.content)

# Execute in memory (simulated - actual execution would be more complex)
# For demonstration, we'll just 'cat' it, but imagine a real execve
print(f"Executing {filename}...")
subprocess.run(["cat", filename])

# Clean up
os.remove(filename)
print(f"Cleaned up {filename}.")

Now, let’s overlay this with Falco’s perspective. Falco watches system calls, the fundamental interface between user-space programs and the kernel. When our Python script runs, Falco sees a sequence of syscalls. It’s the pattern and context of these syscalls that reveal malicious intent.

Here’s how Falco can detect a fileless attack:

1. Unexpected Network Connections from Unprivileged Processes:

• Diagnosis: Look for connect syscalls from processes that don’t typically initiate network connections, especially those running as non-root users.
  • # falco -r -j | jq '. | select(.rule.id=="1000001")' (This is a hypothetical rule ID for network connections)
• Fix: Implement strict network egress filtering at the firewall or network security group level. Deny all outbound connections by default and only allow known-good destinations and ports for specific applications.
• Why it works: Fileless malware often needs to download its payload or communicate with a command-and-control server. By restricting network access to only essential services, you choke off this communication channel.

2. Execution of Shellcode from Memory Regions:

• Diagnosis: Monitor for mmap calls that allocate executable memory (PROT_EXEC) followed by writes (write) to that memory, and then an execve or clone syscall targeting that region.
  • # falco -r -j | jq '. | select(.rule.id=="1000002")' (Hypothetical rule for suspicious memory allocation)
• Fix: Configure SELinux or AppArmor policies to prevent processes from executing code within memory regions they have written to. This is often achieved by disallowing mmap with PROT_EXEC for non-standard memory mappings or by marking memory pages as non-executable after writes.
• Why it works: Legitimate applications rarely allocate memory, write shellcode into it, and then execute that shellcode directly. This pattern is a strong indicator of in-memory code execution.

3. Unusual Process Spawning and Inheritance:

• Diagnosis: Detect processes being spawned (clone, fork, execve) by unexpected parent processes, especially when the child process has unusual arguments or environment variables. Look for processes like sh, bash, powershell, or cmd.exe being launched by applications that shouldn’t be invoking a shell.
  • # falco -r -j | jq '. | select(.rule.id=="1000003")' (Hypothetical rule for suspicious process spawning)
• Fix: Employ strict process execution policies. Use tools like auditd or OS-level controls to whitelist legitimate process creation chains and deny all others. For example, only allow cron to spawn shell scripts, not your web server.
• Why it works: Malware often uses legitimate system utilities (like powershell.exe or sh) to execute commands or scripts in memory, but these utilities are typically invoked by specific parent processes (e.g., explorer.exe for cmd.exe on Windows, or systemd for sh on Linux). Deviations are suspicious.

4. Direct Kernel Object Manipulation (DKOM):

• Diagnosis: Monitor for attempts to directly write to kernel memory structures, often by processes that have elevated privileges. This is harder to detect with syscalls alone but can sometimes manifest as unexpected ioctl calls to device files or attempts to modify kernel module lists (lsmod).
  • # falco -r -j | jq '. | select(.rule.id=="1000004")' (Hypothetical rule for suspicious kernel interactions)
• Fix: Enable kernel integrity checking modules (like kprobes or kexec-tools with integrity checks) and ensure that only trusted kernel modules can be loaded. Restrict access to /dev/mem and /proc/kcore.
• Why it works: Advanced malware might try to hide itself by manipulating kernel data structures directly to evade detection. This is a low-level attack that requires specific kernel access.

5. Use of Obfuscated or Unsigned Binaries:

• Diagnosis: Track the execve syscalls for binaries that lack a valid digital signature (on Windows) or are not part of a trusted package repository (on Linux). Also, flag executables that appear heavily obfuscated or have unusual section names.
  • # falco -r -j | jq '. | select(.rule.id=="1000005")' (Hypothetical rule for unsigned/obfuscated binaries)
• Fix: Implement strict application whitelisting that only allows known, signed, and verified executables to run. Regularly scan your systems for unsigned or suspicious binaries that appear in unexpected locations.
• Why it works: Malware authors often use obfuscation to hide their code and avoid signature-based detection. Unsigned binaries on systems where all software should be signed are a red flag.

6. In-Memory Injection into Legitimate Processes:

• Diagnosis: Monitor for ptrace syscalls where a process is attaching to another, seemingly unrelated process, especially if it’s followed by memory writes (process_vm_writev) and thread creation (clone) within the target process.
  • # falco -r -j | jq '. | select(.rule.id=="1000006")' (Hypothetical rule for process injection)
• Fix: Configure security policies that restrict the ability of unprivileged processes to ptrace other processes. On Linux, this can be controlled via kernel.yama.ptrace_scope. Setting this to 1 or 2 significantly limits ptrace capabilities.
• Why it works: Malware often injects its code into legitimate running processes to gain privileges, hide its presence, and leverage the process’s existing network connections or permissions. ptrace is a common mechanism for this.

By correlating these syscall patterns, Falco can identify fileless malware executing in memory, even if it never touches the disk.

The next thing you’ll likely encounter after fixing these is dealing with polymorphic malware that constantly changes its execution patterns.