Building a Custom Live Memory Dumper for YARA Hits in Go

During dynamic malware analysis, timing is everything. Ephemeral payloads often unpack themselves into memory, execute their staging logic, and immediately overwrite or unmap injected memory regions to evade standard endpoint security agents. If you wait until a process exits or rely on periodic scheduled memory dumps, you will miss the payload in its unpacked state.

To solve this, I built a zero-dependency CLI tool in Go that monitors running userland processes, scans their active virtual memory space against compiled YARA rules, and dumps full process memory regions the instant a signature matches.

Mapping Virtual Memory in Userland

The first challenge is safely enumerating process memory without triggering access violations or causing unnecessary system overhead. On Windows, this relies on three core APIs: CreateToolhelp32Snapshot, OpenProcess, and VirtualQueryEx.

We start by walking the running process tree. For each targeted process ID, we request a process handle with PROCESS_VM_READ and PROCESS_QUERY_INFORMATION rights. Before reading memory, we must acquire SeDebugPrivilege within the token context of our scanner tool; otherwise, system processes or processes running under elevated contexts will return an access error.

Once a valid handle is obtained, we iterate through the process address space page by page using VirtualQueryEx. This populates a structure that tells us the state, type, and protection level of each memory region:

• State: We only care about committed memory (MEM_COMMIT). Scanning reserved or uncommitted regions allocates unnecessary CPU cycles and causes read errors.
• Protection: We filter for readable pages (PAGE_READONLY, PAGE_READWRITE, PAGE_EXECUTE_READ, PAGE_EXECUTE_READWRITE). Scanning pages with PAGE_GUARD or PAGE_NOACCESS flags will throw exceptions or alert kernel-level monitoring drivers.

Efficient Pattern Matching in RAM

Once we identify a committed, readable memory block, we read its contents using ReadProcessMemory. Passing entire gigabytes of process memory directly into RAM buffers will quickly exhaust host system memory, especially when monitoring multiple processes simultaneously.

To prevent this, the Go tool processes memory in chunked buffer streams. Each page chunk is passed directly to Cgo bindings wrapping native rule engines. Rather than writing process memory to disk and reading it back into the scanner—which adds heavy I/O latency—we execute direct in-memory scanning:

for address < maxAddress {
    mbi := VirtualQueryEx(hProcess, address)
    if mbi.State == MEM_COMMIT && isReadable(mbi.Protect) {
        buffer := make([]byte, mbi.RegionSize)
        if ReadProcessMemory(hProcess, mbi.BaseAddress, &buffer) {
            matches := yaraScanner.ScanMem(buffer)
            if len(matches) > 0 {
                triggerDump(pid, mbi.BaseAddress, buffer, matches)
            }
        }
    }
    address += mbi.RegionSize
}

When a rule triggers a match, the tool instantly freezes the target thread context using SuspendThread (or NtSuspendProcess for multi-threaded payloads), writes the target memory space to an output raw dump file, and logs the specific offset and rule name that triggered the detection.

Operational Lessons and Optimization

Building and deploying this tool across test lab environments revealed several practical edge cases:

1. Worker Pools and Throttling: Running continuous scans across dozens of rapidly spawning child processes can easily pin host CPU usage to 100%. Implementing a worker pool with a rate limiter ensures the utility stays lightweight and does not distort execution metrics during sandbox runs.
2. Handling Short-Lived Processes: Malware loaders often spawn, inject, and terminate within a few hundred milliseconds. To catch these, the tool subscribes to Process Creation Event Tracing for Windows (ETW) notifications to kick off near-instantaneous scans as soon as a new process ID registers.
3. Dumping Self-Modifying Code: Certain packers alter memory page permissions back to restricted access flags immediately after execution. The tool includes an option to temporarily reprotect pages via VirtualProtectEx to read execution artifacts before restoring original protection flags.

By keeping the utility small, modular, and natively compiled, it acts as an immediate triage step in an automated malware detonation pipeline—capturing ephemeral code in memory without the weight of full virtual machine snapshots.

Photo by Conny Schneider on Unsplash.