マルウェアのアンパッキングと逆解析回避：実世界の手法の深掘り

Malware authors don't make our job easy. Every time we think we've figured out their tricks, they layer on another obfuscation technique, another anti-debugging check, another sandbox evasion. Over the past few weeks, I've been deep in the trenches with some particularly stubborn samples — the kind that detect your debugger, hide their strings behind XOR encoding, and hollow out legitimate processes to hide their payload. This article walks through my hands-on exploration of these techniques. We'll look at how malware detects analysis tools, how it obfuscates its strings, how it unpacks itself in memory, and most importantly — how we can bypass these defenses to see what the malware is actually trying to do. The tools we'll use: x64dbg/x32dbg for dynamic analysis and patching IDA Pro for static disassembly REMnux (Linux toolkit) for string deobfuscation FLOSS, XORSearch, bbcrack for automated string decoding Scylla & OllyDumpEx for dumping unpacked payloads Process Hacker for memory forensics Problem Statement Modern malware is rarely "what you see is what you get." A single executable might be: Packed — the actual malicious code is compressed/encrypted and only revealed at runtime Anti-debug aware — it checks for debuggers and changes behavior or terminates Sandbox-aware — it detects virtualized environments and refuses to execute its payload String-obfuscated — URLs, registry keys, and IOCs are encoded to evade signature detection Process-injecting — it hollows out a legitimate process (like explorer.exe ) and runs its code there Our goal: peel back these layers and extract the real payload for analysis. Exercise 1: Bypassing Debugger Detection in getdown.exe What I Found The first sample, getdown.exe , refused to show any network activity when run inside a debugger. Outside the debugger, it connected to 1.234.27.146:80 . Classic anti-debugging behavior. The Detection Mechanism Using x64dbg, I searched for intermodular calls and immediately spotted IsDebuggerPresent at the top of the list: call qword ptr ds:[<&IsDebuggerPresent>] test eax, eax jne getdown.140001216 ; jumps away if debugger detected When IsDebuggerPresent returns 1 (debugger present), the JNE jumps to code that terminates the process. When it returns 0 , execution continues normally. The Bypass I set a breakpoint on the TEST EAX, EAX instruction after the call. When hit, RAX contained 1 — confirming debugger detection. The fix was simple: patch the conditional jump to unconditional NOPs . ; Before (at 14000102C): jne getdown.140001216 ; After patching: nop nop nop nop nop nop In x64dbg: select the JNE instruction → press Space → type NOP → enable "Fill with NOP's" → OK. Now the malware executes its payload regardless of what IsDebuggerPresent returns. Exercise 2: Deobfuscating Encoded Strings Malware hides its strings using simple encoding algorithms. I experimented with several tools to decode them. Tool 1: XORSearch Searching for the known IP 1.234.27. inside getdown.exe : xorsearch -i -s getdown.exe 1.234.27. Output revealed an XOR key of 0x83 . The -s flag generated getdown.exe.XOR.83 — a fully decoded file. Extracting strings from it: strings getdown.exe.XOR.83 | more This surfaced not just the C2 URL, but also an affiliate ID string — a nice unexpected find. Tool 2: brxor.py & bbcrack.py On hubert.dll , brxor.py automatically found XOR key 0x5 and decoded English-language strings: brxor.py hubert.dll The output suggested this sample was a fake antivirus tool , complete with registry key paths and URL patterns — solid IOCs for further hunting. For deeper analysis, bbcrack.py with single-level transformations: bbcrack.py -l 1 hubert.dll This generated hubert_xor05.dll . Running strings on it confirmed the decoded content matched and extended what brxor.py found. Tool 3: Manual Stack String Decoding in IDA Some malware builds strings character-by-character on the stack at runtime. In 9.exe , between offsets 40133D and 4013B8 , I found a block of MOV instructions pushing single bytes: mov byte ptr [ebp+var_4], 5Ch ; '' mov byte ptr [ebp+var_3], 50h ; 'P' ; ... etc In IDA, highlighting each hex value and pressing R converts it to ASCII. Doing this for the entire block revealed the string: \Program Files\Common Files\ Tool 4: Automated Stack String Extraction For 9.exe , strdeob.pl on REMnux decoded these automatically: strdeob.pl 9.exe | more This found more strings than FLOSS managed to extract, showing the value of combining multiple tools: floss 9.exe > 9-floss.txt Exercise 3: Unpacking drtg.exe Using RtlDecompressBuffer Static Recon with FLOSS Before touching the debugger, I ran FLOSS on drtg.exe : floss drtg.exe > drtg-floss.txt FLOSS decoded 27 obfuscated strings, including: NtAllocateVirtualMemory ZwProtectVirtualMemory ZwWriteVirtualMemory RtlDecompressBuffer These APIs scream "unpacking and injection." The presence of RtlDecompressBuffer was particularly interesting — it decompresses a buffer using a known compression algorithm. Dynamic Unpacking in x32dbg Enable ScyllaHide (Plugins → Scylla Hide → Options → check all first-column boxes) to cloak the debugger. Set breakpoint on RtlDecompressBuffer : SetBPX RtlDecompressBuffer Run (F9) until the breakpoint hits inside ntdll.dll . Follow the UncompressedBuffer parameter (second arg, [esp+8] ) in the Dump panel. Initially all zeros. Set a return breakpoint at 402D66 (where execution returns after RtlDecompressBuffer ): Go to 402D66 (Ctrl+G) Toggle breakpoint (F2) Run (F9) After returning, the Dump panel shows decompressed data. Step over (F8) nine times until offset 4022CC . The dump now shows: 4D 5A 90 00 03 00 00 00 04 00 00 00 FF FF 00 00 MZ.ÿÿ.. B8 00 00 00 00 00 00 00 40 00 00 00 00 00 00 00 ¸.......@....... That MZ header and "This program cannot be run in DOS mode" confirms a valid PE file was decompressed into memory. Dump the memory region : Right-click Dump → Follow in Memory Map → right-click region → Dump Memory to File → save as drtg-dumped.exe . Verification Loaded drtg-dumped.exe in PeStudio — parsed successfully. Checked imports and found NtQueryInformationProcess — another anti-debug API. Bonus: Finding the Anti-Debug Check In IDA, at offset 4054B7 : push 7 ; ProcessInformationClass = 7 (debug port) lea eax, [ebp-4] push eax ; ProcessInformation push 4 ; ProcessInformationLength call GetCurrentProcess push eax ; ProcessHandle call NtQueryInformationProcess cmp dword ptr [ebp-4], 0 jnz short loc_4058C9 ; jumps to ExitProcess if debugged The value 7 for ProcessInformationClass retrieves the debugger port number . Non-zero means a debugger is attached. To bypass: patch the JNZ at 4054C1 to JZ — invert the logic. Exercise 4: Behavioral Analysis of WinHost32.exe Before unpacking, I wanted to see what this sample actually does when it runs. Setup REMnux : Wireshark capturing Windows REM Workstation : Process Hacker + Process Monitor running Infection Ran WinHost32.exe for ~30 seconds, then terminated it. Network Findings (Wireshark) DNS queries attempting to resolve google.com . No direct C2 connections in the short window — likely a connectivity check before revealing true behavior. Process Monitor → ProcDOT Exported the ProcMon log as CSV, loaded it into ProcDOT : Selected the initial WinHost32.exe PID as the launcher Generated the graph The graph revealed process hollowing : the initial process spawned a child process with the same name. The child process showed registry activity (likely false positives, but worth noting). Exercise 5: Unpacking WinHost32.exe (Process Hollowing) Static Analysis in IDA At offset 4021AE , CreateProcessA is called with dwCreationFlags = 4 : push 4 ; CREATE_SUSPENDED push ... call CreateProcessA CREATE_SUSPENDED is a red flag — the process starts with its main thread frozen, ready for manipulation. Following the code: 40220D : VirtualAllocEx with flProtect = 0x40 (PAGE_EXECUTE_READWRITE) 402237 : WriteProcessMemory — writing into the suspended process 4021F7 : call edi where EDI = NtUnmapViewOfSection This is the process hollowing pattern: unmap the legitimate executable's memory, allocate new RWX memory, write the malicious payload, then resume the thread. Dynamic Extraction in x32dbg Load WinHost32.exe , go to 402237 (Ctrl+G) Set breakpoint on the call ebp (WriteProcessMemory) Run (F9) until breakpoint The lpBuffer parameter (3rd arg, [esp+8] ) points to the payload. Follow in Dump Dump shows MZ header — the payload is a full PE Follow in Memory Map → Dump Memory to File → winhost32-dumped.exe Verification PeStudio confirmed it's a valid PE. Strings revealed: Multiple suspicious URLs google.com (the connectivity check we saw in Wireshark) Many more plaintext strings than the packed version Exercise 6: Anti-Sandbox via Mouse Hooks ( vbprop.exe ) Some malware refuses to execute until a real user interacts with it. This sample uses SetWindowsHookExA to wait for mouse events. The Hook Setup (IDA) At 40103E : push 0 ; hmod = 0 (current process) push offset fn ; lpfn = 4010B0 (our hook function) push 0Eh ; idHook = WH_MOUSE (0x0E) call SetWindowsHookExA The hook function at 4010B0 checks wParam : 0x200 (WM_MOUSEMOVE) → pass to next hook 0x201 (WM_LBUTTONDOWN) → pass to next hook 0x202 (WM_LBUTTONUP) → unhook and execute malicious payload ( sub_401170 ) The malware only reveals itself when you release the left mouse button . Automated sandboxes that don't simulate real mouse clicks will never see the payload execute. Exercise 7: Sandbox Evasion via Connectivity Check ( winhost32-dumped.exe ) The unpacked WinHost32.exe checks for internet connectivity before executing its payload. It doesn't just check if the network is up — it downloads http://google.com and verifies the response starts with <!do (the beginning of <!doctype html> ). The Logic Flow sub_401A90(): download http://google.com check if buffer[0:4] == "<!do" return 1 if match, 0 otherwise Caller at 4023B0: call sub_401A90 test eax, eax jnz loc_4023C6 ; success path push 0EA60h ; 60000 ms