When security researchers analyze suspicious software, they typically employ three primary methodologies: debugging with tools like x64dbg or WinDbg, executing within virtual machines for isolation, and observing behavior in automated sandboxes. Each methodology reveals different aspects of a program's behavior, but each also leaves distinctive fingerprints that can be detected. Anti-analysis techniques exploit these fingerprints to identify when code is being examined, allowing malware to alter its behavior or refuse to execute entirely.
Understanding these techniques serves a dual purpose. For red team operators, they provide methods to protect sensitive tooling from reverse engineering and sandbox detonation. For defenders and analysts, understanding evasion techniques helps design more robust analysis environments and recognize when samples are actively evading examination.
ANTI-ANALYSIS TECHNIQUE HIERARCHY
==================================
┌─────────────────────┐
│ ANTI-ANALYSIS │
│ TECHNIQUES │
└──────────┬──────────┘
│
┌──────────────────────┼──────────────────────┐
│ │ │
▼ ▼ ▼
┌───────────────┐ ┌───────────────┐ ┌───────────────┐
│ ANTI-DEBUG │ │ ANTI-VM │ │ ANTI-SANDBOX │
│ │ │ │ │ │
│ • API checks │ │ • CPUID │ │ • Timing │
│ • Timing │ │ • MAC address │ │ • Environment │
│ • Breakpoints │ │ • Registry │ │ • Interaction │
│ • PEB flags │ │ • Processes │ │ • Resources │
│ • Self-debug │ │ • Files │ │ • Delay │
└───────────────┘ └───────────────┘ └───────────────┘
DETECTION PHILOSOPHY:
┌────────────────────────────────────────────────────────────────┐
│ │
│ Real systems have: Analysis environments have: │
│ • User activity • Automation │
│ • Many processes • Minimal processes │
│ • Real hardware • Virtualized hardware │
│ • History/artifacts • Fresh/clean state │
│ • Debugging flags OFF • Debugging flags ON │
│ │
│ The goal: Identify characteristics that distinguish │
│ real operational environments from analysis contexts │
│ │
└────────────────────────────────────────────────────────────────┘Debuggers are the primary tool for dynamic analysis, allowing analysts to pause execution, inspect memory, and trace program flow. However, debuggers must integrate with the operating system's debugging infrastructure, and this integration leaves detectable traces throughout the system.
When a debugger attaches to a process, Windows sets several flags within the Process Environment Block (PEB) to indicate the debugging state. The most straightforward check examines the BeingDebugged flag, which is precisely what the IsDebuggerPresent API reads internally.
#include <windows.h>
// The simplest approach - calls kernel32!IsDebuggerPresent
BOOL CheckIsDebuggerPresent(void) {
return IsDebuggerPresent();
}
// Direct PEB access bypasses potential API hooks
BOOL CheckPEBDebugFlag(void) {
#ifdef _WIN64
// On x64, the TEB is at GS:[0], and TEB.ProcessEnvironmentBlock
// is at offset 0x60 from the TEB base
PPEB pPeb = (PPEB)__readgsqword(0x60);
#else
// On x86, the TEB is at FS:[0], and TEB.ProcessEnvironmentBlock
// is at offset 0x30 from the TEB base
PPEB pPeb = (PPEB)__readfsdword(0x30);
#endif
// PEB.BeingDebugged is at offset 0x2 (single byte)
return pPeb->BeingDebugged;
}The direct PEB access method is more robust because it bypasses any hooks placed on the IsDebuggerPresent API. Many anti-anti-debugging tools work by hooking this function to always return FALSE, but they cannot easily intercept direct segment register reads.
While the PEB flag is easily manipulated, Windows maintains additional debugging state that is harder to forge. The NtQueryInformationProcess function can query several debugging-related information classes that provide more reliable detection.
#include <windows.h>
typedef NTSTATUS (NTAPI* fnNtQueryInformationProcess)(
HANDLE ProcessHandle,
ULONG ProcessInformationClass,
PVOID ProcessInformation,
ULONG ProcessInformationLength,
PULONG ReturnLength
);
// Information classes for debugging detection
#define ProcessDebugPort 7 // Returns debug port handle
#define ProcessDebugObjectHandle 30 // Returns debug object handle
#define ProcessDebugFlags 31 // Returns debug inheritance flag
BOOL CheckDebugPort(void) {
fnNtQueryInformationProcess pNtQueryInformationProcess =
(fnNtQueryInformationProcess)GetProcAddress(
GetModuleHandleW(L"ntdll.dll"),
"NtQueryInformationProcess"
);
DWORD_PTR dwDebugPort = 0;
NTSTATUS status = pNtQueryInformationProcess(
GetCurrentProcess(),
ProcessDebugPort,
&dwDebugPort,
sizeof(dwDebugPort),
NULL
);
// When debugged, the debug port is set (typically -1)
// When not debugged, the value is 0
return (NT_SUCCESS(status) && dwDebugPort != 0);
}The debug port check is particularly effective because the debug port is maintained by the kernel and cannot be easily modified from user mode. Even if an analyst patches the PEB flags, the debug port will still reveal the attached debugger.
BOOL CheckDebugObjectHandle(void) {
fnNtQueryInformationProcess pNtQueryInformationProcess =
(fnNtQueryInformationProcess)GetProcAddress(
GetModuleHandleW(L"ntdll.dll"),
"NtQueryInformationProcess"
);
HANDLE hDebugObject = NULL;
NTSTATUS status = pNtQueryInformationProcess(
GetCurrentProcess(),
ProcessDebugObjectHandle,
&hDebugObject,
sizeof(hDebugObject),
NULL
);
// If we can retrieve a debug object handle, we're being debugged
if (NT_SUCCESS(status) && hDebugObject != NULL) {
CloseHandle(hDebugObject);
return TRUE;
}
return FALSE;
}
BOOL CheckDebugFlags(void) {
fnNtQueryInformationProcess pNtQueryInformationProcess =
(fnNtQueryInformationProcess)GetProcAddress(
GetModuleHandleW(L"ntdll.dll"),
"NtQueryInformationProcess"
);
DWORD dwNoDebugInherit = 0;
NTSTATUS status = pNtQueryInformationProcess(
GetCurrentProcess(),
ProcessDebugFlags,
&dwNoDebugInherit,
sizeof(dwNoDebugInherit),
NULL
);
// ProcessDebugFlags returns 1 if NO debugger, 0 if debugged
// (Indicates whether child processes should inherit debug port)
return (NT_SUCCESS(status) && dwNoDebugInherit == 0);
}Modern processors provide hardware debugging registers (DR0-DR7) that allow setting breakpoints without modifying code. These registers are part of the thread context and can be examined programmatically.
HARDWARE DEBUG REGISTERS
========================
┌─────────────────────────────────────────────────────────────┐
│ DEBUG REGISTERS │
├─────────────────────────────────────────────────────────────┤
│ │
│ DR0 ─────► Breakpoint 0 address │
│ DR1 ─────► Breakpoint 1 address │
│ DR2 ─────► Breakpoint 2 address │
│ DR3 ─────► Breakpoint 3 address │
│ │
│ DR6 ─────► Debug status (which BP triggered) │
│ DR7 ─────► Debug control (BP enable/conditions) │
│ │
├─────────────────────────────────────────────────────────────┤
│ Detection: If DR0-DR3 contain non-zero values, │
│ hardware breakpoints are set = active debugging │
└─────────────────────────────────────────────────────────────┘#include <windows.h>
BOOL CheckHardwareBreakpoints(void) {
CONTEXT ctx = { 0 };
ctx.ContextFlags = CONTEXT_DEBUG_REGISTERS;
// GetThreadContext retrieves the debug register values
if (!GetThreadContext(GetCurrentThread(), &ctx)) {
return FALSE;
}
// DR0-DR3 hold breakpoint addresses
// Any non-zero value indicates a hardware breakpoint is set
if (ctx.Dr0 || ctx.Dr1 || ctx.Dr2 || ctx.Dr3) {
return TRUE;
}
return FALSE;
}Beyond detection, malware can actively clear hardware breakpoints to disrupt debugging sessions:
BOOL ClearHardwareBreakpoints(void) {
CONTEXT ctx = { 0 };
ctx.ContextFlags = CONTEXT_DEBUG_REGISTERS;
if (!GetThreadContext(GetCurrentThread(), &ctx)) {
return FALSE;
}
// Zero all debug registers
ctx.Dr0 = 0;
ctx.Dr1 = 0;
ctx.Dr2 = 0;
ctx.Dr3 = 0;
ctx.Dr6 = 0; // Debug status
ctx.Dr7 = 0; // Debug control (disables all breakpoints)
return SetThreadContext(GetCurrentThread(), &ctx);
}Debuggers typically set software breakpoints by replacing the first byte of an instruction with INT3 (opcode 0xCC). When executed, this generates a breakpoint exception that the debugger catches. By scanning code sections for INT3 bytes, or by checksumming code to detect modifications, programs can detect tampering.
#include <windows.h>
// Scan a code region for INT3 instructions
BOOL CheckCodeIntegrity(PVOID pFunction, SIZE_T sSize) {
PBYTE pCode = (PBYTE)pFunction;
for (SIZE_T i = 0; i < sSize; i++) {
if (pCode[i] == 0xCC) { // INT3 opcode
return TRUE; // Software breakpoint detected
}
}
return FALSE;
}
// More robust: checksum entire code region
DWORD CalculateChecksum(PVOID pStart, SIZE_T sSize) {
PBYTE pData = (PBYTE)pStart;
DWORD dwChecksum = 0;
for (SIZE_T i = 0; i < sSize; i++) {
dwChecksum += pData[i];
dwChecksum ^= (dwChecksum << 5);
}
return dwChecksum;
}
// Store expected checksum (calculated at compile time or first run)
DWORD g_dwExpectedChecksum = 0;
BOOL VerifyCodeIntegrity(PVOID pFunction, SIZE_T sSize) {
DWORD dwCurrentChecksum = CalculateChecksum(pFunction, sSize);
return (dwCurrentChecksum == g_dwExpectedChecksum);
}The checksum approach is more robust because it detects any modification, not just INT3 insertions. Some debuggers use different breakpoint patterns or hook functions with JMP instructions, which INT3 scanning would miss.
Perhaps the most elegant category of anti-debugging techniques exploits timing. When an analyst steps through code instruction-by-instruction, the execution time increases by orders of magnitude. By measuring time before and after operations, code can detect this dramatic slowdown.
TIMING ATTACK PRINCIPLE
=======================
Normal execution: Debugging (stepping):
┌─────┐ ┌─────┐
│Code │ ~1000 CPU cycles │Code │ ~10,000,000+ cycles
└─────┘ └─────┘
│ │
▼ ▼ (analyst examines state)
┌─────┐ ┌─────┐
│More │ │More │
│Code │ │Code │
└─────┘ └─────┘
Time source options:
• RDTSC - CPU timestamp counter (highest resolution)
• QPC - QueryPerformanceCounter (kernel-backed)
• GTick - GetTickCount (millisecond resolution)
• Time API - GetSystemTime (second resolution)#include <windows.h>
#include <intrin.h>
// RDTSC provides the highest resolution timing
BOOL CheckTiming_RDTSC(void) {
ULONGLONG ullStart = __rdtsc();
// Execute some operations that debuggers slow down
for (int i = 0; i < 100; i++) {
__nop(); // No-operation, but must be executed
}
ULONGLONG ullEnd = __rdtsc();
ULONGLONG ullDelta = ullEnd - ullStart;
// Normal execution: typically < 1000 cycles
// Single-stepping: easily > 10000 cycles per instruction
return (ullDelta > 10000);
}
// GetTickCount has lower resolution but is harder to fake
BOOL CheckTiming_GetTickCount(void) {
DWORD dwStart = GetTickCount();
// Operations to measure
Sleep(0); // Force context switch
for (int i = 0; i < 1000; i++) {
GetCurrentProcessId();
}
DWORD dwEnd = GetTickCount();
DWORD dwDelta = dwEnd - dwStart;
// Should complete in < 10ms normally
// Under debugger with breakpoints: much longer
return (dwDelta > 100);
}
// QueryPerformanceCounter for microsecond precision
BOOL CheckTiming_QPC(void) {
LARGE_INTEGER liStart, liEnd, liFreq;
QueryPerformanceFrequency(&liFreq);
QueryPerformanceCounter(&liStart);
// Code to time
for (int i = 0; i < 1000; i++) {
__nop();
}
QueryPerformanceCounter(&liEnd);
LONGLONG llDelta = liEnd.QuadPart - liStart.QuadPart;
double dSeconds = (double)llDelta / liFreq.QuadPart;
// More than 10ms indicates likely debugging
return (dSeconds > 0.01);
}When a process starts under a debugger, Windows enables additional heap debugging features that help analysts track memory corruption. These features are controlled by flags in the PEB that are not normally set.
#include <windows.h>
// Heap debugging flags set when debugging
#define FLG_HEAP_ENABLE_TAIL_CHECK 0x10
#define FLG_HEAP_ENABLE_FREE_CHECK 0x20
#define FLG_HEAP_VALIDATE_PARAMETERS 0x40
BOOL CheckNtGlobalFlag(void) {
#ifdef _WIN64
PPEB pPeb = (PPEB)__readgsqword(0x60);
// NtGlobalFlag is at offset 0xBC in the 64-bit PEB
DWORD dwNtGlobalFlag = *(PDWORD)((PBYTE)pPeb + 0xBC);
#else
PPEB pPeb = (PPEB)__readfsdword(0x30);
// NtGlobalFlag is at offset 0x68 in the 32-bit PEB
DWORD dwNtGlobalFlag = *(PDWORD)((PBYTE)pPeb + 0x68);
#endif
// Check for heap debugging flags
DWORD dwDebugFlags = FLG_HEAP_ENABLE_TAIL_CHECK |
FLG_HEAP_ENABLE_FREE_CHECK |
FLG_HEAP_VALIDATE_PARAMETERS;
return (dwNtGlobalFlag & dwDebugFlags) != 0;
}The heap structures themselves also contain flags that indicate debugging state:
BOOL CheckHeapFlags(void) {
#ifdef _WIN64
PPEB pPeb = (PPEB)__readgsqword(0x60);
// PEB.ProcessHeap at offset 0x30
PVOID pProcessHeap = *(PVOID*)((PBYTE)pPeb + 0x30);
// HEAP.Flags at offset 0x70, ForceFlags at 0x74
DWORD dwFlags = *(PDWORD)((PBYTE)pProcessHeap + 0x70);
DWORD dwForceFlags = *(PDWORD)((PBYTE)pProcessHeap + 0x74);
#else
PPEB pPeb = (PPEB)__readfsdword(0x30);
PVOID pProcessHeap = *(PVOID*)((PBYTE)pPeb + 0x18);
DWORD dwFlags = *(PDWORD)((PBYTE)pProcessHeap + 0x40);
DWORD dwForceFlags = *(PDWORD)((PBYTE)pProcessHeap + 0x44);
#endif
// Normal: Flags = HEAP_GROWABLE (0x2), ForceFlags = 0
// Debugged: Additional flags set, ForceFlags != 0
return (dwForceFlags != 0);
}The OutputDebugString function sends a string to any attached debugger. When no debugger is present, the function behavior differs in a subtle way related to the last error code:
#include <windows.h>
BOOL CheckOutputDebugString(void) {
// Set a known error code
SetLastError(0x12345678);
// Call OutputDebugString
OutputDebugStringA("Anti-debug test string");
// If no debugger: error code remains unchanged
// If debugger: error code changes (typically to 0)
return (GetLastError() != 0x12345678);
}This technique exploits an implementation detail where the presence of a debugger changes the function's error-setting behavior. It's a subtle check that many anti-anti-debugging tools miss.
Beyond detecting debugger attachment, malware often searches for common analysis tools running on the system. The presence of IDA, x64dbg, Wireshark, or Process Monitor strongly suggests the machine is being used for analysis.
#include <windows.h>
#include <tlhelp32.h>
const wchar_t* g_wszDebuggerProcesses[] = {
L"x64dbg.exe",
L"x32dbg.exe",
L"ollydbg.exe",
L"ida.exe",
L"ida64.exe",
L"idaq.exe",
L"idaq64.exe",
L"windbg.exe",
L"dbgview.exe",
L"processhacker.exe",
L"procmon.exe",
L"procmon64.exe",
L"procexp.exe",
L"procexp64.exe",
L"ghidra.exe",
L"fiddler.exe",
L"wireshark.exe",
L"dumpcap.exe",
NULL
};
BOOL CheckDebuggerProcesses(void) {
HANDLE hSnapshot = CreateToolhelp32Snapshot(TH32CS_SNAPPROCESS, 0);
if (hSnapshot == INVALID_HANDLE_VALUE) return FALSE;
PROCESSENTRY32W pe32 = { sizeof(PROCESSENTRY32W) };
BOOL bFound = FALSE;
if (Process32FirstW(hSnapshot, &pe32)) {
do {
for (int i = 0; g_wszDebuggerProcesses[i] != NULL; i++) {
if (_wcsicmp(pe32.szExeFile, g_wszDebuggerProcesses[i]) == 0) {
bFound = TRUE;
break;
}
}
} while (!bFound && Process32NextW(hSnapshot, &pe32));
}
CloseHandle(hSnapshot);
return bFound;
}Window enumeration provides another angle, checking for debugger windows even if the process name has been changed:
BOOL CALLBACK EnumWindowsProc(HWND hwnd, LPARAM lParam) {
wchar_t wszTitle[256] = { 0 };
GetWindowTextW(hwnd, wszTitle, 256);
// Check for common debugger window title patterns
if (wcsstr(wszTitle, L"x64dbg") ||
wcsstr(wszTitle, L"x32dbg") ||
wcsstr(wszTitle, L"OllyDbg") ||
wcsstr(wszTitle, L"IDA") ||
wcsstr(wszTitle, L"WinDbg") ||
wcsstr(wszTitle, L"Ghidra")) {
*(PBOOL)lParam = TRUE;
return FALSE; // Stop enumeration
}
return TRUE;
}
BOOL CheckDebuggerWindows(void) {
BOOL bFound = FALSE;
EnumWindows(EnumWindowsProc, (LPARAM)&bFound);
return bFound;
}Windows permits only one debugger per process. By debugging a child process (or having a parent process debug us), we prevent other debuggers from attaching.
#include <windows.h>
// Start ourselves as a child and debug it, preventing other attachments
BOOL SelfDebug(void) {
STARTUPINFOW si = { sizeof(si) };
PROCESS_INFORMATION pi = { 0 };
wchar_t wszSelf[MAX_PATH];
GetModuleFileNameW(NULL, wszSelf, MAX_PATH);
// Create child process in debug mode
if (!CreateProcessW(
wszSelf,
NULL, NULL, NULL,
FALSE,
DEBUG_ONLY_THIS_PROCESS, // Attach as debugger
NULL, NULL,
&si,
&pi
)) {
return FALSE;
}
// Handle debug events (must pump the debug loop)
DEBUG_EVENT de;
while (WaitForDebugEvent(&de, INFINITE)) {
if (de.dwDebugEventCode == EXIT_PROCESS_DEBUG_EVENT) {
break;
}
ContinueDebugEvent(de.dwProcessId, de.dwThreadId, DBG_CONTINUE);
}
CloseHandle(pi.hThread);
CloseHandle(pi.hProcess);
return TRUE;
}Virtual machines provide isolated analysis environments, but the virtualization layer introduces detectable artifacts. Modern hypervisors try to hide their presence, but perfect concealment is difficult when the guest must interact with virtualized hardware.
The x86 CPUID instruction returns information about the processor. Hypervisors set a specific bit (bit 31 of ECX after CPUID with EAX=1) to indicate their presence. Additionally, a special CPUID leaf (0x40000000) returns the hypervisor's vendor string.
CPUID HYPERVISOR DETECTION
==========================
CPUID EAX=1:
┌────────────────────────────────────────────────────────────┐
│ ECX Register: │
│ │
│ Bit 31: Hypervisor Present │
│ 0 = Running on bare metal │
│ 1 = Running under hypervisor │
└────────────────────────────────────────────────────────────┘
CPUID EAX=0x40000000 (Hypervisor leaf):
┌────────────────────────────────────────────────────────────┐
│ Returns hypervisor vendor string in EBX:ECX:EDX │
│ │
│ "VMwareVMware" - VMware │
│ "VBoxVBoxVBox" - VirtualBox │
│ "Microsoft Hv" - Hyper-V │
│ "KVMKVMKVM" - KVM │
│ "XenVMMXenVMM" - Xen │
└────────────────────────────────────────────────────────────┘#include <windows.h>
#include <intrin.h>
BOOL CheckCPUID_Hypervisor(void) {
int cpuInfo[4] = { 0 };
// CPUID with EAX=1 returns feature flags
__cpuid(cpuInfo, 1);
// ECX bit 31 indicates hypervisor present
return (cpuInfo[2] & (1 << 31)) != 0;
}
BOOL CheckCPUID_VendorString(void) {
int cpuInfo[4] = { 0 };
char szVendor[13] = { 0 };
// Hypervisor vendor string leaf
__cpuid(cpuInfo, 0x40000000);
// Vendor string is in EBX:ECX:EDX
memcpy(szVendor, &cpuInfo[1], 4); // EBX
memcpy(szVendor + 4, &cpuInfo[2], 4); // ECX
memcpy(szVendor + 8, &cpuInfo[3], 4); // EDX
// Known hypervisor signatures
if (strcmp(szVendor, "VMwareVMware") == 0 ||
strcmp(szVendor, "VBoxVBoxVBox") == 0 ||
strcmp(szVendor, "Microsoft Hv") == 0 ||
strcmp(szVendor, "KVMKVMKVM") == 0 ||
strcmp(szVendor, "XenVMMXenVMM") == 0) {
return TRUE;
}
return FALSE;
}Network interface cards have MAC addresses where the first three bytes (the OUI - Organizationally Unique Identifier) identify the manufacturer. Virtual network adapters use predictable OUI prefixes that reveal the virtualization platform.
#include <windows.h>
#include <iphlpapi.h>
#pragma comment(lib, "iphlpapi.lib")
// Known VM MAC address prefixes (OUIs)
BYTE g_VMwareMac[] = { 0x00, 0x0C, 0x29 }; // VMware
BYTE g_VMware2Mac[] = { 0x00, 0x50, 0x56 }; // VMware alternate
BYTE g_VBoxMac[] = { 0x08, 0x00, 0x27 }; // VirtualBox
BYTE g_HyperVMac[] = { 0x00, 0x15, 0x5D }; // Hyper-V
BYTE g_ParallelsMac[] = { 0x00, 0x1C, 0x42 }; // Parallels
BOOL CheckMACAddress(void) {
ULONG ulSize = 0;
GetAdaptersInfo(NULL, &ulSize);
PIP_ADAPTER_INFO pAdapterInfo = (PIP_ADAPTER_INFO)HeapAlloc(
GetProcessHeap(), 0, ulSize);
if (GetAdaptersInfo(pAdapterInfo, &ulSize) != ERROR_SUCCESS) {
HeapFree(GetProcessHeap(), 0, pAdapterInfo);
return FALSE;
}
PIP_ADAPTER_INFO pAdapter = pAdapterInfo;
BOOL bIsVM = FALSE;
while (pAdapter) {
if (pAdapter->AddressLength >= 3) {
BYTE* mac = pAdapter->Address;
// Compare first 3 bytes against known VM prefixes
if (memcmp(mac, g_VMwareMac, 3) == 0 ||
memcmp(mac, g_VMware2Mac, 3) == 0 ||
memcmp(mac, g_VBoxMac, 3) == 0 ||
memcmp(mac, g_HyperVMac, 3) == 0 ||
memcmp(mac, g_ParallelsMac, 3) == 0) {
bIsVM = TRUE;
break;
}
}
pAdapter = pAdapter->Next;
}
HeapFree(GetProcessHeap(), 0, pAdapterInfo);
return bIsVM;
}Virtual machine guest tools (VMware Tools, VirtualBox Guest Additions) install services and drivers that register themselves in the Windows registry. These registry keys provide reliable VM indicators.
#include <windows.h>
const wchar_t* g_wszVMRegistryKeys[] = {
// VMware
L"SOFTWARE\\VMware, Inc.\\VMware Tools",
L"SYSTEM\\CurrentControlSet\\Services\\vmci",
L"SYSTEM\\CurrentControlSet\\Services\\vmhgfs",
L"SYSTEM\\CurrentControlSet\\Services\\vmmouse",
L"SYSTEM\\CurrentControlSet\\Services\\vmx86",
// VirtualBox
L"SOFTWARE\\Oracle\\VirtualBox Guest Additions",
L"SYSTEM\\CurrentControlSet\\Services\\VBoxGuest",
L"SYSTEM\\CurrentControlSet\\Services\\VBoxMouse",
L"SYSTEM\\CurrentControlSet\\Services\\VBoxSF",
NULL
};
BOOL CheckRegistryArtifacts(void) {
for (int i = 0; g_wszVMRegistryKeys[i] != NULL; i++) {
HKEY hKey;
if (RegOpenKeyExW(
HKEY_LOCAL_MACHINE,
g_wszVMRegistryKeys[i],
0,
KEY_READ,
&hKey
) == ERROR_SUCCESS) {
RegCloseKey(hKey);
return TRUE; // VM artifact found
}
}
return FALSE;
}The BIOS information in the registry often contains VM-specific strings:
BOOL CheckBIOSStrings(void) {
HKEY hKey;
if (RegOpenKeyExW(
HKEY_LOCAL_MACHINE,
L"HARDWARE\\DESCRIPTION\\System\\BIOS",
0,
KEY_READ,
&hKey
) != ERROR_SUCCESS) {
return FALSE;
}
wchar_t wszValue[256];
DWORD dwSize = sizeof(wszValue);
BOOL bIsVM = FALSE;
// Check SystemManufacturer value
if (RegQueryValueExW(hKey, L"SystemManufacturer", NULL, NULL,
(LPBYTE)wszValue, &dwSize) == ERROR_SUCCESS) {
if (wcsstr(wszValue, L"VMware") ||
wcsstr(wszValue, L"VirtualBox") ||
wcsstr(wszValue, L"QEMU") ||
wcsstr(wszValue, L"Xen") ||
wcsstr(wszValue, L"Microsoft Corporation")) {
bIsVM = TRUE;
}
}
RegCloseKey(hKey);
return bIsVM;
}Guest tools run background processes that can be detected through process enumeration:
#include <windows.h>
#include <tlhelp32.h>
const wchar_t* g_wszVMProcesses[] = {
L"vmtoolsd.exe", // VMware
L"vmwaretray.exe",
L"vmwareuser.exe",
L"VGAuthService.exe",
L"VBoxService.exe", // VirtualBox
L"VBoxTray.exe",
L"xenservice.exe", // Xen
L"qemu-ga.exe", // QEMU
L"prl_tools.exe", // Parallels
L"prl_cc.exe",
NULL
};
BOOL CheckVMProcesses(void) {
HANDLE hSnapshot = CreateToolhelp32Snapshot(TH32CS_SNAPPROCESS, 0);
if (hSnapshot == INVALID_HANDLE_VALUE) return FALSE;
PROCESSENTRY32W pe32 = { sizeof(PROCESSENTRY32W) };
BOOL bFound = FALSE;
if (Process32FirstW(hSnapshot, &pe32)) {
do {
for (int i = 0; g_wszVMProcesses[i] != NULL; i++) {
if (_wcsicmp(pe32.szExeFile, g_wszVMProcesses[i]) == 0) {
bFound = TRUE;
break;
}
}
} while (!bFound && Process32NextW(hSnapshot, &pe32));
}
CloseHandle(hSnapshot);
return bFound;
}Similarly, VM drivers leave files on the filesystem:
const wchar_t* g_wszVMFiles[] = {
L"C:\\Windows\\System32\\drivers\\vmmouse.sys",
L"C:\\Windows\\System32\\drivers\\vmhgfs.sys",
L"C:\\Windows\\System32\\drivers\\VBoxMouse.sys",
L"C:\\Windows\\System32\\drivers\\VBoxGuest.sys",
L"C:\\Windows\\System32\\drivers\\VBoxSF.sys",
L"C:\\Program Files\\VMware\\VMware Tools\\",
L"C:\\Program Files\\Oracle\\VirtualBox Guest Additions\\",
NULL
};
BOOL CheckVMFiles(void) {
for (int i = 0; g_wszVMFiles[i] != NULL; i++) {
if (GetFileAttributesW(g_wszVMFiles[i]) != INVALID_FILE_ATTRIBUTES) {
return TRUE;
}
}
return FALSE;
}Virtual machines typically have limited resources compared to physical workstations. While not definitive, very low CPU counts or RAM sizes can indicate a VM or sandbox.
#include <windows.h>
BOOL CheckHardwareSpecs(void) {
SYSTEM_INFO si;
GetSystemInfo(&si);
MEMORYSTATUSEX ms;
ms.dwLength = sizeof(ms);
GlobalMemoryStatusEx(&ms);
// VMs often have minimal resources
// Less than 2 CPUs is suspicious for modern systems
if (si.dwNumberOfProcessors < 2) {
return TRUE;
}
// Less than 2GB RAM suggests VM or sandbox
if (ms.ullTotalPhys < (2ULL * 1024 * 1024 * 1024)) {
return TRUE;
}
return FALSE;
}
// Screen resolution check
BOOL CheckScreenResolution(void) {
int cx = GetSystemMetrics(SM_CXSCREEN);
int cy = GetSystemMetrics(SM_CYSCREEN);
// Common sandbox/VM default resolutions
if ((cx == 800 && cy == 600) ||
(cx == 1024 && cy == 768)) {
return TRUE; // Suspicious resolution
}
return FALSE;
}Automated sandboxes analyze samples by executing them and observing behavior. However, sandboxes have inherent limitations: they must complete analysis quickly, they run in controlled environments, and they typically lack the rich state of a real user's system.
Most sandboxes analyze samples for only a few minutes. By delaying malicious activity, samples can wait until the sandbox times out before executing their payload.
#include <windows.h>
// Simple sleep - sandboxes may fast-forward
void DelayExecution_Sleep(void) {
Sleep(60000); // 1 minute delay
}
// Alertable sleep is harder to accelerate
void DelayExecution_SleepEx(void) {
SleepEx(60000, FALSE);
}
// Wait on an event that never signals
void DelayExecution_WaitEvent(void) {
HANDLE hEvent = CreateEventW(NULL, FALSE, FALSE, NULL);
WaitForSingleObject(hEvent, 120000); // 2 minute wait
CloseHandle(hEvent);
}
// Native API delay
typedef NTSTATUS (NTAPI* fnNtDelayExecution)(
BOOLEAN Alertable,
PLARGE_INTEGER DelayInterval
);
void DelayExecution_Native(void) {
fnNtDelayExecution pNtDelayExecution =
(fnNtDelayExecution)GetProcAddress(
GetModuleHandleW(L"ntdll.dll"),
"NtDelayExecution"
);
LARGE_INTEGER li;
li.QuadPart = -600000000LL; // 60 seconds (100ns units, negative = relative)
pNtDelayExecution(FALSE, &li);
}Advanced sandboxes accelerate sleep calls to complete analysis faster. By checking whether the actual elapsed time matches the requested sleep duration, samples can detect this acceleration.
#include <windows.h>
BOOL DetectSleepAcceleration(void) {
DWORD dwStart = GetTickCount();
Sleep(10000); // Request 10 second sleep
DWORD dwEnd = GetTickCount();
DWORD dwElapsed = dwEnd - dwStart;
// If less than 9 seconds actually elapsed, sleep was fast-forwarded
return (dwElapsed < 9000);
}
// System time is harder to fake than tick count
BOOL DetectSleepAcceleration_SystemTime(void) {
SYSTEMTIME stStart, stEnd;
GetSystemTime(&stStart);
Sleep(10000); // 10 seconds
GetSystemTime(&stEnd);
// Convert to seconds and compare
LONG lStartSec = stStart.wSecond + stStart.wMinute * 60;
LONG lEndSec = stEnd.wSecond + stEnd.wMinute * 60;
if (lEndSec < lStartSec) lEndSec += 3600; // Handle hour wrap
return ((lEndSec - lStartSec) < 9); // Less than 9 seconds = accelerated
}Rather than sleeping, malware can consume time by calling APIs repeatedly. This creates legitimate CPU work that is harder for sandboxes to accelerate.
#include <windows.h>
// CPU-bound delay through API calls
void APIHammering(void) {
for (int i = 0; i < 10000000; i++) {
GetCurrentProcessId();
GetCurrentThreadId();
GetTickCount();
}
}
// I/O-bound delay through file operations
void FileIOHammering(void) {
wchar_t wszTempPath[MAX_PATH];
GetTempPathW(MAX_PATH, wszTempPath);
wchar_t wszTempFile[MAX_PATH];
GetTempFileNameW(wszTempPath, L"tmp", 0, wszTempFile);
// Create and write large files repeatedly
for (int i = 0; i < 100; i++) {
HANDLE hFile = CreateFileW(
wszTempFile,
GENERIC_WRITE,
0,
NULL,
CREATE_ALWAYS,
FILE_ATTRIBUTE_NORMAL,
NULL
);
if (hFile != INVALID_HANDLE_VALUE) {
// Write ~10MB of data
BYTE bBuffer[4096];
for (int j = 0; j < 2560; j++) {
DWORD dwWritten;
WriteFile(hFile, bBuffer, sizeof(bBuffer), &dwWritten, NULL);
}
CloseHandle(hFile);
}
DeleteFileW(wszTempFile);
}
}Real systems have active users who move mice, type on keyboards, and create documents. Sandboxes typically lack this activity, providing a distinguishing characteristic.
#include <windows.h>
#include <shlobj.h>
// Check if mouse has moved
BOOL CheckUserInteraction_Mouse(void) {
POINT pt1, pt2;
GetCursorPos(&pt1);
Sleep(5000); // Wait 5 seconds
GetCursorPos(&pt2);
// If mouse hasn't moved, likely automated environment
return (pt1.x == pt2.x && pt1.y == pt2.y);
}
// Check time since last input
BOOL CheckUserInteraction_Input(void) {
LASTINPUTINFO lii;
lii.cbSize = sizeof(LASTINPUTINFO);
if (!GetLastInputInfo(&lii)) {
return FALSE;
}
DWORD dwIdle = GetTickCount() - lii.dwTime;
// More than 10 minutes idle is suspicious
return (dwIdle > 600000);
}
// Real users have recent documents
BOOL CheckRecentDocuments(void) {
wchar_t wszRecentPath[MAX_PATH];
if (SHGetFolderPathW(NULL, CSIDL_RECENT, NULL, 0, wszRecentPath) != S_OK) {
return FALSE;
}
WIN32_FIND_DATAW fd;
wchar_t wszSearch[MAX_PATH];
swprintf_s(wszSearch, MAX_PATH, L"%s\\*", wszRecentPath);
HANDLE hFind = FindFirstFileW(wszSearch, &fd);
if (hFind == INVALID_HANDLE_VALUE) {
return FALSE;
}
int nCount = 0;
do {
if (!(fd.dwFileAttributes & FILE_ATTRIBUTE_DIRECTORY)) {
nCount++;
}
} while (FindNextFileW(hFind, &fd) && nCount < 50);
FindClose(hFind);
// Sandboxes have few or no recent documents
return (nCount < 10);
}Sandboxes often use predictable usernames, computer names, and have minimal running processes:
#include <windows.h>
#include <tlhelp32.h>
BOOL CheckUsername(void) {
wchar_t wszUsername[256];
DWORD dwSize = 256;
if (!GetUserNameW(wszUsername, &dwSize)) {
return FALSE;
}
// Common sandbox usernames
if (_wcsicmp(wszUsername, L"sandbox") == 0 ||
_wcsicmp(wszUsername, L"virus") == 0 ||
_wcsicmp(wszUsername, L"malware") == 0 ||
_wcsicmp(wszUsername, L"test") == 0 ||
_wcsicmp(wszUsername, L"user") == 0 ||
_wcsicmp(wszUsername, L"CurrentUser") == 0 ||
_wcsicmp(wszUsername, L"admin") == 0) {
return TRUE;
}
return FALSE;
}
BOOL CheckComputerName(void) {
wchar_t wszComputer[256];
DWORD dwSize = 256;
if (!GetComputerNameW(wszComputer, &dwSize)) {
return FALSE;
}
// Common sandbox naming patterns
if (wcsstr(wszComputer, L"SANDBOX") ||
wcsstr(wszComputer, L"VIRUS") ||
wcsstr(wszComputer, L"MALWARE") ||
wcsstr(wszComputer, L"TEST") ||
wcsstr(wszComputer, L"SAMPLE")) {
return TRUE;
}
return FALSE;
}
// Real systems have many running processes
BOOL CheckProcessCount(void) {
HANDLE hSnapshot = CreateToolhelp32Snapshot(TH32CS_SNAPPROCESS, 0);
if (hSnapshot == INVALID_HANDLE_VALUE) return FALSE;
PROCESSENTRY32W pe32 = { sizeof(PROCESSENTRY32W) };
int nCount = 0;
if (Process32FirstW(hSnapshot, &pe32)) {
do {
nCount++;
} while (Process32NextW(hSnapshot, &pe32));
}
CloseHandle(hSnapshot);
// Minimal sandboxes have fewer than 30 processes
return (nCount < 30);
}After completing their objectives, implants may want to remove themselves from disk to hinder forensic analysis. Windows normally prevents deleting a running executable, but several techniques can work around this limitation.
The FILE_FLAG_DELETE_ON_CLOSE flag marks a file for deletion when the last handle closes:
#include <windows.h>
BOOL SelfDelete_CreateFile(void) {
wchar_t wszSelf[MAX_PATH];
GetModuleFileNameW(NULL, wszSelf, MAX_PATH);
// Open with delete-on-close flag
HANDLE hFile = CreateFileW(
wszSelf,
DELETE,
FILE_SHARE_DELETE, // Allow deletion while open
NULL,
OPEN_EXISTING,
FILE_FLAG_DELETE_ON_CLOSE,
NULL
);
if (hFile == INVALID_HANDLE_VALUE) {
return FALSE;
}
// Close handle - file marked for deletion
// Will be deleted when all handles (including the loader's) close
CloseHandle(hFile);
return TRUE;
}A more sophisticated technique renames the executable's default data stream to an alternate data stream, effectively deleting the file content while the process continues:
#include <windows.h>
typedef struct _FILE_RENAME_INFO {
BOOLEAN ReplaceIfExists;
HANDLE RootDirectory;
ULONG FileNameLength;
WCHAR FileName[1];
} FILE_RENAME_INFO, *PFILE_RENAME_INFO;
#define FileRenameInfo 3
BOOL SelfDelete_ADSRename(void) {
wchar_t wszSelf[MAX_PATH];
GetModuleFileNameW(NULL, wszSelf, MAX_PATH);
// Open file for deletion
HANDLE hFile = CreateFileW(
wszSelf,
DELETE | SYNCHRONIZE,
FILE_SHARE_READ | FILE_SHARE_DELETE,
NULL,
OPEN_EXISTING,
FILE_ATTRIBUTE_NORMAL,
NULL
);
if (hFile == INVALID_HANDLE_VALUE) {
return FALSE;
}
// Rename to alternate data stream (effectively deletes main stream)
wchar_t wszNewName[] = L":deleteme";
SIZE_T sSize = sizeof(FILE_RENAME_INFO) + sizeof(wszNewName);
PFILE_RENAME_INFO pRenameInfo = (PFILE_RENAME_INFO)HeapAlloc(
GetProcessHeap(), HEAP_ZERO_MEMORY, sSize);
pRenameInfo->FileNameLength = sizeof(wszNewName) - sizeof(WCHAR);
memcpy(pRenameInfo->FileName, wszNewName, sizeof(wszNewName));
BOOL bResult = SetFileInformationByHandle(
hFile,
FileRenameInfo,
pRenameInfo,
(DWORD)sSize
);
HeapFree(GetProcessHeap(), 0, pRenameInfo);
if (!bResult) {
CloseHandle(hFile);
return FALSE;
}
// Now mark for deletion
FILE_DISPOSITION_INFO fdi = { TRUE };
bResult = SetFileInformationByHandle(
hFile,
FileDispositionInfo,
&fdi,
sizeof(fdi)
);
CloseHandle(hFile);
return bResult;
}Understanding anti-analysis techniques is essential for building robust analysis environments and detecting evasive samples.
┌────────────────────────┬────────────────────────────┬──────────────────────┐
│ Technique Category │ Detection Method │ Telemetry Source │
├────────────────────────┼────────────────────────────┼──────────────────────┤
│ Anti-Debug API calls │ Hook IsDebuggerPresent, │ ETW, API hooks │
│ │ NtQueryInformationProcess │ │
├────────────────────────┼────────────────────────────┼──────────────────────┤
│ Timing checks │ Monitor RDTSC, QPC patterns│ CPU tracing │
├────────────────────────┼────────────────────────────┼──────────────────────┤
│ Hardware BP detection │ Context access monitoring │ Kernel callbacks │
├────────────────────────┼────────────────────────────┼──────────────────────┤
│ VM detection │ Registry/file/process scan │ Behavioral analysis │
├────────────────────────┼────────────────────────────┼──────────────────────┤
│ Sleep delays │ Fast-forward, wall clock │ Time correlation │
├────────────────────────┼────────────────────────────┼──────────────────────┤
│ API hammering │ I/O rate limiting, CPU │ Resource monitoring │
│ │ throttling │ │
└────────────────────────┴────────────────────────────┴──────────────────────┘SANDBOX HARDENING BEST PRACTICES
================================
┌─────────────────────────────────────────────────────────────────────────┐
│ │
│ ENVIRONMENT REALISM: │
│ • Use realistic hardware specs (4+ CPUs, 8+ GB RAM) │
│ • Install common applications (Office, browsers, utilities) │
│ • Populate with realistic user data and documents │
│ • Use physical network adapters or change MAC addresses │
│ │
│ HIDING ANALYSIS: │
│ • Mask CPUID hypervisor bit │
│ • Rename/remove VM guest tools │
│ • Patch PEB debugging flags │
│ • Hook anti-debug APIs to return benign values │
│ │
│ SIMULATING ACTIVITY: │
│ • Generate random mouse movements │
│ • Simulate keyboard input │
│ • Create browser history │
│ • Update system uptime │
│ │
│ ANALYSIS IMPROVEMENTS: │
│ • Use longer timeouts (10-30 minutes) │
│ • Run multiple analysis passes │
│ • Implement anti-evasion hooks │
│ • Correlate wall clock with internal time │
│ │
└─────────────────────────────────────────────────────────────────────────┘YARA rules can identify samples using anti-analysis techniques:
rule Anti_Debug_Techniques {
meta:
description = "Detects anti-debugging techniques"
strings:
// API-based detection
$api1 = "IsDebuggerPresent" ascii
$api2 = "CheckRemoteDebuggerPresent" ascii
$api3 = "NtQueryInformationProcess" ascii
$api4 = "OutputDebugString" ascii
$api5 = "GetThreadContext" ascii
// Direct PEB access patterns
$peb1 = { 64 A1 30 00 00 00 } // mov eax, fs:[0x30] (x86)
$peb2 = { 65 48 8B 04 25 60 00 00 00 } // mov rax, gs:[0x60] (x64)
condition:
3 of ($api*) or any of ($peb*)
}
rule Anti_VM_Techniques {
meta:
description = "Detects anti-VM techniques"
strings:
// VM vendor strings
$vm1 = "vmware" ascii nocase
$vm2 = "virtualbox" ascii nocase
$vm3 = "vbox" ascii nocase
$vm4 = "qemu" ascii nocase
$vm5 = "xen" ascii nocase
// CPUID instruction
$cpuid = { 0F A2 }
// Registry paths
$reg1 = "SOFTWARE\\VMware, Inc." wide
$reg2 = "SOFTWARE\\Oracle\\VirtualBox" wide
condition:
2 of ($vm*) or ($cpuid and any of ($reg*))
}Anti-analysis techniques represent an arms race between attackers seeking to protect their code and defenders building analysis environments. Understanding both sides of this dynamic is essential for security professionals.
TECHNIQUE EFFECTIVENESS SUMMARY
===============================
┌──────────────────────────┬─────────────┬───────────────┬───────────────┐
│ Technique │ Purpose │ Effectiveness │ Detectability │
├──────────────────────────┼─────────────┼───────────────┼───────────────┤
│ IsDebuggerPresent │ Debug check │ Low │ Easy │
│ NtQueryInformationProcess│ Debug check │ High │ Medium │
│ Timing checks │ Detect step │ Medium │ Hard │
│ Hardware BP detection │ Detect BP │ High │ Medium │
│ CPUID/Registry VM check │ Detect VM │ Medium │ Easy │
│ Process enumeration │ Detect tools│ Medium │ Easy │
│ Sleep delays │ Timeout │ Medium │ Easy to bypass│
│ API hammering │ Timeout │ Medium │ Medium │
│ User interaction │ Detect auto │ High │ Hard │
└──────────────────────────┴─────────────┴───────────────┴───────────────┘Key Principles for Evasive Code:
Key Principles for Analysis Environments: