A collection of anti-analysis techniques for Windows

March 11, 2021

This post is a brief collection of anti-debugging and anti-analysis techniques that I found while reversing some samples. Anti-analysis techniques' purpose is to slow down the analysis as much as possible. By using these techniques a malware could detect if the malicious code is running in a monitored environment and decide to not execute at all or to behave differently, causing the analysis to be mislead.

An example of different behavior could be contacting a different C2 to alert the attacker that some analysis is conducted on the binary (or to make superficial analysis lead to the wrong IOCs). Another example could be to behave non maliciously, leading the analyst to think that the binary is legit. Finally the most obvious behavior is to simply exit the program without any further action.

The idea behind this post is to help junior analysts understand the meaning of some specific constants, API calls or sequence of API calls.

NtSetInformationThread to hide a thread from the debugger

This function call can be encountered when a new thread is created, more specifically to set the priority of the thread.

__kernel_entry NTSYSCALLAPI NTSTATUS NtSetInformationThread(
  HANDLE          ThreadHandle,
  THREADINFOCLASS ThreadInformationClass,
  PVOID           ThreadInformation,
  ULONG           ThreadInformationLength
);

The undocumented enum value 0x11 for the argument ThreadInformationClass corresponds to ThreadHideFromDebugger. So, by setting this value, a malware could create a debugger-invisible thread. If a breakpoint is hit by a hidden thread the program crashes, because there is no debugger registered that can handle the trap.

Example:

mov eax, [ebp+thread_handle]
mov [esp+thread_information_len], 0
mov [esp+thread_information], 0
mov [esp+thread_information_class], 11h
mov [esp+th_handle], eax
call NtSetInformationThread
sub esp, 10h

In this specific example, to get around this anti-analysis technique it is sufficient to reach the call instruction and then skip it (ctrl+N in IDA). The following sub instruction will clean up the stack.

Detecting malware analysis software running on the machine by using FindWindowA

Some malware analysis tools are very common among the analysts. A technique of anti-analysis we can find in malware consists in detecting the programs running on the machine and checking if any of those are commonly used by malware analysts.

The function FindWindowA retrieves a handle to the top-level window whose ClassName and WindowName match the specified strings (this is not case sensitive!). The general idea behind this anti-analysis technique is that if, for example, WinDbg is running, FindWindowA will retrieve a handle on the WinDbg window, whereas if it is not running it will return an error.

HWND FindWindowA(
  LPCSTR lpClassName,
  LPCSTR lpWindowName
);

The function’s first argument is a pointer to a string that specifies the window class name and usually is very similar to the software name. Some interesting class names are:

OLLYDBG
TIdaWindow
WinDbgFrameClass
and more

The second argument is the window name and if it is NULL all windows match.

Detecting malware analysis software running on the machine by using CreateToolhelp32Snapshot

The WinAPI CreateToolhelp32Snaphot takes a snapshot of the specified (or all) processes running on the machine, as well as the heaps, modules and threads used by these processes.

The function returns an open handle on the specified snapshot. This snapshot can be walked through by using two other WinAPIs: Process32First once and Process32Next for all the following processes.

Finding those three API calls in a malicious binary could mean, for example, that it is looking for a specific process from whom to hide or to inject into.

HANDLE CreateToolhelp32Snapshot(
  DWORD dwFlags,
  DWORD th32ProcessID
);

Detecting malware analysis software from DLLs in memory

The majority of software has external dependencies, some of which are unique. If this is the case for an analysis software, the malware could detect the presence of a specific DLL in memory.

To do this, it is necessary to know the name of said DLL. Then, invoking GetModuleHandleA, the code obtains a handle to the module or an error.

HMODULE GetModuleHandleA(
  LPCSTR lpModuleName
);

BeingDebugged

To detect whether a process is being debugged or not, some malware checks the appropriate flag in the PEB table (Process Environment Block), which is a user-mode data structure that can be used by applications to get information on the process. In particular it indicates whether the program is being debugged. This flag is at offset 0x02 from the PEB base address. If it is set to 1, then it means that a debugger is attached to the process.

Example of compiled code:

mov eax, large fs:30h	    // gets a pointer to the PEB base address
cmp byte ptr [eax+2], 1	    // compares the flag BeingDebugged to 1
jz short loc_4010E1

The previous assembly code shows how to get the value of the flag. First it gets the base address of the PEB table which is stored in the registry FS at offset 0x30 (more details here). Next, it accesses the offset 0x02 form the base of PEB, which corresponds to BeingDebugged.

struct _PEB {
    0x000 BYTE InheritedAddressSpace;
    0x001 BYTE ReadImageFileExecOptions;
    0x002 BYTE BeingDebugged;
    ...
}

IsDebuggerPresent

Determines whether the calling process is being debugged by a user-mode debugger. This function has a very simple signature: does not take any parameters and returns a bool.

BOOL IsDebuggerPresent();

The return value of this function is 0 if the process is not being debugged, otherwise the return value is nonzero. Here is what the compiled implementation could look like (32 bit):

kernelbase_IsDebuggerPresent:
mov     eax, large fs:18h    	// gets pointer to TEB          
mov     eax, [eax+30h]		// gets pointer to PEB
movzx   eax, byte ptr [eax+2]	// gets value of BeingDebugged
retn

In the previous assembly code the function accesses the FS registry at offset 0x18 which contains a pointer to TEB (Thread Environment Block). Source table:

TEBTable

Then it accesses offset TEB+0x30 which corresponds to a pointer to PEB (Process Environment Block). Here a definition of TEB:

struct _TEB {
    0x000 _NT_TIB NtTib;
    0x01c void* EnvironmentPointer;
    0x020 _CLIENT_ID ClientId;
    0x028 void* ActiveRpcHandle;
    0x02c void* ThreadLocalStoragePointer;
    0x030 _PEB* ProcessEnvironmentBlock; 
    ... 
}

Then it accesses the offset 0x02 from the base address of PEB, which corresponds to the variable BeingDebugged. A definition of PEB can be found here:

struct _PEB {
    0x000 BYTE InheritedAddressSpace;
    0x001 BYTE ReadImageFileExecOptions;
    0x002 BYTE BeingDebugged;
    ...
}

Detecting the username

This technique could be used in both blocklist and allowlist mode.

Some sandboxes have default usernames. Malware could bundle a list of the most common usernames used by the sandboxes and detect if the current username matches one of them.

In case of a blocklist behavior, the malware would stop execution or drop any malicious act, while for the allowlist, the malicious code might want to execute only if the username matches the one of a target person.

To obtain the username the function GetUserNameA can be used.

BOOL GetUserNameA(
  LPSTR   lpBuffer,
  LPDWORD pcbBuffer
);