Attackers use buffer overflow exploits to run executable code, which allows the attacker to take over the target computer or compromise its data.
Buffer overflow exploits overflow the fixed-size memory buffer reserved for an input process. A large percentage of attacks is buffer overflow attacks that try to overwrite adjacent memory in the stack frame.
The two types of buffer overflow exploits are:
Stack-based attacks use the stack memory objects to store user input (most common).
Heap-based attacks flood the memory space reserved for a program (rare).
The fixed-size stack memory object is empty and waiting for user input. When a program receives input from the user, the data is stored on top of the stack. The data includes the return address memory information required by the application when calling internal functions. When the stack is processed, the called application function processes the user's input stored on the stack. The return address memory information is used to determine the code address of the caller that the application returns to once processing of the called function finishes.
The following process describes a stack-based buffer overflow attack:
Overflow the stack
When the program is written, a specific amount of memory space is reserved for the data. The stack overflows if the data written is larger than the space reserved for it in the memory stack. This situation is only a problem when combined with malicious input.
Exploit the overflow
The program waits for input from the user. If the attacker enters an executable command that exceeds the stack size, that command is saved outside the reserved space.
Perform malicious actions
The payload of the exploit, also called shellcode performs malicious actions on the system. These actions can include adding new users, changing user permissions, creating or changing files on the system, or downloading and running malware.
Initially, the program starts to crash because of the buffer overflow. If the attacker provided a return memory address that references the malicious payload, the program tries to recover by using the return address. If the return address is valid, the malicious payload is executed.
Exploit the permissions
The payload now runs with the same permissions as the application that was compromised. Because programs usually run in kernel mode or with permissions inherited from a service account, the attacker can now gain full control of the operating system.