Exploring the function control problem and its benefits

Modern disassemblers, such as IDA Pro, and NSA Ghidra, are highly effective at analyzing function calls and deducing high-level information by understanding the relationships between functions. This type of analysis is effective when it’s applied to code written in a conventional programming style and compiled with a standard compiler. However, it can be easily bypassed by the creator of malware.

Function pointers are widely used in the C programming language and play a significant role in C++. However, they continue to present challenges to disassemblers.

When function pointers are used correctly in a C program, they can significantly limit the amount of information that can be automatically inferred about the program’s flow. When function pointers are utilized in handwritten assembly or implemented in a nonstandard manner in source code, it can pose challenges in reverse-engineering the results without the...