Like any other activity, reverse engineering is also a process. There is a guide that we can follow to help us generate information that can be helpful to both the analyst and stakeholders.
Reverse engineering as a process
Seeking approval
Ethics requires anyone carrying out reverse engineering of software to have approval from the owner of the software. However, there are a lot of instances where software shows its bugs upfront, while the operating system reports it. Some companies are more lenient about their software getting reversed without approval, but it is customary today that any vulnerabilities found should be reported directly to the owner and not publicized. It is up to the owner to decide when to report the vulnerability to the community. This prevents attackers from using a vulnerability before a software patch gets released.
It is a different story when malware or hacking is involved. Of course, reversing malware doesn't need approval from the malware author. Rather, one of the goals of malware analysis is to catch the author. If not sure, always consult a lawyer or a company's legal department.
Static analysis
Without any execution, viewing the file's binary and parsing each and every byte provides much of the information needed to continue further. Simply knowing the type of file sets the mindset of the analyst in a way that helps them to prepare specific sets of tools and references that may be used. Searching text strings can also give clues about the author of the program, where it came from, and, most likely, what it does.
Dynamic analysis
This type of analysis is where the the object being analyzed gets executed. It requires an enclosed environment so that behaviors that may compromise production systems do not happen. Setting up enclosed environments are usually done using virtual machines, since they can then easily be controlled. Tools that monitor and log common environment actions are implemented during dynamic analysis.
Low-level analysis
There is some information that may be missed out during static and dynamic analyses. The flow of a program follows a path that depends of certain conditions. For example, a program will only create a file only if a specific process is running. Or, a program will create a registry entry in the Wow6432Node key only if it were running in a 64-bit Windows operating system. Debugging tools are usually used to analyze a program in low-level analysis.
Reporting
While doing analysis, every piece of information should be collected and documented. It is common practice to document a reverse engineered object to help future analysis. An analysis serves as a knowledge base for developers who want to secure their upcoming programs from flaws. For example, a simple input can now be secured by placing bounds validation, which is known about as a result of a prior reverse-engineered program that indicated possible buffer overflow.
A good report answers questions regarding the following:
- How a reversed engineered object works
- When specific behavior triggers
- Why specific codes were used in the program
- Where it was intended to work on
- What the whole program does
