Identifying protected binaries

Identifying a protected binary is the first step in reverse-engineering it. We discussed the common anatomy of protected ELF executables in Chapter 5, Linux Binary Protection. Remember from what we learned that a protected binary is actually two executables that have been merged together: you have the stub executable (the decryptor program) and then the target executable.

One program is responsible for decrypting the other, and it is this program that is going to typically be the wrapper that wraps or contains an encrypted binary within it, as a payload of sorts. Identifying this outer program that we call a stub is typically pretty easy because of the blatant oddities you will see in the program header table.

Let's take a look at a 64-bit ELF binary that is protected using a protector I wrote in 2009 called elfcrypt:

$ readelf -l test.elfcrypt

Elf file type is EXEC (Executable file)
Entry point 0xa01136
There are 2 program headers, starting at offset 64

Program...