Exploring advanced math algorithms in malware
In previous chapters, we looked at popular and well-studied encoding and encryption algorithms such as XOR, AES, RC4, and Base64. In recent years, I’ve wondered, “What if we used other advanced encryption algorithms that are based on simple ones?” I decided to conduct research and apply various encryption algorithms that were presented to the public in the '80s and '90s and see how using them affects the VirusTotal score result. So, can they be used in malware development? Let’s look at some algorithms and cover some practical examples of payload encryption.
Tiny encryption algorithm (TEA)
Tiny encryption algorithm (TEA) is a symmetric-key block cipher algorithm that operates on 64-bit blocks and uses a 128-bit key. The basic flow of the TEA encryption algorithm is as follows:
- Key expansion: The 128-bit key is split into two 64-bit subkeys.
- Initialization: The 64-bit plaintext block...