The use of prime numbers and modular arithmetic in malware

Let’s dive into an example of implementing the practical use of prime numbers and modular arithmetic in cryptography algorithms. This is typically done to generate keys for RSA encryption.

Practical example

When it comes to key generation, you must select two primes, denoted as p and q, and compute their product, n = p*q. RSA’s security is predicated on the difficulty of deducing p and q from n. The greater the sizes of p and q, the more challenging it is to locate them given n.

The full source code is available in this book’s GitHub repository at https://github.com/PacktPublishing/Malware-Development-for-Ethical-Hackers/blob/main/chapter12/02-prime-numbers/hack.c.

The main logic is pretty simple:

1. Choose two large prime numbers.
2. Compute n (modulus) and phi (Euler’s totient function).
3. Choose a public exponent, e.
4. Compute the private exponent, d.
5. Encrypt a message...