Understanding log management challenges

Logs are nothing but streams of events coming from a running process. For traditional JEE applications, a number of frameworks and libraries are available for logging. Java Logging (JUL) is an option off-the-shelf from Java itself. Log4j, Logback, and SLF4J are some of the other popular logging frameworks available. These frameworks support both UDP as well as TCP protocols for logging. The applications send log entries to the console or the filesystem. File recycling techniques are generally employed to avoid logs filling up all disk space.

One of the best practices for log handling is to switch off most of the log entries in production due to the high cost of disk IOs. The disk IOs not only slow down the application, but can also severely impact the scalability. Writing logs into the disk also requires a high disk capacity. Running out of the disk space scenario can bring down the application. Logging frameworks provide options to control logging...