11.5 Reading complex formats using regular expressions

Many file formats lack the elegant regularity of a CSV file. One common file format that’s rather difficult to parse is a web server log file. These files tend to have complex data without a single, uniform separator character or consistent quoting rules.

When we looked at a simplified log file in the Writing generator functions with the yield statement recipe in Chapter 9, we saw that the rows look as follows:

[2016-05-08 11:08:18,651] INFO in ch09_r09: Sample Message One 
 
[2016-05-08 11:08:18,651] DEBUG in ch09_r09: Debugging 
 
[2016-05-08 11:08:18,652] WARNING in ch09_r09: Something might have gone wrong

There are a variety of punctuation marks being used in this file. The csv module can’t parse this complexity.

We’d like to write programs with the elegant simplicity of CSV processing. This means we’ll need to encapsulate the complexities of...