Understanding data in the State of Union addresses

In this chapter, we'll apply topic modeling to the (SOTU) State of the Union addresses presented by the presidents of the United States of America. Each January or February, the President addresses the US Senate and the House of Representatives either in person or in writing, and talks about how the country is doing as well as outlining his agenda for the coming year. The speeches can be fairly short, but the written reports can be much longer. George Washington's first State of the Union address from 1790 had less than 500 words. Barack Obama's latest SOTU (at the time of this writing in 2013) had over 3,000 words. Jimmy Carter had the longest SOTU address, which he delivered in writing in 1981. It is almost 14,000 words long.

The gradual increase in the length of the SOTU address, which climaxed around 1910, was because starting from Thomas Jefferson's 1801 address up until William H. Taft's 1912 address, the SOTU address was a written...