In this section, we will train NTM on Enron emails to produce topics. These emails were exchanged between Enron, an American energy company that ceased its operations in 2007 due to financial losses, and other parties that did business with the company.
The dataset contains 39,861 emails and 28,101 unique words. We will work with a subset of these emails – 3,986 emails and 17,524 unique words. Additionally, we will create a text file, vocab.txt, so that the NTM model can report the word distribution of a topic.
Before we get started, make sure that both docword.enron.txt.gz and vocab.enron.txt have been uploaded to a folder called data on the local SageMaker compute instance. Follow these steps:
- Create a bag-of-words representation of emails, as follows:
pvt_emails = pd.pivot_table(df_emails, values='count', index='email_ID&apos...
