LDA Modeling

In Chapter 9, Understanding Discrete Distribution, and Chapter 10, Latent Dirichlet Allocation, we learned about the Dirichlet distribution and the structure of the LDA model, which equipped you with a sound theoretical background. In this chapter, we will go over the code to build an LDA model. I will touch upon the key decisions in building an LDA model, including text preprocessing, model hyperparameters, the determination of the number of topics, and how to use the model in production to score new documents. This is a special feature in this book that focuses on model implementation in production. In short, we will cover the following topics:

• Text preprocessing
• Experimenting with LDA modeling
• Building LDA models with a different number of topics
• Determining the optimal number of topics
• Using the model to score new documents

With the completion of this chapter, you will be able to develop LDA topic models independently. You will also be...