Removing HTML tags during indexing
There are many real-life situations when you have to clean your data. Let's assume that you want to index web pages that your client sends you. You don't know anything about the structure of the page; one thing you know is that you must provide a search mechanism that will enable searching through the content of the pages. Of course, you can index the whole page splitting it by whitespaces, but then you will probably hear the client complain about the HTML tags being searchable, and so on. So, before we enable searching on the contents of the page, we need to clean the data. In this recipe, we will see how to remove the HTML tags with Solr.
How to do it...
Now, let's take a look at the steps needed to remove the HTML tags from our data.
We start by assuming that our data looks like this:
<add> <doc> <field name="id">1</field> <field name="html"><
