Gensim is a popular open source library for processing raw, unstructured human-generated text created by Radim Řehůřek. Some of the features that Gensim boasts are:
- Memory independence is one of the core value propositions of Gensim, which is that it should be scalable and not hold all the document in the RAM. Hence, you will be able to train documents that are significantly larger than the memory of your machine.
- Gensim has efficient implementations of various popular vector space algorithms. There has been a recent implementation of fastText in gensim as well.
- There are IO/wrappers and converters around several popular data formats as well. Remember that fastText only supports UTF-8 formats and hence Gensim might be a good choice if you have data that is in different formats.
- Different algorithms for similarity queries. So, you are not stuck with the...
