Understanding the limitations of autoencoders

As we saw previously, neural networks such as autoencoders are used to automatically learn representative features from data, without explicitly relying on human-engineered assumptions. While this approach may allow us to discover ideal encoding schemes that are specific to different types of data, this approach does present certain limitations. Firstly, autoencoders are said to be data-specific, in the sense that their utility is restricted to data that is considerably similar to its training data. For example, an autoencoder that's trained to only regenerate cat pictures will have a very hard time generating dog pictures without explicitly being trained to do so. Naturally, this seems to reduce the scalability of such algorithms. It is also noteworthy that autoencoders, as of yet, do not perform noticeably better than the JPEG...