These are the models that most fascinate me. From their wide range of applications to their brilliant origin—everything seems marvelous to me. Note that there is no such a thing as an all mighty model and neural nets are not it. They can be distinguished by being very flexible models that can perform both unsupervised and supervised learning.
Even though it's a very powerful method, it's certainly not all powerful. Neural nets are able to capture linear and non-linear relations. Yet, everything will depend on how you design the networks (researcher's ability and experience), how complex is the problem at hand, and how many observations do you have.
Computational constraints can also be a bottleneck. As powerful as they are known to be, neural networks are not remembered as computationally inexpensive methods. It's not hard to find problems...