Future research directions to address limitations
What might our human-machine civilization do to LLMs to remove and mitigate more of their limitations and drive technological advancements?
Let’s consider a few ideas here.
Continuous learning
If we could enable LLMs to constantly take in new data and re-train frequently (e.g., every day), they would not be out of date for long and could go through many iterations of improvement in short time spans.
Novel architectures
Exploring new neural network architectures and hybrid models can lead to breakthroughs in LLM capabilities.
New hardware devices and coding and testing practices have always been important for machine learning advancement, but what really drives AI power is new neural network architecture.
The neural network gave us the ability to train software to make its own decisions and be more adaptable, rather than every scenario being programmed and hardcoded in.
Before deep learning, neural networks...
