![](https://crypto4nerd.com/wp-content/uploads/2023/10/05tLF9HN4VhnFIdPg.jpeg)
Artificial intelligence has come a long way in recent years. Systems like ChatGPT show how far language models have progressed when it comes to conversing naturally and responding to open-ended prompts.
However, there’s still ample room for improvement, especially when it comes to factual accuracy. Even powerful AI like ChatGPT often generate plausible-sounding but incorrect or unverified information. This is because they solely rely on the knowledge encapsulated in their parameters rather than external sources.
Enter a paradigm called retrieval-augmented generation (RAG). RAG enhances language models by first retrieving relevant knowledge — say from Wikipedia — before generating a response. This equips the AI with external references to ground what it says in factual information.
But as with any new technique, RAG has its limitations.
A core one is that existing RAG systems retrieve information indiscriminately, whether it’s needed or not. They also lack mechanisms to verify that the information they produce is fully supported by the retrieved evidence.
Now, researchers from the University of Washington, Allen Institute for AI, and IBM have introduced a breakthrough new framework called SELF-RAG that could address these issues and significantly improve RAG.
SELF-RAG trains language models to critique and reflect upon their own generations using special tokens. This gives them two key capabilities completely missing in standard RAG:
- Adaptive retrieval: The model learns to retrieve knowledge precisely when needed based on the context, by predicting whether appending evidence would improve the response.
- Self-critiquing: The model evaluates whether the retrieved passage is relevant and whether its own output is fully supported by the evidence.