LLMs have very favourable exploration<>consolidation dynamics. An LLM alternative would need to display fundamental improvements at relatively large scales which is extremely challenging for new research.