Apple Machine Learning Research has a new preprint: “The Illusion of Thinking: Understanding the Strengths and Limitations of Reasoning Models via the Lens of Problem Complexity.” [Apple, PDF] “Lar…
As the bioware nerd I am it makes my heart glad to see the Towers of Hanoi doing their part in this fight. And it seems like the published paper undersells how significant this problem is for the promptfondlers’ preferred narratives. Given how simple it is to scale the problem complexity for these scenarios, it seems likely that there isn’t a viable scaling-based solution here. No matter how big you make the context windows and how many steps the system is able to process it’s going to get out scaled by simply increasing some Ns in the puzzle itself.
Diz and others with a better understanding of what’s actually under the hood have frequently referenced how bad Transformer models are at recursion and this seems like a pretty straightforward way to demonstrate that and one that I would expect to be pretty consistent.
So I don’t know if it’s a strong link, but I definitely learned to solve the Towers when playing through KotOR, then had it come up again in Mass Effect, and Jade Empire, both of which I played at around the same time. From a quick “am I making this up?” search, it’s also used in a raid in SW:TOR, and gets referenced throughout the dragon age and mass effect franchises even if not actually deployed.
Right, yeah i just recall that for a high enough bit of towers the amount of steps needed to solve it rises quickly. The story, “Now Inhale”, by Eric Frank Russell, uses 64 discs. Fun story.
Min steps is 2 to the power the number of disks minus 1.
Programming a system that solves it was a programming excersize for me a long time ago. Those are my stronger memories of it
It’s a solid intro CS puzzle for teaching recursion. I think the original story they invented to go with it also had 64 disks in a temple in, well, Hanoi. Once the priests finished it the world was supposed to end or something.
As the bioware nerd I am it makes my heart glad to see the Towers of Hanoi doing their part in this fight. And it seems like the published paper undersells how significant this problem is for the promptfondlers’ preferred narratives. Given how simple it is to scale the problem complexity for these scenarios, it seems likely that there isn’t a viable scaling-based solution here. No matter how big you make the context windows and how many steps the system is able to process it’s going to get out scaled by simply increasing some Ns in the puzzle itself.
Diz and others with a better understanding of what’s actually under the hood have frequently referenced how bad Transformer models are at recursion and this seems like a pretty straightforward way to demonstrate that and one that I would expect to be pretty consistent.
Sorry what is the link between bioware and towers of hanoi? (I do know about the old “one final game before your execution” science fiction story).
So I don’t know if it’s a strong link, but I definitely learned to solve the Towers when playing through KotOR, then had it come up again in Mass Effect, and Jade Empire, both of which I played at around the same time. From a quick “am I making this up?” search, it’s also used in a raid in SW:TOR, and gets referenced throughout the dragon age and mass effect franchises even if not actually deployed.
Right, yeah i just recall that for a high enough bit of towers the amount of steps needed to solve it rises quickly. The story, “Now Inhale”, by Eric Frank Russell, uses 64 discs. Fun story.
Min steps is 2 to the power the number of disks minus 1.
Programming a system that solves it was a programming excersize for me a long time ago. Those are my stronger memories of it
It’s a solid intro CS puzzle for teaching recursion. I think the original story they invented to go with it also had 64 disks in a temple in, well, Hanoi. Once the priests finished it the world was supposed to end or something.