<
https://pivot-to-ai.com/2024/07/06/llms-can-solve-any-word-problem-as-long-as-they-can-crib-the-answer/>
"AI companies keep claiming that LLMs do reasoning. The AI can think! The AI
can figure out word puzzles! The AI can pass the LSAT! You’re fired!
So our good friend Diz tested this by throwing logic puzzles and paradoxes at
ChatGPT — the sort where you have a fox and a chicken to take across a river
without the fox eating the chicken.
You’ll be unsurprised to hear that LLMs do well at puzzles if the answers are
already in their training data. If you vary the puzzle even slightly, the LLM
fails hard.
The LLM doesn’t actually form an internal representation of any problem. It
only outputs words that would statistically follow you stating such a problem."
Cheers,
*** Xanni ***
--
mailto:xanni@xanadu.net Andrew Pam
http://xanadu.com.au/ Chief Scientist, Xanadu
https://glasswings.com.au/ Partner, Glass Wings
https://sericyb.com.au/ Manager, Serious Cybernetics
