You’re not thinking about how an LLM works: That would be a super easy question for it because there are tons of resources about imaginary numbers and i specifically online.
But ask it to add two random 4-digit numbers (an easy task for a computer or an elementary school student) and it might get the answer wrong just because those precise numbers haven’t been added enough on the web, and they treat numbers the same way they do words.
I gave DeepSeek conniptions by saying “I am somewhere undetermined on Earth. I walk 4m straight south. I then turn and walk 4m due west. I finally turn and walk 4m straight north. How far am I from my starting point?”
It first gave a facile answer based on being in a flat space, but I reminded it I was on planet Earth which is a sphere.
It hasn’t stopped its “reasoning” yet, about ten minutes after I started. It’s churning out a bunch of pseudo-reasoning complete with pseudo-maths that is hilarious to observe.
I guess it really doesn’t ‘want’ to say “not enough information” - which I think is the true answer:
If your starting point was the North Pole then you are back exactly where you started, if it was along the equator then you are within rounding error of 4m west of where you started. Start 4m from the South Pole and “4m west” ends up undefined. At the SP, “4m south” is undefined, etc.
The full range of defined answers is 0 to 10.546m, roughly. Then there’s the undefined ones when you get closer to the south pole than 4m.
The key to fucking with a “reasoning” LLMbecile is to ask it a question that involves:
Spatial reasoning; and,
Multiple steps for full reasoning to play out.
Any one of these alone may or may not fuck it up, but if you give it both I’ve never seen one pass it unless it’s one that’s common knowledge that it basically already has the answer to.
It hasn’t stopped its “reasoning” yet, about ten minutes after I started. It’s churning out a bunch of pseudo-reasoning complete with pseudo-maths that is hilarious to observe.
You’re not thinking about how an LLM works: That would be a super easy question for it because there are tons of resources about imaginary numbers and i specifically online.
But ask it to add two random 4-digit numbers (an easy task for a computer or an elementary school student) and it might get the answer wrong just because those precise numbers haven’t been added enough on the web, and they treat numbers the same way they do words.
I gave DeepSeek conniptions by saying “I am somewhere undetermined on Earth. I walk 4m straight south. I then turn and walk 4m due west. I finally turn and walk 4m straight north. How far am I from my starting point?”
It first gave a facile answer based on being in a flat space, but I reminded it I was on planet Earth which is a sphere.
It hasn’t stopped its “reasoning” yet, about ten minutes after I started. It’s churning out a bunch of pseudo-reasoning complete with pseudo-maths that is hilarious to observe.
Yeah, that tracks.
I guess it really doesn’t ‘want’ to say “not enough information” - which I think is the true answer:
If your starting point was the North Pole then you are back exactly where you started, if it was along the equator then you are within rounding error of 4m west of where you started. Start 4m from the South Pole and “4m west” ends up undefined. At the SP, “4m south” is undefined, etc.
The full range of defined answers is 0 to 10.546m, roughly. Then there’s the undefined ones when you get closer to the south pole than 4m.
The key to fucking with a “reasoning” LLMbecile is to ask it a question that involves:
Any one of these alone may or may not fuck it up, but if you give it both I’ve never seen one pass it unless it’s one that’s common knowledge that it basically already has the answer to.
Hope it’s worth killing the environment for