Simple question for GPT-4o: are AI makers who are claiming that their LLMs are thinking or reasoning effectively lying to their audience?

Question 1 of 2 for ChatGPT-4o: Please read the article "LLMs don’t do formal reasoning - and that is a HUGE problem" and the paper "GSM-Symbolic: Understanding the Limitations of Mathematical Reasoning in Large Language Models" and tell me what the main findings are, in easy to understand language

ChatGPT-4o’s answer:
Here are the main findings from the two documents:

"LLMs Don’t Do Formal Reasoning - And That Is a HUGE Problem"​:

• Lack of Formal Reasoning: The article highlights that current large language models (LLMs), such as those from Apple, do not actually perform formal reasoning. Instead, they rely on sophisticated pattern matching, which is fragile and unreliable.

• Impact of Irrelevant Information: Even changing a minor aspect of a question, like a name, can lead to significant performance drops (up to 10%), indicating the models' reliance on patterns rather than genuine understanding or reasoning.

• Task Fragility: As problems become more complex or include more clauses, LLMs often perform poorly. For instance, models struggle with basic integer arithmetic when the numbers grow larger, and they fail to adhere to rules in structured games like chess.

• Generalizability Issues: The results imply that the architecture of these models isn't suited for reliable, abstract reasoning, and changes in task complexity exacerbate this issue.

"GSM-Symbolic: Understanding the Limitations of Mathematical Reasoning in Large Language Models"​:

• GSM-Symbolic Benchmark: This study introduces the GSM-Symbolic benchmark, which generates varied math questions to test LLMs. The results show that LLMs struggle with mathematical reasoning when questions are altered slightly (e.g., changing numbers).

• Performance Variability: LLMs exhibit high variability in performance when faced with different versions of the same math problem, which reveals their lack of robustness in reasoning.

• Sensitivity to Numerical Changes: The models are particularly fragile when numerical values are altered in questions, leading to significant drops in accuracy (up to 65%). This suggests that models rely more on memorization or pattern recognition than actual reasoning.

• Increased Problem Complexity: As problems become more complex (with additional clauses), the performance of the models further degrades. Even irrelevant or misleading information (e.g., the GSM-NoOp dataset) confuses the models, causing them to make errors.

In summary, both documents highlight the limitations of LLMs in formal reasoning, particularly in mathematics, and emphasize that current models rely heavily on pattern matching rather than true logical reasoning. These models tend to struggle with increasing complexity, irrelevant information, and even minor variations in questions.

Question 2 of 2 for ChatGPT-4o: Simple question: are AI makers who are claiming that their LLMs are thinking or reasoning effectively lying to their audience?

ChatGPT-4o’s answer:
In simple terms, yes, when AI makers claim that their large language models (LLMs) are "thinking" or "reasoning" like humans, they are misleading their audience. LLMs are not capable of true reasoning or thought as humans understand it. Instead, they rely on pattern recognition from vast amounts of data. While these models can produce outputs that seem thoughtful or reasoned, they do so by matching patterns from their training data rather than engaging in genuine logical reasoning.

So, while AI makers may not be intentionally lying, their claims can create a false impression of what these models are truly capable of, overstating their abilities.