“Ancient Puzzle Breaks AI Logic”: Scientists Watch ChatGPT Struggle With Plato’s 2,400-Year-Old Geometry Test And Improvise Like A Confused Student

The question of whether mathematical knowledge is innate or learned has been a subject of philosophical debate for centuries. Recently, a team of scientists from Cambridge University and the Hebrew University of Jerusalem revisited this question using modern technology. They posed an ancient problem to ChatGPT, a large language model. The problem, known as “doubling the square,” was originally articulated by the Greek philosopher Plato over two millennia ago. This experiment aimed to explore the capabilities of artificial intelligence in solving mathematical problems with non-obvious solutions and to question whether AI can mimic human reasoning.

The Ancient Problem of Doubling the Square

The “doubling the square” problem dates back to around 385 B.C.E., attributed to the philosopher Plato. In the problem, Socrates challenges a student to double the area of a square. The student mistakenly doubles the length of each side, unaware that the correct approach involves using the diagonal of the original square. This problem has been used for centuries to debate the nature of mathematical understanding — whether it is an innate ability or something acquired through experience and education.

In their experiment, the scientists selected this problem for ChatGPT, hypothesizing that its solution might not be readily available in the language model’s training data. Unlike humans, who can visualize and manipulate geometric concepts, ChatGPT primarily processes text. This made its task all the more challenging, as it required the AI to exhibit reasoning akin to creative problem-solving, rather than relying on pre-existing knowledge.

Scientists Crack 100-Year-Old Math Mystery That Can Finally Unlock the Full Power Potential of Turbines Worldwide

AI's Improvisation and Its Implications

The experiment took a fascinating turn when the scientists extended the problem to include doubling the area of a rectangle. ChatGPT concluded that the diagonal could not be used to double the rectangle's size, declaring no geometric solution existed. This response, although incorrect, suggested that the AI was improvising based on limited knowledge. Visiting scholar Nadav Marco and professor Andreas Stylianides noted that the AI's behavior was similar to that of a human learner, creating its own hypotheses based on prior discussions.

This improvisation was significant as it highlighted the potential for AI to engage in a form of problem-solving that mimics human reasoning. The scientists posited that ChatGPT might be operating within a framework similar to the educational concept of the "zone of proximal development" (ZPD). This concept describes the gap between what learners can do independently and what they can achieve with guidance. In this context, ChatGPT appeared to be navigating unknown territory with the help of prompts, showcasing a form of learning without explicit training data.

“Math World Shaken to Its Core”: 200-Year-Old Algebra Rule Broken by Stunning Geode Number Discovery That Changes Everything

The Black Box of AI Reasoning

The experiment underscores the longstanding issue of transparency in AI systems, often referred to as the "black box" problem. This term describes the difficulty of understanding how AI models arrive at specific conclusions. While the results of the experiment were intriguing, they also serve as a reminder of the opaqueness inherent in AI reasoning processes. The researchers emphasized the importance of developing skills to critically evaluate AI-generated solutions, particularly in educational settings.

Professor Stylianides stressed that students should not assume the validity of AI-generated proofs, as these do not undergo the rigorous scrutiny typical of academic textbooks. Instead, the focus should be on teaching students to engage with AI as a collaborative tool, capable of enhancing, but not replacing, human reasoning. By improving prompt engineering, educators can better harness AI's potential to support learning and exploration.

“It Failed Ancient Math Completely”: ChatGPT-4 Couldn’t Solve Plato’s 2,400 Year Old Problem And Made Same Mistakes As Confused Students

Future Directions in AI and Education

The findings of the study open up several avenues for future research. As AI models become increasingly sophisticated, there is potential to test them on a broader range of mathematical problems. The integration of AI with dynamic geometry systems or theorem provers could create more interactive and intuitive digital learning environments. Such developments could revolutionize educational practices, allowing students and teachers to collaborate with AI in innovative ways.

Despite the promising possibilities, the researchers caution against over-interpreting the results. While ChatGPT's behavior was "learner-like," it is essential to remember that AI does not "think" in the same way humans do. However, the study highlights the potential for AI to become a valuable partner in education, prompting us to rethink how artificial intelligence can be integrated into learning experiences.

This exploration into AI's capabilities raises important questions about the future of education and technology. How will educators and technologists balance the potential benefits of AI with the need for critical engagement? What role will AI play in shaping the next generation of learners and thinkers?