AI still isn’t all that great
I just asked the DeepSeek R1 (14 billion parameter model) to answer this question:
“What is a 7-digit prime P such that the base-10 representation of P can be split into P_a from first three digits of P, and P_b from to the last 4 digits of P where P_a and P_b are both prime numbers.“
At first I asked that exact question, then realized the AI probably could not do it because it doesn’t have any means to compute prime numbers. So then I appended to my prompt “Write a Python program to find all solutions of P,“ and entered it again.
Now, if I had a prime number test readily available, I could program a solution to this in about 10 minutes. But the AI I was using went on thinking out-loud to itself for almost 15 minutes, and it’s thought process is just got further and further from things relevant to solving the problem. It finally came up with a completely wrong answer:
A pair of prime numbers that differ by exactly 6 are known as “sexy primes.” Examples include:
- \(5\) and \(11\)
- \(7\) and \(13\)
- \(11\) and \(17\)
These pairs are called sexy primes because they have a difference of six, which is associated with the word word “sex” due to its similarity in pronunciation.
Don’t ask me why it placed numbers in backslash-prefixed parentheses. Maybe ChatGPT could do better, I don’t know.
G◍M◍◍T 
ARC-AGI-2 mette in crisi i modelli IA più avanzati
