Discussion about this post

User's avatar
Eliezer Yudkowsky's avatar

You needed a better step-by-step prompt! ChatGPT generates answers one word at a time, sequentially; as soon as it answered "No" right off the bat, all its later "step-by-step" reasoning was useless and played no role in generating the "No." This of course is a mistake humans also make, but if you want to get ChatGPT to be any smarter than humans about it, you need some prompt that tells it to not answer right away until after it's reasoned step-by-step. As soon as the AI answers "No", it's done reasoning step-by-step; if it answers that right away, the prompt failed.

The original step-by-step prompt is in GPT-3 format, not ChatGPT format, and has the answerer *itself* reply "Let's reason step-by-step." before the rest of the answer gets filled in.

Expand full comment
Simon's avatar

Interestingly, it knows what mistake it made

Me: is the number of letters in the word 'prime' prime?

ChatGPT: No, the number of letters in the word "prime" is not a prime number. Prime numbers are positive integers that have only two positive integer divisors: 1 and themselves. The number 5, for example, is a prime number because it can only be divided evenly by 1 and 5. The number of letters in the word "prime," which is 5, can also be evenly divided by 2 and 5, so it is not a prime number.

Me: spot the mistake in your answer

ChatGPT: I apologize if my previous response was incorrect. I'll review my response and try to identify any mistakes.

In my previous response, I stated that the number of letters in the word "prime" is not a prime number because it can be divided evenly by 2 and 5. However, this is incorrect. The number of letters in the word "prime," which is 5, is actually a prime number because it can only be divided evenly by 1 and 5. I apologize for the mistake in my previous response.

Expand full comment
9 more comments...

No posts