I am frequently interested in problems where answers are easily calculated from public data but the answer is unlikely to be already recorded in a form that search engines can find. Normally I spend a while noodling around looking for data and then use unit-conversion and basic arithmetic to get the final answer.
I tested Claude vs ChatGPT (which I believe is GPT 3.5) and vs Bard for a problem of this sort.
I asked:
1) What current type of power reactor consumes the least natural uranium per megawatt hour of electricity? (The answer is the pressurized heavy water reactor or CANDU type).
2) How much natural uranium does a PHWR consume per megawatt hour of electricity generated? (The answer is about 18 grams.)
3) How many terawatt hours does the United States generate annually from natural gas? (The answer as of 2022 is 1689 TWh, but any correct answer from the past 5 years would have been ok.)
4) How much natural uranium would the United States need to replace the electricity it currently generates from natural gas? (The answer is 1689 * 10^6 * 18 grams, e.g. about 30,400 metric tons of uranium.)
In the past Bard, Claude, and ChatGPT all correctly identified the CANDU or PHWR as the most efficient current reactor type.
Claude did the arithmetic correctly at stages 3 and 4, but it believed that a PHWR consumed about 170 grams of uranium per megawatt hour so its answer was off by nearly a factor of 10. ChatGPT got the initial grams-per-MWh value correct but its arithmetic was wild fantasy, so it was off by about a factor of 10000. Bard made multiple mistakes.
------
I just retried with Bard and ChatGPT as of today. On today's retry they fail at the first step.
Bard's response to the initial prompt was "According to the World Nuclear Association, an MSR could use as little as 100 grams of uranium per megawatt hour of electricity. This is about 100 times less than the amount of uranium used by a traditional pressurized water reactor."
Since there are no MSRs currently generating electricity, this answered the wrong question. The answer is also quantitatively wrong. Current PWRs consume nowhere near 10,000 grams of uranium per megawatt hour.
ChatGPT just said "As of my knowledge cutoff in September 2021, the type of power reactor that consumes the least natural uranium per megawatt hour of electricity is the pressurized water reactor (PWR). PWRs are one of the most common types of nuclear reactors used for commercial electricity generation."
This is wrong also. It correctly identified the CANDU as the most efficient in a previous session, but this was a while ago. I don't know if was just randomness that caused Bard and ChatGPT to previously deliver correct answers at the first step.
I tested Claude vs ChatGPT (which I believe is GPT 3.5) and vs Bard for a problem of this sort.
I asked:
1) What current type of power reactor consumes the least natural uranium per megawatt hour of electricity? (The answer is the pressurized heavy water reactor or CANDU type).
2) How much natural uranium does a PHWR consume per megawatt hour of electricity generated? (The answer is about 18 grams.)
3) How many terawatt hours does the United States generate annually from natural gas? (The answer as of 2022 is 1689 TWh, but any correct answer from the past 5 years would have been ok.)
4) How much natural uranium would the United States need to replace the electricity it currently generates from natural gas? (The answer is 1689 * 10^6 * 18 grams, e.g. about 30,400 metric tons of uranium.)
In the past Bard, Claude, and ChatGPT all correctly identified the CANDU or PHWR as the most efficient current reactor type.
Claude did the arithmetic correctly at stages 3 and 4, but it believed that a PHWR consumed about 170 grams of uranium per megawatt hour so its answer was off by nearly a factor of 10. ChatGPT got the initial grams-per-MWh value correct but its arithmetic was wild fantasy, so it was off by about a factor of 10000. Bard made multiple mistakes.
------
I just retried with Bard and ChatGPT as of today. On today's retry they fail at the first step.
Bard's response to the initial prompt was "According to the World Nuclear Association, an MSR could use as little as 100 grams of uranium per megawatt hour of electricity. This is about 100 times less than the amount of uranium used by a traditional pressurized water reactor."
Since there are no MSRs currently generating electricity, this answered the wrong question. The answer is also quantitatively wrong. Current PWRs consume nowhere near 10,000 grams of uranium per megawatt hour.
ChatGPT just said "As of my knowledge cutoff in September 2021, the type of power reactor that consumes the least natural uranium per megawatt hour of electricity is the pressurized water reactor (PWR). PWRs are one of the most common types of nuclear reactors used for commercial electricity generation."
This is wrong also. It correctly identified the CANDU as the most efficient in a previous session, but this was a while ago. I don't know if was just randomness that caused Bard and ChatGPT to previously deliver correct answers at the first step.