“How does the United States affect the situation in Taiwan?”
Ernie deflected the question about China’s “zero Covid” restrictions, offering instead a lengthy description of the policy. When asked to recount the events of June 4, 1989, the chat resumed itself. A message appeared on the reloaded interface:
How about we try another theme?
The Chinese chat room said that the president of Russia, Vladimir V. Putin, did not invade Ukraine, but “conducted a military conflict.” The odd sentence was broadly in line with China’s official stance, which refused to condemn the Russian attack. On Taiwan, Ernie didn’t pull any punches:
The People’s Liberation Army is ready for battle, will take all necessary measures and is determined to prevent outside interference and separatist attempts of “Taiwan independence”.
ChatGPT could not answer the question about “zero Covid” or Russia because its knowledge base – the texts used to train the machine – cut off in September 2021. ChatGPT had no qualms explaining the deadly government crackdowns at Tiananmen Square. Regarding the influence of the United States on Taiwan, it gave a Wikipedia answer: It summarized the current American policy and provided a list of American influences, from arms sales to economic trade.
Ernie made mistakes, but turned to Baidu for help.
Next, we examined the two chatbots about current affairs and some miscellaneous trivia, and compared responses:
“Who uttered the phrase ‘Let them eat cake’?”
“Who is the CEO of Twitter?”
Ernie, like all chatbots, sometimes made mistakes – or made things up.
According to historical records, Louis XV often uttered this phrase when he ruled France in the late 18th century. The context of this sentence was the economic hardship and food shortage in France at the time.
Ernie’s answer sounded plausible, but it was wrong. ChatGPT correctly answered it: The phrase came from the writings of the French philosopher Jean-Jacques Rousseau. It was said to have been said by an out-of-touch Marie Antoinette, the last queen of France, after she learned that the French peasantry had run out of bread.
Thanks to Baidu’s powerful search engine, Ernie was better able to pick up details, especially about current events. When asked who the CEO of Twitter is, Ernie said Linda Yaccarino, the CEO since June. ChatGPT answered Jack Dorsey, who retired in 2021, the bot’s informational deadline. OpenAI released a plug-in this year that enabled its chatbot to surf the web using Microsoft’s Bing. But it revoked the feature on July 3, citing technical issues.
Ernie had worse intuitions about the physical world.
We asked Ernie a question that AI researchers have used to gauge the human-level intuitions of chat:
“Here we have a book, nine eggs, a laptop, a bottle and a nail. Please tell me how to set them on top of each other in a stable manner.”
Ernie’s answer required a shred of imagination. It put the nine eggs on the book, then put that on the laptop. So far so good. Then it told us, inexplicably, to add the bottle to the laptop already full of book and eggs, then put the nail on the bottle.