Elon Musk’s Wikipedia rival Glokipedia got off to a “rocky start” in its public debut, but Wikipedia founder Jimmy Wales didn’t have to see the AI’s output to know what he was expecting.
“I’m not optimistic that he’ll produce anything very useful right away,” Wales said Tuesday at the CNBC Technology Executive Council Summit in New York City.
Wales had a lot of choice words for Musk, especially regarding claims that Wikipedia has “woke bias”. “He’s misunderstood about it,” Wales said. “His complaint about Wiki is that we focus on mainstream sources, and I have no qualms about that. We don’t treat random crap the same way we treat the New England Journal of Medicine. That doesn’t mean we’re woke,” he said at a CNBC event. “It’s a paradox. We’re radical enough to quote the New York Times.”
“I haven’t had time to take a look at Grokipedia, so it would be interesting to look at it, but apparently it contains a lot of praise for the genius of Elon Musk, so I’m sure it’s completely neutral,” he added.
Mr. Welsh’s investigation at Grokipedia, which has its own Wiki page, was not about his ongoing battle with Mr. Musk, but rather his grave concerns about efforts by all large language models to create authoritative online information sources.
“The LLM he’s using to write it will make significant errors,” Wales said. “We know that ChatGPT and all other LLMs are not good enough to create Wiki entries.”
Musk seemed equally convinced of the opposite outcome, saying in a post Tuesday night that “Glocipedia will surpass Wikipedia in breadth, depth, and accuracy by several orders of magnitude.”
Mr. Wales gave several examples of why he does not trust LLMs to rebuild what Wikipedia’s global community has been building for decades at a fraction of the cost. The organization’s hard technology costs are $175 million a year, compared to the tens of billions of dollars that big tech companies consistently pour into AI efforts, and one Wall Street estimate puts so-called hyperscalers’ expected total AI spending next year at $550 billion.
One example that Mr. Wales gives of LLM’s inaccuracies concerns his wife. Mr Wales said he often asked his new chatbot model to investigate obscure topics to test its capabilities, and when he asked it who his wife was, a “not famous but known” person who worked in British politics, he said the answer was always “plausible but wrong”. Whenever you ask an LLM to dig deep, “it’s a mess,” Wales added.
He also gave the example of a German Wiki community member who wrote a program to verify the ISBN numbers of cited books and was able to track down a person’s notable mistakes. The person eventually confessed that he had been using ChatGPT to find citations for text references, but the LLM was “happy to make a book for you,” Wales said.
Wales said the battle between masks and AI that he has become embroiled in certainly reinforces an important message for Wikipedia. “It’s very important for us and the Wiki community to respond to criticisms like this by doubling down on our neutrality and paying close attention to our sources,” he said. “We shouldn’t be ‘Wakepedia.’ That’s not what we should be or what people want us to be. That undermines trust.”
Wales believes that the public and media often overestimate Wikipedia. He says that in the early days of the site, it wasn’t as bad as people joked about it. But now, he says, “We’re not as good as they think. Of course we’re much better than before, but there’s still a lot of work to do.”
And he expects the challenges posed by technology and misinformation to become even worse as the ability to use LLM to create fake websites with plausible text and possibly deceiving the public increases. But he says it will be difficult to fool the Wiki community, which has spent 25 years researching and discussing reliable sources. “But that ends up deceiving a lot of people, and that’s a problem,” he said.
In some cases, he said, the same new technology that “creates something completely useless” could be useful to Wikipedia. Wales is working to find limited areas where AI can find additional information to add to the wiki from existing sources, something he says is “sort of okay” with Gen AI’s usage right now.
“Maybe it will help us do our job faster,” he said. That feedback loop could be very helpful for sites if they developed their own LLMs that they could train, but the costs associated with it have kept them from making a formal effort while they continue to test the technology, he added.
“We’re really happy that Wiki is now part of the world’s infrastructure. This is a pretty big burden on us. So when people say we’re biased, we need to take that seriously and address everything related to that,” Wales said.
But he couldn’t help but say otherwise. “We’re talking about the errors ChatGPT makes. Imagine an AI trained on Twitter alone. That would be a crazy, angry AI trained on nonsense,” Wales said.
