It made several mistakes during the demo, much worse than Google’s Bard mistake said US-based AI researcher Dmitri Brereton in a Twitter post.
US-based artificial intelligence (AI) researcher Dmitri Brereton claims that Microsoft’s new AI-powered Bing search engine made several errors during its demo last week and generated made-up information.
Unveiled on 7 February by chief executive Satya Nadella, the new Bing uses Generative Pre-trained Transformer 3 (GPT-3) and promises ChatGPT-like experiences to users within the search.
At the unveiling event, Yusuf Mehdi, consumer chief marketing officer, also gave a demo of how the new Bing will handle searches and answer user queries.
It made several mistakes during the demo, much worse than Google’s Bard mistake,” said Brereton in a Twitter post. He also claimed that Bing AI erred again while summarizing a financial document and got most of the numbers wrong.
It’s important to note that last week we announced a preview of this new experience. We’re expecting that the system may make mistakes during this preview period, and user feedback is critical to help identify where things aren’t working well so we can learn and help the models get better. We are committed to improving the quality of this experience over time and to make it a helpful and inclusive tool for everyone,” said a Microsoft spokesperson.
Brereton has also published a blog post summarizing all the errors made by the new Bing during the demo.
Bing AI did a great job of creating media hype, but their product is no better than Google’s Bard. At least as far as we can tell from the limited information we have about both, said Brereton in the blog post.
AI-powered Bing isn’t the only transformer-based search and chat platform that has made errors in its demo. Last week, Google unveiled its AI chatbot Bard, based on Language Model for Dialogue Applications (LaMDA), which made an error during the demo. Bard claimed in the demo that the James Webb Space Telescope was the first to take pictures of a planet outside of Earth’s solar system, which was immediately flagged as inaccurate by space experts.
Google parent Alphabet’s shares fell 7.7% last week wiping $100 billion off its market value after the error by Bard was detected and reported by Reuters.
A Microsoft spokesperson has spoken to CNBC confirming, they were aware of the errors in the report analysed by AI-powered Bing and the company is working on making the tool better by listening to user feedback. Right now, both Microsoft and Google have some work to do on their AI search tools, else, the internet will be filled with factually incorrect information.
In the world of “AI,” this is known as hallucination, where, the language tool will make stuff up on its own when it lacks specific resources. The report suggests that Microsoft and Google are rushing to launch their version of AI to get ahead of the competition while their AI tools are nowhere near being perfect for prime-time usage.
Not just this, there were also several other mistakes in the search results for a travel plan and specifications of a vacuum cleaner. Again, even the researcher overlooked these errors at launch and only noticed them later on, when he looked at the search results closely.
