Following up on last year’s impressive comparison of personal AI solutions, Loup Ventures today released the results of its 2019 Digital Assistant IQ Test, and there’s good news if you enjoy giving voice commands to your phone, tablet, or speaker: All of the leading digital assistants are getting better at their jobs.
Using a test comprised of the same 800 questions across each AI system, Google Assistant once again led the pack, understanding a full 100% of the questions it was asked, just like last year, and correctly answering 92.9% of them. That’s up from 85.5% correct last year, and rapidly approaching a level of accuracy where errors won’t be a common occurrence.
By contrast, Apple’s Siri jumped in both categories, rising from a 99% understanding level last year to 99.8% this year, and 2018’s 78.5% correct answer level to a 83.1% correct level for 2019. Another way of looking at that — even though it may conflict with real-world Siri user experiences — is that Siri is nearly as likely to respond correctly this year as Google Assistant was last year.
Amazon’s Alexa once again took third place, but made major strides this year, understanding 99.9% of the questions and answering them correctly 79.8% of the time, better than last year’s Siri performance. That’s a sharp rise in correct answers for Alexa, which jumped from a surprising low of 61.4% last year, and Loup notes that it’s the largest jump it has seen between years since it started recording results.
Notably, Loup left out Microsoft’s Cortana this year, which isn’t hugely surprising as the fourth-place AI has been disappearing from Microsoft’s products and third-party accessories. Cortana had only answered 52.4% of last year’s questions correctly, which is to say that you’d be just as well off flipping a coin or guessing if your question could be answered in a binary fashion.
Read more here: