“We recently tested four smart speakers by asking Alexa, Siri, Google Assistant, and Cortana 800 questions each,” Gene Munster and Will Thompson write for Loup Ventures. “Google Assistant was able to answer 88% of them correctly vs. Siri at 75%, Alexa at 73%, and Cortana at 63%. Last year, Google Assistant was able to answer 81% correctly vs. Siri (Feb-18) at 52%, Alexa at 64%, and Cortana at 56%.”
“We separate digital assistants on your smartphone from smart speakers because, while the underlying tech is similar, the use cases and user experience differ greatly,” Munster and Thompson write. “Google Home continued its outperformance, answering 86% correctly and understanding all 800 questions. The HomePod correctly answered 75% and only misunderstood 3, the Echo correctly answered 73% and misunderstood 8 questions, and Cortana correctly answered 63% and misunderstood just 5 questions.”
“Note that nearly every misunderstood question involved a proper noun, often the name of a local town or restaurant. Both the voice recognition and natural language processing of digital assistants across the board has improved to the point where, within reason, they will understand everything you say to them,” Munster and Thompson write. “Google Home has the edge in four out of the five categories [Local, Commerce, Navigation, and Information] but falls short of Siri in the Command category. HomePod’s lead in this category may come from the fact that the HomePod will pass on full SiriKit requests like those regarding messaging, lists, and basically anything other than music to the iOS device paired to the speaker. Siri on iPhone has deep integration with email, calendar, messaging, and other areas of focus in our Command category. Our question set also contains a fair amount of music-related queries, which HomePod specializes in.”
“Over a 12-month period, Google Home improved by 7 percentage points, Echo by 9 points, Siri (9-month) by 22 points, and Cortana by 7 points in terms of questions answered correctly,” Munster and Thompson write. “We continue to be impressed with the speed at which this technology is making meaningful improvement.”
Much more in the full article – recommended – here.
MacDailyNews Take: Anytime anyone claims Siri’s not that bright, tell them they’re lucky they never dealt with Microsoft’s Cortana.
SEE ALSO:
AI guru John Giannandrea named to Apple’s executive team – December 20, 2018
Former Apple employees on Eddy Cue: Siri and Eddy were ‘a bad fit’ – September 5, 2018
Apple’s Siri improved by 11 percentage points in correct answers over the last 15 months – July 25, 2018
Apple combines machine learning and Siri teams under John Giannandrea – July 10, 2018
Apple’s ‘personal assistant activation’ patent application hints at improved Siri – May 10, 2018
iOS 11.4 will allow Siri to recognize AirPlay commands – May 3, 2018
iPhone X owners are extremely satisfied with basically everything except Siri – April 20, 2018
Apple’s Siri, HomePod and the voice assistant showdown – April 6, 2018
Apple’s A.I. efforts get shot of adrenaline with Giannandrea coup – April 6, 2018
A.I. defector gives Apple access to Google’s secrets – April 6, 2018
Gene Munster: Poaching AI chief John Giannandrea from Google a win for Apple – April 5, 2018
Apple hires Google’s A.I. chief to improve Siri – April 4, 2018
Siri will soon surpass all of them. The “terrible Siri” meme will take awhile to die, but die it will.
From your mouth to Gods ears !
When I bought the 4S a key feature was the debut of Siri as the first voice assistant on a smartphone.
It was magical and worked very well at first, yet somehow seriously degraded over time. The attention to precision door knobs and glass walls in the spaceship must have zapped resources for years.
Years ago it got so bad it could not directly dial my parents mobile number without asking a few questions. That’s when I noticed the downgrade in quality, gee what a surprise under Cook.
The article points out Siri improved 22% since the last test. While almost a quarter in improvement is certainly good news, proves my point how it fell behind.
When you are repeatedly disappointed, disillusioned and let down by Apple, it is not easy to recover as a satisfied customer. That same, albeit far MORE INTENSE sinking feeling, also applies to the Mac Pro…
GoeBlow, It seems highly unlikely that you will ever “…recover as a satisfied customer.” If your sinking feeling is that INTENSE, then I suggest that you move on. Perhaps the grass will be greener for you on the Wintel PC and Android side. Please try it for a few years and let us know how it turns out in 2021 or so. Or never…
Uh no…GoeBlow is immune to common sense, as his over-active whingeing record admits no possibility of ever connecting the dots. He’s now more of a worn seat cover.
Oh look, the KingSmell sycophant lapdog right on Q…
Oh look, KingSmell snowflake fanboy can’t handle constructive criticism. Go back to playing games and watching movies leaving reality to the adults around here…
@KingMel,
GoeB is a homophobic, shyte disturber. It is the only reason he stays on this forum, to disturb shyte.
Some people are disturbed like that. Go figure.
Best to ignore them.
“GoeB is a homophobic”
Congrats, the most irresponsible FALSE ACCUSATION ever leveled against me on MDN!
Hey Buster, you pull the “homophobic” trigger like a gun against anyone that criticizes the performance of Cook.
It is TOTALLY FALSE, BRAINLESS OPINION, and not an atomic particle of FACT.
Pay close attention: YOU ARE DEAD WRONG! Don’t do it again…
We read it only yesterday in an article on here. I just couldn’t be bothered to answer it with regard to tests defying the perception of the author, it’s like banging your head against a wall. 24 hours later further proof of the invalidity of this perception. Yep if tales a long time for reality to overcome prejudice especially when it’s in the interest of so many damn trolls and purveyors of misinformation to persist with the lies.
from my experience? not it’s not
Agreed. I regularly get frustrated with Siri.
That’s not what’s interesting. What is interesting is that Siri and Cortana both use Bing as the search engine, so the difference between them is…. ? Speech recognition? What else?
Yes, Siri’s speech recognition is the bane. Amazingly curious are the errors in “recorrecting.” Many times words are interpreted correctly, but then changed to error.
Currentinterest: yes, let’s hope so, but one has to ask, why in the world would Apple let a major handicapped service run wild in the 1st place and, for so long? Maps’ release was ill-timed, but the recovery has been solid.
I love my Apple kit and always will – but Echo is so far ahead of Apple in voice – it may be uncatchable. Tim and the exec failed in realising that home automation wouldn’t be a chance to create niche expensive kit – but mass adoption. I’ve now got 2 dots and a plus planned soon. So easy, works and great value.