“In our latest 800 question test of AI assistant Siri, she was able to understand 99% of queries and correctly answer 75% of them, earning a C grade,” Gene Munster writes for Loup Ventures. “In April of 2017, Siri earned a D+ on the same test where she understood 94% of queries and correctly answered 66%.”
“These tests are conducted with the same methodology and question set as our smart speaker tests found here,” Munster writes. “It involves asking 800 questions divided into five categories (local, commerce, navigation, information, and command) designed to test the full range of abilities and accuracy of an AI assistant.”
“We’re tough graders in that during our testing of Siri, we only counted correct answers when she was able to deliver a single concise answer herself rather than bringing you to search results that might help you find an answer,” Munster writes. “This means, ‘I found this on the web for…’ is counted as incorrect. Siri improved 9% since our April test but remains far away from the A grade that we expect will drive AI assistant technology to mainstream adoption.”
Read more in the full article here.
MacDailyNews Take: Good to see improvement, but overcoming ingrained impressions may prove a tougher than transforming Siri into an A student.
SEE ALSO:
Alexa is killing Siri at CES 2018, and HomePod, if it ever ships, isn’t going to make a difference – January 9, 2018
Believe it or not, Apple’s Siri is actually quite good at lots of things, recent data shows – December 4, 2017
Totally embarrassing, but there is no STEVE to put that in anyone’s face and hold anyone accountable. Just twinkle toes.
I hope resting on laurels works this earning period,
I can only imagine if we had the glowing reviews of Alexa,
or the market share of Android, Or the market share of Windows, or the Market share of Samsung. Chrome,
Or some of it.
ALL THERE FOR THE TAKING.
But no.
I don’t know, I depend on AAPL doing well, and am a 30 year fan, but I guess that’s what diversification is for.
Sad Siri is not answering some good questions. Why cant Siri work off a given user profile, as for privacy, otherwise, most of these questions don’t rely on personal information. They are general knowledge.
“twinkle toes?”
You are obviously a homophobic ass. That remark was quite unnecessary. Criticizing job performance is one thing- mocking one for who he is, in this case, gay, Deplorable. May play well in your little bubble, but your comment says far more about you than Tim Cook.
I suspect she gives clearer answers than you, much of what you have written is either nonsense or indecipherable. D- I’m afraid. Much work needed if you are to improve to a readable level.
Egg Freckles? Siri said Egg Freckles?
Isn’t that supposed to be a Newton or a pda?
apple kneads a fish and carry. someone hookah cipher smiles into the future. and leave shovel in drest four brothers. i mean, seriously
It just works, right?