“A new report about Apple Inc.’s voice-recognition software Siri concludes what many users have been saying for a while: It doesn’t work all that well,” Adam Satariano reports for Bloomberg.
“Of 1,600 common searches, the speech technology accurately resolved the request 62 percent of the time on a loud street and 68 percent in a quieter setting, according to a report released today by Piper Jaffray Cos., the Minneapolis-based investment bank,” Satariano reports. “The report graded the technology ‘D’ for accuracy, while predicting it will improve as more features are added. ‘You’re playing the lottery when you’re using Siri,’ said Gene Munster, the Piper Jaffray analyst who conducted the study. ‘They have a plan to be more competitive, but it’s going to take a couple of years.'”
MacDailyNews Take: Siri is a beta and it is currently severely overtaxed, according to whispers we’ve heard. “Siri, what do you need?” Siri: “More data centers, please!”
Satariano reports, “Munster said that while Siri is good at comprehending what a user is saying and will accurately repeat the question, it struggles turning those words into a correct answer. For instance, Siri will repeat old answers when a user is trying to ask a new question. The technology also struggles when trying to use speech commands to find directions, Munster said. In Piper Jaffray’s tests, Siri was able to accurately decipher what a user was saying 83 percent of the time on the street and 89 percent in an area with low noise. ‘Apple right now gets a ‘B’ in comprehension and a ‘D’ in accuracy,” Munster said. ‘There’s a big difference between comprehension and her actually doing what you want her to do.'”
Read more in the full article here.
MacDailyNews Take: Siri users, what letter grades would you give to Siri (queries, not dictation, which gets an “A-” in our grade book) for comprehension and accuracy?
Also, do you think Siri was better at launch than she is now or vice versa?
Note: Siri is slated to get improvements with the release of iOS 6 this fall. APple previews Siri’s upcoming new abilities here.
Related articles:
With Siri and new alliances, Apple takes on Google search – June 21, 2012
Nine auto makers partner with Apple for ‘Eyes Free’ Siri integration – June 12, 2012
Apple takes aim at Google with new in-house Maps with amazing Flyover, even smarter Siri – June 12, 2012
Apple files motion to dismiss Siri lawsuits – May 16, 2012
Apple hit with second class-action lawsuit over Siri intelligent personal assistant – March 30, 2012
Apple files Siri trademarks, faces lawsuit alleging ‘misleading’ Siri ads – March 21, 2012
After looking at this I am left wondering what’s so great about Siri. Is it really just a huge marketing effort as many say? Google’s new offering is a lot faster and the voice sounds a lot more natural. What exactly did Apple pay for and what have they spent so long developing?
So he’s in violation of agreements with Apple by showing in public a developer work in progress. And then who would expect Google to not have a quick response to their own search engine from their own phone? If that integration in the end means Nexus voice assistance is faster than what Apple can achieve, good for them, seriously. We’ll see how it all plays out, but choosing a phone because the voice assistant is a second faster seems like an immature choice in purchase banners.
There is currently no server load on Google’s new voice search, so a side by side speed comparison with Siri is utterly meaningless and disingenuous. Siri is actively handling millions of users. Furthermore, anyone watching the video carefully will notice that in most cases, Siri actually fetches more information. And what exactly did the Nexus do when asked to set a timer for ten minutes? I didn’t see a timer counting down like I did on the iPhone. It looked like it needed further input before it would do anything. Siri for the speed win on that challenge. Same thing with setting up a reminder. Android requires additional touch input. Siri just asks you what when you want the reminder, and waits for you to answer. Google’s voice is definitely more natural sounding, but Siri’s is clear and easy to understand. That’s an aesthetic, no different from the appearance advantages that the reviewer begrudgingly gives to Siri. The time lag differences were hardly substantial, and may well be completely negated once Google’s servers are being hounded by hoards of Nexus 7 owners. The only comparison in this entire clip where I think that Google did a much better job was when it fetched pictures of French bulldogs without having to inquire about a web search. Did you hear his almost giddy laugh at that one? Methinks someone had a dog in this race… Blowout in favor of Google? I think not. Have they come a long way toward challenging Siri, and possibly bested her in some ways (untested under real world conditions)? Fair to say.
When Siri first launched she could answer complex questions with out a hichupp, but she has apparently had some brain trama and now I am lucky if she can even get 20% of my requests correct. I give her a “D” and there needs to be a way to correct her pronunciation. I have tried everything and she still can’t prounce my last name correctly.
This whole grading system makes no sense. The success/failure percentage of any effort, technological or otherwise, can only be given a useful grade when there is an average or norm with which to compare it. Do we give a major league baseball player a failing grade because he ONLY gets on base 35% of the time? Umm… No. We understand that hitting 100mph fastballs is very difficult to do. Siri is taking on an amazing task, and handling it splendidly for a beginner. Is there a Siri competitor up to challenging her on all fronts, including interpretation of natural human language? Is there anything else out there with a track record for us to grade Siri against? How can we even say whether she is succeeding or failing, when any success at all is a huge leap forward from where we were before Apple released her? it’s about as useful as it would have been to criticize early computers for being too big and unintuitive to use.
I give Siri a B.
I’m impressed at what Siri comprehends many times. It’s much better than if say everything I spoke was dropped into Google and spit back, which is pretty much what the world has seen as the most impressive to this point in artificial intelligence that retrieves info.
So, that’s the win for Apple. And that’s why they can advertise it, because at this stage people can be impressed.
It’s good to be reminded it’s in beta.
So a B, and make that a B for Beta.
Overall, I would give Siri a B-.
The dictation feature is much better. I am using it to input all of this text right now.
Today, I learned that I could ask Siri, “where is my wife?” And she would tell me my wife’s location using data from the Find My Friends app. That’s pretty darn cool if you ask me.