“Compared to Bing and Siri, Google’s artificial intelligent system is a cut above the competition, according to new research at Cornell University,” Paul Lilly reports for HotHardware.
“Researchers Feng Liu, Yong Shi, and Ying Liu set out to test and rank the intelligence quotient (IQ) of various natural and artificial intelligent systems, including humans, Google, Bing, Baidu, and Siri. Based on the tests conducted, Google’s AI has an IQ of 47.28,” Lilly reports. “That ranks just below a six-year child with an IQ of 55.5, but is twice as high as Siri with an IQ of 23.9. It’s also higher than Bing (31.98) and Baidu (32.92).”
Lilly reports, “‘Although this work is still in progress, the results so far indicate that the artificial-intelligence systems produced by Google, Baidu, and others have significantly improved over the past two years but still have certain gaps as compared with even a six-year-old child,'” the researchers wrote in a paper published on ArXiv.”
Read more in the full article here.
MacDailyNews Take: Again, as we wrote earlier today, Apple’s dedication to privacy hamstrings Siri. Google et al. have no such issue. To them, users’ privacy is to be trampled.
It’s not at all apparent that the general public values their privacy enough or even knows that Apple’s privacy is paramount, but the average Joe/Jane does seem to regard Siri as not too bright, putting into question whether Apple’s commitment to privacy will every really pay off; i.e. translate to increased product sales.
Apple product users seem to value their privacy. Non-Apple product users, by definition, do not value their privacy (or they’d be Apple product users).
So, what’s the inflection point? Do Google and the others need to have an Equifax event befall it for their product users to wake up? Would they even wake up if Google etc. did have a cataclysmic breach? We have our doubts.
SEE ALSO:
Siri, why have you fallen behind other digital assistants? – October 5, 2017
[Thanks to MacDailyNews Reader “TJ” for the heads up.]
I don’t quite understand MDN’s point that privacy protections somehow keep Siri from being as efficient and reliable as Google’s regarding the ability to respond with correct information from queries.
I believe their point is that Google’s AI prowess comes as a result that Google machines read all gmail, track all web searches, etc. They have orders of magnitude more human communication and interaction data for their AI to learn from. Since Apple doesn’t pore over user data like Google does, they are at a disadvantage at machine learning as compared to the brute force that Google wields.
thanks.
WOW! A Thanks for botvirnnik… really – I have never seen this!
It’s a kinder gentler botvinnik these days.
because of the swirling Bot-Hatred, people with animus flowing within, still proactively give him the middle finger down-votes even when comments (like this) are pedestrian. Diaper Salesman; where are you? There are many damp and uncomfortable toddlers needing a change.
P.S. dude you can get rid of your super lame trump avatar we get that you have a boner for the guy…
I will on January 20th, 2025.
How is Google’s use of data to teach Google AI any different in result with Apple aggregating data from their own users? In the end no personal data is actually used in the general ‘smarts’ of the system. Personalization will demand private data but that is another part of the system that isn’t being judged with this article just like MDN commented similarly in an earlier article. Lack of access to private data is not an excuse for not being at least as competent on non-private related queries.
That indeed seems to be what MDN is attempting to assert but that’s not how intelligence quotient works.
Big Data can rapidly tell marketers what the gossip is, what is selling, etc. Intelligence requires the ability to use logic and follow rules of grammar, math,etc.
Google and its fans think it is smart because it knows what you put in your email and can regurgitate it back at you. Apple fans know better. Unfortunately we also know that Siri is as dumb as a box of rocks. Apple can’t even offer a good spell checker, for goodness sake.
All true. I prefer the thought that Siri, simple lass that she is, will not pick your pocket or finger you to the police.
I live inTenafly NJ, a borough of 15,000 ppl and probably twice as many Apple products.
Siri doesn’t believe it’s a town. Ask Siri for directions to Tenafly NJ and it will give you directions to “ten a fly” somewhere in the world. I kid you not!
I have reported this every new release. Tenafly still exists but not according to Siri.
OTOH Google AI has no problem with Tenafly. Try it yourself.
Siri is such a stupid assistant on Sierra. I asked several times for Tenafly, New Jersey and it kept giving me directions to 10 Fay Place, Oradell, New Jersey. FAIL!
Google Search on Chrome using voice got it right the first time in an instant. PASS!
Apple really disappointed me. How can Siri not understand something as simple as that? I’m really disgusted. It should be smart enough to know if asked a number of times that it’s not giving the right response. It’s no wonder Apple is valued below the FANG stocks.
You know, I tested your statement. Lo and behold. Siri wants to take me to 10 For Drive, Hamilton, NJ. LOL!
10 Foy Drive….
Siri simply doesn’t have the sound of Tenafly listed as a city and so tries to convert it to something else. Not good at all, but understandable. It doesn’t seem all that simple to me.
A long time ago when I was a naive teenager I had a summer job in a travel agency. A client called up and needed tickets to Hot Foot, Connecticut. This was before computerization, and so I grabbed to giant route book and was searching and searching. I finally had to tell him that I couldn’t find any Hot Foot, Connecticut. He said “No! Hot Foot. Hot Foot!” and slammed down the phone.
I’m sure that man who wanted to go home to Hartford thought I was just as much an idiot as Siri (and others) can be now.
Maybe it’s just because Siri is an expense, rather than a revenue item.
It is not unusual to find a focus on profits when the CEO is an accountant.
What’s your point? Apple has never focused on profits first, and Tim Cook is not an accountant!
I would like to know this study’s methodology, as everything else in the world of Google search is about advertising revenue, and therefore crap. And Siri is improving dramatically in iOS 11, something these results (somewhat suspicious timing!) clearly don’t reflect.
Cook graduated from Robertsdale High School. He earned a Bachelor of Science (B.S.) in industrial engineering from Auburn University in 1982, and his Master of Business Administration (MBA) from Duke University’s Fuqua School of Business in 1988.
botvinnik graduated from the Oklahoma School for the Underprivileged
1) Many of the problems of Siri are just general and have nothing to do with privacy although I do agree privacy has a part to play (see below) . For example understanding basic commands for device function, access to general information etc. is not due to privacy.
For the general info part Apple has moved Siri to Google from Bing. (Bing from my experience is a poorer search engine, but how this will play with privacy I don’t know… )
They have to work on the other issues like broaden developer involvement etc (see various articles on the web on this) , better integration among all Apple devices etc.
2) Privacy. (First of all to Flamers who keep one starring one for this, I’m not AGAINST privacy or Apple , I’m just explaining why it hobbles Apple vs Google etc. Some Siri engineers if the reports are true has quit over this. They say that Apple’s policy for example of flushing a lot of personal data quickly makes it hard for Siri. )
Privacy. Google etc knows a lot about a person from web searches, gmail (they have bots that scan your gmail, apparently even the link photos) , your map data, they store data from your previous use their voice assistant etc. gives them an edge for better, faster responses for CERTAIN , not all, things.
A person who knows you can react to you better. For example if I tell my spouse “Im going to get milk”, she knows WHERE I’m going from previous experience, what type of milk I’ll probably be buying, that I’ll be driving etc. She would seem ‘more intelligent’ than a total stranger to this errand. If I ask her ‘do you want me to get anything else’ she would know all these details to tailor a response.
When you say ‘Where to play ball’, an assistant that knows you probably knows you mean ‘BASE’ ball vs something else and locate al the BASE ball fields etc . it might recognize your voice, realize you are ‘East Indian’ (or something) and have proper translation tools , ability to understand accent, particular grammar patterns etc. (I’m not saying Google does all this, I’m just giving general examples).
Like i said not all Siri issues are due to this privacy thing but people can see it’s effect. I’m always astonished how quickly Google changes the ads that appear on webpages after I’ve done a Google search (like for example recently i did searches for a trip to the Far East and now I continuously get hotel etc ads in the specific regions and price ranges etc I searched for. Some of the ads are down like eerie. They even know the attractions for the cities that i might like, I’m an artist and also do a lot of art searches for example and they’ve linked the two together … )
2) I ALSO note that Siri has been taken from Eddy Cue and given to Federighi.
( btw Federighi was also the guy who spoke up about the Mac Pro issue, although the MP was a HARDWARE Jony Ive problem and not his software… . I’m pointing this out because at least a few Apple SVPs seem to have ‘checked out’ mentally and Apple product issues might be related to this , and this is a TIM COOK leadership problem… )
Keep beating that “privacy” horse, it’ll be dead soon. Apple = Derivative Excuses. Steve warned us what would happen when the product guys no longer run the company.
So rather than sitting here spouting your snark, why not fuck off and buy yourself some Androcrap and Winblows?
@Sean:
As a site where people care about Apple’s product quality, that includes complaining when we think it’s making mistakes. Snark is a fairly common way to express frustration.
What’s your issue? You like to pretend problems don’t exist? You don’t like comment sections of websites?
If that’s correct, then you’d probably be wise to follow the first portion of your own advice and avoid the comments here.
How long has Siri been around? About six long years. Apple didn’t seem to do very much with it. I hated the fact that Siri never made it into Mac OSX until recently. So what happened? Apple let Amazon come along with Alexa and eat their lunch. So foolish for a company with so much money. Now Amazon and Google get all the praise for their intelligent assistants and Siri became the world’s stupidest assistant. Apple is definitely playing to lose or they just don’t care.
Apple is so concerned about privacy while all the other tech companies are sucking user information like crazy and no one is complaining about it. All the FANG companies are harvesting user data from everyone and they’re the most prized companies on the planet. I honestly don’t understand why Apple is so concerned about privacy but I suppose it’s just some Apple policy decided upon to be different from other companies.
Don’t fool yourself that Apple isn’t sucking up all your info too. The only difference is that Apple claims to anonymize it, ie tracking you with MAC address rather than your name. But the data is definitely being collected by Apple and their iCloud providers. Guess who those companies are!
Something negative happened after Apple bought Siri. This was evident when Apple hobbled Siri’s capabilities and when Siri’s chief inventor, Dag Kittlaus, left the company to create Viv, “The Global Brain”. Siri has slowly improved in the years that followed, but obviously not enough to keep up with the competition.
[Side Note 1: Samsung bought Viv a year ago, but have yet to integrate it into any of their products.]
[Side Note 2: All current personal assistant software is ‘stupid’ compared to the goal of actual artificial intelligence. That includes everything from ELIZA through Deep Blue, Watson and Google Brain. Performing circus computers do not equate to intelligence.]
The first thing that struck me after the acquisition was no longer needing to train Siri to recognise my specific voice. Apparently the speech detection algorithmics got fundamentally altered to recognise a generic human’s speech patterns. I suspect Steve was behind that decision, demanding that Siri serve all members of the household, not just the Master/Mistress. But it degraded Siri’s understanding of my diction in an alarming way, and in years since she has suffered a long, slow climb back to the keen level of comprehension that I experienced when first we met.
And there were other modifications. Her wells of knowledge were sealed off as untrusted or license-heavy, and Siri seemed to become addled, like my maternal grandmother in her dotage.
All the while Apple engineers were commanded to respect personal security — privacy — above utility. It was a tall order, and one that none of their competitors really bothered with. Apple consequently did cede ground to them but did so strategically, in a play for the long term, which privacy advocates like me hope will end in landmark legislation mandating an end to tin-can telephones and bootleg snooping.
An IQ of 23.9? What an absolute disgrace. Apple has the nerve to call it a virtual assistant when it’s half as smart as a six year old child? Who would hire such a person to work for them? You’re better off with Sloth from the Goonies holding a pen and a legal pad. No wonder Siri is so useless.
“…whether Apple’s commitment to privacy will every really pay off”?
Dear MDN,
When did pay off for Apple start equating to sales? Maybe Tim Cook isn’t the only one who’s lost sight recently.
It’s not about sales—if that was the case there’d be no problem at all since the sales of products running Siri are through the roof.
This is about quality of product. Apple has the worst “virtual assistant” of any mobile operating software maker. How is that acceptable? These premium iOS devices are supposed to be the cream of the crop, with one (Apple Watch) very much dependent on voice assistance to navigate its UI, and yet in Siri they feature a bottom of the barrel competitor in the artificial intelligence race? It’s shameful.
At this point in time, we know that typical “I.Q.” tests are blunt instruments of questionable value. They’re based on military interests in the potential quality of soldiers within a limited range of purposes. They’re another part of what I consider to be the still primitive nature of human psychology. Good luck proving me wrong.
Then add on the silliness of “I.Q.” testing a machine. (Yes, software is a component part of any computerized machine).
Therefore, taking seriously any “I.Q.” testing of advanced expert systems (versus the still elusive and mythic “A.I.” system) is approaching nonsensical.
Nonetheless, it’s interesting to see the results of this bludgeon testing if only to kick Apple in the backside regarding their lackadaisical improvement of Siri. Google is taking personal assistant software seriously. Apple is allowing their personal assistant to languish and antiquate by comparison.
Apply the Apple Prod now.
Siri is worse because it’s the most used 🙂 Think about it, everyone either has a story about Siri messing up or know someone that has a story about Siri messing up. I don’t have a story about Alexa or Google because I don’t use those often enough to come across areas of difficulty.
In our current Age of Marketing, perception is everything. Therefore, your point is worthy of consideration. Thank you!
I can believe most used per person for Siri, but in terms of most requests made of, Google probably tops all comers. For Siri people have their idiosyncrasies that may affect result, but with Google you have many more people with their quirks to deal with.
I think it is more likely that it is because there are less ‘different’ people using Siri, that could be a reason for not learning as fast or encountering errors often enough that they are brought quickly to the attention of the developers for a fix.
an AI machine with the IQ of a 6 year old hooked up directly to google can do a lot more damage than a vegetable hot-wired to the net.
Apple has no choice if they want be relevant down the line..
They either have to compromise their mantra or find a way around it to amass huge amounts of data , develop the proper Analytics and Algorithms…. and try to CATCH UP BIG TIME!
The present difference in AI is not acceptable or even an option! Imo
It’s a path to oblivion….
I think a big part of the problem isn’t Siri by itself but hardware and software issues. A large proportion of my “Hey Siri” commands through my Apple Watch leave me hanging with “Hold on. I’ll tap you when I’m ready.” Commands that used to work nearly 100% of the time, like starting a workout or timer, are now a 50/50 coin toss. Doing it manually is less frustrating.
For many years, while driving, I would ask Siri to “Read me my stock list.” and she would read aloud my stock list from the stocks app.
About a year and a half ago she stopped responding by reading it aloud and instead only displayed the information. She stopped understanding the difference between “show me” and “read me”.
Complaining to Apple eventually brought me into contact with an Apple engineer, so I hoped the problem would soon be solved.
The solution now involves asking Siri two questions.
“Read me my stock list” (She displays it) followed by “Read it to me.” (She reads it aloud)
Not very encouraging.
My take is that Apple is not trying very hard- they are mailing it in on Siri, just like Maps. They both still suck and are inferior to the competition years later.
Yes, Siri is absolutely useless, but I will not use goog mess.
“…the average Joe/Jane does seem to regard Siri as not too bright, putting into question whether Apple’s commitment to privacy will every really pay off; i.e. translate to increased product sales.”
Apple should safeguard their customers’ privacy because it’s the right thing to do, not because they hope to make more money by doing so.