Apple’s upcoming Vision Pro headset will feature a new augmented reality (AR) version of the Visual Lookup feature found in iPhones and iPads. This feature, dubbed “Visual Search,” will allow wearers to get information about various items by simply looking at them.
The feature was discovered this week by MacRumors contributor Steve Moser, who inspected code within the Xcode beta of visionOS, Apple’s AR and reality operating system that will power the Vision Pro headset. Moser found references to Visual Search in the code, which suggests that the feature is still under development.
Visual Search is expected to work by using the Vision Pro headset’s cameras to scan the environment and identify objects. Once an object is identified, the headset will then display information about the object, such as its name, description, and price.
Visual Search could be a powerful tool for users who want to learn more about the world around them. For example, users could use Visual Search to identify plants and animals, learn about historical landmarks, or even find product information while shopping.
I think the new Visual Search tool will be a killer Vision Pro feature. Not only that, but I expect Visual Search to grow into a massive threat to Google Search. It may actually be an even bigger threat to Google than ChatGPT…
[T]he Vision Pro will be extremely important to Apple’s post-iPhone era. It’ll set the principles of spatial computing, which Apple will use to build towards sophisticated AR glasses. And Visual Search will sit at the center of it.
As I’ve explained, Vision Pro will be much faster than other computers. It tracks your eyes and hands to deliver instant results. Add voice control, and the Vision Pro might help you become even more productive.
Visual Search will let you search the world around you through the Vision Pro’s cameras. You’ll be able to point at things and ask for information about them. And grab information from the real world, like texts and web addresses. All of that will deliver Google Search-like results.
MacDailyNews Take: Google should be very afraid of Apple Vision Pro’s ‘Visual Search’ capabilities IF Apple finally stops taking Google’s billions to be the Vision Pro’s default search engine and instead inks a deal with Bing, DuckDuckGo, or, much better yet, finally debuts their own in-house search engine.
As we wrote over a year ago last June:
First of all, in order to launch a standalone search engine, Apple would likely be forgoing tens of billions of dollars per year, hoping to replace that with advertising revenue (Google currently pays Apple an estimated $18 billion – $20 billion annually for their search engine to be the default in Apple’s operating systems).
Apple does have an installed base of some 1.8 billion devices (and growing) and far superior user demographics than do competing, derivative OSes (which is why Google pays Apple so much to have access to people who have money and the proven will to spend it), making advertising via “Apple Search” much more valuable, so the math might work out.
In addition, governments and other entities are gunning for Apple and Alphabet, of which Google is a subsidiary, and antitrust authorities are questioning the two companies’ search deal, so Apple could be working on search as a contingency or even preparing to sever the deal and launch a first party search engine in order to preempt any antitrust actions.
There is also the privacy aspect: Apple’s reliance of Google search is a privacy weak point. “Apple Search” could tout and deliver privacy in search to differentiate itself against Google. Open sourcing the Apple search engine’s algorithms would also increase trust.
Before Apple Maps, many said it would be impossible to take on Google. Even with Apple Maps disastrous should’ve-been-tagged-beta launch, that canard has long since been disproven. Apple, the world’s most valuable company, could compete with Google in search as well.
Removing Google’s default access to Mac, iPad, and iPhone users would seriously impact Google’s advertising rates.
The one thing online that needs competition the most is Web search engines. Google’s monopoly in search hampers and affects everything online from publishing to privacy to politics and beyond. A single gatekeeper for finding things online is a prime example of why antitrust law exists (Microsoft’s also-ran Bing with 3.08% of worldwide search, and tiny niche engines like Yahoo, YANDEX, DuckDuckGo, etc. with ~1% or less are not competitive).
As we wrote of the “Apple Search” idea long ago:
If you really want to wage thermonuclear war, wage thermonuclear war. – MacDailyNews, May 30, 2014
See also:
• Google pays Apple $20 billion annually to be Safari’s default search engine – February 21, 2023
• Is Apple preparing to launch its own search engine? – June 2, 2022
Please help support MacDailyNews. Click or tap here to support our independent tech blog. Thank you!
Support MacDailyNews at no extra cost to you by using this link to shop at Amazon.
Apple can’t be in a position where they make money on your info, otherwise called advertising.