Apple has filed for a U.S. patent (number 20210042010) for a “3D document editing system” that would work with the widely-expected smart AR glasses (“Apple Glasses”) and a Smart Keyboard or iPad which would allow for manipulation of virtual apps including word processing, games, entertainment, and more.
Dennis Sellers for Apple World Today:
Conventional VR, AR, and MR systems may allow content consumers to view and interact with content in a 3D environment. They may provide tools and applications that allow VR content creators to create and edit 3D objects, and may provide a text generation and editing system with a conventional 2D GUI that allows content creators to generate text content that can be attached to 3D objects.
However, Apple says these conventional VR systems typically don’t provide text generation and editing systems with GUIs that allow content creators to generate and edit text with 3D effects in a VR 3D environment. The tech giant’s solution is Apple Glasses used in conjunction with a keyboard or iPad.
Here’s the summary of the patent filing: “A 3D document editing system and graphical user interface (GUI) that includes a virtual reality and/or augmented reality device and an input device (e.g., keyboard) that implements sensing technology for detecting gestures by a user. Using the system, portions of a document can be placed at or moved to various Z-depths in a 3D virtual space provided by the VR device to provide 3D effects in the document. The sensing technology may allow the user to make gestures while entering text via a keypad, thus allowing the user to specify 3D effects in the document while typing…”
MacDailyNews Take: With the lightweight and, presumably, stylish “Apple Glasses” expected to have the iPhone do the heavy-lifting (processing) during normal application, any device capable of paring with Apple’s AR smartglasses could conceivably handle those chores; certainly an iPad and even a “Smart Keyboard for Apple Glasses” which could contain the “brains” (Apple A16?) required by the smart glasses. One would assume, however, that the Apple Glasses user would have his iPhone nearby anyway, so the “Smart Keyboard” (with trackpad) could be rather “dumb” (and more affordable) as the glasses could simply continue to use the connected iPhone for processing.
This boils down to the AR glasses simulating a computer screen and using server-side processing plus iCloud to make the simulated computer screen work like it’s actually attached to your computer.
Imagine a Smart Keyboard that folds up and goes with you. Literally anywhere that has 5G access you could have your computer there, along with all your apps and data. Plus the “screen” could be as large as you like and the processing power could be pretty incredible, depending on lag.
Now realize this could be any computing device — so a gaming system, a “tv,” a VR rig, a Mac Pro.
Now imagine how much more this will be once developers get it in their hands.
It’s revolution-level stuff.
Seems like the end of the PC, and the smartest way into a larger AR/VR experience.
Yet another brilliant Apple go-to-market strategy.
10 years away…
If you’re thinking of a Mac Pro doing the heavy lifting then it could be as little as 2 – 3 years away. If you’re thinking of an iPhone doing the heavy lifting as a stand alone device
I’d just settle for an EFC Global in the meantime.
Ah, it reminds of of Apple’s Xspace GUI for the Internet. It was a 3D interface that you “flew through” to see pages in servers and websites. You could choose the depth level you could see at any given moment. It was much faster to use than the myriad links per web page we still have to day. I thought it was brilliant. It let you very quickly get to specific pages in a site to which you’d never been before.
The downside was that it required web developers to specifically code their sites to support it. This meant that extremely few sites supported it. (I only found a few dozen when I was trying to use it.) Thus it died after a couple year. It’s not easy to find references to it on the ‘net these days.
THIS is how “holographic” displays from sci-fi movies and TV shows will work in the real (near-future) world. I can’t conceive of a way to create an actual sharp well-defined hologram screen “out of thin air.” But this trick with AR glasses will look and feel (to the user) like a sci-fi holo-display.
THEN the next step is to link multiple users (wearing AR glasses) into the same illusion; they’re all standing in an augmented “holo-room,” seeing and interacting with the same display (and other things) in real time. MUCH later, with enough network bandwidth and distributed processing power, expand that “room” to cover an entire community, then the world. By then, the AR method will go far beyond wearing glasses.