This story is part of, CNET’s complete coverage from and about Apple’s annual developers conference.
Apple didn’t announce a virtual or augmented reality headset at its latest WWDC developer conference, but some new software tools were still announced that push AR and social connection further.
Why it matters
Apple’s expected headset will likely work with Macs, iPads, and iPhones, building off ideas already being developed on the company’s software. Meanwhile, many other companies including Google and Qualcomm are advancing their AR ambitions.
A VR headset may not come until 2023, although it’s possible one could be announced sooner.
Anyone following Apple’sfor news on its long-expected was bound to be disappointed. The event had news about , , , , the smart home and even , but AR was barely mentioned.
After reports of delays, Apple may not end up releasing this headset. That could mean another year of waiting… or it could mean an event later this year previewing what’s to come. Right now we’re left with a lot of unknowns, but that doesn’t mean there weren’t clues and new software that seem like they’ll be very useful for a headset to come.
Apple already has a well-developed set offor iPhones and iPads, and depth-sensing that can do the sorts of real world “meshing” that VR and AR headsets need to overlay virtual objects convincingly. There is a set of tools that text and objects from camera feeds, much like Google Lens. The Apple Watch even has some accessibility-focused .
What WWDC 2022 showed off were a handful of refinements that, the more I think about them, seem aimed at laying down additional groundwork before a headset arrives.
FaceTime and Messages: tools for Apple’s metaverse?
As companies have pivoted to metaverse messaging over the last year, it’s generally been code for rethinking massively social cross-platform interactions. “Social” is a weird thing for Apple, which isn’t a social network-focused company like Meta or Snap or Niantic.
Apple does have FaceTime and Messages, which form a proprietary network across Apple’s devices that could be the glue to connect with people on headsets. Apple’sframework, which was introduced in 2021’s iOS 15, is trying to collaboration and watching/playing content with others feel more instant. SharePlay’s reach has expanded in iOS 16. A lot of it looks like the sort of adhesive that Apple’s metaverse ambitions need.
Apple already has Memoji avatars, but Apple has also been increasingly adding sharing tools that link apps and collaborative content through Messages and FaceTime. Those added features in iOS 16, iPadOS 16 and MacOS Ventura could make sharing things easier, but on a headset they could be essential shortcuts for connecting quickly with others. Meta’s VR headsets lean on friends and party-based connections via Messenger for social; Apple could be heading down the same path. SharePlay coming to Game Center, Apple’s overlooked social gaming hub, seems like a similarly useful tool for future cross-play experiences.
Could RoomPlan be a stepping-stone to in-room mixed reality?
Apple announced one stealthy new tool in its upcoming ARKit 6: a room-scanning technology called RoomPlan. At first glance, the feature looks like Apple’s own version of room-scanning tech using lidar, similar to what other companies . The tool recognizes furniture, walls, windows and other room elements, and also quickly creates a 3D scan of a space for things like home improvement or construction.
Or, maybe, it could enable new forms of mixed reality. “Don’t forget about using people’s surroundings as a canvas for your experiences,” says the WWDC developer video detailing RoomPlan, adding you can “even bring people’s spaces into the game you’re building.” While a depth map with lidar could overlay AR objects already, what this new tech could enable is bringing a room into VR and overlaying virtual things into that layout. I saw this idea a while ago in VR headsets that used cameras to scan environments and, creating a sort of mixed reality feeling, and it could well be the sort of mixed reality that Apple’s expected camera-studded VR headset starts enabling.
4K video for AR sounds like a path to headsets
Another new ARKit 6 feature seems notable: AR effects will now work with 4K video input. That’s a weird feature at first glance for phones, which have screens too small to appreciate 4K AR. For capturing video with AR effects overlaid, it could be useful. But increasing the video quality for AR would be extremely helpful in VR headsets that use onboard cameras to combine VR with video feed of the outside world, a technique called passthrough mixed reality.
VR headsets that use passthrough mixed reality, like the hardware Apple is expected to eventually release, rely on a high-quality video feed to make overlaid virtual objects look realistic. On the, is grainy and in black and white. On the professional-grade Varjo XR-3, where I’ve to try a more advanced passthrough mixed reality, it’s in color and much higher-res.
The new ARKit features also look to be faster at recognizing room details for quick AR overlays and motion-capture tracking. Both would also be helpful and necessary in a headset with AR features, too.
Apple’s still expanding cities where location-specific AR could work
A number of companies have recently expanded their mapping initiatives to work with AR so that future glasses could recognize “permanent” AR objects in real-world places. Google is expanding AR to work across many of its, and Niantic’s building a of playable areas for AR games. Snap has been with lidar. Apple has as well, but only a handful. More are being added this year, but that means there are only certain places where location-specific AR will work reliably with Apple’s AR toolkits. Apple is not expected to have any everyday-wearable AR glasses for a long while, and that makes sense: Besides concerns about battery life, cost or safety, AR glasses will need a worldwide mapping framework that is only half-built right now.
Still a lot we don’t know
Despite many reports of a headset being imminent (or arriving in 2023), we still know nothing definitive about what Apple has planned. These little bits and pieces of speculation from Apple’s new software tools are hardly proof of anything… but they show many of the necessary pieces are assembling in plain sight.
Source from www.cnet.com