Apple is enabling third-party motion controllers and tracked accessories in visionOS 27

With the introduction of VisionOS 27, Apple opens the door for third parties to develop their own high-performance controllers and accessories for Vision Pro.

news

VisionOS 27, available to developers starting today, adds new support for actively tracked accessories like motion controllers, which Apple calls ‘spatial accessories’. Previous updates to visionOS passively tracked objects by their appearance, but not actively interacting with the headset. On the other hand, visionOS 27 allows Vision Pro to track parts based on known IR LED patterns and position data streamed from an accessory’s on-board IMU via Bluetooth. As described in a newly released developer session from Apple, spatial accessories can also send input data from buttons and thumbsticks.

A combination of IR LEDs and an IMU is the defacto standard for tracking VR motion controllers. This approach enables higher accuracy and lower latency than trying to passively track an object based on its geometric shape alone. IR LEDs are more robust to varying lighting conditions, while IMUs can provide hundreds of updates per second, increasing accuracy by providing the latest updates on the controller’s movement. Apple says the spatial components can track the headset’s display rate (nominally 90Hz, but up to 120Hz).

While this news doesn’t mean that any old VR controller will be compatible with Vision Pro, it does mean that companies that make controllers (or other tracked accessories) will be able to add official Vision Pro support to their current or upcoming products. Apple says that a spatial accessory tracking profile can be added to the visionOS app, which “records across your accessory system, so any app on Apple Vision Pro can use it.” However, third-party tracked accessories created this way cannot support the full VisionOS interface, such as PSVR 2 controllers and the Logitech stylus.

Apple is working with DFRobot and MIKROE to develop off-the-shelf trackers that can be used as “reference hardware and development tools” to create Vision Pro-compatible tracked parts.

While other details like pricing weren’t shared at the session, the trackers will be available “later this year.”

VisionOS 27 adds support for actively tracked spatial accessories and improves support for passively tracked objects. Apple says a new high-frame-rate tracking mode and extended training for tracking models bring lower latency and more reliable object tracking to the headset.

My tag

Apple has completed the motion controllers for the Vision Pro. When headsets first launched, the user’s hands (and technically their eyes too) were the only input device that was tracked. That flew in the face of nearly every major AR and VR headset launched before the Vision Pro, and ensured that even the most well-considered VR games of the time had no chance of running on the headset. This is definitely not an oversight on Apple’s part, but rather a deliberate choice.

It’s true that the Vision Pro’s rejection of motion controllers led Apple to the Vision Pro’s ‘look + pinch’ input system, which I personally consider one of the headset’s most powerful innovations. But while that system is great for quickly and accurately controlling flat content and interfaces, it’s no substitute for the reliable, high-performance motion controller tracking that makes VR’s best games possible.

As a company that often stands by tough design choices, I wouldn’t rule it out of the realm of possibility that Apple would stick to its guns and hope that VR game developers figure out how to work within the limitations of the Vision Pro’s look + pinch input system. Thankfully, it looks like the company heard the feedback loud and clear.

Apple laid the groundwork for motion controller support last year when it made the PSVR 2 controllers, and tracked accessories for the Vision Pro officially supported the Logitech stylus. But until now, those are the only actively tracked accessories that can work with the headset.

While it’s good news that Apple is opening the door for third-party motion controllers to be actively tracked in the Vision Pro, it remains to be seen how important this will be to the Vision Pro in its current form. Even with official PSVR 2 controller support, well-known VR games aren’t exactly coming to the PC. But given the headset’s hefty price tag, it’s unlikely Apple thought so.

The company responds to the headset’s key use cases in the enterprise and near-term demand derived from other non-consumer systems such as training and simulation. It’s true that Apple usually adds features and developer capabilities with the intention of supporting them in the long run. So while third-party motion controller support might not be a huge boon for the current Vision Pro, it could have a more meaningful impact on future VisionOS headsets, especially if the headset is available at launch two and a half years after it first hit the market.