foreign Pro I had about a 30 minute demo just using this new headset I had about a trillion questions so I tried to ask as many of them as I possibly could of Apple to get all the answers I could for you and for myself so without any further Ado let’s just jump right in this is everything you need to know so far about Apple’s new AR VR headset.

what is it

the first question naturally what is it so first things first this is a first generation device from Apple which we don’t see very often but it is an augmented reality slash virtual reality headset with a bunch of cameras and sensors inside that you strap to your face it’ll start at three thousand five hundred dollars and it’ll go on sale

starting sometime early next year nobody’s been able to shoot video of using it yet so I’m gonna have to do my best with descriptions here but fundamentally this isn’t something that’s paired to your iPhone or your Mac by default it’s a standalone computer with an M2 Chip inside its own Wi-Fi connection a roughly 4K micro OLED display for each eye a set of speakers right above each ear on the band and then a dedicated new R1 chip which is doing all of the real-time processing from all the sensors that make this headset pretty special but we’ll get to that in a second.

How dose it work

so all right then how does it work well basically we’ve seen VR headsets and AR headsets before it’s uh typically there’s screens on the inside and then on the outside there’s some sort of shell which has some sensors on the outside that help detect what is in your room around you and help to see the controllers in your hands but with this one there are no controllers at all they’re not making any for it there aren’t going to be any optional accessory they’re not even going to design any controllers you are controlling this thing entirely with your hands and your eyes and your voice so you put this headset on and basically

all over the headset there are an absolute ton of sensors you have two main forward-facing cameras two down downward facing cameras two more cameras for the sides two infrared illuminators a lidar scanner a true depth camera these are all to keep track of what’s  happening around the headset and also to track your hands in front of your face and then on the inside there’s also a bunch of infrared illuminators and infrared cameras that are purely there just there to track your eyes so there’s actually a setup process before you even get the headset you basically measure

you sort of scan your face with the true depth camera on the iPhone and it uses that depth information about your face to give you sort of a custom cut out this will be a process that you can do in Apple Stores but basically you get the headset with the custom cut out for your face if you have glasses there are some Zeiss lenses but you finally get the headset out and you put it on and there’s some nice meshing on the back some fabric that stretches and is pretty breathable it’s built extremely well with all this metal and glass and then once you get it going the most impressive thing about this headset the most impressive thing is the eye tracking.

I’m not even kidding this this eye tracking is sick so basically the eye tracking in this headset as it looks at your eyes and keeps track of where your eyes move around is the closest thing that I’ve experienced to like magic like I don’t I normally don’t call Tech things sort of magical or surreal like this but this was even for a pre-release product

kind of unbelievable how well it does anytime you move your eyes around the UI it would immediately highlight and select exactly what you’re looking at no matter how small the target was or what you’re looking at and then when you want to select something all you do is take your fingers like this and just touch them together that’s how you click so you look at something click look at something click so there’s actually there’s a calibration process.

when you first set up the headset you put it on for the first time where you basically just look around with your eyes at a bunch of different dots so it learns your eyes and then yeah from that point on you’re just cruising you’re sailing through everything by just looking at things and it feels like telepathic you just look at something and select it and that’s it so this is how the whole UI works pretty much every other headset has the ability to use your hands to select things but most of them rely on the sensors on the outside to keep track of a set of controllers for maximum accuracy but this massive array of sensors on the outside of Apple’s headset have done an impressively good job of picking up what my hand is doing kind of no matter where it is so like I said there’s sensors all the way around the thing they’re facing forward but they’re facing sideways and facing down so I can just kind of rest my hand like wherever and look at stuff and click I don’t even have to move my head I can just look at something and just click my fingers I kind of did it by accident a few times like I just sort of be sitting there and my fingertips would actually touch and I’d select whatever I was looking at but pretty quickly you get used to just

just this just just doing this now there are also some text boxes occasionally you want to type something and so you can look at the text box select it and then a keyboard will pop up and you can look at the letters but also you can just look at a search box field and just start talking and it’ll start filling in whatever you’re saying because there’s microphones on the headset and there’s also something they’re calling optic ID which is basically kind of just the fingerprint ID of the headset because it’s looking at your eyes and everyone’s eyes is a different unique biometric pattern and so you log in just by putting a headset on and it looks at your eyes so optic ID this thing knows your eyes really well so you’ve probably seen all these weird looking frames where you can see people’s eyes through it like this this super weird thing that I’ve never seen in any other headset so what is happening here well spoiler alert it’s not actually transparent it kind of looks like it’s a glass transparent thing but it’s not it appears to be because there’s actually an OLED screen facing out this is what I would call a 2 way pass-through that’s what I’m calling it see a normal VR headset we’ve seen is of course completely opaque usually it’s like black plastic or something like that so in order to see through to your actual environment without taking the headset off like for picking up your controllers or walking around a room without hitting a wall they have a pass-through mode so it’ll turn on a camera on the outside so you can see what’s going on around you and some pass-through modes are better than others

some will do it from a single black and white camera some will do stereo pass through from two cameras some will do color pass through the pass-through on Apple’s headset is again the best I’ve ever seen in a headset it’s stereo it’s color it’s sharp it’s real time it’s really good so then the eyes you see from the outside are actually an OLED display showing what your eyes look like to the cameras inside the headset so it shows your eyes like this whenever you’re using any sort of pass-through mode and if you can see your environment it’ll show your eyes so you’re not ,actually like looking at someone’s eyes

but it kind of feels like you are ,kind of creepy a little bit I don’t know ,it’s the simplest way of explaining it ,if they’re doing something fully immersive in VR like watching a movie or ,playing a game then it won’t show their eyes it’ll show this graphic over their eyes instead it’s a really interesting look it’s a it’s a bold look I just have this like thought of like you’re uh you’re on a airplane and the flight attendant walks up and Taps you hey do you want any water and you just look up and your eyes shine through the headset it’s just a weird thing I don’t think it’s smart but I don’t think other headsets are gonna do it because most of ,them don’t have it in the budget to have a screen facing the outside.