You can tell a lot about someone from their eyes. They can indicate how tired you are, the type of mood you're in, and potentially provide clues about health problems. But your eyes could also leak more secretive information: your passwords, PINs, and messages you type.
Today, a group of six computer scientists are revealing a new attack against Apple's Vision Pro mixed reality headset where exposed eye-tracking data allowed them to decipher what people entered on the device's virtual keyboard. The attack, dubbed GAZEploit and shared exclusively with WIRED, allowed the researchers to successfully reconstruct passwords, PINs, and messages people typed with their eyes.
"Based on the direction of the eye movement, the hacker can determine which key the victim is now typing," says Hanqiu Wang, one of the leading researchers involved in the work. They identified the correct letters people typed in passwords 77 percent of the time within five guesses and 92 percent of the time in messages.
To be clear, the researchers did not gain access to Apple's headset to see what they were viewing. Instead, they worked out what people were typing by remotely analyzing the eye movements of a virtual avatar created by the Vision Pro. This avatar can be used in Zoom calls, Teams, Slack, Reddit, Tinder, Twitter, Skype, and FaceTime.
The researchers alerted Apple to the vulnerability in April, and the company issued a patch to stop the potential for data to leak at the end of July. It is the first attack to exploit people's "gaze" data in this way, the researchers say. The findings underline how people's biometric data -- information and measurements about your body -- can expose sensitive information and be used as part of the burgeoning surveillance industry.
Your eyes are your mouse when using the Vision Pro. When typing, you look at a virtual keyboard that hovers around, and can be moved and resized. When you're looking at the right letter, tapping two fingers together works as a click.
What you do stays within the headset, but if you want to jump on a quick Zoom, FaceTime some friends, or livestream, you'll likely end up using a Persona -- the sort of ghostly 3D avatar the Vision Pro creates by scanning your face.
"These technologies ... can inadvertently expose critical facial biometrics, including eye-tracking data, through video calls where the user's virtual avatar mirrors their eye movements," the researchers write in a preprint paper detailing their findings. Wang says the work relies on two biometrics that can be extracted from recordings of a Persona: the eye aspect ratio (EAR) and eye gaze estimation. (As well as Wang, the research was completed by Siqi Dai, Max Panoff, and Shuo Wang from the University of Florida, Haoqi Shan from blockchain security company CertiK, and Zihao Zhan from Texas Tech University.)