Did Apple just invent the invisibility cloak with Vision Pro?

Is the Apple Vision Pro a proof of concept for the world’s first invisibility device, asks Satyen K. Bordoloi as he looks into the studies searching for the elixir that can hide you in plain sight.

In what is perhaps the first story of invisibility: Plato tells of an innocent shepherd who finds a ring that makes him invisible. He quickly goes and seduces the queen, kills the king, and establishes his own dynasty. Plato’s tale is allegorical, meant to highlight one simple precept: power corrupts.

H G Wells, inspired by it, wrote The Invisible Man 126 years ago. Since then, the world has not stopped fantasizing about invisibility. Attempts have been made using magical portions and later science. Now, for the first time, modern technology seems to have cracked it and Apple Vision Pro – even though it wasn’t intended as such – might be proof of concept for an invisibility device.

But first, let us look at what an invisibility cloak is.

INVISIBILITY SO FAR

Invisibility – at a basic level – is of two types: psychological and physical. All of us have experienced psychological invisibility, where we feel unnoticed, that our opinions and ideas don’t matter. It is a state of powerlessness. The other – literal invisibility – as in Plato’s allegory, gives us the power of the gods. What would you do if you could get away from the accountability of anything you do? Plato believed the average human would shake off all morality to turn evil.

Human life has been an attempt to stave off psychological invisibility and figure out the literal one. While the former needs counselling and comes under the domain called perceptual psychology, the latter has been a domain of physics that understands that invisibility is an optical illusion.

You see an object because it reflects light that hits your eyes. Everything becomes invisible in the dark because there is no light to reflect. Real invisibility is when you don’t see an object even in the light. This can happen if the object neither absorbs or reflects light i.e., like glass.

What we mean by wanting to make something invisible, is to use optical and visual factors to make a viewer or sensor not ‘see’ or ‘sense’ that thing – like we were camouflaging it. E.g., as in the image below, by using two parabolic cylindrical mirrors and one plane mirror, the image of the background is directed around an object, making the object itself invisible if viewed from two sides.

INVISIBILITY SO FAR

In 2006, a team of researchers at Duke University, US, built the world’s first device to make an object invisible to microwaves. It might not seem a big deal, but this demonstrates that in principle, objects could be made to disappear from detection devices like radars and cameras using engineered metamaterials. This was based on a theory developed by John Pendry in the 1990s where he postulated that like water bends and flows over rocks – light rays bend around an object placed in the centre of a metamaterial shell, and recombine on the other side to give the illusion of invisibility.

Japanese roboticist and computer engineer Susumu Tachi of Keio University, meanwhile, devised a simple disappearing trick on the streets of Japan: he used a camera to record video that is projected live on the body of the person in real-time. Thus, to the person looking at it from the right angles, the person appears near invisible since the image of one side is projected on the other.

Illusionists have used their own tricks to make large objects ‘invisible’ to the eye. Scientists, however, have been studying real methods to do so. Italian computer scientists Franco Zambonelli and Marco Mamei, in this fascinating 2002 paper postulate how digital technology could be used to make a cloak of invisibility. In a nutshell, their hypothesis is that given half a million Euros it is “possible to create a cloak of invisibility – a flexible artefact that can make anything inside it invisible and preserve invisibility despite mobility and deformation.” The paper explores “the algorithmic and technological challenges involved.”

My postulation is that the Apple Vision Pro is the world’s first proof-of-concept invisibility device in line with what Franco and Marco postulated, and Susumu Tachi demonstrated. It gets over the engineering, technological, computational and technical challenges to prove the two scientists right.

WHAT IS THE APPLE VISION PRO

The first truly personal computer, the first real smartphone, tech and gadgets to render animation, and personal devices like the iPod and Watch – Apple has a galaxy of stars that spearheaded our digital age. Yet, none of them has been as complex, as brilliant, as the Apple Vision Pro. And this I say as someone who thinks VR i.e. Virtual Reality and AR i.e. Augmented Reality, will never be as big as Meta and others think.

In Apple’s own words, Vision Pro is a spatial computer. It is the crown jewel of what I call ADU – Apple Digital Universe. But having studied its specs well, I am confident its many uses have yet to be figured out and will go beyond just a VR or AR set or Spatial computer. Firstly, it’ll herald the age where AI truly becomes our personal assistants. But that’s a long discussion for another time. Here we talk about Vision Pro as a proof of concept for an invisibility cloak.

To understand why I make this astounding claim, we have to understand what makes Apple Vision Pro a technical marvel. The Vision Pro has 23 sensors. It has four inward infrared cameras, inward LED illuminators to track eye movement, two main cameras, four downward cameras, six microphones, two external IR illuminators, two side cameras, a LiDAR Scanner, and a TrueDepth camera to track hands and environment.

The complex array of information gathered by these sensors is processed by two chips: a new R1 chip and the M2 chip found in Apple laptops. It has other cool features, like cramming 11.5 million pixels into the size of a postage stamp screen, making it have more pixels than a 4K TV for each eye. Yet one of its marvels is how it solved the isolation and separation of a wearer of VR glasses from the outside world.

Since Apple has arranged all its sensors in the perimeter of the device – both on the inside and outside – it leaves space in the centre for an outward-facing display. Thus if the wearer is looking at something in the real world, this display screen will show a lie feed of their eye position, movement and blinks that look eerily like their real eyes and movements. But it isn’t, for it’s just a recording of the person’s eyes shown on the display, just as what the wearer is seeing is not the actual outside world but its live feed on both the screens of the eyes.

Thus, though the Apple Vision Pro is an opaque device you cannot see through, the illusion it creates for both the wearer and someone looking at it is of a transparent device: the wearer can see the outside world, just as someone looking at the person can see their eyes.

APPLE VISION PRO AND INVISIBILITY CLOAK

Calling it an invisibility cloak 1.0 might seem a stretch, but if you really consider what Apple has managed to do, you’ll change your mind. Consider the curved lenticular display that projects the video of the eyes in real-time. It’s a first-of-its-kind display which shows the image from the viewing angle of the observer. Thus, different people looking at it from different angles will see the correct perspective of the eye.

Now imagine making an entire suit made of everything that the Apple Vision Pro is made of: hundreds of sensors and cameras are placed all over the body of the suit, and all around it are lenticular displays which curve with the contours of the human body. They are programmed to project the live feed of the camera on the other side of the suit. Because this projection is lenticular, it means anyone watching from any angle, gets the right view making the person or thing inside appear transparent – like glass, or the monster from the movie Predator.

Apple can thus make a sophisticated version of what Japanese roboticist Susumu Tachi was doing on the streets of Tokyo.

How much would such a suit cost? If the portion of your body that the Apple Vision Pro covers is about 2%. To scale it up to the whole body, you need to multiply it by a factor of at least 50. This means it will take hundreds of sensors and cameras, scores of lenticular displays like scales in the Ironman suit all over the body and cost $3500 multiplied by 50 i.e. 175,000 i.e. nearly a third of what Franco and Marco proposed. Considering how digital technology has developed in the 21 years since their calculations and costs gone down correspondingly, you realise the two scientists weren’t much off the mark.

Of course, it’s not as easy as that. First of all, the complexity and amount of processing with inputs from hundreds of cameras and sensors synced and projected in perfection to scores of displays is a mind-numbingly complex technological challenge. Secondly, the amount of energy such a suit would require to power up would make for a battery bigger than the suit itself, yet not last long.

Thus, though technically it is possible to make just such a suit, we are still a decade or two behind such a feat. Perhaps the advances in quantum computing and new battery technologies will make it possible one day.

Even if we can, and do make the cloak of invisibility, how would we use it? As expected, the first buyers and funders would be the military. Would we allow the already powerful to get more so? Or will we learn from Plato’s 2400-year-old parable and find more creative uses for it? Either way, since the technological blocks for it have been removed, it seems inevitable – as the Apple Vision Pro shows – that we’ll find out sooner, rather than latter.

In case you missed:

• AIoT Explained: The Intersection of AI and the Internet of Things
• Personal AI’s Future Is Here: Not What You Expected; Close to What You Need
• Apple Intelligence – Steve Jobs’ Company Finally Bites the AI Apple
• Kodak Moment: How Apple, Amazon, Meta, Microsoft Missed the AI Boat, Playing Catch-Up
• The Path to AGI is Through AMIs Connected by APIs
• 2023 Developments That’ll Alter Your Future Forever
• The Rise of Personal AI Assistants: Jarvis to ‘Agent Smith’
• How Old Are We: Shocking New Finding Upends History of Our Species
• Susan Wojcicki: The Screaming Legacy of The Quiet Architect of the Digital Age
• ‘Now and Then’: Time Travelling with The Beatles’ Last AI Song