The conviction of those promoting the military metaverse has only grown as of late. However, attempts to integrate the virtual and actual worlds have run into difficulties as well. Now, even the US military is building its own metaverse.
On May 10, two fighter pilots conducted a proto-metaverse experiment at high altitude. They wore special AR headsets to link to a system that projected a ghostly, shimmering picture of a refueling aircraft traveling alongside them in the sky a few thousand feet above the California desert in a pair of Berkut 540 planes. Then, one of the pilots conducted a refueling procedure with the virtual tanker while the other watched. Hello and welcome to the nascent military metaverse.
These days, metaverse craze is gripping more than just Silicon Valley. As digital companies and corporations scramble to build virtual world initiatives, numerous defense startups, contractors, and financiers are increasingly touting the metaverse, even though its definition and utility are not always obvious.
The fundamental technologies required for the metaverse—augmented and virtual reality, headmounted displays, 3D simulations, and artificial intelligence-powered virtual environments—are currently in use in the military industry. The end result is less refined, charming, and expansive than Mark Zuckerberg’s virtual world concept, but it is part of the goal. And, even if the fundamental technology stumbles in the civilian sphere, there is a significant chance it will take off.
For example, a combination of augmented reality, artificial intelligence, and video game visuals has allowed fighter pilots to exercise dogfighting against virtual opponents, including Chinese and Russian jets, while pulling several Gs. According to Red 6, the firm developing the technology, this provides a significantly more comprehensive assessment of a pilot’s ability than a traditional flight simulator. “We can fly against whatever threat we want,” says Red 6 founder and CEO Daniel Robinson. “And that threat could be controlled either by an individual remotely or by artificial intelligence.”
Subscribe to GreatGameIndia
Red6’s AR technology must function in harsher environments, with reduced latency and greater dependability than consumer AR or VR headsets. Robinson goes on to say that the company is currently developing a platform that will permit many different scenarios to be depicted in augmented or virtual reality. “What we’re building is really a military metaverse,” he says. “It’s like a multiplayer video game in the sky.”
Some of the most recent military systems include metaverse-related concepts. The new F-35 fighter jet’s high-tech helmet, for example, incorporates an augmented reality screen that overlays telemetry data and target information on top of video footage from around the aircraft. The US Army stated in 2018 that it would pay Microsoft up to $22 billion to construct a version of its HoloLens augmented reality system called the Integrated Visual Augmentation System for warfighters (IVAS).
In recent years, virtual reality and augmented reality have become standard components of military training. Project BlueShark, a system that enabled sailors to control vessels and collaborate in a virtual environment, was developed in 2014 by the Office of Naval Research and the Institute for Creative Technologies at the University of Southern California. Another project, Project Avenger, is now being utilized to train US Navy pilots. The United States Air Force is utilizing virtual reality to train pilots how to control aircraft and missions. VR is also utilized to treat veterans suffering from chronic pain and post-traumatic stress disorder. In addition, Boeing has designed an augmented reality environment in which mechanics can practice working on planes before boarding a real one.
The US military has recently started to investigate more complicated virtual worlds. There is also an increasing interest in integrating and mixing virtual worlds in a manner similar to metaverse thinking. The US Air Force hosted a high-level conference in December 2021, comprising approximately 250 participants from the US to Japan via a virtual environment. “The promise is integrating these technologies,” says Caitlin Dohrman, general manager of Improbable’s defense division, which has generated sprawling virtual battlefields with over 10,000 independently controlled characters for the UK’s military wargames and also works with the US Department of Defense (DOD). “It is an extremely complex type of simulation, especially given the fidelity that the military demands,” Dohrman says. “You can either have live players who are participating in the simulation or [characters] can be AI-enabled, which is often what the military does.”
Palmer Luckey, the creator of Oculus, a virtual reality company bought by Facebook in 2014, says Zuckerberg’s choice to go all-in on VR and the metaverse raised a lot of expectations in the commercial sector. “Everyone on their quarterly corporate calls, like a week or two later, they’re being asked by investors, ‘What’s your metaverse play?’,” he says.
Luckey cofounded the defense firm Anduril in 2017. Despite the recent metaverse excitement, he believes there is significant defense potential, mainly due to the importance and cost of military training. He does, however, believe that the technology does not need to be hyper-realistic to be beneficial, and that Anduril should only employ it when absolutely necessary. “Everything we’re doing with VR is something where it is uniquely better than any other option,” he explains. This may involve employing virtual reality to teach people to operate Anduril’s drones or displaying information about a region based on data from sensors on the ground, according to him.
Newer military systems, like Zuckerberg’s projected metaverse, depend significantly on AI to be effective. Red6’s AR technology was utilized in October 2020 to pit a real fighter pilot against an AI-controlled aircraft produced as part of a Defense Advanced Research Projects Agency (DARPA) AI dogfighting study. Another business, EpiSci, developed the AI top gun, which learnt how to outmaneuver and outgun an adversary through trial and error. The AI pilot finally achieved superhuman abilities and was able to consistently outperform its human opponent.
Another DARPA project, Perceptually-enabled Task Guidance, attempts to develop an AI assistant that monitors what a soldier is doing and provides guidance via speech, sound, or visuals. Unlike Boeing’s augmented reality system, which only functions in a restricted environment, such a system would have to understand the real world. The actual benefit of the technologies being examined by the military, according to Bruce Draper, the DARPA program manager in charge, lies in the melding of the real and the virtual. “The metaverse is mostly virtual, and virtual worlds are useful for training, but we live in the physical world,” he says. “The military domain is inherently physical, it’s not about an abstract metaverse.”
However, attempts to integrate the virtual and actual worlds have run into difficulties. A leaked Microsoft letter from March 2022 allegedly revealed that staff working on IVAS, the US Army’s equivalent of the HoloLens AR headgear, expected it to be poorly welcomed by consumers. And, according to a DOD audit revealed in April 2022, the US Army may be wasting money as a result. Jason Kuruvilla, a Microsoft senior communications manager, published multiple statements from high-ranking army officers extolling the IVAS’s capabilities. He also cited a DOD report from 2021 that emphasizes the significance of creating IVAS quickly so that issues can be dealt out along the road.
The conviction of those promoting the military metaverse has only grown as a result of such high-profile and costly undertakings. “I know that this is the future of military training,,” says Doug Philippone, Palantir’s global defense lead, who has invested in both Anduril and Red6. “But I also see it as the future of the way that the military fights and makes decisions. So it’s not just about fighting, it’s about making decisions.”
Anduril, according to Luckey, is already developing technologies that might be used in training missions and combat. “The next big step for us, which I am really excited about, is taking from our core product and piping that data to heads-up displays that troops on the front line are going to be able to wear,” he says.
But it is unknown how much of this cutting-edge technology makes it to the front lines—or even into training drills. Sorin Adam Matei, a professor at Purdue University in West Lafayette, Indiana, who has designed virtual warfare training platforms for the US military, believes that the technology deployed will be far simpler than proponents of the metaverse believe. He speculates that a less complex version of the IVAS headset could be fitted with an AR rifle scope in the future. “When you are out there shooting and being shot at, the last thing you want to worry about is another piece of equipment,” he says. And technology does not have to be as big as the metaverse to be beneficial. “We need to think a bit more about this metaverse metaphor—which is powerful but also has its limitations.”