Rand Waltzman, Rand Corporation
Cognitive warfare can be functionally defined as “the weaponization of public opinion, by an external entity, for the purpose of (1) influencing public and governmental policy and (2) destabilizing public institutions” [Bernal et.at. 2020]. From the point of view of technique we can say that “cognitive warfare is therefore the art of deceiving the brain or making it doubt what it thinks it knows” [Claverie 2021]. Deception is no less than an attack on cognitive abilities. In discussions of cognitive warfare emotions are often described as limitations to cognitive abilities that must be overcome. But research of the last several decades suggests that rather than being a limitation to cognitive abilities, emotions are integral to them. This in turn suggests that a major attack vector against cognitive abilities is through emotional manipulation. For example: “Neuroscientific and psychological research is now clear: No reasonable or rational decision-making is possible without emotions. Therefore, emotions are necessary for effective decision-making, including in war” [Zilincik 2022]. So to compromise an adversary’s decision making capability, one approach is to appropriately manipulate and compromise the emotions integral to decision-making.
Concepts of cognitive warfare draw heavily from the field of cognitive science. The neuroscientist Joseph LeDoux said of cognitive science that “a pure cognitive approach, one that omits consideration of emotions, motivations, and the like, paints an artificial, highly unrealistic view of real minds. Minds are not either cognitive or emotional, they are both, and more. Inclusion of work on emotion within the cognitive framework can help rescue this field from its sterile approach to the mind as an information-processing device that lacks goals, strivings, desires, fears, and hopes” [LeDoux 2000].
The Allied Command for Transformation (ACT), located in Norfolk, Virginia (USA), is responsible for the preparation and development of future Alliance capabilities. In a recent NATO ACT study on Cognitive Warfare there was great emphasis on Neuroscience and Technology and justifiably so. However, there is another set of technologies on the horizon that needs to be brought to the forefront of any discussion of Cognitive Warfare. Virtual Reality Environments (VREs), like Meta’s Metaverse, of today and in the near future do and will offer tremendous capabilities for cognitive attacks based on emotional manipulation. NATO will have to learn to recognize and mitigate such attacks if its forces are to achieve and maintain cognitive superiority when operating in such environments. In addition to drawing on the already recognized areas of Human Factors and Medicine (HFM), Information Systems Technology (IST), System Analysis and Studies (SAS) and the Modelling and Simulation Group (NMSG) to understand the potential of VREs, we must look to the field of psychology.
In the last several decades, numerous experiments and psychotherapy development have clearly demonstrated the capabilities of even today’s primitive VREs for purposes of emotional manipulation. VREs have been shown to be effective at inducing negative emotions such as fear and anxiety. For example, [Diemer et. al. 2015] demonstrated treatment for phobias using VREs to elicit fear in patients in a controlled way. And while the ability to elicit fear could be used as part of therapy, it can also be used to make a target audience susceptible to harmful messaging like false rumors or conspiracy theories.
Consider the following fictional scenario taking place in a not-so-distant future metaverse: A white supremacist is giving a speech to an audience through one of the new immersive VREs. He is encouraging them to attend an upcoming “Save the Race” rally. Using the VRE he can not only paint his audience a picture to excite them, but he can actually put them in it – not as observers, but as active participants. They now see, hear and feel they are in the midst of a “Save the Race” demonstration that is becoming more and more violent. They feel the exhilaration of being able to take out their frustrations as part of a mob. They themselves are defending the homeland against hordes of invading immigrants, and they feel good about themselves. It all seems so real, except that it isn’t – it is all virtual and it has the intended effect. They are excited and ready to experience those same feelings in the physical world, to participate in a real and potentially violent “Save the Race” demonstration. Next, suppose that tailored coordinated versions of this scenario are simultaneously taking place across NATO member states. Effectively fostering extreme racist, nationalist and anti-NATO sentiment leading to physical violence at this scale could be a major threat to NATO unity and security.
Now imagine a scenario where a political candidate is giving a speech to millions of people in a VRE. While each viewer thinks they are seeing the same version of the candidate, they are each seeing a slightly different version. The difference is that for each and every viewer, the candidates face has been subtly modified to resemble the viewer. This is done by blending features of the viewers face into the candidate’s face in a such a way that the viewer is consciously unaware of any manipulation of the image. The result is that each viewer is more favorably disposed to the candidate than they might have been otherwise. This is not speculation. It is an experimental fact. It has long been known that mimicry can be exploited as a powerful tool for influence.
A series of studies from Stanford University [Bailenson et. al. 2008] has shown that slightly changing the features of a political figure to resemble each voter in turn would result in a significant influence on how people voted. The experiments took pictures of study participants and real candidates in a mockup of an election campaign. The pictures of each candidate were modified to resemble each participant in turn. They found that if 40% of the participant's features were blended into the candidate’s face, the participants were entirely unaware the image had been manipulated. Even so, the blended picture significantly influenced the intended voting result in favor of the candidate. In one experiment, they demonstrated a 20% shift in voting between a control group that did not see the doctored images compared to a group that did. In a VRE, this type of mimicry effect can be produced at a massive scale.
At the heart of all deception, disinformation (a type of deception) and influence operations is emotional manipulation. VREs, like Facebook’s (now Meta’s) Metaverse, will enable psychological and emotional manipulation and privacy violations of its users at a level unimaginable in today’s communications media: social media, film, television and print. Malicious actors will take the arts of deception and influence to new heights – or depths depending on one’s point of view. And as of this writing, we are not able to defend users against the threats posed either technically or legally.
According to the Encyclopedia of Multimedia [Furht 2008], “Virtual Reality (VR) is the technology that provides almost real and/or believable experience in a synthetic or virtual way. The goal of Immersive VR is to completely immerse the user inside the computer generated world, giving the impression to the user that he/she has ‘stepped inside’ the synthetic world.” This definition seems straightforward and innocent enough. However, one can exploit the very same features that make VREs so attractive as communication environments to cause harm to their users. When it comes to emotional manipulation, two features of VREs are particularly important - presence and embodiment. The idea of presence is that participants feel they are communicating with one another directly without any type of computer interface. This is sometimes referred to as “the illusion of non-mediation.” The sense of presence created by a VRE is an important element in eliciting strong emotional responses [Riva et. al. 2004]. The idea of embodiment is that the user has the feeling that his avatar or virtual body – his representation in the VRE - is his actual body.
So, how do presence and embodiment enable emotional manipulation? To understand this, we turn to the theory of embodied cognition [Shapiro and Spaulding 2021]. According to this theory, cognition depends on having a body that has certain perceptual and motor capabilities that are essential to reasoning, memory, language and emotion. Our bodily experiences and processes are the basis for understanding our own emotions as well as the emotions and intentions of others. Emotions are formed from multi-modal inputs. Body language is an important part of our communication. It includes a variety of nonverbal signals like eye gaze, gestures and facial expressions that we use to communicate intentions and emotions. Unlike verbal language, we often produce and perceive body language subconsciously. But it makes up a significant part of our communication. And our perceptual capabilities are not just for communicating with other people. They allow us to pick up on a broad range of emotions and intents from our environments.
Combining the ideas of embodied cognition with the VRE features of presence and embodiment allows interaction between people and between people and environments that exploits the full range of human communication capabilities. Interaction at this level has not been possible in traditional social media environments. And that, of course, is both good news and terrible news. Good, because it allows for better communication. Terrible because it opens users not only to the full range of deceptive, dis-informative and influence techniques used in the physical world, but to what might be even more intense versions of them. So intense, in fact, that the users’ virtual experiences seamlessly meld into real world experiences, memories, and understandings. This changes the way they see the world and understand it and affects the way they behave in it.
The power of VREs has been recently demonstrated by showing that VREs can be used to produce effects similar to those produced by Psychedelic drugs like LSD and psilocybin. These drugs have been shown to be effective for use in therapies for psychological disorders including obsessive-compulsive disorder, addiction, post-traumatic stress disorder, and depression [Kiro 2022]. In fact, the US Food and Drug Administration designated psilocybin as a ‘breakthrough therapy’ for treating depression in 2019 [BusinessWire 2019]. The most significant improvement of patients in psychedelic clinical trials were shown by those who most intensely experienced feelings of self-transcendence. [Yaden et. al. 2017] coined the term ‘self-transcendent experiences’ (STEs) to describe “transient mental states marked by decreased self-salience and increased feelings of connectedness.” [Glowacki et. al. 2022] demonstrated that a VRE can be used to elicit self-transcendence effects that are indistinguishable from those elicited using psychedelics. These effects were measured by four different self-report scales previously used to measure subjective psychedelic experiences. These are the community in self scale, ego-dissolution inventory, communitas scale, and the MEQ30 mystical experience questionnaire. The question that needs further investigation is, what kind of malicious suggestions and messaging are people vulnerable to while experiencing VRE induced STEs?
As immersive technologies become more sophisticated, they will allow the collection of a greater variety of biometric data. Current VR headsets, for example, are mostly limited to tracking body movement. Technology for tracking eye movements, however, is currently being adapted from domains like the aerospace industry. As sensors become smaller and headsets and other wearable devices become lighter and more manageable, virtual reality systems will be able to gather multiple streams of biometric data. VR devices that combine these streams of data will be able to measure and predict user behaviors and attitudes [Heller and Bar-Zeev, 2021]. There will be many who say they will use this capability to enhance the user experience. But rest assured that there will others who, for example, use the technology to effectively manipulate a mob to burn the home of perceived Frankenstein like monster when there was no more than an innocent scientist within.
Consider a few examples of the types of biometric data that an adversary could exploit [Outlaw 2018]. Galvanic skin response can discern how intensely a user is feeling an emotion. Facial muscles are known to reveal feelings like sadness, fear, surprise, anger, contempt, disgust and joy. Muscle tension can be measured using electromyography (EMG). One can use EMG to detect extremely rapid micro-expressions that would be almost impossible for someone to fake. The eye tracker can be used to see what the subject finds visually salient. Electroencephalography can measure how attentive or distracted a user is at any time. A system performing a combined analysis of these biometric data streams, and others will provide a comprehensive picture of the emotional state of a user in real time. One can use that dynamic picture as a feedback loop to modify stimulus to the user for maximum impact. Physiological characteristics related to walking and other movement are useful for gait tracking where one can identify a person by the particular way they walk. Other types of bodily gestures that a VRE could measure might be as revealing about the identity of a user as a fingerprint, retinal scan or a voice print.
The wide variety of biometric measurements will collectively enable the monitoring of how users act, interact and react at a level of detail previously not practical. Such monitoring will support the detailed profiling of their behaviors far beyond what has been possible with traditional social media. This in turn will support manipulation of users more effectively than ever before [Rosenberg 2022].
One consequence of the immersive character of VREs is that a malicious actor could make it extremely difficult for a user to distinguish between authentic and agenda-based manufactured experiences. Immersive environments put the user in the position of a first person actor instead of being a third person observer as in traditional social media. They can take users out of the traditional role of observer and put them at the center of any story [Heller 2021]. This makes it easier to make persuasive messaging seem natural and part of the environment so that the user doesn’t realize what is happening. In other words it is possible to create an “advertisement” that does not appear to the user to be an “advertisement.”
An important question is how users can be helped to make sure they do not misinterpret targeted persuasive content as natural organic encounters. This problem has not even been solved in traditional social media environments where the levers of manipulation are not nearly as extensive as those that will be readily available in VREs. Consider, for example, a VRE where soldiers in an active combat area are able to gather, discuss events and generally commiserate with each other. The adversary could hack into such an environment and plant content, like two soldiers having a discussion that is designed to look authentic and whose purpose is to spread panic and demoralize the troops. This will be very difficult to detect with current technology, especially if their conversation makes use of a broad range of emotional communication channels like gestures, voice tone, pupil dilation, facial expressions, etc.
Conclusion
Cognitive Warfare is about degrading the adversary’s cognitive capability. A key element of maintaining cognitive superiority in a Cognitive Warfare context is an effective cognitive defense. Emotions are integral to cognitive capability so, a major attack vector against cognitive capability is emotional manipulation. Virtual Reality environments (VRE) will be a major battlefield of the future as current social media transitions into VREs [Bazin 2022]. Because of their immersive characteristics, VREs offer opportunities for cognitive attacks based on emotional manipulation at a level far beyond what is possible in today’s social media environments like TikTok, Twitter and Facebook. Capabilities of VREs for emotional manipulation have already been demonstrated in the psychology and psychotherapy communities where current VREs are being used as part of a variety of psychotherapy treatments. In order to adequately defend its forces against such cognitive attacks and ensure cognitive superiority, NATO will have to learn to automatically recognize instances of such attacks. NATO must continuously develop techniques to inoculate its forces against them as well as organizational structures and mechanisms to deliver awareness of and inoculation against such attacks on an ongoing and continuous basis to NATO personnel at all levels.
As a first step towards meeting the requirements of maintaining cognitive superiority in a VRE, NATO must develop a comprehensive taxonomy and catalogue of emotional manipulation techniques in VREs as they exist today and as they might evolve with VRE technology to serve as a foundation for the development of appropriate mitigation strategies to protect future NATO forces. As a first step, one approach to this would be to conduct a comprehensive survey and study of psychology and psychotherapy literature where there are reported findings of a wide variety of experiments and practical experiences using existing VREs for emotional manipulation. Compelling data and evidence will be in the form of reported proven VRE techniques for emotional manipulation that have been used as part of various psychotherapy treatments together with results of VRE experiments with emotional manipulation. The results of this analysis and study will be foundational to the development of mitigation and defensive strategies for maintaining cognitive superiority in Cognitive Warfare environments of the future. In parallel with these developments, NATO must begin to design the organizational and command structures that will be needed to implement and execute the mitigation strategies for defense against cognitive attacks that will be required to maintain cognitive superiority. Such structures will require highly specialized expertise and skills that need to be simultaneously developed.
There are without doubt many characteristics of VREs that will be greatly beneficial to NATO [Mcardle and Dohrman 2022] [Eskiadi 2022]. But while NATO must prepare to take advantage of the benefits of this new technology, it must also prepare to face its potential dangers.
Bibliography
General Eric Autellet, “Cognitive Warfare – Contribution of the French Armed Forces Deputy Chief of Defense” in “Cognitive Warfare: The Future of Cognitive Dominance”, First NATO scientific meeting on Cognitive Warfare (France) ‒ 21 June 2021, Scientific Editors: B. Claverie, B. Prébot, N. Buchler and F. Du Cluzel
Jeremy N. Bailenson, Shanto Iyengar, Nick Yee, Nathan A. Collins, “Facial Similarity Between Voters and Candidates Causes Influence”, Public Opinion Quarterly, Vol. 72, No. 5 2008, pp. 935–961
Bazin, A. “The Metaverse: A New Domain of Warfare”, Small Wars Journal, 3/4/2022, https://smallwarsjournal.com/jrnl/art/metaverse-new-domain-warfare
Bernal, A., Carter, C., Singh, I., Cao, K., Madreperla, O. (2020). “Cognitive Warfare: An Attack on Truth and Thought”, NATO and Johns Hopkins University: Baltimore MD, USA.
Lea Kristina Bjørgul, “Cognitive warfare and the use of force”, Stratagem, November 3, 2021, https://www.stratagem.no/cognitive-warfare-and-the-use-of-force/
BusinessWire, “FDA grants Breakthrough Therapy Designation to Usona Institute's psilocybin program for major depressive disorder”, November 22, 2019, https://www.businesswire.com/news/home/20191122005452/en/FDA-grants-Breakthrough-Therapy-Designation-to-Usona-Institutes-psilocybin-program-for-major-depressive-disorder
Pr. Bernard Claverie, “What is Cognition? And How to Make it One of the Ways of War” in “Cognitive Warfare: The Future of Cognitive Dominance”, First NATO scientific meeting on Cognitive Warfare (France) ‒ 21 June 2021, Scientific Editors: B. Claverie, B. Prébot, N. Buchler and F. Du Cluzel
Diemer, J., Alpers, G. W., Peperkorn, H. M., Shiban, Y. & Mühlberger, A., “The impact of perception and presence on emotional reactions: a review of research in virtual reality”, Front. Psychol. 6, doi:10.3389/fpsyg.2015.00026 (2015)
Eskiadi, I.G., “The use of technology to modernize and strengthen NATO 2030: How the Military Metaverse can support NATO?”, NATO 2030 Global Fellowship Program, 2022, https://www.nato2030fellowship.org/how-military-metaverse-support-nato?utm_content=buffer60abb&utm_medium=social&utm_source=bufferapp.com&utm_campaign=buffer
Felnhofer, A. et al., “Is virtual reality emotionally arousing? Investigating five emotion inducing virtual park scenarios”, Int. J. Hum. Comput. Stud 82, 48–56, doi:10.1016/j.ijhcs.2015.05.004 (2015)
(2008). Immersive Virtual Reality. In: Furht, B. (eds) Encyclopedia of Multimedia. Springer, Boston, MA. https://doi.org/10.1007/978-0-387-78414-4_85
Glowacki, D.R., Williams, R.R., Wonnacott, M.D. et al. Group VR experiences can produce ego attenuation and connectedness comparable to psychedelics. Sci Rep 12, 8995 (2022). https://doi.org/10.1038/s41598-022-12637-z
Brittan Heller, Watching Androids Dream of Electric Sheep: Immersive Technology, Biometric Psychography, and the Law, 23 Vanderbilt Law Review 1 (2021), https://scholarship.law.vanderbilt.edu/jetlaw/vol23/iss1/1
Brittan Heller and Avi Bar-Zeev, “The Problems with Immersive Advertising: In AR/VR, Nobody Knows You Are an Ad”, Journal of Online Trust and Safety, October 2021.
Hana Kiros, “VR is as good as psychedelics at helping people reach transcendence”, MIT Technology Review, August 6, 2022
Joseph E. LeDoux, “Emotion Circuits in the Brain”, Annu. Rev. Neurosci. 2000. 23:155–184
Mcardle, J. and Dohrman, C., “The Full Potential of a Military Metaverse”, War on the Rocks, 2/18/2022, https://warontherocks.com/2022/02/the-full-potential-of-a-military-metaverse/
Jessica Outlaw, “How Can my Biometric Data be Used?”, Jessica Outlaw Blog, April 26, 2018, https://jessica-outlaw.medium.com/how-can-my-biometric-data-be-used-c221fe7d759
Riva, G., Waterworth, J. A. & Waterworth, E. L., “The layers of presence: a bio-cultural approach to understanding presence in natural and mediated environments”, Cyberpsychol. Behav. 7, 402–416, doi:10.1089/cpb.2004.7.402 (2004)
Rosenberg, L.B. (2022). Regulating the Metaverse, a Blueprint for the Future. In: De Paolis, L.T., Arpaia, P., Sacco, M. (eds) Extended Reality. XR Salento 2022. Lecture Notes in Computer Science, vol 13445. Springer, Cham. https://doi.org/10.1007/978-3-031-15546-8_23
Serino, S. et al. “Toward a validation of cyber-interventions for stress disorders based on stress inoculation training: a systematic review”, Virtual Real 18, 73–87, doi:10.1007/s10055-013-0237-6 (2014)
Shapiro, Lawrence and Shannon Spaulding, "Embodied Cognition", The Stanford Encyclopedia of Philosophy (Winter 2021 Edition), Edward N. Zalta (ed.), https://plato.stanford.edu/archives/win2021/entries/embodied-cognition/
Samuel Zilincik, “The Role of Emotions in Military Strategy”, Texas National Security Review: Volume 5, Issue 2 (Spring 2022)
Yaden DB, Haidt J, Hood RW, Vago DR, Newberg AB, “The Varieties of Self-Transcendent Experience”, Review of General Psychology. 2017;21(2):143-160. doi:10.1037/gpr0000102