Anthony McCosker and Rowan Wilken (2020).
Our new book sets out a challenge for the age of digital automation and an environment increasingly saturated with machine vision systems. How do we account for the consistent ambivalence that citizens, companies, governments have toward automated vision technology? How do we chart a path through the visceral response some people have to facial recognition technologies in some situations, alongside the enthusiasm for the security, safety and monitoring solutions those systems offer? And how do we manage the utopian drive for autonomous vehicles against the persistent skepticism and uncertainty that a car will see and avoid dangers on the road?
Automating Vision explores the rise of seeing machines through four case studies: facial recognition, drone vision, mobile and locative media, and driverless cars. Proposing a conceptual lens of camera consciousness, which is drawn from the early visual anthropology of Gregory Bateson and Margaret Mead, Automating Vision accounts for the growing power and value of camera technologies and digital image processing. This idea points us toward the kinds of agency, or even literacy, that are created in the intelligent processing power of machine vision, and the complicated new social relationships they generate.
Behind the smart camera devices examined throughout the book lies a set of increasingly integrated and automated technologies underpinned by artificial intelligence, machine learning and image processing. Seeing machines are now implicated in growing visual data markets and are supported by emerging layers of infrastructure that they coproduce.
In Automating Vision book we address the social impacts, the disruptions and reconfigurations to existing digital media ecosystems, to urban environments, and to mobility and social relations. We explore the prehistory and socio-technical processes that have created the conditions for the increasing automation of vision and consider whether it is possible to create a safe and equitable future as we learn to see with and negotiate the interventions of seeing machines.
The core concept of the book - camera consciousness - targets the expanding potential of 'smart cameras' (just how smart are they?) and automated image processing to reshape society. Our wager is that the changes are as much about the technology as they are about people's responses to and relationships with them.
Camera consciousness is an old concept originating in the early cinema as stage actors had to adapt to addressing cameras rather than audiences. When that caused problems for actors it was seen as condition to be overcome through 'training'.
How does the early cinema actor's quest to learn to see themselves through the camera lens as an imagined audience might resemble the 'training' of algorithms, and the training of deep learning AI systems?)
We see camera consciousness as a conceptual and practical tool for sniffing out the power of smart cameras in the age of AI, automation, visual data analysis. With a bit of effort, camera consciousness can be the basis of an adaptive digital literacy.
This is not a book about surveillance - but the surveillance is ever present and an easy critical target when it comes to this topic. It's more about how we might learn about digital vision and visuality together, with machines. It's about how we see ourselves through technology, to borrow Jill Walker Rettberg's phrase, and how we might learn to see alongside and with seeing machines with all the augmentations and anxieties that are unfolding in the AI, machine vision landscape.
Autonomous vehicles, remote sensing, deepfakes, face recognition, machine vision decision making in health imaging, mobile sensing and mapping, drones... These are our case studies, but the field is expanding rapidly. We are as interested in those dramatic applications, as we are in the use of machine vision for aiding cancer diagnosis or structural maintenance on a bridge or tower.
Labeled medical images. Source: extract from the ChestX-ray8 dataset.
What is camera consciousness?
This somewhat anachronistic term, once used to describe the self-conscious effect of cameras when stage performers became cinema actors, holds a new relevance in the age of seeing machines. Camera consciousness designates both the augmenting power and shift in vision and visibility that computer processing contributes to the work that cameras do. It also refers to the multiplying instances of anxiety and vulnerability that follow.
Something has shifted in how we make sense of seeing, vision and visibility. In fact, with the proliferation of automated cameras and machine vision systems, we no longer hold the privilege of a self-sustained human-centered concept of seeing or vision (if there ever was one). There are cameras that see, act and intervene, and there are the new relationships of visibility they create, the augmentations and states of anxiety that they affect.
The problem of camera consciousness is one of awareness and attention. Sometimes it’s too much, sometimes it’s not enough. Of course, it has become commonplace to consider contemporary digitally mediated life to be encompassed by an “attention economy.”[i] The duplicitous character of camera consciousness in the age of seeing machines is about the new value in the power to manage that attention, or automate it and profit from it.
We look back to the pioneering visual anthropology of Gregory Bateson and Margaret Mead to think through the relevance of the concept of camera consciousness today. Bateson and Mead's photographic and video work in Bali in the 1930s can be understood as an early version of big data visual analysis.[ii]
In Bateson and Mead’s attempt to produce new anthropological knowledge from their visual dataset, they encounter two parallel components to camera consciousness concerning on the one hand the camera and its operator, and on the other the photographed (human) subject. This could be understood equally today in the tension around the dual sense of anxiety and augmentation in the relationship we have with cameras, in their potential for revisioning the world, and in the exchange and analysis of images as visual data.
In fact, camera consciousness foreshadows something of the contemporary discomfort in the dual-use inherent in AI technologies that have begun to automate machine attention and action. For every development that automates vision or manages the use and distribution of digital images, there is a potential application for surveillance, governance, or terror. So, it almost goes without saying that the components and the implications of camera consciousness have greatly expanded.
Camera consciousness describes an often-vexed emotional relationship with personal and impersonal cameras that has heightened with their proliferation and their embedded or associated data technologies. Points of view have become vastly expanded and often inter-connected through networked visual systems or via visual social media platforms.
Digital images carry a new operational and communicative power laden with information that can be newly exploited. Because of this, cameras have become “intelligent” (perhaps in some sense “conscious”) through their capacity for machine vision, deep learning, and real time image processing aiding decision making. Those same cameras can generate image-based logistical systems, and, as we will show, bring into play a new race to commercialize and secure visual and sensory data infrastructures, that positions cameras as a key technology of the fourth industrial revolution and a functioning Internet of Things (IoT).
The automation of vision in seeing machines represents a long sought-after pinnacle of artificial intelligence – the pairing of cameras and judgement. But there are human and social implications. Ultimately, visibility remains as it was in Bateson and Mead’s Bali fieldwork: an unevenly distributed commodity within the digital economy and experienced as the site or territory of social, political and economic struggle.
As a practical application of these ideas, the next step is to explore how to build better digital, data and visual literacies (both human and technical) to manage the emerging landscape of automated vision.
References
[i] Tiziana Terranova, “Attention, Economy and the Brain,” Culture Machine 13 (2012), https://culturemachine.net/wp-content/uploads/2019/01/465-973-1-PB.pdf. [ii] Bateson, G., & Mead, M. (1942). Balinese character: A photographic analysis. New York, 17-92.