5

A Baroque Sensibility for Big Data Visualisations

Evelyn Ruppert

Sensory Sociology and the Empirical Baroque

The sensibilities and varieties of expression gathered under what John Law describes as resources of the baroque do not suggest a method but an orientation to thinking and research that departs from dominant modes of knowing in the social sciences and humanities. This includes concerns with excess, non-coherence, the senses, Otherness, affect, embodiment, movement, and heterogeneity as explored in cultural, postcolonial, feminist, and science and technology studies (STS). As he notes, these concerns resonate with intellectual and political currents that implicitly or explicitly draw from Gilles Deleuze. Coleman and Ringrose’s (2013) edited collection, for example, point to recent Deleuzian inspired empirical research in the social sciences that is concerned with methods as messy, mobile, creative, open-ended, sensory, affective, and performative.

These concerns also resonate with another area of research that attends to the developing field of sensory studies in the social sciences and humanities (Puwar and Sharma 2011). For example, sensory studies have been taken up to conceive of a ‘sensory ethnography’, which takes as its starting point ‘the multisensoriality of experience, perception, knowing and practice’ and how this is integral both to the lives of the researched and how ethnographers practise their craft (Pink 2009: 1). While ‘sensory experience and perception have “always” been central to the ethnographic encounter’ (Ibid: 10), a sensory ethnography involves the explicit inclusion of new forms of sensory experience in the practices of doing ethnography. A key point coming out of Pink’s approach, which focuses on perception, memory, embodiment, and place, is the dual attunement to the sensory of both the researched and the researcher.

There are many more examples of research that takes up the sensory as summarised by Puwar, Sharma, and Pink. What I set out to do here is to understand the sensory as not confined to the senses but to a broader repertoire of ways to apprehend and know worlds. I elaborate this repertoire under the description of a ‘sensory sociology’ with a specific attention to digital mediums.1 This is not to exclude applicability to other mediums but to attend to one that presents specific challenges to methodological sensibilities in sociology. The repertoire is necessarily selective and partial. It is not intended to be a comprehensive survey and it is largely focused on recent writings; indeed, many of the concepts and understandings such as the performativity of methods that I elaborate below can be attributed to many different authors. Instead it is a synthesis of work in the social sciences and humanities that experiments with alternative modes of knowing and what this collection refers to as the resources of the baroque.

The second part of the chapter then exemplifies these sensibilities through an engagement with an empirical object, visualisations of ‘excessive’ digital data, or Big Data. I do this through a discussion of three approaches to visualisation: Anna Munster’s (2013) work on network experience as perception rather than the perceptible; Adrian Mackenzie’s (2010) concern with network theorising that de-animates relations and elides the flow of experience; and Lev Manovich’s (2011) comparison between information visualisation and direct visualisation.

Part 1: A Sensory Sociology

That digitisation is having an impact on how social, political, cultural, and economic worlds are constituted, researched, and known is well documented. Within the social sciences one evaluation of the impact is that it is challenging the expertise of sociologists in both the generation and analysis of social data and social life (Savage and Burrows 2007). Another assessment is that digitisation offers an opportunity to rethink the sociological craft and is leading to a revitalised concern with what ‘the empirical is and how it matters’ in the discipline (Adkins and Lury 2009: 4). For Adkins and Lury, digitisation is destabilising sociology and closing a gap between the practices of sociologists and those of social worlds. The challenge then is to invent sociological methods in ways that can adapt, repurpose, and engage with such digital mediums and the data that they generate (Adkins and Lury 2009; Back 2012). For one, they argue that digital mediums involve continuous feedback loops that change the relations that make up research, and call for

a sociology ‘not of social fact but of the sensate empirical’[…] not at ideal conditions but ‘real existing stuff, real existing social processes’, that can ‘deal with uncertainty, chaos, complexity and multiplicity’ […] ‘multiple, contingent and in process rather than fixed and a priori’ […] ‘requires the discipline to confront a newly co-ordinated reality, one that is open, processual, non-linear and constantly on the move’ (Adkins and Lury 2009: 18).

All of the qualities that Lury and Adkins cite here, from uncertainty and chaos to a reality that is constantly on the move, call for sensibilities that are quite different from the crafted and strategic methodological approaches typical of the sociological craft. For one, digital mediums reconfigure the relations between researchers and the researched by offering new opportunities for engagement through innovative forms and modes of presenting and conducting sociology in ‘live’ mediums (e.g. social media platforms). Through a variety of examples, Adkins and Lury raise questions about how sociology might be reconfigured to do this and how we might then reimagine data not as simply a conversation and dialogue with publics ‘out there’ but engaged in different ways and in relation to the very methods deployed.

This conception of relations to publics and mediums is of course not entirely new. The Mass Observation project is one well-cited example.2 It involved studying the ordinary lives of ordinary people in the UK to counteract the stereotypes that held sway in the British media in the early part of the twentieth century. It could be considered a form of crowdsourcing since a key medium was photographs produced by volunteers documenting intimate aspects of their lives, from their lounges and closets to their gardens. What digital mediums have done is mobilised and made possible the multiplication of forms of engagement that are more recursive, multisensory (image, text, sound, and so on), interactive, live, and happening.3

Digital mediums have also folded the everyday lives of publics into research methods. On the one hand, social media platforms are mediums of digital sociality and the doing of social relations. The data they generate in the cultural sphere on platforms such as Facebook, Spotify, and Flickr are also part of everyday popular cultural forms that are actively both produced and consumed via myriad acts of ‘playbour’ (Beer and Burrows 2013). Such data is lively as it is recursively taken up and reappropriated as a part of contemporary popular culture. At the same time social researchers and others develop methods for analysing and interpreting the data these platforms generate to make sense of, interpret, and know digitally mediated lives (Ruppert, Law, and Savage 2013). Thus digital mediums both open up the possibilities for creative, interactive, and collaborative research engagements with publics and at the same time can render them unknowing research subjects. Their engagement is thus variably configured by the method relations of which they are not separate from but are a part.4 Digital mediums are thus implicated in both the performance and knowledge of contemporary sociality, and changing who subjects are, how they are known, or apprehended, and their forms of engagement.

The relation between the researcher and researched is but one aspect that digital mediums open up. More generally, the concerns that Lury and Adkins raise call for reimagining methods with a different set of sensibilities that are not part of the standard repertoire. Conceptually, I capture these sensibilities, which necessarily overlap and play off each other, under the description of a sensory sociology: medium specific, live, performative, and inventive.

Medium: Specific

Different mediums are different ways of ‘telling about society’, as Becker (2007) has expressed it. From statistical graphs, historical narratives, and novels to films, maps, and theoretical discourses, there are myriad ways of telling about society that are not only the province of social science researchers. While visual mediums such as photography are modes of knowing that have a long history in academic disciplines, publics also do photography and produce visualisations as part of their everyday lives and now are also generators, observers, and analysts of their digital lives. Building on Becker’s argument, Back then asks what is sociological about the images, recordings, and traces generated by digital mediums. If ‘one of the first principles of social science is that nothing speaks for itself’ (Back 2012: 33), then critical and ethical questions must be asked about the methods deployed to speak for things. But, as he asserts, then we must attend to how mediums – the digital, visual, sonic, textual, and so on – are specific and different ways of sensing worlds and not simply add-ons to research methods or to which we can apply already existing methods. Instead, methods need to be (re)imagined in relation to their mediums.

Such a sensibility alerts us to mediums as ‘just openings’; we need to go beyond them because ‘data pictures do not give up their meaning in a single glance […] the pictures require more work before they give up their full meaning’ (Becker 2007: 169). How photos are taken to how they are interpreted involve specific framings; the task then is to make the invisible chain of translations involved in generating the photo visible and part of the analysis (Guggenheim 2013). Attending to the medium thus calls for going ‘behind’ the arrangements of the photo, the digital image, or the sound recording and attending to the different human, material, and technical organisation of mediums and how these participate in and configure concepts and modes of knowing (Ruppert, Law, and Savage 2013).

So, for instance, changes in the relations between researched and researcher are not simply a matter of differently positioned or engaged people but also their relations to what is assembled in a medium. This includes particular forms of expertise, theories, and conceptions of the social to different materialities, tools, and technologies that get folded into the methodological apparatuses organised and taken up by the researcher. Modes of knowing are saturated with all that compose the medium and which make it distinctive.5 For instance, digital mediums such as social media platforms are made up of specific forms of expression such as keyword selections, hashtags, or hyperlinks that are ‘born online’ (Rogers 2013) and generative of distinct modes of sociality (Marres and Weltevrede 2013). The same forms of expression can be repurposed to become both the object of and actively inform social research through ‘native’ methods that follow the medium (Rogers 2013). Rather than importing techniques and concepts used to research other mediums, medium-specific sensibilities involve attending to the unique cultures of mediums and all that compose them. A sensory sociology is thus attuned to medium specificities as not simply add-ons but as constitutive and inside modes of sociality – how the social is done – and also social research – how knowing is done. At the same time, the attention to medium specificities is not a call for separate subfields or to suggest what Guggenheim (2013) criticises as a ‘media-determinism’. Instead, it is to recognise specific forms of expression that may or may not correspond to other mediums but yet share other qualities of a sensory sociology elaborated below.

Live: Qualities

A sensory sociology understands ‘sociology in-the-making’ and focuses on the process of sociological research rather than its products (Lury 2012). It is attuned to what Back has called ‘live’ rather than ‘dead’ sociology (Back 2012). The latter is objectifying, comfortable, disengaged, and parochial. Instead, live sociology draws attention to many of the qualities of the social that are not captured by existing social science methods which are dominated by narrative/words and number/statistics, and which deal poorly with qualities. In characterising live sociology Back summarises Law and Urry’s call for the social sciences to renew the sensibilities of its methodological inheritance, which are ‘under pressure from an alternative, complex and performative sense of social inquiry’ (2004: 403). Twenty-first century methods are, for example, not well positioned to address the

  • fleeting: is here today and gone tomorrow, but only to reappear the day after tomorrow,
  • distributed: that which is to be found here and there but not between or which slips and slides between one place and another,
  • multiple: that which takes different shapes in different places,
  • complex: of the non-causal, the chaotic,
  • sensory: that which is subject to vision, sound, taste, smell,
  • emotional: of time-space compressed outbursts of anger, pain, rage, pleasure, desire, or the spiritual, and
  • kinaesthetic: of pleasures and pains that follow movement and displacement of people, objects, information, and ideas.

The different ways of sensing the social call for reimagining social research not in a way that is against narrative or number but to ‘extend the range, texture and quality of what passes as academic representational practice and writing’ (Back 2012: 28). For example, an equality of the senses and ‘attentiveness to the multiple registers of life’ (Ibid: 29) includes approaches such as ‘sonic geographies’ (Kanngieser 2012) that open up the affective and ethical political force of voice and those with a sensitivity to the multisensory ambiences, sensibilities, and work of the senses of everyday urban life (Rhys-Taylor 2010). Or the ways that Puwar (2011) has explicitly demarcated the sensory as a field within postcolonial studies to engage with questions of difference, alterity, migration, and belonging through the media of musical composition, poetry, and film.

There are numerous other possible approaches and meanings that Back and Puwar (2004: 403) document in their ‘manifesto for live methods’, which consists of eleven provocations for how sociology can be done. Building on the understanding of live methods and the changing nature of the empirical, their collection of articles covers ‘real-time’ and ‘live’ modes of research (Marres), speculative designs that actively seek out empirical objects and events that are ‘idiotic’ (Michael), an ‘amphibious sociology’ that lives in at least two media (Lury), new forms of curating public performances and collaborations (Puwar and Sharma), attentiveness to the doing of social life (Back), and how the ‘make believe’ might aid the sociological imagination (Motamedi Fraser). These are just some of the qualities of what live methods mean for paying attention to the social world through a range of sense-making. But as the contributions also establish, this involves ‘placing critical evaluation and ethical judgement at the centre of research craft’ and ‘debat[ing] the forms of work we are doing, the kinds of academics we are producing, and the institutional and life worlds we occupy as well as make’ (Back and Puwar 2004: 15). These issues are confronted in the final two qualities of a sensory sociology.

Performative: Effects

The entanglement of methods, mediums, researched, and researcher is captured in the understanding that methods have social lives. Interwoven in the foregoing is an implicit argument that reimagining methods in relation to different mediums and sensibilities does not simply involve representing worlds differently but also enacting them in new ways. Methods do not stand apart as representations of social worlds; they also perform those social worlds and enact them. This is different from an epistemological argument that practices construct representations of realities which depend on holding the world (acted upon, and constructed) separate from practices such as methods and their representations – or as perspectives on a stable, accessible ‘real’ (Law and Urry 2004). Instead enactment is based on an ontological claim that both representations and the phenomena they describe come into being through methods. Theory, method, and social worlds are thus entangled and methods are social theories in practice, whether or not this is made explicit or remains implicit.

A sensory sociology thus not only involves going behind the medium to interrogate its specificities, but to also understand how it is inside and part of enacting modes of being and knowing. This includes the performative effects of the medium when it is part of a method apparatus. One approach to this is to think of methods as having double social lives, shaped by and shaping of the social worlds of which they are a part (Law, Ruppert, and Savage 2011). For the former we can ask, ‘what are the specific sensibilities demanded and afforded by a medium when we fold it into our methods’? For the latter, ‘what then are the performative effects of our methods so composed’? This calls for rethinking theoretical assumptions and attuning to a number of qualities and their performative effects that are specific to a medium. The following are some questions about how the qualities and affordances of digital mediums can come to shape the performative effects of methods and the social worlds they enact:6

  • Transactional actors: by making sense of doings – of movements, actions, transactions, interactions, choices, and statements – how do methods enact worlds as the outcome of not only what people say but also do?
  • Heterogeneous relations: by attending to the associations, links, switches, connections, exchanges, flows, and networks between not only people but also things, how do methods enact worlds as mixes of relations?
  • Visualisation of patterns: by making sense not in words or numbers but through images, graphics, and designs that reduce ‘excessive’ information into visual forms, how do methods enact worlds as patterned arrangements?
  • Continuous time: by following the ongoing, real-time, dynamic, and continuous digital traces of the activities and movements of people and things, how do methods enact worlds as processual rather than static entities?
  • Granular being: by focusing on particularistic identifiers of people such as their microscopic details, unique profiles, and specifications, how do methods enact them as monads and particulars?
  • Mobile and mobilising: by engaging people actively in digital mediums that circulate between numerous sites, how do methods enact worlds in participatory yet configured, designed, and perhaps dominating ways?
  • Non-coherent accounts: by taking up digital mediums along with other analysts (governments, businesses, publics) and for different purposes, how do social science methods enact worlds that are implicated in the normative purposes of others and multiply and compete with their accounts?

There are many examples of how digital mediums provoke these questions and constitute specific modes of sense-making. For example, Gabrys (2012) explores environmental sensing as a complex ecology of networked devices and citizens actively engaged in sensing and knowing pollution, flora, fauna, and sustainability. As she further articulates, rather than approaching ‘the senses’ as given, she considers how ‘sensing-as-practice’ is differently articulated in relation to specific entanglements of technologies, data, and humans.7 Sense making is thus an entanglement between people, devices, and data that perform environments in complex ways. Such ecologies draw attention to modes of knowing as ‘happenings’ that are full of contingencies, adjustments, and oriented to the open-endedness of social worlds (Lury and Wakeford 2012).

Inventive: Answerable

Methods understood as performative and happening introduce a fourth sensibility: the inventiveness of sociology as a craft. If methods are performative, then we can think of them as creative and inventive, but in ways that may not be intended or knowable in advance. The openness of the social world and of methods means that there is indeterminacy about how either plays out and which cannot be anticipated or pre-determined. Through their liveliness, worlds and methods are inventive in ways that are beyond the determinations and expectations of the researcher.

Like the other qualities of a sensory sociology, to think of methods as inventive is not new and does not simply apply to contemporary methods that mobilise digital mediums. Methods more generally are means by which the social world is not only investigated, but may also be engaged. However, such inventiveness is often not attended to or acknowledged. That is, researchers are often not attentive to the unexpected and instead explain these as anomalies, or ignore them because they don’t fit the methodological paradigm, expectations, or original problem (Lury and Wakeford 2012). Lury and Wakeford thus suggest that ‘the inventiveness of methods is to be found in the relation between two moments: to a specific problem, and the capacity of what emerges in the use of that method to change the problem’ (2012: 7). Regarding the latter, they argue that this calls for answerablility to what is provoked and emerges when a method is activated:

Inventiveness is a matter of use, of collaboration, of situatedness, and does not imply the ineffectiveness of methods, only that their inventiveness – their capacity to address a problem and change that problem as it performs itself – cannot be secured in advance (Lury and Wakeford 2012: 7).

Answerability to inventiveness means to excavate the versions of social worlds initially implied or explicit in methods but then brought to life by them and made to happen including that which cannot be anticipated. The performativity and inventiveness of methods means they are not innocent but political: ‘They help make realities. But the question is: which realities? Which do we want to help make more real, and which less real?’ (Law and Urry 2004: 404).

There are multiple possible moments of answerability. In addition to how a problem is formulated and what this then provokes, there is accountability to the sensibilities that methods attend to and make more real. By attending to ‘that which has conventionally escaped or troubled social science – the virtual, the affective, the ephemeral’, methods can possibly ‘expand the actual, invent and/or strengthen particular worlds’ (Coleman and Ringrose 2013: 8). There is also answerability to the ethical relations to the researched that methods enact. Rather than the ‘intrusive empiricism’ of confessional and voyeuristic methods such as interviews (Back 2012), answerability can be built into methods. For example, Puwar (2011) has created a call-and-response methodology that engages artists and creative practitioners in music, poetry, and film as active collaborators in the research process and publics not simply as audiences but as dialogic participants. Feminist methodologies have attended to these folded relations between researcher and the researched in the process of sense-making as an ethics of research (Coleman and Ringrose 2013: 12). Barad (2007) names this agential realism: at issue is not representations of a separate real but the real consequences, interventions, creative possibilities, and responsibilities of intra-acting within worlds.

Taken together, the four sensibilities – medium specific, live, performative and inventive – are attuned to what this collection has set out as baroque modes of knowing. As modes of knowing, the sensibilities are far from prescriptive or programmatic, as the examples I have referenced attest to. Much of what I have called a sensory sociology is more suggestive than specific, and that is one of the reasons I work with all of the variations of ‘sense’: from sensory, sensibility, to sense-making, and the myriad meanings associated with these words: to perceive, feel, experience, detect, understand, comprehend, grasp, test, or expound the meaning of worlds.8

In the next part of this chapter I will engage with what I have described as a sensory sociology to think through some examples of dominant and alternative visualisations of Big Data generated by social media platforms. I first start by outlining how visualisation has become a key method through which sense is made out of excessive data. I then turn to examples of work that treat visualisations as openings to the challenge of the empirical that I set out at the beginning of this chapter: the articulation of sensibilities that depart from dominant modes of knowing in the social sciences and humanities.

Part 2: Encounters with Data Excess

St. Pierre describes how she also began to ruminate on the status of ‘data’, and how certain kinds of data ‘were uncodable, excessive, out-of-control, out-of-category’. Paying attention to such ‘transgressive data’, St. Pierre argues, is an ethical project, in that it is concerned with responsibility, with ‘theorising our own lives, examining the frames with which we read the world, and moving toward an ongoing validity of response’ (Coleman and Ringrose 2013: 12, citing St. Pierre 1997: 179, 186).

When we imagine a network these days, it is hard to stave off the flood of visualisations – tangled threads, fractal webs, uneven distributions of interconnected circles and lines – that populate our contemporary connectionist imaginary (Munster 2013: 1).

The ‘data deluge’ is one common term used to describe the volumes of accumulating digital data on the internet and in database archives. Increasingly referred to as Big Data, both the volume and digital format of data has led to visualisation emerging as a key method of social analysis. While numbers and text have long dominated the social sciences, digital visualisation has become a means of reducing ‘excessive’ data to forms that are stabilised and represented in patterns that can be ‘more easily’ interpreted. That visualisation is increasingly being taken up as a tool of social enquiry is also a matter of concern, especially in relation to how they are ‘guiding’ attention in particular ways (Madsen 2013). But the problem of and solutions for analysing and making sense of large volumes of data is not only a challenge of Big Data. Ethnographers, for example, struggle with the flood of data that field sites present and problematise the necessity of drawing boundaries around what is to be included and enacted (Candea 2013; Law and Singleton 2013). What they suggest is that attending to excess is not simply a technical matter of magnitude and its processing but about the choices made to reduce, bound, sort, organise, categorise, represent, and interpret data. Yet these explicit ordering and rationalising techniques also encounter what Saint Pierre calls the ‘uncodable, excessive, out-of-control, out-of-category’ and that which is ‘transgressive’, all that is in excess or beyond the frames of enacted worlds. This includes the ‘ordinary affects’ of the everyday that exceed and evade meaning and representation and that which can only be sensed, as it cannot be grasped by structures of signification.9

Encounters with Big Data as excess – of what is included in sense-making and what is absent, out of control, and beyond – can be interpreted through a sensory sociology. I explore this by contrasting two modes of visualising excessive data that enact social relations, which really are two different empirical strategies. One is a popular mode called the network diagram, which reduces and controls excess to make it perceptible and interpretable. It has become a dominant mode of knowing and showing how ‘excessive’ information can be reduced to a form that can be meaningfully, if partially, rendered for interpretation (Ruppert, Law, and Savage 2013). Another conveys the uncodable, which is beyond signification and involves a mode of knowing that provokes and calls for the sensibilities of a sensory sociology. This comparison is inspired by Anna Munster’s (2013) work on network experience as perception rather than the perceptible, Adrian Mackenzie’s (2010) conception of the embodied and sensory experience of networking and ‘wirelessness’ and Lev Manovich’s (2011) differentiation between information visualisation and direct visualisation. Their work provides a number of openings for doing a sensory sociology through visualisation.

As the quote from Munster at the beginning of this part of the chapter notes, network diagrams are a particular visual form that uses nodes, lines, and simple geometrical shapes to represent relations between data objects and is used to render data visualisations on almost every aspect of life, from social networks to financial transactions. Gephi is one popular open source platform for generating such network diagrams out of social media, from billions of tweets to hundreds or thousands of Facebook friends: ‘It helps data analysts to intuitively reveal patterns and trends, highlight outliers and tells [sic] stories with their data’.10 Entering the words ‘Gephi’ and ‘Twitter’ in a search engine will immediately return thousands of images of network designs generated by individuals, organisations, and researchers.

Fig. 5.1 Network diagram by Martin Hawksey (2011)11

While these are not of the scale of some large quantitative network analyses (e.g. Watts (2007)), the form is similar. As Manovich (2011) describes, they typically do two things: use ‘graphical primitives’ (points, lines, curves) that reduce the specificities of each data object to only about 1% of its characteristics.12 In this way they follow the dominant science paradigm that involves the calculation of totals and averages and their representation in histograms, scatter plots, and line graphs. The second thing they do is use spatial variables such as relative positions to reveal particular patterns and relations. Spatial variables are privileged over other properties while less important qualities are represented in tones, shading, and colours. While there are some practices that veer from this, the majority are based on this configuration. Manovich suggests that this is due to the privileging of everyday ways of seeing spatial properties as well as the affordances of available graphic technologies.13 In this regard, they constitute particular ‘web-visions’ or valuations of what is attention-worthy in data, such as provoking sensitivity to invariants and anomalies that are within and internal to the framing of the diagram (Madsen 2013).

While Manovich critiques the reduction and privileging of spatial dimensions, he doesn’t attend to the kind of experience and sense of the social that network diagrams enact, promote, and provoke. This is something that Anna Munster (2013) takes up when she argues that the network diagram blinds the viewer to qualities such as unevenness and asymmetry and constitutes a particular kind of network experience. Its pervasiveness, sameness, and repetition are generative of a visual and conceptual slide into what she names network anaesthesia: ‘a numbing of our perception that turns us away from their unevenness and from the varying qualities of their relationality’ (Munster 2013: 3). Instead, she argues for ways of sensing the patchiness of the network, its particularities, and complexities, such as the closing and opening of relations and their changes and durations over time. That is, she seeks to undo the frozen quality of the network diagram, its coherence, tidiness, and ‘repetitive mode of managing quantity’ (Ibid: 5) that is anaesthetised and numbing and a normalising dispositif that does not maintain processuality but codifies relations into recognisable patterns.

To accomplish this she proposes a shift from the perceptible to perception, by drawing on pragmatist philosopher William James’ work on experience where perception involves the making of worlds where sensing itself is done ‘as we go’ or a kind of happening. In contrast, the perceptible is the outcome of perception-action, where ‘to recognise is to see something already seen: a pattern seen in data is an example of the perceptible’ (Ibid: 5). It involves generalising experience by making something recognisable through recurring patterns that humans can comprehend and experience. What is perceptible often comes to stand for what is perceived.

A second move she makes is to avoid reducing perception to human experience but to understand network experience as happening through the ‘heterogeneous togetherness’ of humans and nonhumans. This does not involve understanding how humans experience technology, as typically done in studies of human-technology interaction. Rather, she inverts the relation to focus on how networks experience, or what she calls the aesthesia of networks. Network experience is not given but involves recursive relations between humans and nonhumans that are actively forming and which tend toward repetition and difference. She takes up James’ concept of concatenation – the ‘determinately varied hanging together’ – to express network experience as the radically novel and individuated moment that has no prior convention for understanding it:

The point is not to map, model, and systematize the network or experience but to account for, to sense and to encounter novel network aesthesias…. But networking – processes, proto-formations, and imperceptible human/machine currents that conjoin social, info-technical, and aesthetic elements in novel ways – is what generates an aesthesia of networks (Munster 2013: 8–9; emphasis in original).

While the novel is imperceptible it is a becoming perceptible, an emergence that signals the presence of something else, a radical empiricism that is inventive, and creative, and ‘proffers new sensibilities, and so the possibility that other ways of sensing, relating, and indeed living might thereby emerge’ and of ‘generating novel networking sensibilities’ (Ibid: 10). For Munster, the ‘felt force’ of networking comes out of expressing the heterogeneous, inventive, and novel relationality, and collective experience that makes up visualisations that are not ontologically given nor prior to dynamic experience.

Mackenzie also critiques network-oriented theorising and argues that since the 1980s it has turned ‘every scale, order, and variety of phenomena’ into a network form (2010: 9). Like Munster, he takes up William James’ conception of experience but attends more to the specific material and infrastructural elements of networks and their mediations of sense-making. Experience is understood as immersive entanglements with and often fleetingly felt sensations of the flow of wireless networks, things, gadgets, and infrastructures that escape codification, symbolisation, or quantification in network diagrams. He names this experience ‘wirelessness’, which he argues captures James’ conception of ‘the processes of moving, making, changing, altering, and connecting of feelings, things, events, images, textures, ideas, and places’ (2010: 13) and a ‘feeling of incompletion or openness’ (Ibid: 14). For Mackenzie, then, a radical empiricism needs to ‘engage with experiences ranging from the infrastructural to the ephemera of mediatised perception and feeling’ (Ibid: 17).

Mackenzie also attends not to networks but networking as processual where experience involves what James called ‘transitions’. It is the sense of continuous or discontinuous transitions that allow experience to flow, and holding onto this flow from one experience to another is what makes empiricism radical (2010: 18). As Munster also argues, this means attending to tendencies, trajectories, emergence, and inventiveness. To an extent this is what Manovich does when he identifies challenges to the visualisation paradigm that has dominated the last 300 years of practice. With the advances in computing in the twenty-first century, new forms of visualisation, especially in the humanities, are fundamentally changing and foregrounding animation, interactivity, and more complex visualisations. Specifically he calls these emerging forms ‘direct visualisation’, where data objects such as an image are not quantified, reduced, and translated into points and graphic signs but maintained to varying extents in their original complexity, such as visualisations of whole texts or complete corpuses of photos.14 They also can be dynamic, animated, and interactive rather than frozen in time and include sound and textures. Direct visualisation maintains complexity and excess where revealing patterns is still central, but vision can be mixed with sound, tones, interactions, and movements and thus generative of multisensorial and affective experiences of data.

There are many experiments with visualisations that in varying ways and degrees go beyond the calculative and reductionist approach of the network diagram to explore network experience in the ways Munster, Mackenzie, and Manovich suggest. I state this as ‘varying ways and degrees’ to capture how they involve a play between the perceptible and perception, are simultaneously recognisable and novel, but at the same time make the boundary between the two experientially present. This is what Mexican artist Agnes Chavez does in her visualisations of the same kind of social networking data used in the social sciences to generate network diagrams. Her visualisations enact not the network but networking, and call on empirical sensibilities that are radically different. The choice of an artwork to explore this could be criticised since art has stronger claims to the sensibilities I have outlined.15 But it is because Chavez takes up the same digital data that is very popular in social science analyses and network visualisations that I think the example is compelling for exploring how alternative visualisations could be enacted.

She describes her (x)trees project as an exploration of data visualisation and video-mapping technologies to create immersive contemplative spaces around the theme of trees. One version (Fig. 5.2), integrates the data mining of social networks and text messaging to generate branches of messages containing designated keywords (trees, nature, rights) on a wall in ‘real time’ as they happen online.16 When exhibited at the Albuquerque Museum in 2012–13, people could text or tweet live in front of the installation and see their messages as they floated up and disappeared. The trees also incorporated quotes about nature collected from the internet and the articles of the Universal Declaration of Rights of Mother Earth. Finally, the installation included interactive audio created by Alessandro Saccoia out of sounds collected from nature, crowds, and a repeating Prana breath to instil an awareness of breath. The (x)trees were designed as an experience and catalyst for contemplative states of being for people to sense their global and local relationship to others, and to contemplate the value of trees and the rights of nature.

There are of course many explorations of art as a mode of knowing. In drawing on this example I am not arguing that social scientists should become visual artists or that art introduces the subjective and makes up for its erasure in objective network diagrams.18 My point is to draw attention to the sensibilities that this other form of data visualisation and mode of knowing enact and open up. It is a mode of knowing that has interesting crossovers with data-driven approaches in the social sciences, but how it is ‘driven’ by data involves different sensibilities. For while there are many examples of data visualisations in the social sciences that innovatively experiment with the possibilities of digital mediums, they are limited in their engagement with the sensibilities I have outlined above.19 Most importantly, they do not capture the immersive and embodied encounter of (x)trees. For example, they typically retain the network diagram as a container and mode of knowing even though they may modify it by making it more dynamic or interactive.

Fig. 5.2 (x)trees by Agnes Chavez and Alessandro Saccoia (2012)17

Instead, I want to think about how (x)trees does a network experience. The researched – who I will now call a subject – stands outside and observes but can also become part of and see herself as a subject together with others and things in the making of a network experience; she doesn’t simply interact with a given dataset but performs and is part of the experience through her networking with various technological devices. She is folded into and experiences a kind of communion with people, trees, and technologies. As Law writes in his introductory chapter, she is entangled and asked and required to submit and participate. Yet it is through familiar elements of trees, branches, colour, sounds, lines, screens, light, points, and text that data is rendered perceptible as a ‘heterogeneous togetherness’. And while a repetition of what is already recognisable and thus also perceptible to others, difference is introduced – if ever so slightly – by the subject’s presence and intervention that now becomes part of what the network experiences. But this I think is also Chavez’s point. The subject is but one within a complex ecology, whose utterance is a fleeting moment compositionally part of the network and simultaneously collective and novel. The relation is dynamic, temporal, and dependent on the networking experience between the interface as framed by the artist and the flow of words of distant subjects. Temporality trumps spatial relations (as diagrammed in connections between text) in the dynamic formation and flow of fleeting but sometimes repeating text.

While reducing social media data through keyword selection, the (x)trees also maintain the whole data object (tweets, quotes, messages) as streaming text. The scale and quantity of this content are not enumerated – there is no possibility of knowing how much – and it is instead experienced as qualities. Quantity thus becomes a sensory quality to be immersed in and feel but also to see oneself reflexively and individually as part of its very generation and experience. While ‘small data’ such as thick and detailed ethnographic accounts are often put forward as an answer to what is subjectively ‘missing’ from the reductionist and thin analyses of Big Data, (x)trees performs quantity as a subjective and sensory experience. In this way it also complicates the conventional boundaries between quantitative and qualitative methods and accounts.

Nature and its qualities are both a matter of concern and of experience. Living, breathing, moving, and flowing are heard and seen, and the subject is inside of these. While the subject also reads text, it is with the same qualities of the sound and image. That is, text flows and moves quickly, like sounds, and some of it repeats such as quotes; some is never to be seen again, and some flows by without the possibility of being apprehended. Excess is not contained but flows, is part of a process rather than a product, and disappears as part of a lifecycle like its matter of concern. The entirety is thus not graspable or analysable, but that is one of its qualities: to perceive the collective as a sensation of ‘the presence of something else’ (Munster), of ephemera, surpluses, and overflows (Mackenzie), and of not-knowing that can be made ‘experientially real’.20

It is hard not to imagine that the artist intentionally sought to create an experience that enacts an ecological sensibility.21 And for this reason the mediums she chooses – visual, sonic, textual – are not external but internal and consequential to the very ecological phenomenon she seeks to enact. What I think Chavez has achieved is a merging of mediums, subjects, and objects, and that these together constitute the network experience. She prepares a stage that is configured yet happening as it compositionally changes such that each performance is new. Through such a merging the force of networks is felt and experienced as momentary, dispersed, multiple, and multiplying, elusive, chaotic, normative and affectual, and alive, and in motion. Yet, at the same time, much is reasonably and recognisably conveyed through words and repetitions.

There is more that can be said here. I could make explicit how the four sensibilities of a sensory sociology are actualised by the (x)trees. Some of this I have already done above and could extend to the enactment of live, lively, and existing social processes, of attuning to complexity, heterogeneity, and the performativity of the (x)trees. Its juxtaposition against the network diagram certainly has done some of this work. Instead I will reflect on the quality of inventiveness and answerability and offer some comments on the sensibilities they make more real.

In the simplest terms, the network diagram closes what (x)trees opens up. Through reduction, classification, and stabilisation, it settles on a version of networking that makes unproblematic its own terms of closure and its performative effects. These are terms that render the object – relations – calculable and actionable. In these ways they reproduce dominant modes of analysing ‘quantitative’ data in the social sciences. The view is informed but not informing of all that has come to compose it, and it is productive of passive viewers whose data traces may be part of the data network but which they cannot see.

(x)trees provokes, activates, and enacts something else, a different way of sensing, relating, and knowing that is open and changeable not by one subject but by an unseen yet sensed collective of humans and non-humans. In this way (x)trees opens up and activates the emergence of what Munster calls the becoming perceptible while resisting its taming and containing and what Mackenzie describes as the peripheral and overflowing. What is knowable escapes, yet at the same time is felt and apprehended as relations between humans and non-humans that are enacted but irreducible to the figure of the network. Rather than reduction, the subject perceives the irreducibility of knowing and its matter of concern – nature. Yet the encounter with such uncertainty and unknowing enacts something. That something includes experiencing the possibility of other ways of sensing and relating to worlds that open up experience to the multisensorial, contemplation, and the ineffable, and how we are entangled in our matters of concern.

The concerns about nature and ecology and our place within them that (x)trees enacts are often appealed to in declarations and political narratives. Such appeals beckon through words and rhetoric the activation of reason and responsibility to protect nature, and sometimes also mobilise and play on affect to achieve the same purposes. Knowledge politics are thus advanced as a deliberative competition between argument, persuasion, and normative affirmations that subjects receive and evaluate. I think (x)trees performs politics instead as an encounter that is a multiple, complex, and perpetual composition made up of the self in relation to other people, things, ideas, technologies, and nature. The real it does, opens up, and makes more real is one that compels the active, contemplative, and engaged subject. It is a real that is not simplified, classified, and organised but the ‘felt force’ of a collective that beckons a sensibility of the subject as a bit player but also composer of the real. In this way it ends with a question rather than answers: what will you contribute, and what will you create?

Acknowledgements

I am thankful for the feedback on earlier versions of this chapter from John Law, Adrian Mackenzie, and Rebecca Coleman.

Notes

1 I am grateful for the insights, ideas, and contributions towards thinking about a sensory sociology in the writings of colleagues in the Department of Sociology, Goldsmiths, both past and present. In particular, Noortje Marres and Nina Wakeford conceived of and prepared the general outlines of a Masters course called ‘Sensory Sociology: Imagining Digital Social Research’ that I taught with Rebecca Coleman in 2013 and through which I developed my understanding.

2 Founded in 1937 by anthropologist Tom Harrisson, the poet and journalist Charles Madge, and the surrealist painter and filmmaker Humphrey Jennings.

3 #Citizencurators, for example, is a history project that recorded the experience of Londoners during the Olympic fortnight in 2012. Created for the Museum of London, it involved the ‘real-time’ collecting of tweets, objects, and images compiled by citizens to ‘tell the story of everyday life in the capital’ <http://citizencurators.com/about> [accessed 16 July 2014].

4 This is captured by the concept ‘agencement’, the French version of assemblage as elaborated by Deleuze, which focuses attention on how agency and action are configured by and contingent upon the sociotechnical arrangements that make them up (Ruppert 2009).

5 Compare for example the mundane and distributed devices such as statistical procedures, skilled interviewers, and clipboards of surveys, focus groups, or interviews to the digital platforms, algorithms, APIs, software designers, and so on, of digital mediums.

6 These are selected and re-cast from nine propositions about how we might rethink the theoretical assumptions of social science methods to take into account the specificities of digital devices and data (Ruppert, Law, and Savage 2013).

8 ‘Sense, v.’. OED Online. <www.oed.com.catalogue.ulrls.lon.ac.uk/view/Entry/175955?rskey=XKORSF&result=2> [accessed 01 March 2014].

9 I have taken the understanding of ordinary affects from Coleman and Ringrose’s (2013) discussion of Kathleen Stewart’s (2007) book by the same title. On beyond representation, see van de Port in this collection, and his engagement with Žižek and Eagleton on the Real to think about that which is always beyond representation and what structures of signification cannot grasp.

10 ‘Gephi: makes graphs handy’ <https://gephi.org/about/> [accessed 10 Jan 2015].

12 Manovich adopts the term ‘information visualisation’ rather than data visualisation. I agree with the former, since visualisations are the outcome of particular procedures and operations on data; data is not visualised, but it is information – the outcome of these procedures – that is made perceptible and ‘seen’.

13 Colour, for example, became more prominent with the adoption of computers for design which allowed for millions of unique colours. Despite this, colour is not a significant quality of network visualisations because their focus is on discovering spatial variables, which are not known a priori.

14 Manovich provides a number of examples from the humanities that he notes involve different degrees of reduction in data objects.

15 Adrian Mackenzie raised this in his review of an earlier version of this chapter. While I take his point, there is a convergence of interest in visualisations as a method of analysing digital data, from that of data-driven art to that of data-driven science. It is this convergence that I think calls for investigating how visualisations are comparatively being done in the arts and social sciences.

16 The (x)trees are presented in different versions and cities. One version was first on display at the Albuquerque Museum of Art and History from September 2012 to January 2013, and more recently, on 26 September 2014, another version was displayed on the streets of Taos, New Mexico: <http://www.agneschavez.com/xtreeproject/projections>. The installation is best experienced by watching the video clips.

17 See <https://agneschavez.see.me/>. Permission to reproduce this image courtesy of the artist: (x)trees v.2. Data Visualization Projection: Agnes Chavez and Alessandro Saccoia, 2012 [accessed 14 March 2015].

18 This is a point Guggenheim (2013) makes about claims that visual mediums such as photographs are more subjective than objective and thus compensate for what is not rendered by other mediums such as text.

19 This includes interactive visualisations; some specific and innovative examples include controversy and issue mapping; see the work of Marres (2013) and Venturini (2010).

20 I take this from van de Port’s discussion in this collection of the collective as a sensation, not a representation or of being representable.

21 The artist provides very little in the way of such an interpretation and I have been unable to find any produced by others. Minimal text and explanation are provided, perhaps reasserting the emphasis on experiencing data.

Bibliography

Adkins, L., and C. Lury, ‘Introduction to Special Issue “What Is the Empirical”’, European Journal of Social Theory, 12 (2009), 5–20

Back, L., ‘Live Sociology: Social Research and Its Futures’, The Sociological Review, 60 (2012), 18–39

Barad, K., Meeting the Universe Halfway: Quantum Physics and the Entanglement of Matter and Meaning (Durham, NC, and London: Duke University Press, 2007)

Becker, H., Telling About Society (Chicago: University of Chicago Press, 2007)

Beer, D., and R. Burrows, ‘Popular Culture, Digital Archives and the New Social Life of Data’, Theory, Culture & Society, 30.4 (2013), 47–71

Candea, M., ‘The Fieldsite as Device’, Journal of Cultural Economy, 6.3 (2013), 241–258

Coleman, R., and J. Ringrose, eds., Deleuze and Research Methodologies (Edinburgh: Edinburgh University Press, 2013)

Gabrys, J., ‘Sensing an Experimental Forest: Processing Environments and Distributing Relations’, Computational Culture, (2012)

Guggenheim, M., ‘What Was Visual Sociology?’, in CSISPOnline (Goldsmiths: 2013)

Kanngieser, A., ‘A Sonic Geography of Voice: Towards an Affective Politics’, Progress in Human Geography, 36.3 (2012), 336–353

Law, J., E. Ruppert, and M. Savage, ‘The Double Social Life of Methods’, in CRESC Working Paper Series, Paper No. 95, 2011

Law, J., and V. Singleton, ‘Devices as Rituals: Notes on Enacting Resistance’, Journal of Cultural Economy, 6.3 (2013), 259–277

Law, J., and J. Urry, ‘Enacting the Social’, Economy and Society, 33.3 (2004), 390–410

Lury, C., ‘Going Live: Towards an Amphibious Sociology’, The Sociological Review, 60.S1 (2012), 184–197

Lury, C., and N. Wakeford, eds., Inventive Methods: The Happening of the Social (London: Routledge, 2012)

Mackenzie, A., Wirelessness: Radical Empiricism in Network Cultures (Boston: MIT Press, 2010)

Madsen, A. K., ‘Web-Visions: Repurposing Digital Traces to Organize Social Attention’, PhD Thesis, Copenhagen Business School, 2013

Manovich, L., ‘What Is Visualisation?’, Visual Studies, 26.1 (2011), 36–49

Marres, N., and E. Weltevrede, ‘Scraping the Social? Issues in Live Social Research’, Journal of Cultural Economy, 6.3 (2013), 313–335

Munster, A., An Aesthesia of Networks: Conjunctive Experience in Art and Technology (Boston: MIT Press, 2013)

Oxford English Dictionary [online], (Oxford: Oxford University Press, 2013) <www.oed.com.catalogue.ulrls.lon.ac.uk/view/Entry/175955?rskey=XKORSF&result=2> [accessed 01 March 2014]

Pink, S., Doing Sensory Ethnography (London: Sage, 2009)

Puwar, N., ‘Noise of the Past: Spatial Interruptions of War, Nation, and Memory’, Senses and Society, 6.3 (2011), 325–345

Puwar, N., and S. Sharma, ‘Introduction: War Cries’, Senses and Society, 6.3 (2011), 261–266

Rhys-Taylor, A., ‘Coming to Our Senses: A Multi-Sensory Ethnography of Class and Multiculture in East London’, PhD Thesis, Goldsmiths University, 2010

Rogers, R., Digital Methods (Boston: MIT Press, 2013)

Ruppert, E., ‘Becoming Peoples: “Counting Heads in Northern Wilds”’, Journal of Cultural Economy, 2.1.2 (2009), 11–31

Ruppert, E., J. Law, and M. Savage, ‘Reassembling Social Science Methods: The Challenge of Digital Devices’, Theory, Culture & Society, Special Issue on the Social Life of Methods, 30.4 (2013), 22–46

St. Pierre, E. A. (1997). ‘Methodology in the Fold and the Eruption of Transgressive Data’, International Journal of Qualitative Studies in Education, 10 (2), 175–189

Savage, M., and Roger Burrows, ‘The Coming Crisis of Empirical Sociology’, Sociology, 41.5 (2007), 885–899

Stewart, K., Ordinary Affects (Durham, NC, and London: Duke University Press, 2007)

Venturini, T., ‘Building on Faults: How to Represent Controversies with Digital Methods’, Public Understanding of Science, iFirst (2010), 1–17

Watts, D. J., ‘A Twenty-First Century Science’, Nature, 445 (2007), 489