Tactical Tech

Adventures in Mapping Digital Empires

26 minutes read | First published: March 17, 2023
home-2x.jpeg

Marek Tuszynski, January 2023.

Google Empire 2016

Google Empire 2016, Tactical Tech & LaLoma, The Glass Room New York produced by Mozilla (1)

What you will learn here:

  • What it takes to better understand and visualise Big Tech's impact on society.
  • How to tell a story when the data is unreliable and fragmented because these companies are not required to report details of their acquisitions, mergers and investments to the public.
  • How to think about narratives and formats that engage and stimulate critical thinking.
  • What we mean by provocation design, and see it for yourself.
  • You will also read a fragment of a great poem at the very end.

The Simple Task of Mapping an Empire

One way to map an empire – a non-digital one, to begin with – is to create a map so detailed that it would be larger than the empire itself. In which case, the usefulness of such a map would be rather questionable. Still, the idea of creating a map like this belongs in the category of pure speculation, based on the premise that one would have total access to all possible information to create it and to process such an improbable amount of data. It is so easy to debunk this notion it took Jorge Luis Borges a paragraph to describe it in his 1946 "On Exactitude in Science." (2) Yet the idea of Total Information Awareness still prevails (3), from the national security services to Silicon Valley and Shenzhen. The idea can be summed up in one sentence: If only we could collect all the information out there, we could prevent what needs to be prevented and predict what needs to be predicted – while making piles of profit and living a peaceful and beautiful life in the name of preventing terror.

When you map an empire from a disadvantaged position, you have a situation that is a bit like using binoculars the other way around. The empire is looking at us (the subjects) through a ridiculously large telescope, seeing all the little details, while we are looking at it from the other side, obviously seeing it at an impossible distance, with minimal detail, even further away than it actually is...(4) But perhaps this disadvantage can be turned into a useful method.

So what are you left with when you try to map the empire of the five Big Tech companies, one of them, or other companies that benefit from the sophisticated digital infrastructure that the big five have amassed and control? You are left trying.

Before I describe some of it, let's first clarify why we call the Google (Alphabet) empire, or GAFAM (an acronym for Google, Amazon, Facebook, Apple and Microsoft).

“Our understanding of empire here is very basic: empire controls three basic layers of knowledge: 1) collection of data/intelligence, 2) storage of that intelligence, as well as 3) capacity of processing it. These three unique characteristics give them powerful and unprecedented insights into how our societies operate.” (05)

It's somewhat arbitrary to use the term “empire” at this point, but let's stick with it for a while, just for the sake of argument. The same goes for the use of the term “knowledge” or ”intelligence,” but isn't that what collected data becomes when used to train various machine-learning and AI models?

After all, we are in the business of provocation design (06). This is what we do here - we turn limited data, corrupted information and grey areas into engaging and stimulating visual prompts for critical thinking.

The idea of mapping the unprecedented influence of big tech companies, using all the methods available, including licencia poetica (07), is to try to demystify, on the one hand, who these companies are and, on the other hand, the powerful tools they have at their disposal. These are tools they use to influence how we perceive what is important, how we understand, for example, what a crisis is through the lens of technology, or simply how we are going to vote in the next election.

We have an interesting history of many fragile, semi-confusing, but amusing and successful attempts at such mapping. We are a tiny fish in this vast ocean. But we have our tools and our ideas. We are persistently putting together different narratives to engage different audiences in different ways with a subject that is often seen as slightly over the top ("it's the market, stupid"), mostly boring ("another network map, ugh"), and at best too technical ("data?") and at worst too politically biased ("you just hate corporations, don't you?").

Moreover, mapping what we can see against what we cannot see can be beautiful at first, but in the long run, it can also risk representing truly problematic views. Let me take an old example, Edward Quin’s Maps of Empire (08). Here are three examples from his original atlas:

Edward Quin, Maps of Empire, 1830 Edward Quin, Maps of Empire, 1830 (here Alexander, Constantine and Discovery of America) images after David Rumsey Historical Map Collection - https://www.davidrumsey.com

In Quin’s case, mapping is presented as a rather romantic candlelight that, over time, dispels the cloud of ignorance, showing the progress of knowledge directly related to the power of cartographers representing a very specific imperial viewpoint. First, it is assumed that expansion equals knowledge - as if the territories unknown to the cartographers and their inhabitants did not exist before. The act of cartography is subjective and political (09). The other thing we learn from this example is that visualising is associated with immediate discovery for the viewer. It is a very empowering act. In Quin's case, it ended with reprinting his atlas, omitting all his writing in favour of just the images in many future atlas editions.

We like to see things in linear progress, from darkness to light, from not knowing to enlightenment, etc. But there is something else to learn from this example, and that is how much we can learn from so-called negative space - not from what is visible, but from what is bluntly invisible. One of these things is the bias and limited perspective of the mapmaker himself. It is crucial to be aware of this limitation. And perhaps the final and somewhat exaggerated lesson from this example is that no empire should map itself.

The Growing Pains of Mapping an Empire

Let's go back to our journey through our various attempts to explore how to visualise and represent the unprecedented position that the big tech companies have gained in recent decades in terms of their impact on our society. We explored how to tell the story using different data sets, which data sets to choose and how to present them in an interesting way.

We started with the obvious - the “follow the money” mantra. Why follow the money (10)? Because when money is present, there are transactions; when there are transactions, there are values (not only monetary), and there is power. So when we talk about Big Tech, we, in fact, talk about a sector that processes lots of money, more than we can actually imagine:

  • GAFAM’s market capitalisation counts in the trillions (11)
  • GAFAM is made out of the most profitable businesses in existence (12)
  • GAFAM holds a massive number of patents (13)
  • GAFAM includes the most valued brands worldwide (14)
  • GAFAM’s owners and CEOs are some of the wealthiest people in the world. (15) and there is more... (16)

Since we know how much money these companies make, and we somehow know how they spend it, we could make some observations here. It turns out that neither is that obvious, and both sets of data (profit, money spent on investment and acquisitions) are riddled with corporate PR, copy/paste journalism and a lack of corporate transparency. But we gave it a go.

Google Empire 2016, Tactical Tech & LaLoma Google Empire 2016, Tactical Tech & LaLoma – photo Andrea Figari (left) & Daniel Desiderio Paez Castillo (right), as exhibited at the exhibition “Nervous Systems“ at HKW Berlin. (17)

The above is our first modest 3D visualisation from 2016, based on a dataset of about 180 companies that Google acquired and 400 companies that they have invested in since 1998. What we learnt very quickly compiling this dataset was that it needed to be more reliable. It was very hard to verify the numbers we had collected; some information contradicted our initial research, and because it was much easier to look at acquisitions than investments, it was impossible to verify accuracy. We decided to simplify (the above-mentioned licencia poetica) the visual representation. It is basically a good-looking timeline where both acquisitions and investments are represented as two types of bars. While the size of the bar roughly represents the amount of money, the colour shows what kind of category a given transaction belongs to in terms of its primary field of business (such as education, robotics, etc.). We made an arbitrary decision that in cases where we had conflicting numbers (actual money spent or invested), we would average them while making it clear in the description. It was essential to show that Google is more than what we think and definitely more than the sum of its parts.

Soon after that exhibition, we were about to show our work in New York and wanted to make a better impression there. We came up with another version based on the same dataset.

Google Empire, 2016 Tactical Tech & LaLoma Google Empire, 2016 Tactical Tech & LaLoma, New York, Glass Room exhibition produced by Mozilla, photo Tactical Tech

This time we decided to go for a network map made of nails and coloured threads, where each investment or acquisition is represented by the dot (nail), and the size of the circle represents the money spent/invested. The colour of the thread indicates the category of business, similar to the previous version. We thought this was a better representation of the size and scope of these nearly 600 deals, and we could not resist embedding Eric Schmidt's infamous quote, "We know where you are" (18) into the data representation itself. The main reason for moving away from the previous version was that it did not do justice to the scale and scope of these operations, and it continued to alienate - rather than embrace - viewers.

Google Empire, 2016 Tactical Tech & LaLoma Fragment, Google Empire at the Glass Room London 2017, photo David Mirzoeff. Fragment, ibidem, photo author

Google Empire in Glass Room New York 2016 Google Empire in Glass Room New York 2016, visitors exploring the data set, photo Nousha Salimi

Google Empire in Warsaw 2017 Google Empire in Warsaw 2017, a visitor using a magnifying glass, photo Alistair Alexander

It was quite a popular piece. Many people who saw it were surprised by the scale and depth of Google's acquisition and investment power, so we decided to make a portable and more participatory version of it in the style of a connect-the-dots game, which we are still showing around the world through our work with many partners.

Alphabet Empire, part of the Glass Room Community Edition Alphabet Empire, part of the Glass Room Community Edition, Tactical Tech & LaLoma 2017

Alphabet Empire in action, ibidem, photo Alphabet Empire in action, ibidem, photo Nikolione, 2017. Participants interacting with the visualisation, Ghana, photo Safa Ghnaim, 2018

It is central to our practice to work closely with others in presenting such projects and exhibitions worldwide. This part of our practice – working with partners to enable small, local exhibitions, sharing our methods and results and opening them up for adaptation, localisation and remixing – is crucial and, indeed, the most impactful. Being part of the enrichment of public knowledge through our work with libraries, schools, hackerspaces, maker spaces, and community centres is also very rewarding. In the last two years, we have co-organised almost 500 events in 61 countries, with over 350,000 participants. (19)

At Cairotronica20 - Cairo Electronic and Media Art Festival 2021 At Cairotronica (20) - Cairo Electronic and Media Art Festival 2021 photo -Ahmad AlNemr

Coda Festival2221, Sao Paulo Coda Festival (21), Sao Paulo, Photo Open Knowledge Brasil 2022

We can say we've exhibited these works quite often, but when we were working on our next big Glass Room exhibition, in San Francisco (22) at the end of 2019, we decided to prepare a new version of the Google piece. We didn't think the scale or the amounts would make an impression in the heart of Silicon Valley. So we set out to create what we called “Google and You”. (23) We updated the datasets from 2016/17 and 2018 to the most recent dataset from the end of 2019. This time we excluded investments and decided to go exclusively with all known acquisitions. We ended up presenting these as a set of 250 products and services owned by Google.

Google and You, Tactical Tech & LaLoma Google and You, Tactical Tech & LaLoma, 2019, photo author

The idea was very simple. We decided first to show the ten most used services offered by Google (we still like to call Alphabet “Google”). We presented them as rotating plates stacked on top of each other; each plate had a dotted pattern representing the known number of current users of that service. You would think this would be easy to find out - it is not. Google sometimes gives some figures, and journalists report slightly different ones. Again, there is no transparency about how many users a particular service has, either in total or how many log on or use it each month. We chose only those that exceeded the magic number of 1 billion or more, starting with Android, which at the time had almost 3 billion users worldwide. Unlike the previous objects and visualisations, we did not want to show Google as a powerful company just because it can afford to invest and spend a lot of money to acquire other companies (which also means acquisition, talent, skills, infrastructure, code, ideas and maybe sometimes competition). Rather, we wanted to show how much we depend on its services and solutions, its hardware and software. And to show that there is much more to Google than the ten most used applications.

Google and You, mock-up of the object

Google and You, mock-up of the object, LaLoma, 2019

The object was quite complicated because the elements rotated. We wanted it to look abstractly like a series of stacked Google lenses, each with a different focus (aka the number of users), hovering over the global number of internet users (24), represented as transparent mini people at the bottom of the stack, each representing 10 000,000 internauts.

This may have been the last time we produce such a fragile object. People loved rotating the lenses and the outer ring with 250 services. Somehow, at the end of the day, tens of the plastic people would be broken off the base plate. For the 21 days of the exhibition, I would come there early with my coffee and a tube of Super Glue and spend 30 minutes glueing them back on day after day.

Google and You close-up, ibidem, picture author Google and You close-up, ibidem, picture author

There was one thing that worked with this semi-interactive 3D visualisation: It brought together three parts - the staggering number of Google users, the incredible advertising revenue and the company's buying power - all growing together with the unique dependency between the majority of the world's internet users and the company. At the same time, The San Francisco Glass Room exhibition won us the Creative Review Annual Award. (25)\ We were ready for the next challenge: most of our public activities were cancelled at the beginning of 2020, except for our collaboration with FERS (26) and a series of exhibitions in Dutch libraries. (27) For this occasion, we rolled up our sleeves and concentrated on producing a piece called Google Society, a better version of Google and You.

Google Society - Tactical Tech & LaLoma, 2020 Google Society - Tactical Tech & LaLoma, 2020, photo author, exhibited in Leeuwarden 2021

Google Society - Tactical Tech & LaLoma, 2020 Google Society-Tactical Tech & LaLoma, 2020, photo author, exhibited in Leeuwarden 2021

We updated the dataset again and rearranged the object, which looks like a modern version of a camera obscura (28) or a giant artificial jellyfish, ready for the digital world to conquer. On one side, we have the user's view in the form of an idealised mobile phone surface, with its shiny mirrored surface with the ten most popular apps on it. On the opposite side, we have the Google eye, where its retina is made up of all sorts of personal data collected as it peers through the web of connections between the hundreds of services it offers. The side view gives more space to contemplate the tangled web of interdependencies among all of Google's services.

The backdrop remained the same, as in the images below:

Google Society - Tactical Tech & LaLoma, 2020 Google Society, fragments of the visualisation key from the baseplate explaining all the elements and data used to produce the piece. We counter-positioned the sophisticated network of apps and services owned by Google (3) against what we users know Google for (10 most popular services 1 and 5), what sort of personal data it has access to (2) and the up-to-date advertising revenue (4).

The object toured the Netherlands and was shown at two major exhibitions, at MOD Adelaide (29) and DigiDic in St. Pölten (30) in Austria. At the same time, we produced a poster version of this object - to make it more accessible and easier to translate and adapt - as well as a series of smaller posters developed with young people as part of our What the Future Wants initiative.

Google Society poster3332, Tactical Tech & LaLoma, 2021. One of the activity posters for the What the Future Wants project Google Society poster (31), Tactical Tech & LaLoma, 2021. One of the activity posters for the What the Future Wants project co-developed with young people, this work is based on the Google Society piece, Tactical Tech (Bernadette Geiger and Yiorgos Bagakis) 2022. (32)

Google Society an outdoor banner version in Lithuanian Google Society an outdoor banner version in Lithuanian, Tactical Tech & LaLoma 2022

Should We Take a Look at It All One More Time?

As we launched hundreds of events and exhibitions, we returned to the original dataset - because we were not satisfied that virtually nobody beyond academia was talking enough about all the acquisitions by the Big Tech companies, and the pandemic had only accelerated the practice (33).

Little did we know that we were not the only ones thinking about this. Practically three weeks before our original idea of launching GAFAM Empire on 21 April 2021, the Washington Post published their “How Big Tech Got So Big: Hundreds of Acquisitions35.” (34) There were far too many similarities between their work and ours that we decided to postpone and rethink our approach, as well as see where we could add something the Washington Post piece did not have.

Initial data analysis of our GAFAM empire from early 2021 Initial data analysis of our GAFAM empire from early 2021, research Patrick Harvey, Data Analysis and categorisation Basile Simon (35).

Their piece focused on two aspects of big tech acquisitions - first, that they are different from what they were when they started. Second, that they get big as Company A, and then they spend the money on acquisitions and become Company B. For example, Amazon started as an online bookstore and became a cloud computing powerhouse, but to everyday users, they are mostly known as a retail company. The data team at the Washington Post did some really great visualisations. But they reduced it to a timeline. More importantly, they made it clear that their efforts, much like ours, were designed to help institutions and policymakers (the Federal Trade Commission and the Department of Justice, respectively) in their fight against these companies using their monopolistic positions to a) weaken competitors and b) strengthen their grip on particular sectors.

We returned to the drawing board and decided to wait and update the dataset and give it more nuance and depth.

One of the visualisations of GAFAM Empire 2022 One of the visualisations of GAFAM Empire 2022, Tactical Tech and Density Design - we called this part Petri dishes - where we observed changes in the landscape of acquisitions in 5-year intervals. (36)

The first thing we focused on, and what makes our work very different from that of the Washington Post, was the problem of categorising the 1,210 acquisitions we were looking at. We quickly learnt that this was purely arbitrary, not very precise and often more PR than anything else - companies acquired might be described as just "software". In contrast, on closer inspection, we could say they were "developers of mobile recognition engines for the mobile phone and consumer electronics industries, and more." We dug wherever we could: press, reports, company filings, and so on. This allowed us to make much more specific observations about what these acquisitions were really about - although still with many limitations - as it is not possible to really determine why these acquisitions were made. Was the company acquiring some expertise, code, patents, ideas, talent, user base, data - some or all of it, or simply neutralising a potential competitor?

The next thing we decided to look at, which was part of the inspiring Washington Post investigation, is what we can learn about these companies through their acquisition strategies.

And, of course, our findings also overlap with the Washington Post. Still, in the case of Amazon, for example, their focus is on Amazon's transition from an e-commerce giant to a cloud computing "landlord." What our observations add is that they are also the most significant player in automation and robotics. In the case of Google, the Washington Post observed that they transitioned from an advertising giant to a dominant force in mobile and artificial intelligence and digital mapping. Our observations also show Google's strategic dominance in education and health.

GAFAM Empire – fragment of Microsoft acquisitions GAFAM Empire – a fragment of Microsoft acquisitions – LinkedIn, Tactical Tech & Density Design 2022 (37).

We also spent significant time trying to decipher their business strategies, looking not only at the actual acquisition in focus - as in the case above of Microsoft buying LinkedIn - but also scrutinising what companies like LinkedIn had bought before becoming part of Microsoft's portfolio, as well as what companies they acquired after being owned by Microsoft. In fact, researchers at the Transnational Institute took note and looked at Meta's series of purchases made via the acquisition of Oculus and how they expanded their reach into VR technologies (38). There are many cases of such complex acquisitions by all of the Big Five.

Finally, the last thing that made it a big bonus for researchers and policymakers is that we published all the data in our Methodology section for scrutiny and use (39).

And this is not the end of our mapping adventures - we are now working on simplified versions of the GAFAM Empire, another poster, an animation and...

Whatever we do next, we could do a lot better (or more generally, we as policymakers, researchers and institutions could do a lot better in terms of better understanding the impact of these companies at large) if a few basic things were available, such as:

  • detailed financial reports from companies that are useful to regulators and researchers (in particular, disclosing the specific purposes of acquisitions and investments, with business plans describing the intended use of acquired assets)
  • detailed non-financial reports explaining the purposes of acquisitions, including inventories listing what has been acquired
  • detailed information about monetisation metrics – it is essential to know the money-making strategies and tools used by companies (what is specifically being monetised for what value? The infamous Zuckerberg classic, "Senator, we run ads" (40) is not enough, never was) (41).

It may sound too much - and perhaps it would be if these corporations were not dealing with the fragile structure of our society, where everything we do is mediated through their dominant and far-from-neutral infrastructures. It also assumes, as we said at the beginning, that all we need is more data, more information. To some extent, it would certainly help to define new rules of the game (new policies, regulations, restrictions, rights and responsibilities). Once again, the map becomes larger than the empire that looms around the corner. The point we are making is that mapping as a process is crucial to finding ways of understanding and measuring power relations. It also helps us to get a basic orientation - where are we in this? And more importantly, all maps are temporary - by the time they are finished, they are already out of date. So it is fascinating to look at the processes and methods behind the maps while enjoying their visual representation.

Meanwhile - we will keep mapping. To put it simply, we like maps and mapping, perhaps awkwardly, for the same reason as the poet Wislawa Szymborska does:

“(…) I like maps, because they lie. Because they give no access to the vicious truth. Because great-heartedly, good-naturedly they spread before me a world not of this world.” (42)

__________________________________________________________

FOOTNOTES