2

Centralised architectures as informal standards for ‘control by design’

This chapter is the first of a series, constituting the central part of the book, that is best read as a triptych. Together, these three chapters are meant to provide an analysis of different architectural choices and their impact on the configuration – social and economic as well as technical – of encrypted messaging tools. Evaluating the consequences of these choices is important, in our view, for the reasons expressed in the introductory discussion of the contributions social sciences can make to the study of encryption and decentralisation. We saw some of this at work in Chapter 1: every technical choice made by developers of secure messaging tools is not about reaching a perfect model, but about implementing, in practice, a series of compromises that are about technical performance, but also about community-building, standard-setting and finding suitable governance and organisational models. It is also about business models and reaching specific publics. Illustrating this interweaving of dynamics has been the core aim of this triptych of chapters.

Furthermore, we understand our review of the three types of architecture as a discussion of different ways to practise specific visions of online freedom: freedom to control the implementation of the technical solution, freedom proposed to users to choose their infrastructure to a more or less large extent, freedom to do without intermediaries or to configure them in specific ways, etc. Indeed, throughout these chapters, the overarching, central concept for this book – ‘concealing for freedom’ – is fleshed out, not as a fixed value, but as something that is performed differently in different settings and defined by the field(s) within which it operates.

This chapter discusses centralisation, as defined in the Introduction, as a ‘control by design’ model for secure messaging protocols and applications. In particular, we discuss the dynamics of centralisation-as-control deployed as a means to respond quickly to technical challenges in a situation of uncertainty. To do so, we explore Signal as a central case study, but we also touch upon the cases of Telegram and Wire. This chapter shows how ‘concealing for freedom’ dynamics unfold for both developers and users in the case of centralised architectures. While centralised tools can offer users suitable solutions to protect their freedom ‘from’ particular adversaries, their stance vis-à-vis active freedoms, such as the freedom ‘of’ choice – of particular incarnations of a protocol, of specific interface elements – is much more problematic. On the developers’ side, centralised architectures can restrict the freedom ‘to’ reuse and implement code in different applications subtending the same protocol (Nyman and Lindman 2013). As Nyman (2015: 116) points out, quoting Raymond (1999), closed source licensing, made easier by centralised arrangements, erects a number of artificial technical, legal or institutional barriers that prevent ‘good solutions from being re-used’. In this case, a certain level of standardisation – be it formal or informal, as we will analyse below – can become a factor of freedom for developers.

As seminal secure messaging protocols such as PGP and OTR started showing their age in terms of security and usability, cryptographers, unsurprisingly, wanted to develop new and better protocols in the wake of the Snowden revelations. Open-source developers started to deploy efforts to create a next-generation secure messaging protocol. The most advanced and popular protocol in this regard is the Signal protocol, used by applications such as Signal (formerly TextSecure).

The Signal protocol is now considered the ‘best practice’ in the field of encrypted messaging and has become a trendsetter for other projects in terms of privacy and security features. Currently, Signal is centralised, as a single server mediates the setup of the protocol in most widespread deployments (e.g. WhatsApp, Google Allo, Facebook Messenger, Wire). A new means of ‘quasi-standardisation’ or ‘standardisation by running code’ is being practised around this protocol. In this process, a quasi-standard is defined as ‘something that works’ and that has been iterated and redeployed by others. Centralisation has allowed Signal developers to swiftly update the protocol in response to research and bugs, and to limit concerns about the technical competence of third-party developers standardising their protocol. However, the ‘bottleneck’ of centralisation also implies that difficulties and tensions have arisen in connection with some attempts to reimplement the Signal protocol, due to the lack of specifications. This chapter will examine how these attempts to ‘control by design’ play out in the context of the development of secure messaging’s foremost protocols and applications, paying particular attention to attempts at informal standardisation as a way to ‘open up’ the protocol.1

From Axolotl to Signal: Between innovation and continuity

As we mentioned in the Introduction to this book, many actors in the field of secure messaging share a tacit agreement that the Signal protocol’s Double Ratchet is currently the leading protocol for instant messaging. In order to better understand how the Signal protocol interacts with previous and potential future developments of encryption protocols, it is useful to recall briefly the historical debates around two major protocols that dominated the ecosystem for many years before Snowden: OTR and PGP.

The main problems with PGP discussed in the cryptographic community, and shared by advanced users, could be summarised according to two main issues: complex key management and lack of repudiation and forward secrecy. The crisis of ‘public key infrastructures’ and of the very concept of cryptographic keys and signatures was also highlighted by digital security trainer communities and international NGOs promoting privacy-enhancing technologies, such as Tactical Tech and the Electronic Frontier Foundation (Musiani and Ermoshina 2017). While still offering one of the most cryptographically robust solutions, PGP clearly faces usability challenges.

The OTR protocol started as a research project at UC Berkeley in 2002 and was first released in 2004. OTR developer Ian Goldberg described himself, in our interview with him, as a very early PGP user and mostly an ‘email person’; it was his student, Nikita Borisov, who drew Goldberg’s attention to the growing field of social networking and instant messaging, thus identifying a new security gap to be filled. Ian’s description of the mission behind OTR relates this protocol to pre-existing technologies, as a way to respond to challenges that were not properly addressed by PGP:

Your choices at that time were either completely unprotected communication neither encrypted nor authenticated, or PGP in which case it’s confidential and authenticated unless your key leaks in which it’s not confidential and the authentication with digital signatures leads to non-repudiation (Interview with Ian Goldberg, May 2018).

While OTR’s solution of using per-conversation keys offered good repudiation and forward secrecy, it did not permit group chat encryption as, in Goldberg’s words, ‘that was how instant communication worked back then: you had to be online at the same time’: OTR’s design was inspired by existing tools that required synchrony (e.g. Aim). However, while popular in high-risk activist communities, OTR-based applications (such as Jabber) were widely criticised for their lack of multi-device support and other usability problems.

These shared concerns were at the core of the efforts subtending the Signal protocol, which successfully managed to address them, according to Goldberg: ‘Text Secure, and later Signal, basically took OTR protocol and added basic features to it to make it work in an asynchronous setting’. Using per-conversation key material in a similar manner to OTR, it did not force complex key management on users. It maintained properties of repudiation and forward secrecy by virtue of the Axolotl Diffie-Hellman key ratchet, but added ‘future secrecy’ so that messages could not be read at any point in the future in the case of a key material compromise (Cohn-Gordon et al. 2016). It also solved the asynchronous messaging problem by allowing longer-term pre-keys managed by the Signal server and offered group messaging implemented as point-to-point messaging. The Signal protocol uses the ‘Axolotl’ key ratchet for initial key exchange and the ongoing renewal and maintenance of short-lived session keys, so there is no long-term key that can be compromised. This provides forward secrecy so that the compromise of a short-term key does not compromise past keys (which would enable an adversary to decrypt past messages).

As the Signal protocol initiated a dialogue with the previous crypto protocol ‘tradition’ (mainly by addressing the aforementioned limits of OTR and PGP), it quickly attracted the attention of the academic cryptographic community, and only minor flaws were found (Frosch et al. 2016). Although alternative approaches were developed and widely deployed, like MTProto by Telegram, these protocols developed their own cryptographic primitives and so received less attention from the academic community; additionally, a number of bugs and usability problems were revealed (Jakobsen and Orlandi 2016; Abu-Salma et al. 2017a). Interestingly, while Signal has deeply influenced the crypto protocol field, it did not depart from previous efforts, but drew from their well-known flaws: it is this continuity that can partly account for the interest from crypto experts.

With minor variants implemented in the vastly popular WhatsApp messenger, the core Signal protocol seems well on its way to replacing the use of XMPP+OTR and to becoming a competitive, if somewhat ‘boutique’, feature for mainstream messaging services (as shown by the adoption of the Signal protocol as an option by both Google Allo and Facebook Messenger). Encrypted messaging applications like WhatsApp, Telegram, and Signal2 are now the default application of this kind for users who consider themselves high risk. Usability studies have shown that although Signal, like OTR, is relatively easy to set up and use, even highly skilled users fail to use verification correctly. Signal is centralised, as a single server mediates the setup of the protocol in most widespread deployments (WhatsApp, Google Allo, Facebook Messenger, Wire).

There are also open-source alternatives that claim to use the Signal protocol or its forks, such as Wire, another centralised application, that uses a fork of Axolotl protocol called Proteus. Parts of the Signal protocol were copied by a draft XMPP-Foundation standard called OMEMO, for use by applications such as Conversations and ChatSecure, which led to usage of Signal’s double ratchet in federated projects. Another decentralised project, called Matrix, has reused parts of the Signal protocol to integrate them into their own cryptographic library, called Olm. While Signal appears to be widely adopted and considered an improvement over both OTR and PGP, the core Signal protocol remains officially unstandardised, even though the protocol’s creators Trevor Perrin and Moxie Marlinspike have produced an informal draft after considerable ‘community pressure’ (in the words of Matrix.org’s lead developer).

A ‘quasi-standardisation’ process

While most users we have interviewed, including high-risk users, do not appear to have standardisation as an explicit priority, developers care deeply about standards as ‘something they would eventually be working on’, to increase the ‘dialogue’ between applications and reduce the silo effect: ‘In the long term I am not opposed to the idea of standardising, it’s great to have a reference for interoperability’ (Briar lead developer).

Standardisation is understood as a reference and thus as an important communication or mediation instrument, that helps members of the security community understand each other and build a ground for common knowledge (such as cryptographic libraries); it also guarantees a smoother development of new applications on top of standardised protocols. Yet a widespread discontent with existing standards bodies is expressed by developers, for several reasons. Developers underline recent transformations of these organisations, referring to a previous ‘golden age’ of standardising bodies, when their mode of existence was closer to that of FOSS communities. Our respondents note the growing importance of private actors as stakeholders within standardising bodies:

My impression of the IETF is that it’s not the same beast it was in the early days. There was a time when it was a group of enthusiastic people who would come to the IETF with an idea that was sort of halfway finished and they’d say look I wanna let everybody know about this, let’s knock it into shape and we’ll all build on it. I think it’s become a much slower moving and more adversarial environment. This area of technologies has attracted more money and more corporate participation and therefore, conflicts of interest (Briar lead developer).

This institutionalisation of standardising bodies and their progressive removal from coding communities creates an environment that is less suitable for experiments and unfinished projects:

I think that [an automated, periodical clearing out of message history] is something that we will implement, it just probably will not be standardised because the XMPP community is very conservative. I don’t think… they don’t fully get it. It’s something that users want… so why?… I don’t know. They end up in that old school stuff (ChatSecure developer).

As Callon (1986) would put it, standardisation implies the ‘translation’ of a protocol as a sociotechnical experiment into a pre-standard, able to ‘enroll’ and convince various agents within evaluation bodies. Standardisation involves collective work that opens up the core-set of protocol authors to include external experts from standardising organisations, some of them being far from users’ experiences and needs, and from the ‘real’ economy of the encrypted messaging field – a process that is hardly appealing to some developers, as it is seen as time-consuming in the early stages of project development:

I wouldn’t really think about submitting something to the IETF on an early stage these days because I think that would probably involve a lot of work to convince other people to allow it to become a standard… and obviously everybody would have their own thoughts of how better to work (Briar lead developer).

Instead, most developers share the philosophy that they would build the application first, and then focus on standardisation and decentralisation via the use of open standards:

I used to work with W3C a long time ago and I am very aware of how they work and that they may have some limitations. We want to get Matrix as mature enough and solid and stable enough, then we can pass it over to a proper governance organization but right now it’s still evolving very rapidly (Matrix.org lead developer).

In the case of secure messaging, it is still felt that more development is needed on the code, and standardisation would only slow down existing development efforts. Indeed, a new way of ‘quasi-standardisation’ or ‘standardisation by running code’ is being practised in the field of end-to-end encrypted messaging applications, around the Signal protocol. In this process, a quasi-standard is defined as something that has been previously iterated and redeployed by others with success. In this sense, all of the various Signal protocol deployments (e.g. Wire, WhatsApp and OMEMO-based apps such as Conversations and ChatSecure) work as crash-tests for the protocol, where the protocol gets forged by usage:

I think the direction to take is [the one] Signal is taking, when you kind of build a system for a while, you iterate a bit until you think you have something that works well and then you start to document it and if other people wanna interoperate, you talk to them about standardizing on that stage, on a much later stage […] I wonder whether they [Signal] will think about standardizing at some point, maybe the idea is about delaying it to a later stage of the project, and not about avoiding it (Briar lead developer).

The Signal protocol, characterised by the double Diffie-Hellman ratchet, is now considered the best practice in the field and has become a trend-setter for other projects in terms of privacy and security features (e.g. forward secrecy and future secrecy for example). Developers, even those working in federated (e.g. Conversations) or peer-to-peer (e.g. Briar) projects – which will be explored in Chapters 3 and 4 – see the Signal solution as one of the best designs available, despite its lack of formal standardisation – and, maybe, precisely because its standardisation is happening informally, in the ‘mundane practices’ and multiple implementations taking place in different projects.

Signal’s multiple implementations: ‘Snowden meets the market’

The field of instant messaging applications has been deeply transformed by the different implementations of the Signal protocol, but also because of the growing popularity of other secure messaging tools, such as Telegram, Threema or Wickr that use their own protocols. The turn to encryption has modified the market and brought considerable changes at the governance level, engaging important private sector players in the game:

What is happening in the last two years [interview was done in 2016] is fantastic, with a number of messengers popping up and also greater publicity around Axolotl or Signal… Snowden also talking about it… So this is really good for the industry. And we’ve seen it’s triggered even the big ones who started using encryption (Alan, Wire CTO).

Post-Snowden, one of the most important manifestations of the ‘turn to encryption’ in messaging applications has been WhatsApp’s – one of the ‘big ones’, in Alan’s words – implementation of end-to-end encryption, which became operational in April 2016 (Metz 2016). However, according to former Pirate Bay founder Peter Sunde – creator of Heml.is, an end-to-end encrypted IM application – the adoption of end-to-end encryption by WhatsApp happened for much more ‘financial’ reasons. Before being purchased by Facebook, WhatsApp’s value was estimated at 19 billion dollars. After an unfruitful attempt to purchase Sunde’s protocol, they turned to Signal and acquired the Signal protocol for 1 million dollars. After the implementation of end-to-end encryption in WhatsApp, the app was sold to Facebook for a better price of 21 billion dollars. Peter Sunde recalls:

[WhatsApp’s CEO] was upset that I and other guys [from Heml.is team] were talking about ideology and politics because he was not interested in politics. He was more interested in the government staying out of his life, and not touching him so much and so on. And he did not wanna call that ideology, which was probably the first problem we had between us. And then he said encryption is not something people care about because no one is interested in what people talk about on WhatsApp because there was nothing important. For him it was more like we need to have something because people are complaining in the media, rather than it’s actually about privacy… It was just a business thing. He has no clue what people are using his app for… I don’t think he was in touch with his user base, he was more interested in user growth (Peter Sunde, Heml.is core developer).

The element of ‘political ideology’ raised by Peter in the above citation is another important factor that influences interactions and collaborations among different instant messaging and cryptographic projects. Our fieldwork within developer communities shows that common ideological grounds may become a crucial factor for the decision whether to adopt specific protocol choices and design decisions. We will show later in the chapter how, in the case of Wire, the importance of specific values cement F/OSS communities, and their influence on fundamental development choices.

However, in the case of Signal’s cooperation with WhatsApp (and, as a consequence, with Facebook), reflections on the social consequences of the adoption of end-to-end encryption are used to justify this controversial deal, that became one of the most important collaborations in the recent history of instant messaging. Several of our respondents underline the ultimately positive consequences of WhatsApp’s turn to encryption, that, according to them, has solved major problems related to its adoption costs by users. Let us recall the Ukrainian security trainer pointing out, in the previous chapter, that since WhatsApp’s adoption of end-to-end encryption she recommended users to ‘stay with it’ if they already used it, and also high-risk users pointing out how a generalist, centralised application such as WhatsApp could in fact make activists less conspicuous than if they used an activist-targeted app.

Forking the Signal protocol: Non-standardisation as a business-model

One of the most well-known and popular forks of the Signal protocol is called Proteus and is used by the application Wire. Wire was launched by ex-Skype developers, who desired, according to its CTO, to respond to ‘one of the biggest gaps that was missing on the market, related to privacy and security’. Wire’s primary targeted user group is identified as privacy-aware consumers:

Our users in the first phase are primarily consumers, who care about privacy. […] I like to compare passive smoking with rising awareness about privacy when people understand how information can be misused. It’s something that needs to be spread more (Alan, Wire CTO).

As Wire is not aimed at activists or at a tech-savvy audience, but at the average user, one of its main concerns is to build a usable interface and integrate new features that will distinguish it from other end-to-end encrypted messengers. Thus, Wire supports drawings, GIF exchange, large end-to-end encrypted group chats, multiple-participant group video calls, disappearing timed messages, file transfer, and so on. A number of our interviewees have underlined the aesthetic aspect of Wire’s user interface as an advantage, favouring Wire’s widespread adoption as opposed to Signal. Another of Wire’s selling arguments is the quality and encryption of its voice calls, as it offers end-to-end encrypted voice calls using a specific protocol based on constant bit rate encoding.

The underlying Wire encryption protocol, called Proteus, is a fork of Axolotl with ‘pieces that were needed to have support for the multiple devices as a standalone’ [Wire CTO]. However, difficulties and tensions have been observed around Wire’s attempts to reimplement the Signal protocol. Some of these difficulties are connected with the lack of specifications – the set of documented requirements that a product must satisfy, often considered as a first form of standardisation.3 A draft specification4 was produced in 2016 – as our respondents explain, not without pressure from other developer communities:

OWS [Open Whisper Systems, the company managing Signal until 2018, now Signal Messenger LLC] did not prioritize standardizing [the protocol] both because it gave them flexibility to change it as well as allow it to be more valuable to them as intellectual property. However, they have just finished standardizing a lot of it, […] and I think to some extent that was because of the pressure coming from community like us (Matrix.org lead developer).

So, this lack of specification obliges developers to re-code from scratch sometimes using other programming languages:

There was, I would even say on purpose, not enough available documentation. But if you wanted to develop your own implementation, you were pushed or bullied that you have copied their implementation. […] I was very naive and went to Moxie last year in June and asked him to review our implementation and we would pay him very good money for that. Instead he said you can pay 1,5 million, and I will keep your binaries and will help you to get the implementation going. And then I was like… yeah, exactly… [A legal dispute happened.] And then it was just settled. He dropped his charges, we dropped our charges and we are using Axolotl the way we do and how we would like (Alan, Wire CTO).

One of ChatSecure’s developers explains this conflict as a consequence of a specific licensing politics that led to tampering and modifications in the legal terms and agreements between the Signal team and other implementers:

The Signal protocol is open source under the GPL, that means you can’t integrate it into a commercial product; that’s why OWS were getting large licensing agreements from Facebook and Google and WhatsApp to integrate without opening up all of their source code. Part of that was incompatibilities with GPL and AppStore specifically. So we needed to get some of the legal language exempted […] Moxie needs to protect his revenue. Part of his arguments with Wire was that they [Signal] hadn’t documented Signal protocol, so there was no open specification, so if you wanted to write a compatible reimplementation, you would have to read the source code which would create a derivative work, which would not allow you to use it commercially because he would argue he still has copyright of the majority of the work (ChatSecure developer).

The Signal developers, for their part, argue that they are concerned about the technical competence of having third-party developers standardise or deploy forks of their protocol (‘Moxie is a very good coder, and his standards are very high’). They are also preoccupied with maintaining the prerogative of updating the protocol rapidly enough in response to research and bugs. This makes it possible to use the non-standardisation of the protocol as part of Signal’s business model, where the expertise and specification necessary for a proper deployment of the protocol can be offered by the Signal team as a service:

You can say OK, we will license this technology, which is not something I am interested in because I would like it to remain free software. But you can also say ‘we are the people who understand this technology, it makes sense to hire us if you want to deploy it’. If people build systems on top of it, then they pay somebody to contribute changes down into that codebase (Briar lead developer).

As our interviews with users and observation of security training sessions have revealed, open source and licensing choices are less covered in sessions destined for high-risk users, as they do not always associate open source with security. Open source is often perceived as a less important criterion in the context of an immediate physical threat; when a proprietary, centralised, but ‘efficient’ and ‘easy to explain’ solution exists, trainers will give it priority. The primary task for trainers in high-risk contexts with low-knowledge users is to help them quickly abandon unencrypted tools as well as tools that collaborate with their adversaries. However, users do care about sources of funding and business models of end-to-end encrypted messaging applications, and particularly of centralised systems, due to the great authority the architecture accords to their creators. This was, and is, the case with Signal, as well; in particular, questions about business models are very frequent on the different chats about cybersecurity that we have been observing since September 2016. Users ask for transparency of funding but at the same time show a certain scepticism regarding crowdfunding models (donations) that seem not sustainable enough for an application to be properly maintained. Recent critiques addressed to Signal concern its dependence on US government funding:

Signal was created by the same spooky regime change outfits that fund the Tor Project. The money primarily comes through the federal government’s premier Internet Freedom venture capital outfit: Open Technology Fund, which works closely with the State Department’s regime change arm and is funded through several layers of Cold War CIA cutouts — including Radio Free Asia and the Broadcasting Board of Governors (Levine 2016).

Telegram creator Pavel Durov’s critique of Signal goes in the same direction, stressing that no US-government funded application can be trusted.

Beyond Signal’s ‘non-standardisation as a business model’, centralised end-to-end encrypted secure messaging applications propose different business models, though it seems that no ideal solution has yet been found. Wire’s project is to propose paid services to users for supplementary storage space (encrypted Cloud services). Wire also proposes business solutions for end-to-end encryption of the Internet of Things. Threema, one of the few end-to-end encrypted applications requiring a financial subscription from users, asks for a $2 contribution per user. Some users underline that as an advantage: ‘All of us use Threema […] I prefer to pay, at least I am sure that I am not the product’ (Cryptoparty organiser, male, Austria).

As in previous histories of software development, for Signal and the ecosystem of applications using variations of the Signal protocol, the licensing, business-model and open/closed source choices turn out to be complex sociotechnical processes that are embedded in both community-related interactions, economic context and legal arrangements.

Centralisation calls for centralisation? Signal’s ‘dependencies’ critiqued

Though the Signal protocol has enjoyed extensive attention from academics and has been formally verified, with only minor flaws found, several users and tech-savvy communities have critiqued the tool – critiques that were highlighted during our interviews. Interestingly, some of these critiques do not concern cryptographic properties of the protocol per se, but are focused on the application’s UI/UX, privacy features and the particular architectural design choices that bind Signal to other applications and protocols that are, in their turn, under scrutiny by particular communities because of their centralised and/or proprietary structures.

De-Googling Signal: The LibreSignal controversy

First of all, tech-savvy users – especially crypto trainers from low-risk countries such as Austria, Germany and France – have pointed out Signal’s dependence on Google services. They see it as a negative from the standpoints of both open-source community ethics and privacy, the two main points of controversy being Signal’s relationship with Google Cloud Messaging (GCM) services and its reliance on Google Play. As a consequence, two initiatives were launched in order to ‘decentralise’ or ‘de-Googlise’ Signal: The F-Droid repository,5 which aimed to distribute Signal outside the Google Play Store; and a Github repository, which aimed at completely removing the Google components of Signal, even from the server side. This second project was called ‘LibreSignal’. Open-source client used the Signal protocol and was lauded as ‘The truly private and Google-Free messenger for Android’.

LibreSignal’s authors stated on their GitHub page that the problem of Signal relying on GCM ‘is only an issue for people who use a custom Android ROM without Google Play Services. For the vast majority of people who do have Google Play on their phone, this issue is completely irrelevant’.6 Thus, the Google dependencies of Signal are problematic only for a very specific user community, both privacy-aware and tech-savvy (those who, like the mail service provider cited in Chapter 1, ‘do not like mainstream on [their] computer’), who opt for decentralised communication tools explicitly designed to be alternatives to those developed by the Silicon Valley tech giants. In this context, the choice of a ‘Google-free’ messenger can be perceived and defined as a ‘lifestyle’ choice. During our interviews, we noticed that this choice often coexists with alternative hardware choices (Linux phone, Fair Phone, CopperheadOS and others), or customised versions of the Android operating system, in particular versions free of Google Play.

LibreSignal was launched in 2015 but its development was stopped in November 2016 after a long online discussion between the LibreSignal team and Moxie Marlinspike. LibreSignal started as a fork of TextSecure,7 that several open-source contributors forked by integrating an alternative to Google Cloud Messaging, called WebSocket.

LibreSignal published a build on F-Droid, the community-maintained software repository for Android, similar to Google Play but containing only free and open-source apps. However, in the course of a discussion on the GitHub repository of the project, Moxie Marlinspike controversially demanded that the name of the fork be changed (not referring to TextSecure or Signal), and that it should not use Open Whisper System servers. He also insisted that no distribution of Signal via FDroid was possible.8 The debate between LibreSignal and OWS raised several issues that are important for the overall understanding of the sociotechnical and legal problems related to the debate about centralisation in secure messaging, and its relationship with projects focused on federated architectures (see also Chapter 4). A first argument concerned the server infrastructure needed for federated projects, which seemed complicated to set up and maintain, compared to the ‘forking’ of the client-side code. The latter does not, as the former does, demand coordination efforts between different agents: developers are able to fork and experiment on their own without having to care for the server infrastructure. However, the process of publishing a build on any of the app repositories – the open-source oriented F-Droid or the proprietary GooglePlay – demands a greater coordination and enrolment of a multitude of actors, as well as adherence to a number of legal and technical standards. This problem of federated protocols, as opposed to centralised ones, which we will come back to in Chapter 4, was also underlined in our interviews:

Obviously, I think that federation is good otherwise I wouldn’t be doing Conversations or using XMPP. But of course it’s more challenging to do. Because you have to coordinate with a lot of different vendor (ChatSecure developer).

In the case of Signal, the problems of forking and federating the protocol – as LibreSignal intended to do – turned out to be legal (disputes concerning the use of the Signal trademark), technical (the quality of LibreSignal builds, questioned by OWS), and financial (the costs of running servers).9 Ultimately, the enflamed debate about the LibreSignal GitHub page led Moxie Marlinspike to publish his standpoint on federation and centralisation in a blog post, ‘The Ecosystem is Moving’, which would become a landmark intervention in the controversy surrounding architectural choices in secure messaging (Marlinspike 2016). The Signal team was accused of not being ‘truly’ open source because of their refusal to draw upon open federated protocols. Comments such as ‘The problem is that Moxie and OWS don’t want Signal to be 100% free/libre software (it is not important for them…)’ or ‘They are not interested in removing the Google GCM dependencies […] Moxie has been quite explicit on this point, several times’ were commonplace.10

Starting from 21 February 2017, Signal could be used without dependencies upon Google services. As Elijah Sparrow, the creator of LEAP (see Chapter 4) explains, the long process of ‘degooglising’ Signal is related to an important technical problem, that of notifications, that demanded infrastructure investment:

LibreSignal was an attempt to avoid the Google services push. The problem with push on mobile is that it’s a very difficult problem to be able to push events to mobile devices without burning the battery and without setting up a complex infrastructure for that. So you can rely on Google infrastructure for that, it’s more efficient and they do it for free. But it’s controversial […] mostly because it requires to install all Google services on your device, which means you basically have to turn your device over to Google that always finds ways to gather your data so you couldn’t run Signal on a Google-free device. That’s recently changed, Signal very recently supports push notifications without Google services [although] Moxie still insists that no one should distribute the Signal application outside the Google Play market because he wants to get bug reports and data who’s running and how they are running it when it’s crushing (LEAP lead developer).

The problem of Google Cloud Messaging dependency and push notifications was addressed by other projects that found different solutions to this problem. For example, ChatSecure uses its own system for push notifications – a complex system, based on randomised tokens – that claims to have better privacy properties than the one used in Signal. For its part, Wire claims to be ‘pretty much Google-free’; however, it relies on another ‘net giant’, Amazon, for push notifications and analytics. Wire’s CTO describes the transition from Google services as a long evolution, marked by a dependency on Google’s ‘reputation’ privacy-wise:

[Being free of Google] took time… It wasn’t easy. As you know maybe, when we started Wire 3+ years ago, Google was already under suspicion but there were some people in the company who were Google fans, now they are completely disillusioned [, nonetheless] it took us some time to take away all of the Google stuff (Alan, Wire CTO).

‘ID’d by phone number’: The privacy weakness of centralised applications?

The second important controversy focused on Signal concerned the usage of phone numbers as identification mechanisms. Although the Signal protocol provides confidentiality, it involves exposing phone numbers to the server and so allows the server – despite efforts by Signal, acknowledged by the tech-savvy public, to minimise logs – or a passive adversary to capture all the metadata, including the social graph of users. This phenomenon is not specific to Signal, but common to the majority of popular IM apps that are based on centralised architectures. As a consequence, having users’ phone numbers as an identification mechanism is considered one of the central problems of post-Snowden end-to-end encrypted secure messaging.

The developer of Heml.is and Pirate Bay, Peter Sunde, as well as Wire’s CTO, Alan, argue that identification via phone numbers and the possibility of sharing a user’s phone contacts are important for contact discovery: they help users to build their network of contacts, and to rapidly find other users in their phonebook who use the same messaging app. An easy contact discovery is important, according to Alan, to build contacts and stabilise users’ relationship with a new messaging app:

In the beginning we didn’t have a possibility to have your phone number to register, we wanted to have just emails. But then we had an amazing two days with hundreds of thousands of downloads, even millions [however] we could not build a good network effect. People were just downloading it individually and not connecting to each other. So we went towards [having] your address book uploaded so it will help you to have some contacts established, and then once people have contacts, they stick with Wire (Alan, Wire CTO).11

Here Alan raises an important issue, also underlined by several users in our interviews, which we call the ‘migration problem’: how do we migrate our contacts from one IM to another? Or inversely, how do we ‘stick with’ a new IM if no one from our social group is using it? A significant number of our respondents among non-tech savvy users explain the weak adoption of PGP and Signal with reference to the unpopularity of these tools among their peer groups (‘no one uses it). Moxie Marlinspike also underlines how secure messaging tools are far from being immune to the network externalities effect, well-known and documented for social networks (e.g. Zhang et al. 2017):

Social networks have value proportional to their size, so participants aren’t motivated to join new social networks which aren’t already large. It’s a paradox where if people haven’t already joined, people aren’t motivated to join (Marlinspike 2014).

This effect also becomes an important factor for advising on IMs choices during security training sessions. For example, several Ukrainian trainers whom we interviewed tended to recommend WhatsApp and Gmail over Signal, and PGP over Thunderbird, partly because users already have some of their contacts using these tools, and the transition/adoption costs are lower (see also Chapter 1).

Eventually, Wire decided to move away from the use of phone numbers as an authentication method, and to give users the option of signing up only with an email address. On 25 May 2017, Wire added to its system the option to delete a previously registered phone number and adopt email-only authentication. This design solution was partly influenced by the critiques addressed to Signal, and as a ‘selling argument’ means of distinguishing itself from the latter. Wire largely promoted this new feature on social media and published a guide on the specific subject of ‘staying anonymous on Wire’. Anonymisation became one of the app’s core features – giving users the possibility to subscribe using a fake email account, over VPN or Tor, providing almost no personal information (Wire 2017). In general discussions, this feature also became a popular argument in favour of adopting Wire over Signal, as could be inferred both from our interviews and from social media and press analysis. Eventually, Edward Snowden came to recommend Wire as an alternative to Signal for those users wishing to stay within the realm of centralised applications.12

As the controversy unfolded, the use of phone numbers as an identification mechanism appeared to be related to the centralised architecture as such, making it more complicated to consider a transition towards federated systems. This issue was closely linked to the critique of telecom operators, perceived by several developers as fundamentally driven by a proprietary culture, and untrustworthy in terms of privacy. Using phone numbers as an authentication method is seen as linked to the increase in metadata collection (see the following section), and as opposed to alternative methods of identification, used in federated systems:

With Signal it’s impossible to create a decentralised system because phone numbers aren’t decentralised. With XMPP it’s like an email address. Even users who aren’t technologically savvy can understand: this is my user ID, and this is my server. We have no idea what server you use and what’s your ID. It’s on the client side, and we don’t need it for pushing messages (ChatSecure developer).

The lack of confidence in telephone numbers as a means of authentication is grounded in specific episodes of digital infrastructure-based attempts at state control in collaboration with the private sector (Musiani 2013). For example, in Russia, the use of phone numbers as IDs has been a source of important leaks and attacks on Telegram, another widely used centralised secure messaging app. On 29 April 2016, the accounts of two opposition activists from Moscow were seized by the Russian government: in collaboration with the telephone company MTS – which both activists subscribed to for their mobile communications – the government deactivated the SMS delivery service for the two users, and reclaimed recovery codes for their Telegram accounts. When these codes reached the MTS servers, the police could intercept them and use them to access the activists’ accounts. Pavel Durov, the creator and core developer of Telegram, highly recommended using two-factor authentication in order to prevent this kind of attack – a method that is also recommended during cryptoparties – and to reinforce protection of accounts on centralised end-to-end encrypted messengers.

Overall, our research demonstrated that high-risk, tech-savvy and privacy-concerned users are aware of and preoccupied by the possibility of their phone numbers being used for identification. Similarly, the usage of existing users’ social graphs – especially using third-party authorisation mechanisms, such as Twitter, Gmail or Facebook authentication services – was widely criticised. Wire’s introduction of a ‘matching’ feature, suggesting new contacts based on a particular user’s Wire contact list, was also attacked as ‘creepy’ by some of our respondents (tech-savvy security trainers from Austria and Germany). Ultimately, the ‘phone numbers as IDs’ controversy suggests that – among specific communities of users and developers in particular, but post-Snowden, more widely among the general public – there is a strong need for an alternative contact discovery mechanism that would give users better control over their social graphs.

Metadata collection: A misunderstood and unsolved issue for centralised applications

Some secure messaging solutions, like the experimental Ricochet,13 attempted to use Tor to hide the IP address of their users. However, none of the popular Signal-based messengers hide metadata, even though its users often believe that Signal indeed protects metadata and keeps their conversations anonymous. Although Signal does not log metadata (besides the last time a user installed the application and the time of her latest connection), it has been argued by one of our developer respondents that a real-world adversary could simply watch the encrypted messages going in and out of the Signal server in order to capture, to a large degree, the social graph of its users.

Even more easily, the physical seizure of a device would reveal the phone numbers of all other Signal contacts. Although applications such as Ricochet attempted to achieve protection against this kind of threat – which is a particular worry for users living in authoritarian countries – high-risk users are in general much more aware of the existence of Signal and find it easy to use, while all interviewed users were unaware of attempts at possible alternatives, such as the anonymised Ricochet application. In short, the issue of metadata collection (see Chapter 1) is one of the most controversial in the field of secure messaging today, as on the one hand, developers recognise that there is currently no satisfactory way to limit it or prevent it altogether, and on the other hand, it is an issue largely misunderstood or misinterpreted by users – even users who are informed and concerned about the problem – who believe they are completely anonymous when in fact they are far from it.

In the absence of solutions that prevent metadata collection, ‘selling arguments’ by different IM services have focused on its minimisation. Wire, for example, claims to reduce metadata storage of its messages to 72 hours, and this only to follow up on abuse or for troubleshooting reasons. In this sense, the minimisation of metadata collection is meant to single Wire out with respect to other IM applications:14

[For Wire] what was important was end-to-end encryption as a first step. But besides, there is a whole new dynamic that needs to happen, related to all of the metadata and what, how, it is used for. So the first step is to absolutely minimize all of the metadata. This is the part we are currently heavily working on, and some of the metadata we have was here for historic reasons. When you launch an application first without end-to-end encryption and then you upgrade it to support end-to-end encryption there is still some stuff left from the previous implementation. This is something that we are cleaning up (Alan, Wire CTO).

Based on our interviews, it’s clear that most developers have come to consider the improvement of privacy via the minimisation of metadata collection to be the second most important feature in IM after the development of end-to-end encryption itself. Yet interestingly, developers themselves sometimes misinterpret the issue of metadata collection, confusing whether or not a third-party adversary could be passively monitoring the communication – and thus collecting metadata – with whether or not they, as developers, personally collect data – as exemplified by one developer who candidly stated, ‘I do not have anything in my code that would let me know how many people are watching the app right now’. Other developers – a majority – declare their full awareness that they do have to collect some metadata in order to interoperate with features such as push notifications of arriving messages; however, they try to limit the collection as much as possible, to minimise harm:

With push messaging, it’s the first time we’re exposed to possible metadata. But we don’t log anything, we don’t know who is talking to who, we don’t log any information. We designed it specifically to be as harmless as possible [and] I think we’re going to enable server operators to run additional components that reduce exposures as well. There’s now some level of registration with our servers but it’s optional. It’s required to use push-messaging, but if you use an account with TOR, we disable this information (ChatSecure lead developer).

Most developers who are aware of third-party data collection of metadata declared themselves supportive of using Tor and disabling the collection of phone numbers in particular, but they lacked a comprehensive plan to minimise the collection of data as such. High-risk and low-risk users alike generally supported reducing data collection and increasing privacy, although often, non-trained users assumed that end-to-end encryption in itself could hide metadata. Thus, as the reader may recall from Chapter 1, developers and information security trainers underline the urgency to find a reliable solution to the metadata collection problem and state that, at present, nothing in the field of end-to-end encrypted instant messaging apps offers good metadata protection. Furthermore, developers and trainers associated the leaking of metadata with centralised servers – which, as Peter Sunde mentioned in a previously-cited interview, become ‘honeypots’.

Politicised (and centralised) alternatives to Signal: The case of Telegram

As this chapter has shown, for a number of reasons that range from its centralised architecture and the perceived technical soundness and novelty of its features to its business model and relationship to standardisation bodies and open-source communities – and despite a number of controversies around its development and implementation – Signal has come to assume a central role in the secure messaging field. Yet the ‘ecosystem’ surrounding it is lively, including other centralised architecture-based applications that – unlike Wire or ChatSecure – are not based on the Signal protocol.

A case in point is Telegram, which was briefly introduced in Chapter 1. Telegram’s client-side code is free software, while its server-side code is closed-source and proprietary, favouring centralised management; furthermore, Telegram has a centralised architecture, as it runs servers on which correspondence is clearly stored. Telegram’s accounts and identity authentication are also tied to telephone numbers. While these features lead both developers and users to situate Telegram unambiguously in the realm of ‘centralised’ services, it remains the most popular secure messaging app for some user populations – in our research, notably Russian users.

According to our interviews, the usage of Telegram may vary quite substantially according to users’ goals and threat models. Several functionalities of the app make it convenient for different user groups: chats, secret chats, group chats, bots and channel broadcasting. Users faced with a low level of threat, not associated with any political activities, tend to adopt Telegram as an alternative to WhatsApp, using it for sending short messages in everyday conversations with their peer groups. Many activists and privacy-concerned users are aware of the absence of ‘privacy by default’ in Telegram chats (client-to-server encryption); however, they do not always opt for a ‘secret chat’ option that offers end-to-end encryption. This user group sometimes adopts two-step authentication and self-destruct timer options. Functions such as ‘Group chat’ are used for group conversations and are popular among activists, journalists or researchers for organisational purposes, as an alternative to Google Groups or mailing lists.

For example, one of our use-cases, a group of researchers working in Eastern Ukraine, use Telegram on a daily basis to coordinate research activities and discuss fieldwork, materials and other organisational information. However, they do not rely on Telegram for very sensitive discussions and prefer face-to-face offline meetings. During Russian anti-Putin protests in spring 2017, Telegram was also very popular, regardless of previously detected security flaws and the absence of end-to-end encryption in group chats. Telegram group chats remain popular among high-risk users despite the fact that the encryption for group chat offered by the app is very basic.

The popularity of Telegram in Russia can be partly explained by the reputation of its founders, Nicolai and Pavel Durov, Russian-born developers and entrepreneurs. Pavel Durov, the founder of Vkontakte, the most famous Russian social network, is colloquially referred to as the ‘Russian Zuckerberg’ and became persona non grata in Russia after his refusal to collaborate with the FSB. Telegram’s swift rise on the market of messaging apps reveals a lot about the socioeconomic factors that influence the success of an innovation in the field of secure messaging: indeed, when Facebook bought WhatsApp (resulting in a lengthy black-out for the latter), the Telegram download rate exploded. In contrast to WhatsApp, Telegram can publicly underline its not-for-profit character and lack of ties with any commercial or governmental services.

While the Russian version of Telegram was released in 2012, before the Snowden revelations, Durov claims that the international version of his tool was inspired by the whistleblower:

In 2012 my brother and I built an encrypted messaging app for our personal use—we wanted to be able to securely pass on information to each other, in an environment where WhatsApp and other tools were easily monitored by the authorities. After Edward Snowden’s revelations in 2013 we understood the problem was not unique to our situation and existed in other countries. So we released the encrypted messaging app for the general public (Durov, quoted in Healy 2015).

Despite its centralised nature, Telegram servers are located in five different countries around the world, with the aim of improving response times and better responding to national or regional jurisdictions (for example, the data of users who subscribe from inside Europe is stored only in the Netherlands). Due to the geographical distribution of its servers, Telegram’s broadcasting function is used by censored media as a way to bypass the blockage, and by bloggers as an alternative to Facebook and traditional blogging platforms (for example, Alexey Navalny’s popular bot on Telegram and the Grani.ru channel and bot, among others). However, unlike private communications on Telegram, public channels may be read and blocked by the Telegram technical team (e.g. as of January 2016, 660 channels attributed to ISIS were blocked).

Recent research in usability showed that Telegram was suffering from a number of important problems in this regard (Abu-Salma et al. 2017a); however, our research shows that this application is widely adopted by high-risk users, in Russia, Ukraine, but also in Iran, and reveals a number of ‘alternative’ practices and room for resistance within a service that has otherwise a number of centralised ‘control points’ (DeNardis 2014). In Russia, Telegram is widely used by Pirate Party activists (240 users as of 17 June 2017). Users from and living in Iran highlighted several aspects in favour of their choice of Telegram, not all of them directly connected with the privacy or security properties of the app. Users were especially happy with Telegram’s functions such as stickers: Telegram makes it possible for users to generate their own stickers, which permits personalised uses of the app. For example, Iranian users explained, in our interviews with them, that adding stickers representing Muslims in their everyday environment, but also specific stickers on the Iranian political situation, was a very important feature that differentiates Telegram from ‘first world’ apps that only focus on Western lifestyles and emoticons. Furthermore, during our web ethnography focused on usages of Telegram, we observed how Telegram stickers were used to attract attention to a specific cause, as an alternative way of spreading information and setting a trend. Thus, after the arrest of the Tor exit-node operator Dmitry Bogatov in Russia on 10 April 2017, a group of Russian Internet freedom activists (some of them members of Pirate Party Russia), who used Telegram group chat to coordinate the #freeBogatov campaign, generated a sticker pack dedicated to his case. Apart from stickers – that become at the same time tools for community-building and for personalising this messaging app – other elements of customisation are present in Telegram, such as the possibility to set any image as wallpaper/background for the app, or to use a wide range of colour themes – functions that are absent in Signal.

Finally – as the quotations from the two ‘Resistance’ group chat participants in Chapter 1 also show – trust in Telegram, according to our interviews, goes beyond the technology to place a special emphasis on Pavel Durov as a ‘charismatic leader’ (Conger 1989) and on his political positioning. The cryptographic protocol and security and privacy properties lose their importance for users, compared to UI/UX features and the ‘aura’ surrounding the app’s creator.

Conclusions: Centralised architectures between control and dependency

As we finish writing this chapter, in the summer of 2020, the controversy about phone numbers as means of identification and authentication in secure messaging platforms still unfolds. In late May 2020, Signal released ‘Signal PINs’, a feature meant to allow users to keep control of their account even if they lose their phone or are forced to switch numbers – a feature which, in the words of a Signal developer, is aimed at ‘facilitat(ing) new features like addressing that isn’t based exclusively on phone numbers’ (randall-signal 2020). Still, the debate about this implying the beginning of the end for phone-number-based identification rages on:

Signal isn’t yet announcing a way to use its product without handing over a phone number at all (…) You still can’t use the laptop versions of the app without setting Signal up on your phone first, and you can’t set it up on your phone without handing over a real, live phone number right at the start of the installation. PINs aren’t a replacement for phone numbers but they do provide a safer way to recover your account in an emergency than a phone number alone (…) It’s a start, not least because it means an interfering government or mobile phone company can’t lock you out of your account simply by cancelling your SIM card. But you still need a phone to get onto Signal in the first place (Ducklin 2020).

Whether Signal’s latest developments indeed herald a post-number-as-ID era for secure messaging tools or not, the fact that this issue – linked with the problem of metadata collection and retention – keeps on being the Apple of Discord for secure messaging systems is revealing of broader controversies around the ‘dependencies’ of centralised systems upon other entities. Interestingly, these dependencies are discussed in ways that largely overlook the soundness (or lack of) of the cryptographic protocols. On one hand, they are about the ways in which the application’s interface, features and particular architectural design choices bind secure messaging tools to other applications and protocols that have, in turn, centralised and/or proprietary structures. On the other hand, they are about the economic and social ‘trade-offs’ that users must accept in order to use the system – handing over to a server-based service, as identifiers, pieces of information that they consider to be excessively sensitive or revealing, or knowing that a weakness of the system may actually come from the collection and retention of the ‘information about information’, metadata, that is more concentrated and exposed in centralised applications.

At the same time, centralised architectures make it easier for IM developers to experiment with, for example, particular forms of business models where some form of control or concentration of prerogatives must be retained. The choice of the Signal team to refrain from standardisation so as to maintain the capacity of updating the protocol rapidly enough as a response to the evolutions of the field, and to provide as a service the expertise and specification necessary for a proper deployment of the protocol, is an interesting case in point.

Centralisation may also mean faith in a charismatic developer, for better or worse. Beyond Telegram’s Pavel Durov – the centrality of Moxie Marlinspike’s figure and interventions in the secure messaging ecosystem is clear in the case of Signal, as well – motivations for the adoption of privacy-enhancing tools are also dependent on the reputation of their creators. In the users’ mind, these figures of core developers become a (sometimes very welcome) ‘centralisation’ point, and their handling of shifting geopolitical alliances that may affect the reach of government agencies and, as a consequence, the users’ threat models, is closely monitored and either fiercely supported or blamed.

The (hi)story of the Signal protocol is also interesting from the standpoint of Internet history. Indeed, it is a story of protocols that mutually influence each other, borrow from each other, and even try to revert to their predecessors and renew them in the light of new norms and requirements brought forward by Signal, such as forward secrecy – nowadays accepted as the necessary minimum. The Signal protocol has deeply influenced the crypto protocol field by introducing a combination of properties, such as forward and future secrecy and non-repudiation, combined with a modern interface, that have become a new minimum required for a secure messaging application. As the inventor of the OMEMO protocol (see Chapter 4) explains,

If you are designing a new protocol for end-to-end encryption now, or even two years ago, for instant messaging having forward secrecy in it is just a good practice. It’s just what all the other IM encryption schemes are doing as well. Signal does it, WhatsApp does it.

Boosted by Signal’s innovations, older protocols have been refurbished, and new standards are now being discussed that aim at bringing some of these properties to a documented and stabilised form. Signal’s impact goes beyond a ‘linear’ leap from older to modern protocols. The turn to mass encryption has shaken the secure messaging community, instigated protocol renewals and raised new challenges, many of them still unsolved.

The story of Signal is also telling in terms of ‘informal’ Internet governance. It shows how, as encryption becomes much more of a public concern than it was a few years ago, several end-to-end encrypted messaging developers are becoming sceptical about traditional arenas of exchange and dialogue on potential standards (such as the IETF, XMPP Foundation, W3C or NIST), which they consider less effective (or more ‘compromised’) than a development-based approach. Does this mean that, as far as widespread adoption of encryption in secure messaging is concerned, we are looking at the ‘end’ of the standardisation era? Will governance of encrypted messaging happen by infrastructure and by code, by ‘something that works’? And if so, is a centralised service more likely to lead the way, due to its ‘control points’ – technical, financial and organisational – that can make control by design more straightforward for those willing to attempt it? The next two chapters, delving into more distributed arrangements for secure messaging applications – peer-to-peer and federated architectures – will keep on addressing this core question.

Notes

1 This chapter partially draws upon Ermoshina, K., and F. Musiani, ‘Standardising by Running Code’: The Signal Protocol and De Facto Standardisation in End-to-End Encrypted Messaging’, Internet Histories: Digital Technology, Culture & Society, 3.3–4 (2019): 343–63.

2 As a reminder (see Chapter 1’s ‘Instant messaging encryption’ for more details), one of the applications built on top of the Signal Protocol is also called Signal and developed by the same team.

3 This independent audit of the OMEMO protocol includes a part dedicated to the Signal protocol and refers to a lack of documentation: https://conversations.im/omemo/audit.pdf.

7 Signal’s former and initial name.

11 The willingness of users to ‘stick’ with Wire was also the reason why its team implemented a (end-to-end encrypted) chatbot called ‘Otto’, intended to become users’ first contact, a way to get familiar with the app and test it – a way to, in Alan’s words, ‘show [users] the potential of Wire so that they feel comfortable inviting their friends’.

12 @Snowden on Twitter, 8 June 2017, 8:18 PM: ‘Wire seems like a reasonable alternative to Signal, it’s just less well-studied. Also lets you register with anon email instead of a phone #’. The tweet is no longer available, although the conversation that followed still is.

13 https://ricochet.im, which stopped development at the end of 2016.

14 In 2017, the Wire team claimed that one-to-one voice calls made via its system no longer collect metadata (Wire, 2017b).

Where to buy cardura in Colorado online Zetia ezetimibe 1 0mg Where to buy synthroid in Thunder Bay Where to buy xyzal 5mg in Lincoln online Side effects of prograf Provigil and skin disorder Buy buspar 5mg from Sydney Buy alphagan online from Michigan Cleocin topical lotion Thunder Bay verapamil shipping