Information About the Silk Transparency Project

Sources

This Silk site uses only data retrieved from the official Transparency Reports of each company. Please note that not all companies publish a report, nor have they stated doing so in the same year. Links to the original reports are posted below. All the data has been carefully copied and double-checked. We cannot guarantee accuracy but have done our best to ensure accuracy. Should you find any inaccuracy, please contact us.

Methodology

Every company has different standards for publishing the information.  We normalized the data to make it comparable. Here we describe the normalization methods we used. 

  • Twitter uses the value "<10" for countries that submitted less than ten requests. All other service providers published the exact number. To carry out calculations and comparisons with the Twitter data, we converted "  <10"  to the medium value of 5.5.
  • Apart from the data included here, Dropbox has received 90 requests from countries other than the U.S. For legal reasons, Dropbox has not yet published the names of the countries that have filed these requests. These requests were not included in the Transparency Report. 
  • Most service providers publish reports covering six months of requests. In some cases service providers published a yearly report. In these cases, the data on the requests has been divided by two. Each half was assigned to six months of the year to properly align with the other Transparency Reports. We used this normalization process in the following cases:
    • AT&T (2013)
    • Cloudfare (2013)
    • CREDO (2013)
    • Dropbox (2012 and 2013)
    • Sonic.net (2011 and 2012)
    • Microsoft (2012)
    • Tumblr (2013)
    • Verizon (2013)
  • Verizon doesn't publish information on the number of requests it complies with. For that reason we could not add Verizon compliance rate data to the Silk Transparency Report Database.
  • AT&T publishes information on the number of requests it complies with but it publishes the data in a relatively complicated manner. The AT&T reports combine requests that don't produce data with requests that lead to the disclosure of "partial data". This is ambiguous and differs from the clearer distinctions offered in the reports of all other companies, where the disclosure of "partial data" is separated from "no data disclosure". For this reason information on AT&T's compliance rate is left empty.
  • A note on the United States, NSL and FISA. The data in the Silk Transparency Reports database contains only requests that the United States Federal Government has submitted through defined legal processes (Court Orders, Subpoenas, Search Warrants, etc.). In January 2014, the US Department of Justice announced that it is now permissible for companies to release information on the number of National Security Letters and FISA court orders to disclose data. These disclosures will be permitted only in broad ranges. While these ranges may provide some insights, in practice it is impossible to carry out any proper data analysis using such broad measures. For this reason,  we chose to exclude this type of data from this database. Additionally, not every company has yet updated their Transparency Reports with this information,

Structure of the Data and Explanation of Variables

"Time Periods" Collection

Contains data for 2009 - 2013 divided in block of six months. The variables for each one are:

  •  Requests Received by Each Service Provider
  • Total Requests Reported
  • Companies Who Have Published a Transparency Report
  • Number of Companies Who Have Published a Transparency Report
  • Average Requests Reported per Service Provider (Calculated by dividing the Total Requests of the six months by the number of companies that have published a report in those six months)

"Companies" Collection

Contains data for the 15 companies that have published a transparency report. The variables for each one are:

  • Requests Reported in Each Six-Month Block
  • Compliance Rate Reported in Each Six-Month Block
  • Total Requests Received
  • Total Compliance Rate

"Countries" Collection

Contains data for the 101 countries that appear in at least one of the published company transparency reports. The variables for each country are:

  • Requests Sent to Each Service Provider
  • Requests Reported in Each Six-Month Period
  • Average Requests per Service Provider in Each Time Period (Calculated by dividing the total requests sent in each time period by the number of companies to have published a report in each time period).
  • Total Requests Reported
  • Population (based on 2012 data from the World Bank)
  • Average Requests Sent per Million Inhabitants (Calculated by dividing the Total Requests Sent by the Population and multiplying for 1 million)
  • Internet Population for 2012 (data from Wikipedia and Internet World Stats)
  • Total Requests per Million Internet Users (Calculated by dividing the Total Requests Sent by the Internet Population and multiplying for 1 million)

  • Compliance Rate for Each Service Provider
  • Total Compliance Rate (for all service providers and all time periods)
  • Companied Solicited

A Note on Transparency Reports

In this project we included only the data that each company discloses in its official Transparency Reports. Please bear in mind that this is by no means comprehensive of all the entities that have accessed users' data. The companies themselves can access the data stored in their own servers, especially for legal purposes. This is, for example, what Microsoft could do before updating its policy in March 2014.

Additionally, Transparency Reports on government information requests are not complete even in terms of governmental access to user's data. The reports contain the requests that law enforcement authorities file through standard legal procedures - thus excluding data collected through bulk surveillance programs and unauthorized interceptions, like NSA's PRISM or GCHQ's Tempora.

Facts collected from tagged text on this page
Facts about this page