Search

Home > News

OONI Highlights: 2017-2019

 
Mon, 2020-03-02 18:21

The following post was authored by Maria Xynou on behalf of the OONI team. You can also view it posted on the OONI blog, here.

Back in 2012, the Open Observatory of Network Interference (OONI) was born with OTF support to create the first open methodologies that would enable anyone around the world to measure internet censorship and other forms of network interference.

Today, OONI comprises a global community that runs its censorship measurement software (OONI Probe) in more than 200 countries every month. Millions of measurements have been collected from tens of thousands of networks in 236 countries since 2012, all of which are openly published on OONI Explorer — arguably the largest publicly available resource on internet censorship to date.

Most of this growth is thanks to OTF support over the years.

In this post, we share highlights from what the OONI team accomplished between 2017 to 2019 with OTF Internet Freedom Fund support, including improved censorship measurement tools, expanded global coverage of censorship events, timely evidence of global censorship events, rapid response to emergent censorship events, and growing the global OONI censorship measurement community.

Improved censorship measurement tools

image3.png

Image: Comparison of OONI Probe mobile 1.0 with OONI Probe mobile 2.0 (both released between 2017-2019)


To enable the internet freedom community to measure censorship events around the world, we first needed to:

  • Make OONI Probe available on more platforms (such as Android and Windows);
  • Improve OONI Probe usability;
  • Make OONI Probe more useful to the internet freedom community (by including new features based on community requests);
  • Improve and expand upon OONI’s methodologies to collect richer measurements and detect more forms of internet censorship.

To this end, we released:

Expanded global coverage of censorship events

image7.png

Image: Monthly OONI measurement coverage worldwide


Every month, millions of OONI measurements are collected from thousands of networks in more than 200 countries, shedding light on censorship events around the world.

This expanded coverage is most likely due to the improvement of OONI’s censorship measurement tools. The following OONI measurement stats show a large uptick in February 2017, following the initial launch of the OONI Probe mobile app (which made censorship measurement much easier).

image2.png

Image: OONI monthly measurement statistics (previously hosted on the OONI API)


Expanded measurement coverage means that more OONI Probe tests are run in more countries and on more networks, enabling the internet freedom community to detect many more cases of internet censorship around the world.

Between 2017 to 2019, we collaborated with our local partners (and other community members) on the publication of the following 32 research reports covering censorship events around the world:

Timely evidence of global censorship events

image1.png

Image: OONI Explorer (revamped in 2019)


Now, OONI measurements from around the world are openly published in near real-time. This means that as soon as anyone runs OONI Probe anywhere around the world, their test results are (automatically) openly published on OONI Explorer within minutes (unless if they have opted-out from publication). It is therefore now possible to track internet censorship events around the world in near real-time, thanks to the creation of a new fast-path pipeline.

Data analysts and researchers may be interested in the OONI API (launched in 2017) for downloading the raw measurements in JSON format to perform their own analysis.

image4.png

Image: OONI API


Given that other tools (such as OONI Explorer) depend on the OONI API, the use of the API should be limited to light-weight queries (to avoid affecting the performance of tools that rely on it).

To support batch consumption of OONI data, we have made a full copy of the OONI PostgreSQL MetaDB available. This enables any third party to setup a read-only replica of the OONI MetaDB, allowing them to run heavy database queries on their own infrastructure for batch analysis. This is the same database that also powers the OONI API and OONI Explorer. To enable adoption, we published documentation that explains how to set-up a copy of the OONI MetaDB.

Thanks to the Amazon Open Data program, the whole OONI dataset can be fetched from the ooni-data Amazon S3 bucket, and we published a blog post explaining how to do this.

Rapid response to emergent censorship events

image5.jpg

Image: OONI Run


To promote rapid response to emergent censorship events, we created a new platform, called OONI Run, to enable the coordination of customized OONI Probe testing.

Through OONI Run, you can generate a link based on specific websites that you are interested in testing, and share that link with OONI Probe users so that they can test those websites. This has been useful in various cases for measuring emergent censorship events around the world.

Last year, for example, IPYS Venezuela and Venezuela Inteligente coordinated an OONI Probe measurement campaign amid protests and political turbulence, sharing OONI Run links with Venezuelans to test YouTube, Instagram, and Twitter across the country. This enabled the rapid collection of relevant measurements, supporting the publication of a joint report providing evidence of social media blocking in Venezuela. Quite similarly, OONI Run links were used to coordinate testing in Egypt to collect evidence on the blocking of bbc.com and alhurra.com amid protests.

As social media, news websites, and circumvention tools often get blocked during political events around the world, we embedded several OONI Run buttons on the OONI website to enable the testing of such websites.

image6.png

Image: OONI Run buttons embedded on OONI website


These sorts of buttons can be generated through the OONI Run platform.

Global OONI censorship measurement community

All of the above requires a community: humans who are interested in defending a free and open internet, and willing to run censorship measurement tests from local vantage points (despite potential risks).

To empower censorship research participation in the internet freedom community, we established 17 new partnerships (between 2017 to 2019) with the following organizations:

We started establishing partnerships with digital rights organizations in mid-2016, and the new partnerships listed above expand our network to 27 partnerships overall. Through these partnerships, we aim to support local efforts in increasing transparency of internet censorship. To this end, we collaborate on OONI network measurement and provide OONI data analysis support.

Between 2017 to 2019, we collaborated with our partners (and other community members) on updating 20 Citizen Lab test lists for the following countries: Senegal, Bolivia, Mexico, Paraguay, Tanzania, Algeria, Vietnam, Indonesia, Cameroon, Democratic Republic of Congo (DRC), Colombia, Argentina, Jordan, Nigeria, South Sudan, Venezuela, Zimbabwe, Sierra Leone, Egypt, and Mali.

To support local OONI community engagement efforts, we created and published several resources, such as OONI workshop slides, a Frequently Asked Questions (FAQ) section, and an OONI Glossary. We also started an OONI Community Interview series and created a short documentary on internet censorship in Cuba.

Thanks to OTF Community Lab support, we had the opportunity to host the OONI Partner Gathering in 2017: a two-day event that brought all of our partners together to share skills and knowledge on censorship measurement research. We subsequently held several other community gatherings, such as our monthly online meetings and in-person meetups at international conferences and events (such as RightsCon and the Internet Freedom Festival).

The OONI team is immensely grateful to the OTF for supporting the growth of the OONI-verse. We look forward to continuing to support the internet freedom community in the fight against internet censorship.

Projects Mentioned