When the Open Technology Fund (OTF) began, we had an idea to extend the culture of openness to security audits within the Internet freedom and broader human rights technology field. We believe the standard for all technology crucial to the digital defense of free expression online should be openly and collaboratively audited by industry professionals. We were definitely not the first ones to think this a good idea. Before OTF came to be, the Department of State’s Internet freedom program began auditing the various technologies it supported. In 2012, Tor released an audit of Ultrasurf. After two years of formulation, the Open Internet Tools Project launched the Peer Review Board in 2014. Even with these developments, the field of Internet freedom digital defenders can still do more to receive open and collaborative audits.
To address these limitations, OTF set up a red team mechanism to be more open about the security audits we fund and tried not making it hard for good projects to receive good security reviews. Despite challenges, the rewards have been significant. To date, OTF has conducted 30 technology code audits identifying 185 privacy and security vulnerabilities in both OTF and non-OTF-funded Internet freedom projects. Each audit was funded by OTF and offered in-kind at no cost to the project. Subsequently, the identified vulnerabilities were addressed by the projects in updated versions of the audited technology. To further maximize the impact of these audits and share information on how to replicate this process, OTF developed and published a methodology and framework for evaluating technical audit reports from the perspective of a funder. The results have been a swift and visible increase of privacy and security throughout the whole field.
As a small team, OTF has faced many challenges in our attempts to be more open about the security of the tools we support. The demand is high and OTF offers these audits to any project within our remit, not just the projects OTF supports. A brief form is all we require to start the process. As such, the program is auditing between 5 and 10 projects at any given time. That’s a lot to manage for a small team. Fortunately, the auditors OTF partners with are capable of working at a high professional standard without much oversight by OTF. While their backgrounds are diverse, each are a part of a mature industry from a commercial sector. They bring significant professionalism to the nascent Internet freedom and human rights technology field. With that comes significant project management experience ensuring each audit is managed collaboratively and bespoke to each project’s unique needs.
The real challenge has been the most unexpected, openness.
At the onset, OTF assumed the auditors would be hesitant to publish their findings and the projects – mostly open source tools – would welcome the public scrutiny. We presumed that auditors would consider their audit reports or disclosure of their unique techniques a form of intellectual property. These concerns have proven to be unfounded. In fact, each partnering auditor had a simple request, audits disclosed publicly must be disclosed in full. From our experience, an auditors primary concern is that an audited project would only disclose those things that make the project look good… a misrepresentation that could make the auditor look bad.
Most unexpectedly, the majority of projects OTF has supported audits for have chosen to not publish the audit reports. Recognizing that improvement of the technology is a critical and necessary incremental improvement, OTF continues to leave the decision to disclose an audit report entirely up to the project being audited. Of the 30, less than 5 have published their audit reports publicly. As a program, OTF conducts these audits to mitigate the risk inherent in funding cutting-edge technologies and strengthen the technical capacity of the project. Fortunately, most of the audited tools are open source. As such, the resulting improvements made because of an audit do increase the capacity of the broader human rights and Internet freedom technology field writ large. Yet, disclosure of the reports in full would augment capacity even more.
Of the 30 audited projects, one stands out for its continued commitment to public scrutiny for the betterment of the tool. Cryptocat has continued to release the technology audits in full and OTF recognizes both the difficulty and value in doing so. In return, Cryptocat continues to weather significant public scrutiny. As a program, we believe this openness can only improve the tools and advance the knowledge of the field. The results for Cryptocat have been notable. For each audit supported by OTF, there have been independent security and privacy researchers who have reviewed Cryptocat code in their own time. They can do this because the code is open source. They can also spend their time looking for new problems rather than duplicating previous efforts because Cryptocat publishes previous audit reports in full.
For now, OTF will continue performing audit reports as it has, leaving full discretion to the projects on whether to disclose reports or not. That said, Cryptocat has demonstrated the importance of open review, stringent repeat audits, and a transparent process. Moving forward, OTF would like to see more projects publish their reports publicly.