Report: How to Evaluate Technical Audits as a Funder
Open Technology Fund (OTF) finances technologies that promote human rights and Internet freedom globally. As a funder of emerging technology, OTF is interested in keeping the quality of its investments high. One component of ensuring its projects meet the highest information security standards includes facilitating independent technology audits. Accordingly, OTF contracts information security auditors to assess the privacy and security limitations in OTF-funded projects and suggest remedial recommendations. These audits both mitigate risk inherent in funding cutting-edge technologies and strengthen the technical capacity of OTF supported projects and the broader human rights and Internet freedom technology community.
To derive the greatest possible value from these audits, OTF has sought to identify a process by which to evaluate technical security audit reports from the perspective of a funder, not a technologist. This document provides a framework for how an organization, such as a human rights funder or an NGO, can effectively and efficiently engage information security auditors, based on OTF’s experience and findings.
Specifically, Section 2 introduces the Open Technology Fund, its security audit process, and defines terminology used through the document. Section 3 provides a methodology for working with information security auditors and assessing security audit reports. In Section 4 we identify areas of inadequacies, problems, or concerns we expect an auditor to evaluate, along with other criteria we expect security auditors to meet, such as recommendations for reducing security vulnerabilities in the future. Section 5 contains the results of OTF security audits, summing up individual reports, identifying best and worst practices, and common themes. The conclusion in Section 6 provides suggestions for interviewing and selecting an audit firm, and offers specific recommendations for OTF, although useful for other organizations, to improve its information security audit process. Finally, appendices and attachments include worksheets, checklists, and templates used by OTF that may assist other organizations with similar needs.
A note on confidentiality and original audit reports
Final reports from auditors are often confidential. To fully explain their findings and help applications resolve any problems, reports will describe tools and methodologies used during the audit. These tools and methodologies are often trade secrets that make an auditor competitive. For this reason, the reports are not available for public review and considered confidential information available only to OTF and its consultants under confidentiality agreements. During the preparation of this report, auditors’ final reports were made available to a single iSEC consultant operating under NDA forbidding disclosure or anti-competitive business practices.
iSEC Partners is a strategic digital security organization, performing application and system penetration testing and analysis for multiple platforms and environments. iSEC regularly provides Application Security and Network Security consulting practices for the most common and trusted components of the Internet ecosystem. Besides regularly performing security assessments of all types at iSEC, the author has extensive experience in OTF’s field of work. The author previously contirbuted suggestions for auditing sensitive applications of these types and made them available via Creative Commons, tracks and comments on the development of many of OTF-funded projects and their peers, participates in the development of Internet security standards and protocols, and has conducted security research and presented at security conferences around the world.