Mozilla has publicly documented 12 major bugs in Facebook’s transparency API – a tool that Facebook has now rolled out globally following the EU elections.
Ahead of the EU parliamentary elections in May, Facebook released an ad archive API, in an attempt to make political advertising on the platform more accessible and transparent to researchers and journalists. When it was released in April, Mozilla took a preliminary review of the API and determined it wasn’t up to snuff. Through a month of rigorous testing we uncovered 12 major bugs, that plagued the tool and were reported to Facebook.
In the run-up to the European Elections, anyone looking was unable to find out who was buying EU election ads on Facebook and who they were targeting, ultimately rendering Facebook’s efforts lip service. Facebook has now rolled this same tool out globally to aid in elections happening around the globe including the US.
For example, Facebook says ‘You can search data for all active and inactive ads about social issues, elections or politics.” But when put to the test, technical or data issues affected a user’s ability to reliably retrieve data from multiple searches.
As Marshall Erwin, Senior Director of Trust & Security at Mozilla comments: “Our documentation of the broken API provides Facebook a clear roadmap to make the necessary improvements to deliver a functioning and useful API. There are no excuses. Important elections are expected to take place almost every month around the globe until the end of the year. We need an API that actually helps – not hinders – researchers and journalists uncover who is buying ads, the way these ads are being targeted and to whom they’re being served. We need Facebook to be better.”
So how broken is the Facebook API?
Here’s how the API actually stacked up against Facebook’s public statements (Source: Mozilla).
What Facebook says: “The Ad Library application programming interface (API) allows you to perform customized keyword searches of ads.”
What Mozilla found: Software errors crippled a user’s ability to do keyword searches, including the following bugs:
- The API is trapped in an infinite loop
- The API returns invalid next pages
- The API fails when returning exactly one page of ads
- The API fails when returning exactly 100 ads per page
- The API randomly terminates a search
What Facebook says: “You can search data for all active and inactive ads about social issues, elections or politics.”
What Mozilla found: Technical or data issues affected a user’s ability to reliably retrieve data from multiple searches:The API returns unreproducible results (identical searches)
- The API returns incorrect results (keywords)
- The API returns inconsistent results (member states)
- The API exhibits exceedingly high error rates
What Facebook says: “We know we can’t do this alone, which is why we’re also rolling out access to our Ad Library API globally so regulators, journalists, watchdog groups and other people can analyze ads about social issues, elections or politics and help hold advertisers and Facebook accountable.”
What Mozilla found: Design limitations that would have prevented users from retrieving a sufficient quantity of data, even if the API had been functional:
- The API provides no guarantee of completeness
- The API places a significant limit on search terms
- The API places a significant limit on bandwidth
What Facebook says: “It helps to be a little familiar with programming to use an API.”,
What Mozilla found: With Mozilla’s own team, it still took us an entire month to figure out how to use the API. Even still, the API delivered incomplete data on most days from its release through May 16, when Facebook fixed a critical bug. The API was broken again from May 18 through May 26, the last day of the elections.