Our three year saga to release 13 million pages of CIA secrets

From videotaping visitors to protecting metadata, Kel McClanahan outlines every excuse the agency had for why the couldn’t put CREST online

Written by Kel McClanahan
Edited by JPat Brown

Kel McClanahan, the lawyer who represented MuckRock in our CIA lawsuit, outlines the three year fight to get the agency to release its declassified database - and all the excuses the agency used for why it couldn’t be done. This post originally lived as a Facebook post, but was too good not to share.


The release of CREST is great news for intel researchers and historians, but the backstory is a case study of the CIA’s stubborn sense of entitlement and DOJ litigation insanity.

1) For over 15 years, the CIA insists that “security reasons” prevent it from allowing people to take electronic records from CREST, only allowing people to print records. This was publicly ridiculed back in 2009 in a Mother Jones article and elsewhere, but CIA won’t back down.

One of the reasons CIA gave for this policy was the need to use surveillance cameras to record all visitors to the CREST terminals in case they turned out to be dastardly evildoers out to get “publicly available” - CIA’s words - unclassified historical CIA documents for various and sundry acts of nefarious evil.

2) In 2011 Jason Smathers decides to try and force the issue, filing a FOIA request for an electronic copy of the entire CREST database that he’ll put online himself (since FOIA requires agencies to release records in electronic format upon request). CIA denies the request (for entirely unclassified records) in full citing Exemption (b)(1) — the exemption covering classified information.

3) In 2014 Michael Morisy and I start talking about litigating this issue. So we file a request in June 2014 functionally identical to Jason’s, and since we know exactly what their response will be, we file suit while they are sitting on it.

4) In January 2015 CIA files a declaration saying “TIFF images are used in CREST, because they are single layer files which are very hard to alter or to supplement with additional data. Therefore, they are more secure.”

The declaration also says that CIA cannot release these TIFF files in electronic form because they can be so easily altered by the mere act of a CIA FOIA analyst looking at them, and that the security measures they must take to remove this accidental metadata for an electronic release (involving editing each file separately by hand) would take 28 years and 1,200 CDs.

Therefore, the DOJ lawyer files a brief breathlessly citing all the Very Important Business that CIA has to do that this request would interfere with and telling the judge that the request is clearly unduly burdensome and absolutely not in the public interest (remember this because it will be relevant later) because people can get printouts of the records from the terminals.

5) A month later, before I get a chance to respond, CIA files a second declaration saying, in effect, “Oops, we forgot to look in the corner.” Not even kidding.

Turns out that the first time anyone bothers to ask the actual CREST people, they reply that there is a big stack of backup CDs in the corner which were created SINCE OUR REQUEST (it includes 1,450 CDs but there were only 1200 CDs’ worth of records as of the request, a point CIA explicitly makes in this declaration to avoid considering those extra 250 CDs responsive to the request).

According to CIA, this means that it will only take six years instead of 28 to process this request, but that’s still too burdensome. Plus, they say they currently intend to have the entire thing online in four years (when evildoers apparently no longer need to be videorecorded) so there’s no point in giving MuckRock “its own personal copy of the CREST database.”

Oh, and by the way, if they were to process the request, they would first need a check for $108,000 because there is absolutely no public interest (are you remembering this?) in these records because people can get printouts of the record from the terminals.

6) In June I file my oppo, including an expert declaration from former CIA IT and records guy Jeff Scudder effectively refuting everything CIA’s declarations said. It includes stuff like

  • all of CREST would fit on one portable 1 TB hard drive, $70 at Staples;
  • there are a vast array of automated metadata removal programs available through a Google search, and that any high school student who took a programming course could write a script for it;
  • the metadata could not be modified the way CIA said anyway;
  • any true bad actor could just as easily get the data by taking the cover off of one of the CREST terminals, plugging in a hard drive for a file dump, and leaving the building before anyone noticed;
  • there is accordingly no way the CIA Office of Security would allow the terminals to simply sit there with a camera if the risk of metadata release was as great as the declaration claims; and overall,
  • “Every claim by [CIA] falsely exaggerates how systems do not work or how they are slow and cumbersome.”

And my personal favorite …

  • “Moreover, when [CIA] estimates that it would take 28 years just to create copies of the CREST database, it begs the question of how CIA loaded CREST in the first place. The system has only been operational since 2000.”

7) A month later CIA files yet a third declaration, this time doubling down on how easy it is to alter these TIFF files that they use because they are so difficult to alter.

8) I file a sur-reply drawing on my extensive experience as a person with a working brain, closing with the following thought:

“Last, CIA claims that ‘the act of a CIA employee opening a document on his or her terminal may cause metadata to embed itself on the image header.’ This is a frivolous statement for two reasons. First, files on PCs are not altered unless they are saved after the alteration. Simply opening a file and then closing it without changing it does not embed metadata on the file. Second, if this were a valid concern, it would apply to every file processed by the CIA FOIA office, not just CREST files. Since it clearly does not (which would paralyze the FOIA office), then the Court should view this claim very skeptically.”

9) In September the judge orders CIA to file another declaration proving exactly what burden it would suffer if it processed the request.

10) In November CIA files a fourth declaration talking about the horrific burden. Then it adds, which is news to everyone:

“Despite the challenges, given the high public interest in the CREST database, the CIA has continued its efforts towards placing the entire CREST database online and has recently surged resources to make it available for the general public as soon as possible. As a result of these efforts, the CIA anticipates that the entire CREST database will be publicly available online in an electronic reading room within the next year, and is preparing to commence uploading documents to that reading room on a periodic basis beginning in the first quarter of 2017.”

11) A couple weeks ago, Michael Best (who had the clever idea to crowdfund an effort to print out and rescan all of CREST) tweets out a photo of a sign posted on one of the CREST terminals which reads, “As of 26 January 2017, this CREST kiosk will be decommissioned. All CREST content will be available electronically on the CIA Electronic Reading Room.”

Let’s take this apart a little.

In 2000, using cutting edge 2000 technology, CIA populates CREST over a matter of months. For the next 15 years they insist that the only way to protect this system is to videotape people who want to access it. In 2015 they say that, using cutting edge 2015 technology, it will take them 28 years to make CD copies of CREST. Then they say that they already MADE CD copies of CREST in a matter of months if not weeks, meaning that it will only take 6 years to copy THOSE copies.

But they can copy those records to the web in 4 years. Less than a year later, they say that because of the huge public interest in CREST over the last several years (that they expressly disavowed less than 6 months before, did you remember?) they will have all of CREST online within a year because nobody needs to be videotaped any more. Then, less than 2 months later, they put all of CREST online.

And when the issue of fees comes up, they are 1000% guaranteed to argue that this lawsuit had absolutely nothing to do with them posting CREST online and that every single sworn declaration saying exactly how impossible it would be to do it faster was completely accurate and should not be considered opportunistic false testimony offered in bad faith.

Oh, and when they criticized Jeff Scudder for not knowing anything and altogether being clueless? Yeah, they stand by that too. Because this is how the government litigates FOIA.

And now you know … the Rest of the Story.


Image by Mike Best via Kickstarter