Reopening enforcement

To Whom It May Concern:

Pursuant to the New York Freedom of Information Law, I hereby request the following records:

For the following records holders/custodians:

Sandra Lay
Peabo Welcome
Argentina Rodriguez
Rohan Russell
Adam Gittlitz
Hai Le
Crystal Leckie
Kwabena Nuamah

For the following terms:

proactive, 311, mask, distancing, soap, sanitizer, cleaning, screening, reopening, HVAC, thermometer, barrier, central air, ventilation, filtration, COVID, airborne, aerosol, respirator, n95, infection, safety, droplets

Between January 1, 2020, and January 1, 2022.

The requested documents will be made available to the general public, and this request is not being made for commercial purposes.

In the event that there are fees, I would be grateful if you would inform me of the total charges in advance of fulfilling my request. I would prefer the request filled electronically, by e-mail attachment if available or CD-ROM if not.

Thank you in advance for your anticipated cooperation in this matter. I look forward to receiving your response to this request within 5 business days, as the statute requires.

Sincerely,

ALEXANDER RICCIO

Please provide with me an update on the status of this request.

I have yet to receive an estimated completion date for this request - please provide me with one.

Ok, that's two days from now, and three days from when you responded. Is there any reason why I may not receive documents by that date?

When I submitted my request on 12/15/2023, I specified the following 8 records holders:
Sandra Lay
Peabo Welcome
Argentina Rodriguez
Rohan Russell
Adam Gittlitz
Hai Le
Crystal Leckie
Kwabena Nuamah

If I read the above query correctly, it appears you have not narrowed to request to those individuals? If that is the case, please do so.

If you have 18 GB of data from a query involving those 8 records holders, please confirm as such and I will narrow.

Thanks for being direct and quick - I've dealt with many state and local agencies which drag out this process for weeks.

Would it be possible for you to tell me how much data each search term generates? I could prioritize it that way.

If running queries is cheap, easy and done in a few minutes or less, you could run a query like
proactive (c:c)(date=2020-01-01..2022-01-01)
(Count those results)
Then a query like 311 (c:c)(date=2020-01-01..2022-01-01)
(Count those)
Etc...

Otherwise, I'll try and narrow it more tonight or tomorrow.

Ok, no fun, but not something either of us can fix. I kinda expect that one or two of those terms is responsible for the majority of the volume of records, but I'm hesitant to drop most of them. I'll start by removing several.

If I narrow to the following 16 terms is it more manageable?
mask OR
distancing OR
sanitizer OR
cleaning OR
reopening OR
HVAC OR
barrier OR
"central air" OR
ventilation OR
filtration OR
COVID OR
airborne OR
aerosol OR
respirator OR
n95 OR
droplets

Hmph. The difficulty is that those are pretty central to what I'm looking for. If you just drop mask and COVID (so 14 terms) how many results?

I think this shows that my earlier work has pointed me in the direction of the right records holders, and that we're making significant progress in narrowing it down (18GB -> 8GB cut it in half)

How about these nine terms:
distancing OR
sanitizer OR
reopening OR
"central air" OR
ventilation OR
filtration OR
airborne OR
n95 OR
droplets

I might be able to drop one or two more, but I've already removed most of the synonyms and coordinate terms I've seen in other documents across the public, federal, state, and even some local, realms...

If this still returns far too many records for the (understandably understaffed) FOIL office to review for release, let me know and I'll haggle with myself to try to further prioritize the relevant terms.

It would be difficult for me to choose which of the remaining to drop unless there'd be a way to later submit a request that returns only records that I had not yet been provided, I think? I know of at least one COVID FOI archive which has a similar problem, with dozens of redundant copies of records from separate requests - it's a real pain to make sense of my searches!

I don't know what software the department has provided for you, but would you be able to later query for only records that were not returned by a query from this request? I imagine you'd be able to use a NOT operator to exclude these terms? Or something like it.

Wow! Again, thanks for your patience in the matter. I must be hitting the nail on the head. I gotta think a bit before narrowing further.

Lemme get back to you in a few hours.

I'm not totally sure if you're only searching emails or if you're doing some kind of eDiscovery search of all records. I hope it's the latter, but I am presuming that all 6 GB are emails, and the 1GB/16,000-60,000 pages ratio holds.

I don't want to use logical AND if I can avoid it, I want to leave that kind of thing to myself later when I have the final records.

I have a gut feeling "reopening" and "distancing" are catching a huge dragnet because so many things fell under the cognitive umbrella of "distancing".

If you drop those and search only for (sanitizer OR "central air" OR ventilation OR filtration OR airborne OR n95 OR droplets), does that get it down to something reasonable?

If *that* search is still massive, then a search for only (ventilation OR airborne OR n95 OR droplets) is kinda acceptable if I bite my tongue, close my eyes, and stub my toe.

I'd really rather not drop germs like "sanitizer" insofar as we went through several years of a predominantly airborne aerosol-inhalation-spread pandemic with authorities directing the general public to clean surfaces instead of the air.... So if you have the patience I'd really appreciate you trying the 7 term search first!!

Hmmm. Let me know the results, keep me posted.

Ok, hmm. Lemme get back to this today.

It *is* difficult to bite the bullet and drop any of the remaining terms.

While I'm haggling over it today, if you just run a query for airborne OR n95, does that get it into reasonable territory?

I'm still thinking about this - don't drop it yet! - I'll ask a few friends and colleagues for advice and try to get back to you Monday.

I guess what I want is something like:

("proactive" OR "airborne" OR "n95" OR "soap" OR "sanitizer" OR "cleaning" OR "screening" OR "barrier" OR "safety" OR "HVAC" OR "thermometer" OR "central air" OR "ventilation" OR "filtration" OR "MERV" OR "aerosol" OR "respirator" OR "infection" OR "droplet" OR "mask") AND ("COVID" OR "COVID-19" OR "Coronavirus" OR "2019-nCoV" OR "Wuhan Coronavirus" OR "2019 NOVEL CORONAVIRUS" OR "reopening" OR "safety" OR "distancing")

...but I don't have any way to test purview KQL on my end. Does that work and return a reasonable volume? or would I need to do something much more like this instead:

("airborne" OR "n95" OR "HVAC" OR "central air" OR "ventilation" OR "filtration" OR "MERV" OR "aerosol" OR "respirator") AND ("COVID" OR "COVID-19" OR "Coronavirus" OR "2019-nCoV" OR "Wuhan Coronavirus" OR "2019 NOVEL CORONAVIRUS" OR "reopening")

If the second (smaller) one is much smaller than the max you'd consider acceptable, I'd like to know how much the larger one returns too!

I guess what I want is something like:

("proactive" OR "airborne" OR "n95" OR "soap" OR "sanitizer" OR "cleaning" OR "screening" OR "barrier" OR "safety" OR "HVAC" OR "thermometer" OR "central air" OR "ventilation" OR "filtration" OR "MERV" OR "aerosol" OR "respirator" OR "infection" OR "droplet" OR "mask") AND ("COVID" OR "COVID-19" OR "Coronavirus" OR "2019-nCoV" OR "Wuhan Coronavirus" OR "2019 NOVEL CORONAVIRUS" OR "reopening" OR "safety" OR "distancing")

...but I don't have any way to test purview KQL on my end. Does that work and return a reasonable volume? or would I need to do something much more like this instead:

("airborne" OR "n95" OR "HVAC" OR "central air" OR "ventilation" OR "filtration" OR "MERV" OR "aerosol" OR "respirator") AND ("COVID" OR "COVID-19" OR "Coronavirus" OR "2019-nCoV" OR "Wuhan Coronavirus" OR "2019 NOVEL CORONAVIRUS" OR "reopening")

If the second (smaller) one is much smaller than the max you'd consider acceptable, I'd like to know how much the larger one returns too!

Damnit.
At least I know I'm looking in the right place.

May I suggest we only use smaller files for the page estimation? It looks like you can use an operator like this to return only items smaller than a megabyte.
size:1..1048567

See:
"The second example returns messages from 1 through 1,048,567 bytes (1 MB) in size."
https://learn.microsoft.com/en-us/purview/ediscovery-keyword-queries-and-search-conditions#:~:text=The%20second%20example%20returns%20messages%20from%201%20through%201%2C048%2C567%20bytes%20(1%20MB)%20in%20size.

I mean, hey, no harm in trying it, it's in the documentation! Worst case is it just fails to parse the query. Best case is that all those giant powerpoints that got sent around without any image compression are excluded, and we get a better idea of how many pages of text actually need to be redacted.

Heh, ok! I appreciate the heads up!

Excellent news!

I think the final step here is to run the same query but with
size:1048568..1074000000

And briefly look at the files that come back

I'd recommend downloading a copy of all the files to your local machine and use software like WinDirStat to examine the numbers, types, and sizes, of the files that are greater than 1 MiB here. I'm guessing that a small number of files are responsible for the vast majority of file space, and I would like to discuss using a different page estimation for those files.

If that's too slow, I also am the author of a very popular fork of that program that's manyfold faster, it's called altWinDirStat. I wrote after getting frustrated with the slowness of WinDirStat 😅. Your IT department may discourage installing altWinDirStat because I never bothered to sign the executable.

Kinda, yeah, but not exactly.

I'm suggesting you run the search again but only with that parameter returning larger files, with the intent of seeing how many files are of the kind that might fit the "1 GB of data can contain anywhere from 16,000 individual pages to over 60,000 pages" estimate. Using size:1048568..1074000000 returns all the results that are between about a megabyte and a gigabyte. I'm making an educated guess that a very small number of files are responsible for the majority of the on disk space.

I suspect that some files are very large on disk (like powerpoints with uncompressed images), but have very few pages that need to be reviewed. As an example, a scientist colleague of mine once had a powerpoint presentation that was more than a gigabyte in size, with something like only 50 slides. She'd included some videos that had never been compressed, which drastically inflated the file size.

I do not know exactly what features the city pays for (the licensing agreement with Microsoft is the subject of another ongoing FOIL request of mine!), but one possible option is you may be able to export a report to see more information about the documents the search returns:
https://learn.microsoft.com/en-us/purview/ediscovery-export-a-content-search-report

An alternative that I provided was to simply download a copy of the larger search results and spot check them to see if there are just a few files that account for the majority of the on-disk space. That's very easy with some common free software, but if you wish to do it manually, increasing the parameter to something like size:52430000..1074000000 should reduce the volume of files even further.

I'd expect that using size:1048568..1074000000 might return several hundred files, and size:52430000..1074000000 might return a few dozen. If you run the 50 MB-1GB search and the size of the responsive records are like 10GB, that means we've found the culprit! And I can ask you to either review the number of pages in that files when estimating the voluminousness of the request as a whole. If it's just a few powerpoints, neither of us have to worry about days spent redacting 16,000 pages, it may only be a hundred.

That's ok then, let's go with the search you identified as reasonable.

🤷‍♂️👍

Thanks! I'll be waiting eagerly!