New York State or New York City Value Added Measures for Teachers

To Whom It May Concern:

Pursuant to the New York State Freedom of Information Law (1977 N.Y. Laws ch. 933), I hereby request the following records:

This request concerns value-added methods (VAMs) for evaluating teacher performance by using statistical methods to measure changes in student test scores over time, while taking into account student characteristics and other factors often found to influence achievement. I am seeking records concerning (1) the VAM that New York State, and New York City School District, used prior to the decision by the New York Supreme Court in Lederman v. King (May 12, 2016) holding that the growth measurements were arbitrary and capricious as to the complaining teacher; and (2) any new VAM that New York State and/or New York City School District, is developing, planning, or implementing. Together, these are the “NY VAM Programs” or “Programs.”

For reference and help retrieving responsive documents, one of the NY VAM Programs was publicly discussed in the following court decisions and other reports:
https://www.washingtonpost.com/news/answer-sheet/wp/2016/05/10/judge-calls-evaluation-of-n-y-teacher-arbitrary-and-capricious-in-case-against-new-u-s-secretary-of-education/
https://mathbabe.org/2014/03/07/an-attempt-to-foil-request-the-source-code-of-the-value-added-model/
http://www.nytimes.com/2016/05/11/nyregion/court-vacates-long-island-teachers-evaluation-tied-to-student-test-scores.html

I am aware of documents made available through https://www.engageny.org/resource/resources-about-state-growth-measures. I am seeking other records that pertain to the underlying software of the Programs, validation studies, development contracts, and other aspects of the Programs (including those being developed) as described below.

I am requesting the following records related to the NY VAM Programs:
* Any software developed for the implementation of the Programs. This includes any software that was developed by or with, given to, used by, purchased or licensed to this agency for the above implementation of the Programs, which encompasses original source code, any compiled binaries (also known as executables), spreadsheets, Programs scripts, and other digital materials used to calculate any data in the above Programs. I will refer to these materials as the Software.
* All instructional materials, presentations and presentation materials (including recorded video and audio, PowerPoint files, prepared remarks, and slides formats), and other guidance on the NYS VAM Programs or the use of the Software. This includes any notes taking during meetings that discussed the implementation of the Programs or the use of the Software, any explanations (whether internally or externally generated) of how the Programs or the Software works, and any other document that has been used to help explain the use of the Programs or the Software to any party, including internal documentation, public relations materials, and executive summaries.
* A copy of any privacy impact assessments, use policies, standard operating procedures, data retention policies, legal opinions, warranties, non-disclosure agreements, contracts, liability waivers, insurance agreements, Requests for Proposals, Responses to Requests for Proposal, Memorandums of Understanding, Letters of Interest, usage policies, or informal agreements that reference the Programs or the Software or otherwise guide it use or the relationship between this agency and the provider or developer of the Programs and/or the Software.
* A copy of any funding opportunity announcements, grant applications and grantor status/progress reports, reports to legislative bodies, annual reports that mention the NY VAM Programs and/or the Software, as well as audit documents, including but not limited to security audits of the software, misuse reports, and reports to oversight bodies. I am requesting records dealing with the revision of the NY VAM Programs in light of the Lederman v. King decision.
* A copy of any validation studies conducted with respect to the NY VAM Programs or with respect to any software or algorithms used in connection with the Programs, and a copy of any records pertaining to inquiries for the need of validation studies, or discussion of potential or actual validation studies. By “validation study,” I mean any study designed to assess how accurate the Programs are in predicting what they are supposed to predict, or to assess whether the Programs may err in the direction of either under- or overestimating likelihoods or predicted outcomes, or may produce any results that are biased or unexpected in any way.
* For the aspects of the Programs or the Software that require an input, for example, to compute a value, we request a copy of the five most recent sets of data that were used for input, as well as the five most recent outputs of the Programs or the Software, in whatever their native format is. If these inputs or outputs include exempt information, we ask that you exempt only that information which is specifically exempted from law, and provide the other material.

The requested records will be made available to the general public, and this request is not being made for commercial purposes.

In the event that there are fees, I would be grateful if you would inform me of the total charges in advance of fulfilling my request. I would prefer the request filled electronically, by e-mail attachment if available or CD-ROM if not.

Thank you in advance for your anticipated cooperation in this matter. I look forward to receiving your response to this request within 5 business days, as the statute requires.

Sincerely,

Michael Morisy

Dear Will:

This concerns F12827. Thank you for your reply and the work you've done so far. We know that our request is pretty broad and perhaps we can helpfully narrow it. What we're really most interested in are any:
- contracts, proposals, RFPs, and letters of intent (not price information);
- training data;
- validity studies;
- correspondence, including emails and other correspondence (a) between the agency and vendor concerning development and implementation of the software and related processes, (b) within the agency concerning the efficacy, reliability, and accuracy of the software and related processes, and c) any other concerns related to the use of the software and related processes; and
- sample output.

All of the above relates to the VAMs described in the second paragraph of our request and is to assist us in understanding the government policies concerning, and implementation of, this form of evaluation. We will look forward to hearing from you.

Happy Holidays,
Ellen Goodman (working with Michael Morisy)