NY State Value Added Measures for Teachers

Algorithmic Transparency filed this request with the New York State Education Department of New York.
Tracking # FL-EM-16/697
Status
Completed

Communications

From: Michael Morisy

To Whom It May Concern:

Pursuant to the New York State Freedom of Information Law (1977 N.Y. Laws ch. 933), I hereby request the following records:

This request concerns value-added methods (VAMs) for evaluating teacher performance by using statistical methods to measure changes in student test scores over time, while taking into account student characteristics and other factors often found to influence achievement. I am seeking records concerning (1) the VAM that New York State, and New York City School District, used prior to the decision by the New York Supreme Court in Lederman v. King (May 12, 2016) holding that the growth measurements were arbitrary and capricious as to the complaining teacher; and (2) any new VAM that New York State and/or New York City School District, is developing, planning, or implementing. Together, these are the “NY VAM Programs” or “Programs.”

For reference and help retrieving responsive documents, one of the NY VAM Programs was publicly discussed in the following court decisions and other reports:
https://www.washingtonpost.com/news/answer-sheet/wp/2016/05/10/judge-calls-evaluation-of-n-y-teacher-arbitrary-and-capricious-in-case-against-new-u-s-secretary-of-education/
https://mathbabe.org/2014/03/07/an-attempt-to-foil-request-the-source-code-of-the-value-added-model/
http://www.nytimes.com/2016/05/11/nyregion/court-vacates-long-island-teachers-evaluation-tied-to-student-test-scores.html

I am aware of documents made available through https://www.engageny.org/resource/resources-about-state-growth-measures. I am seeking other records that pertain to the underlying software of the Programs, validation studies, development contracts, and other aspects of the Programs (including those being developed) as described below.

I am requesting the following records related to the NY VAM Programs:
* Any software developed for the implementation of the Programs. This includes any software that was developed by or with, given to, used by, purchased or licensed to this agency for the above implementation of the Programs, which encompasses original source code, any compiled binaries (also known as executables), spreadsheets, Programs scripts, and other digital materials used to calculate any data in the above Programs. I will refer to these materials as the Software.
* All instructional materials, presentations and presentation materials (including recorded video and audio, PowerPoint files, prepared remarks, and slides formats), and other guidance on the NYS VAM Programs or the use of the Software. This includes any notes taking during meetings that discussed the implementation of the Programs or the use of the Software, any explanations (whether internally or externally generated) of how the Programs or the Software works, and any other document that has been used to help explain the use of the Programs or the Software to any party, including internal documentation, public relations materials, and executive summaries.
* A copy of any privacy impact assessments, use policies, standard operating procedures, data retention policies, legal opinions, warranties, non-disclosure agreements, contracts, liability waivers, insurance agreements, Requests for Proposals, Responses to Requests for Proposal, Memorandums of Understanding, Letters of Interest, usage policies, or informal agreements that reference the Programs or the Software or otherwise guide it use or the relationship between this agency and the provider or developer of the Programs and/or the Software.
* A copy of any funding opportunity announcements, grant applications and grantor status/progress reports, reports to legislative bodies, annual reports that mention the NY VAM Programs and/or the Software, as well as audit documents, including but not limited to security audits of the software, misuse reports, and reports to oversight bodies. I am requesting records dealing with the revision of the NY VAM Programs in light of the Lederman v. King decision.
* A copy of any validation studies conducted with respect to the NY VAM Programs or with respect to any software or algorithms used in connection with the Programs, and a copy of any records pertaining to inquiries for the need of validation studies, or discussion of potential or actual validation studies. By “validation study,” I mean any study designed to assess how accurate the Programs are in predicting what they are supposed to predict, or to assess whether the Programs may err in the direction of either under- or overestimating likelihoods or predicted outcomes, or may produce any results that are biased or unexpected in any way.
* For the aspects of the Programs or the Software that require an input, for example, to compute a value, we request a copy of the five most recent sets of data that were used for input, as well as the five most recent outputs of the Programs or the Software, in whatever their native format is. If these inputs or outputs include exempt information, we ask that you exempt only that information which is specifically exempted from law, and provide the other material.

The requested records will be made available to the general public, and this request is not being made for commercial purposes.
In the event that there are fees, I would be grateful if you would inform me of the total charges in advance of fulfilling my request. I would prefer the request filled electronically, by e-mail attachment if available or CD-ROM if not.

Thank you in advance for your anticipated cooperation in this matter. I look forward to receiving your response to this request within 5 business days, as the statute requires.

Sincerely,

Ellen P. Goodman

From: FOIL

Dear Ms. Goodman:

This is to acknowledge receipt of your Freedom of Information Law (FOIL) request for State Education Department (SED) records regarding NYS Value Added Measures for Teachers. The reference number for this request is FL-HE-16/697. Please be advised that you will hear further from the Department by approximately December 7, 2016 as to whether your request will be granted or denied in whole or in part.

SED charges the statutorily permitted fee of $.25 per page for duplication of records requested under FOIL (Public Officers Law §87[1][b][iii]). There is no provision in law or regulation requiring the waiver of this fee. Payment must be made to the NYS Education Department by check or money order. Do not send any payment until you are notified that your request is granted and informed of the charge for your request. If your request is for electronic records and your request is granted, the records will be provided to you in that format.

If your request is granted one of the following will happen:

1. If your request can be filled immediately, and the total duplication fee is under $25.00, staff will forward the requested records to you with a statement of what you owe for the duplication fee. Or:

2. If the total duplication fee for your request is $25.00 or more (100 pages), you will be advised of the page count and duplication fee owed in advance of duplicating the records. Upon receipt of payment, staff will duplicate and mail the requested records. Or:

3. If it will take time to locate and/or duplicate the records you have requested, you will be given a date by which you will be told the page count and duplication fee for your request.
Regards,
Records Access Officer

From: FOIL

Dear Ms. Goodman:

This is a further response to your Freedom of Information Law (FOIL) request FL-EM-16/697 for State Education Department (SED) records regarding NY State Value Added Measures for Teachers, received on November 1, 2016.

Please be advised that your FOIL request is still under review. Due to the scope and nature of your request, and the effort required to identify and collate the appropriate records, it will take the Department additional time to complete your request. You will hear from the Department by approximately January 4, 2017, as to whether your request will be granted or denied in whole or in part. If your request is granted, we will provide you with the documents or advise you of the page count and duplication costs.

Thank you,
Records Access Officer

From: Michael Morisy

Dear Records Access Officer:

Thank you for your reply and the work you've done so far. We know that our request is pretty broad and perhaps we can helpfully narrow it. What we're really most interested in are any:
- contracts, proposals, RFPs, and letters of intent (not price information);
- training data;
- validity studies;
- correspondence, including emails and other correspondence (a) between the agency and vendor concerning development and implementation of the software and related processes, (b) within the agency concerning the efficacy, reliability, and accuracy of the software and related processes, and c) any other concerns related to the use of the software and related processes; and
- sample output.

All of the above relates to the VAMs described in the second paragraph of our request and is to assist us in understanding the government policies concerning, and implementation of, this form of evaluation. We will look forward to hearing from you.

Happy Holidays,
Ellen Goodman (working with Michael Morisy)

From: FOIL

Ms. Goodman-

Please see attached as response to your FOIL request of November 1, 2016.

Sincerely,

Records Access Officer

Files

pages

Close