Disability benefits algorithm audit (Calif. Dept. of Employment Development)

Good afternoon Todd,

Please see the attached letter regarding your California Public Records Act request.

Thank you,

Nicholas Scheuer
Attorney
Legal Office, MIC 53
Directorate
916-208-9468 Work Cell
916-654-9069 Fax
nicholas.scheuer@edd.ca.gov <mailto:nicholas.scheuer@edd.ca.gov>

State of California
Employment Development Department
PO Box 826880, Sacramento, CA 94280-0001
www.edd.ca.gov<http://www.edd.ca.gov/>

This email may contain attorney work product and may include privileged material protected by the attorney-client privilege or governmental privileges. This email may not be disclosed to third parties without the express consent of the sender. If you think you received this e-mail in error, please notify the sender immediately.

Good afternoon Todd,

Please see the attached letter regarding your California Public Records Act request.

Thank you,

Nicholas Scheuer
Attorney
Legal Office, MIC 53
Directorate
916-208-9468 Work Cell
916-654-9069 Fax
nicholas.scheuer@edd.ca.gov <mailto:nicholas.scheuer@edd.ca.gov>

State of California
Employment Development Department
PO Box 826880, Sacramento, CA 94280-0001
www.edd.ca.gov<http://www.edd.ca.gov/>

This email may contain attorney work product and may include privileged material protected by the attorney-client privilege or governmental privileges. This email may not be disclosed to third parties without the express consent of the sender. If you think you received this e-mail in error, please notify the sender immediately.

Good afternoon Todd,

Please see the attached letter regarding your California Public Records Act request.

Thank you,

Nicholas Scheuer
Attorney
Legal Office, MIC 53
Directorate
916-208-9468 Work Cell
916-654-9069 Fax
nicholas.scheuer@edd.ca.gov <mailto:nicholas.scheuer@edd.ca.gov>

State of California
Employment Development Department
PO Box 826880, Sacramento, CA 94280-0001
www.edd.ca.gov<http://www.edd.ca.gov/>

This email may contain attorney work product and may include privileged material protected by the attorney-client privilege or governmental privileges. This email may not be disclosed to third parties without the express consent of the sender. If you think you received this e-mail in error, please notify the sender immediately.

Good afternoon Todd,

Please see the attached letter regarding your California Public Records Act request.

Thank you,

Nicholas Scheuer
Attorney
Legal Office, MIC 53
Directorate
916-208-9468 Work Cell
916-654-9069 Fax
nicholas.scheuer@edd.ca.gov <mailto:nicholas.scheuer@edd.ca.gov>

State of California
Employment Development Department
PO Box 826880, Sacramento, CA 94280-0001
www.edd.ca.gov<http://www.edd.ca.gov/>

This email may contain attorney work product and may include privileged material protected by the attorney-client privilege or governmental privileges. This email may not be disclosed to third parties without the express consent of the sender. If you think you received this e-mail in error, please notify the sender immediately.

Good afternoon Todd,

Please see the attached letter regarding your California Public Records Act request.

Thank you,

Nicholas Scheuer
Attorney
Legal Office, MIC 53
Directorate
916-208-9468 Work Cell
916-654-9069 Fax
nicholas.scheuer@edd.ca.gov <mailto:nicholas.scheuer@edd.ca.gov>

State of California
Employment Development Department
PO Box 826880, Sacramento, CA 94280-0001
www.edd.ca.gov<http://www.edd.ca.gov/>

This email may contain attorney work product and may include privileged material protected by the attorney-client privilege or governmental privileges. This email may not be disclosed to third parties without the express consent of the sender. If you think you received this e-mail in error, please notify the sender immediately.

Good afternoon Todd,

Please see the attached letter regarding your California Public Records Act request.

Thank you,

Nicholas Scheuer
Attorney
Legal Office, MIC 53
Directorate
916-208-9468 Work Cell
916-654-9069 Fax
nicholas.scheuer@edd.ca.gov <mailto:nicholas.scheuer@edd.ca.gov>

State of California
Employment Development Department
PO Box 826880, Sacramento, CA 94280-0001
www.edd.ca.gov<http://www.edd.ca.gov/>

This email may contain attorney work product and may include privileged material protected by the attorney-client privilege or governmental privileges. This email may not be disclosed to third parties without the express consent of the sender. If you think you received this e-mail in error, please notify the sender immediately.

Good afternoon Todd,

Please see the attached letter regarding your California Public Records Act request.

Thank you,

Nicholas Scheuer
Attorney
Legal Office, MIC 53
Directorate
916-208-9468 Work Cell
916-654-9069 Fax
nicholas.scheuer@edd.ca.gov <mailto:nicholas.scheuer@edd.ca.gov>

State of California
Employment Development Department
PO Box 826880, Sacramento, CA 94280-0001
www.edd.ca.gov<http://www.edd.ca.gov/>

This email may contain attorney work product and may include privileged material protected by the attorney-client privilege or governmental privileges. This email may not be disclosed to third parties without the express consent of the sender. If you think you received this e-mail in error, please notify the sender immediately.

Good afternoon Todd,

Please see the attached letter regarding your California Public Records Act request.

Thank you,

Nicholas Scheuer
Attorney
Legal Office, MIC 53
Directorate
916-208-9468 Work Cell
916-654-9069 Fax
nicholas.scheuer@edd.ca.gov <mailto:nicholas.scheuer@edd.ca.gov>

State of California
Employment Development Department
PO Box 826880, Sacramento, CA 94280-0001
www.edd.ca.gov<http://www.edd.ca.gov/>

This email may contain attorney work product and may include privileged material protected by the attorney-client privilege or governmental privileges. This email may not be disclosed to third parties without the express consent of the sender. If you think you received this e-mail in error, please notify the sender immediately.

Good afternoon Todd,

Please see the attached letter regarding your California Public Records Act request.

Thank you,

Nicholas Scheuer
Attorney
Legal Office, MIC 53
Directorate
916-208-9468 Work Cell
916-654-9069 Fax
nicholas.scheuer@edd.ca.gov <mailto:nicholas.scheuer@edd.ca.gov>

State of California
Employment Development Department
PO Box 826880, Sacramento, CA 94280-0001
www.edd.ca.gov<http://www.edd.ca.gov/>

This email may contain attorney work product and may include privileged material protected by the attorney-client privilege or governmental privileges. This email may not be disclosed to third parties without the express consent of the sender. If you think you received this e-mail in error, please notify the sender immediately.

Good afternoon Todd,

Please see the attached letter regarding your California Public Records Act request.

Thank you,

Nicholas Scheuer
Attorney
Legal Office, MIC 53
Directorate
916-208-9468 Work Cell
916-654-9069 Fax
nicholas.scheuer@edd.ca.gov <mailto:nicholas.scheuer@edd.ca.gov>

State of California
Employment Development Department
PO Box 826880, Sacramento, CA 94280-0001
www.edd.ca.gov<http://www.edd.ca.gov/>

This email may contain attorney work product and may include privileged material protected by the attorney-client privilege or governmental privileges. This email may not be disclosed to third parties without the express consent of the sender. If you think you received this e-mail in error, please notify the sender immediately.

Good afternoon Todd,

Please see the attached letter regarding your California Public Records Act request.

Thank you,

Nicholas Scheuer
Attorney
Legal Office, MIC 53
Directorate
916-208-9468 Work Cell
916-654-9069 Fax
nicholas.scheuer@edd.ca.gov <mailto:nicholas.scheuer@edd.ca.gov>

State of California
Employment Development Department
PO Box 826880, Sacramento, CA 94280-0001
www.edd.ca.gov<http://www.edd.ca.gov/>

This email may contain attorney work product and may include privileged material protected by the attorney-client privilege or governmental privileges. This email may not be disclosed to third parties without the express consent of the sender. If you think you received this e-mail in error, please notify the sender immediately.

Good afternoon Todd,

Please see the attached letter regarding your California Public Records Act request.

Thank you,

Nicholas Scheuer
Attorney
Legal Office, MIC 53
Directorate
916-208-9468 Work Cell
916-654-9069 Fax
nicholas.scheuer@edd.ca.gov <mailto:nicholas.scheuer@edd.ca.gov>

State of California
Employment Development Department
PO Box 826880, Sacramento, CA 94280-0001
www.edd.ca.gov<http://www.edd.ca.gov/>

This email may contain attorney work product and may include privileged material protected by the attorney-client privilege or governmental privileges. This email may not be disclosed to third parties without the express consent of the sender. If you think you received this e-mail in error, please notify the sender immediately.

Good afternoon Todd,

Please see the attached letter regarding your California Public Records Act request.

Thank you,

Nicholas Scheuer
Attorney
Legal Office, MIC 53
Directorate
916-208-9468 Work Cell
916-654-9069 Fax
nicholas.scheuer@edd.ca.gov <mailto:nicholas.scheuer@edd.ca.gov>

State of California
Employment Development Department
PO Box 826880, Sacramento, CA 94280-0001
www.edd.ca.gov<http://www.edd.ca.gov/>

This email may contain attorney work product and may include privileged material protected by the attorney-client privilege or governmental privileges. This email may not be disclosed to third parties without the express consent of the sender. If you think you received this e-mail in error, please notify the sender immediately.

Good afternoon Todd,

Please see the attached letter regarding your California Public Records Act request.

Thank you,

Nicholas Scheuer
Attorney
Legal Office, MIC 53
Directorate
916-208-9468 Work Cell
916-654-9069 Fax
nicholas.scheuer@edd.ca.gov <mailto:nicholas.scheuer@edd.ca.gov>

State of California
Employment Development Department
PO Box 826880, Sacramento, CA 94280-0001
www.edd.ca.gov<http://www.edd.ca.gov/>

This email may contain attorney work product and may include privileged material protected by the attorney-client privilege or governmental privileges. This email may not be disclosed to third parties without the express consent of the sender. If you think you received this e-mail in error, please notify the sender immediately.

Good afternoon Todd,

Please see the attached letter regarding your California Public Records Act request.

Thank you,

Nicholas Scheuer
Attorney
Legal Office, MIC 53
Directorate
916-208-9468 Work Cell
916-654-9069 Fax
nicholas.scheuer@edd.ca.gov <mailto:nicholas.scheuer@edd.ca.gov>

State of California
Employment Development Department
PO Box 826880, Sacramento, CA 94280-0001
www.edd.ca.gov<http://www.edd.ca.gov/>

This email may contain attorney work product and may include privileged material protected by the attorney-client privilege or governmental privileges. This email may not be disclosed to third parties without the express consent of the sender. If you think you received this e-mail in error, please notify the sender immediately.

Good afternoon Todd,

Please see the attached letter regarding your California Public Records Act request.

Thank you,

Nicholas Scheuer
Attorney
Legal Office, MIC 53
Directorate
916-208-9468 Work Cell
916-654-9069 Fax
nicholas.scheuer@edd.ca.gov <mailto:nicholas.scheuer@edd.ca.gov>

State of California
Employment Development Department
PO Box 826880, Sacramento, CA 94280-0001
www.edd.ca.gov<http://www.edd.ca.gov/>

This email may contain attorney work product and may include privileged material protected by the attorney-client privilege or governmental privileges. This email may not be disclosed to third parties without the express consent of the sender. If you think you received this e-mail in error, please notify the sender immediately.

Good afternoon Todd Feathers,

This email is the Employment Development Department's (EDD) response to your Public Records Act request, received March 15, 2023. The text of your request is copied below:
Pursuant to the California Public Records Act, I request the following records regarding algorithmic tools used by your agency to guide decision making for the administration of Social Security Disability Insurance (SSDI), Supplemental Security Income (SSI), and/or supplemental state disability benefits:

For the purposes of this request, the term algorithmic tools refers to computer software or code that employs machine learning techniques (sometimes described as artificial intelligence or predictive analytics). It does not apply to other types of algorithms, such as Excel formulas.

My request applies to all algorithmic tools developed in-house by your agency, purchased for use by your agency, or otherwise used by your agency since Jan. 1, 2018 to help administer Social Security Disability Insurance (SSDI), Supplemental Security Income (SSI), and/or supplemental state disability benefits. Such algorithmic tools include, but are not limited to, models for:

- Fraud detection
- Rate setting
- Benefit use prediction (e.g., models that predict for how long individuals will require benefits or what the total cost will be)
- Enrollment prediction (e.g., models that predict enrollment rates among the larger population)
- Case management (e.g., models that group similar cases together or determine caseloads for adjudicators)
- Investigations (e.g., models that prioritize complaints for investigation or flag claims for additional review)
- Advertising/outreach prioritization (e.g. models used to decide where or to which groups to advertise)

For each algorithmic tool your agency has used since Jan. 1, 2018, I request the following records:

1) Executive summaries, project overviews, or comparable records that describe the scope and purpose of the algorithmic tool.

2) Contracts, statements of work, successful RFP submissions and bid packets, or comparable records that describe the purchase and/or construction of the algorithmic tool.

3) User handbooks, guides, training manuals, agency policies, or comparable records that describe the tool's functions and its appropriate uses..
Social Security Disability Insurance (SSDI) and Supplemental Security Income (SSI) are not administered by EDD, so EDD does not possess any responsive documents regarding those two programs. EDD administers the California State Disability Insurance (SDI) program, the "supplemental state disability benefits" referenced above. This response addresses each of your three numbered requests as they apply to SDI, below.

1) Executive summaries, project overviews, or comparable records that describe the scope and purpose of the algorithmic tool.
EDD used two services that utilized algorithmic tools during the time frame specified in your request. Here are the executive summaries for both:
Thomson Reuter Executive Summary:
Thomson Reuter's Fraud Detect solution will be an enterprise system consolidated with the EDD Unemployment and Disability Insurance Fraud Detect System. An enterprise system will allow users to view the Customer's Data for Unemployment Insurance Claimants, Disability Insurance Claimants, Employers, and Providers. Thomson Reuter's Fraud Detect suite was designed to provide a one-stop solution comprised of varied modular components combined with powerful analytics, special investigation services, and intuitive design to deliver the most important leads to analysts and investigators. Employing advanced prediction and machine-learning algorithms allows Thomson Reuter to continually improve detection accuracy to identify outliers and fraud schemes to provide valuable data insights. Intuitive analytics consisting of trends, patterns, and clusters are delivered through easy to interpret graphs, link charts, and tables through a multi-varied and visually representative dashboard and profiles.

ID.me Executive Summary:
ID.me Identity verification solution was purchased through a third party reseller, V3Gate LLC. It is an enterprise system consolidated with the EDD Unemployment and Disability Insurance Online Systems. The Identity Verification Solution provides higher confidence in the identity of users filing an Unemployment Insurance claim or users registering into Disability Insurance Online than the current data match based approach. ID.me allows to lower false classification of valid claimants as potentially fraudulent users, higher detection of users actually providing fraudulent identity information. The capability of identity verification solution is integrated with EDD's online Unemployment Insurance, providing real-time transaction during the claim filing process so that, after identity has been verified, the claim can be automatically processed. The capability of identity verification solution is integrated with EDD's online Disability Insurance, providing real-time transaction during the account registration process so that, after identity has been verified, the user is allowed to complete the account registration which allows them to file for disability claims. ID.me Solution follows NIST 800-63B standard and be capable of meeting IAL/AAL2 levels of assurance for both identification and authentication. ID.me solution complies with all Federal and State of California privacy laws in safeguarding identity data.

2) Contracts, statements of work, successful RFP submissions and bid packets, or comparable records that describe the purchase and/or construction of the algorithmic tool.
EDD purchased services that use algorithmic tools from two vendors, Thomson Reuters and V3Gate LLC. The responsive Thomson Reuters contracts are attached to this email. There are five responsive V3Gate LLC contracts: the original agreement and four subsequent amendments. You may access the original V3Gate LLC agreement and Amendment 1 by clicking this link, or by copying it into your web browser: https://covid19.ca.gov/contracts/. Amendments 2 through 4 are attached to this email.

Federal Employer Identification Numbers (FEINs) have been redacted from the attached contracts under Government Code § 7922.000, because the public interest in revealing a vendor's FEIN is outweighed by the public interest in protecting a vendor's identifying tax information. For the contracts that include a "Contract Request" attachment, Contract Requests are completed by EDD before a contract or a contract amendment is executed and contains deliberative language. For that reason, such language has been redacted under Government Code § 7922.000. Finally, detailed descriptions of Pondera's FraudCaster service, per the California State Legislature's finding that the interest in the public disclosure of EDD's fraud deterrence information is outweighed by the interest in maintaining the confidentiality of this information. (2021 AB 138, sec. 19)

Statements of work have been withheld for the reason listed for item 3, below.

3) User handbooks, guides, training manuals, agency policies, or comparable records that describe the tool's functions and its appropriate uses.
Producing unredacted statements of work, user handbooks, guides, training manuals, agency policies, or comparable records for EDD's fraud prevention and deterrence tools risks revealing information that may enable bad actors to work around those tools, making it easier to commit fraud. Additionally, the California state legislature found the interest in the public disclosure of EDD's fraud deterrence information is outweighed by the interest in maintaining the confidentiality of this information. (2021 AB 138, sec. 19) Therefore, these records have been withheld.

Thank you,

Nicholas Scheuer
Attorney
Legal Office, MIC 53
Directorate
916-208-9468 Work Cell
916-654-9069 Fax
nicholas.scheuer@edd.ca.gov <mailto:nicholas.scheuer@edd.ca.gov>

State of California
Employment Development Department
PO Box 826880, Sacramento, CA 94280-0001
www.edd.ca.gov<http://www.edd.ca.gov/>

This email may contain attorney work product and may include privileged material protected by the attorney-client privilege or governmental privileges. This email may not be disclosed to third parties without the express consent of the sender. If you think you received this e-mail in error, please notify the sender immediately.

Good afternoon.

Please refer to the attached document regarding your PRA request.

Thank you,
EDD Media Services