RI child welfare predicitive analytics models
| Submitted | June 18, 2021 |
MuckRock users can file, duplicate, track, and share public records requests like this one. Learn more.
Communications
From: Todd Feathers
To Whom It May Concern:
Pursuant to the Rhode Island Access to Public Records Act, I hereby request the following records:
1) Mathematical documentation of all predictive analytics models that test for factors that predict child welfare outcomes developed by the Division of Performance Improvement Data Analytics and Evaluation. This should specifically include the raw algorithms and statistical equations underlying the models, the data dictionaries for these models, and any performance evaluations of these models. These models are directly referenced by DCYF in its response to my recent public records act request (see attached)
2) All policy documents, instruction manuals, guidelines, or other documents explaining how DCYF's child welfare predictive analytics models should be developed and/or used by DCYF employees.
3) Aggregate data--broken down by the below listed demographic groups-- from Jan. 1, 2016 until the date this request is processed for the child welfare outcomes measured by DCYF's predictive analytics models.
- Race
- Gender
- Disability
- Race AND gender
Please note that I am asking for aggregate demographic data and not data that could be used to identify any individuals. DCYF maintains this type of data, as demonstrated by the publicly available data analytics reports that can be found here: http://www.dcyf.ri.gov/data-evaluation/permanency-data-analytics-report.php
I ask that all fees be waived as I am a journalist and intend to use the requested records to publish articles in the public interest. In the event you choose to impose fees, I request a detailed breakdown of the fees, including the hourly wage of each employ involved and an explanation justifying the employee hours required to fulfill the request.
Should you choose to reject this request or redact portions of it, I ask that you provide a detailed breakdown of the statutory exemptions and associated case law underlying your decision to withhold each/any portions from public review.
In the event that there are fees, I would be grateful if you would inform me of the total charges in advance of fulfilling my request. I would prefer the request filled electronically, by e-mail attachment if available or CD-ROM if not.
Thank you in advance for your anticipated cooperation in this matter. I look forward to receiving your response to this request within 10 business days, as the statute requires.
Sincerely,
Todd Feathers
From: Department Of Children, Youth And Families
Hello:
We have attached the DCYF response to your APRA Request. By way of informal narrative and further discussion ( not APRA) please note that The Department does not use any algorithms for predictive analytics. There are no algorithms in the Department's MIS database used for predictive analytics.
RI DCYF's Division of Performance Improvement Data Analytics and Evaluation unit performs statistical analysis including predictive analytic models that test for factors that predict child welfare outcomes. RI DCYF DPI Division is led by Dr. Colleen Caron who holds a Ph.D. in Public Health Epidemiology and is also an Assistant Professor of Practicing Epidemiology at Brown University School of Public Health. RI DCYF DPI Data Analytics and Evaluation unit is comprised of 5 master level epidemiologists. The predictive analytic models are designed by Dr. Caron and the epidemiologists based on the research question which guides the selection of the population, the factors entered into the statistical model, the model type and the outcome. RI DCYF DPI epidemiologists use SAS software for statistical analysis. RI DCYF does not use a software that auto generates or machine learning predictive models. The Department process allows for only the DPI epidemiologists to perform data analysis and advanced statistical analysis including predictive analytic models.
The epidemiologists meet on a weekly basis and design, plan and review the analysis to ensure the following (not an exhaustive list of steps):
-What is the research question(s)
-What is the research question based on (trend data routinely monitored, question from the practice, senior team, etc.)
-Based on the research question staff identify the methodology- qualitative vs quantitative or both; descriptive statistics and advanced statistics; design-longitudinal, cross sectional, retrospective, etc.; the statistical tests-multivariate regression, survival analysis, GEE, chi-square, t-test etc.
-If quantitative and the data exists, based on the research question and design staff identify the data to have MIS pull the data and how to pull the data
-MIS pulls the data
-Epidemiologists conduct a data quality check via frequencies, cross tabs, etc. and data checks with the raw data pull and the SACWIS data system (SAS is used for statistical analysis)
-Any modifications based on the data quality check, another data pull may be needed
-Epidemiologists go through standard data analytic procedures in part depending on the design and statistical tests, such as descriptive statistics, correlations, t-tests, multivariate regression etc.
-Results are reviewed among the epidemiologists and data code checked.
-All predictive models are accompanied with a presentation to staff associated with the particular program(s) the data impacts with complete description on the methods, data sources, data notes and caution with Ecological fallacy. Related to ecological fallacy is the recommendation for staff to rely on the individual assessments that caseworkers and\or providers administer to avoid ecological fallacy.
Sincerely
Deborah Buffi
Deborah Buffi
Associate Director/Acting Deputy Director
Department of Children, Youth and Families
101 Friendship Street
Providence, RI
401 528-3582
Files
pages