Allegheny Family Screening Tool
The Allegheny Family Screening Tool (AFST) is a predictive risk model that supports child welfare call screeners and supervisors in deciding how to respond to reports of suspected child abuse or neglect. DHS began using the AFST in August 2016 in the child welfare intake office, which followed a multi-year process that included procurement, community engagement, validation, ethical review, and independent evaluation.
The AFST was the first tool of its kind used in child welfare decision-making and has received national and international attention. DHS has since provided technical assistance to other jurisdictions exploring predictive risk modeling, helping them assess whether these tools are appropriate and how to design them responsibly.
How It Works
The AFST analyzes integrated data from the Allegheny County Data Warehouse and generates a risk score that estimates the likelihood of a child experiencing harm within two years. Staff use the score alongside their experience and clinical expertise to decide whether to open a formal investigation, provide services or referrals, or offer other less intrusive supports.
The AFST is designed to support, not replace, professional judgment. By quickly analyzing large amounts of data, the AFST helps staff make more informed decisions, which would otherwise take significant time to do manually.
Development
In 2014, with support from the Richard King Mellon Foundation, DHS launched a highly competitive procurement process to solicit proposals(PDF, 2MB) to explore how we could better use these data to improve decision making through predictive risk modeling.
The contract was awarded to a team from Auckland University of Technology (AUT) led by the co-director of the Centre for Social Data Analytics, Rhema Vaithianathan, and including Emily Putnam-Hornstein from the University of Southern California, Irene de Haan from the University of Auckland, Marianne Bitler from the University of California, Irvine, and Tim Maloney and Nan Jiang from AUT.
The video below explains DHS’s partnership with Rhema Vaithianathan in the development and implementation of a predictive risk model for child welfare.
Evaluation and Ethical Review
Formal evaluation of the Allegheny Family Screening Tool implementation process and impact began in 2015, following a DHS issued RFP for assessments in these areas. Two separate contracts were awarded.
Hornby Zeller Associates, Inc., a public sector evaluation, research and consulting firm, was awarded the task of examining the process by which the tool was implemented. The firm analyzed how the tool has changed the experience of call screeners, the practice and policy implications of its use, and perceptions and reactions to the tool. Their report, Allegheny County Predictive Risk Modeling Tool Implementation: Process Evaluation(PDF, 735KB) was published in January 2018.
Jeremy Goldhaber-Fiebert, Assistant Professor of Medicine at Stanford University, examined the impact of the tool on the accuracy of decisions, disparity rates, overall referral rates and workload and more. A summary of the impact evaluation(PDF, 190KB) and the full evaluation report(PDF, 2MB) were published in April 2019.
An independent ethical analysis conducted by Tim Dare (University of Auckland) and Eileen Gambrill (UC Berkeley) found the tool to be ethically appropriate, particularly because it improved accuracy compared to prior approaches. DHS also published a formal response to this analysis(PDF, 75KB). A detailed methodology and implementation report was released in May 2019.
For more information about the AFST and its development, implementation and evaluations, visit Allegheny Analytics.
Research and Policy
Media Coverage