Research Recent work with the PSC
National Quality Forum: The PSC has been endorsed by the National Quality Forum (NQF) as an outcome measure to improve the quality of care provided to children. The NQF is a voluntary and non-profit consensus standards-setting organization whose mission is to improve the quality of health care delivered in the United States. To that end, the NQF embodies a three-part mission -- to set goals for performance improvement, to endorse standards for measuring and reporting on performance, and to promote educational and outreach programs (www.qualityforum.org). In January 2011, the PSC received provisional approval from the NQF as a performance measure in child behavioral healthcare. In doing so, the PSC became one of only fifteen endorsed child health measures, and the only one to focus exclusively on child mental health. NQF gave the PSC its full endorsement on April 12, 2013.
Studies exploring the validity of the PSC
In a number of validity studies, PSC case classifications agreed with case classifications on the Child Behavior Checklist (CBCL), Children's Global Assessment Scale (CGAS) ratings of impairment, and the presence of psychiatric disorder in a variety of pediatric and subspecialty settings representing diverse socioeconomic backgrounds (Jellinek et al., 1988; Jellinek et al., 1991; Rauch et al., 1991; Murphy et al., 1992a; Murphy et al., 1996). When compared to the CGAS in both middle and lower income samples, the PSC has shown high rates of overall agreement (79%; 92%), sensitivity (95%; 88%) and specificity (68%; 100%), respectively (Jellinek, 1986; Jellinek et al., 1988; Murphy et al., 1992a). Studies using the PSC have found prevalence rates of psychosocial impairment in middle class or general settings (~12%) that are quite comparable to national estimates of the prevalence of psychosocial problems (Jellinek et al., 1988; Jellinek et al., 1991; Little et al., 1994; Murphy et al., 1992a; Murphy et al., 1996; Rauch et al., 1991).
Factor analysis has found that the measure loads significantly onto three brief subscales for use in identification of attentional, internalizing (depression/anxiety), and conduct problems (Gardner at al., 1999), which have relatively high rates of agreement were found with specific diagnoses on a structured psychiatric rating scale (the K-SADS-PL) and with some of the most commonly used outcome measures in child psychiatry, including the Children’s Depression Inventory (CDI) for depression, the Screen for Child Anxiety Related Emotional Disorder (SCARED) for anxiety, and the ADHD scale of the Child Behavior Checklist for attention problems. Additionally, a youth self report version of the measure has been developed and validated for use (Gall et al., 2000; Pagano et al., 2000).
One reason that the PSC continues to be listed as a recommended instrument is that virtually all of the studies that have assessed it have found the measure to be valid and reliable. With one important caveat, all two dozen or so studies that assessed the validity of the PSC in various heterogeneous or homogeneous subpopulations have confirmed such. This one caveat concerns the specific cut-off score to use to indicate dysfunction and/or the need for further evaluation in any given population. Although two of the largest and most diverse samples studied to date (Bernal et al., 2000; Kelleher et al., 1998) found that the originally recommended cut-off score of 28 for school aged children appeaed to be optimal and that this cut-off appeared to be correct for most of the many subgroups studied (Jellinek et al., 1999), a number of other studies have suggested the need to determine the optimal cut-off score when new populations are studied. These findings will be reviewed below.
Although the PSC was well-validated in the 1980’s and 1990’s, new studies assessing the validity of the PSC have been necessitated as it has continued to be used with new populations, in both ethnic subgroups in the US and in non-US samples. Over the past decade, the PSC has been translated into many languages and used in studies in more than a dozen countries on all of the major continents. Published studies have documented the validity of the PSC in Japan (Ishizaki et al., 1997; Ishizaki et al., 2000; Ishizaki et al., 2002; Ishizaki et al., 2005a; Ishizaki et al., 2005b; Ishizaki et al., 2006; Oyama et al., 2002), Holland (Reijneveld et al., 2006), Austria (Herzog et al., 2007),Chile (de la Barra et al., 2005), the Philippines (Cancecko-Llego, 2009), and Botswana (Lowenthal et al., 2011). Unpublished studies have also been conducted in the Philippines, India, Israel, Korea, Tanzania, and Australia.
The PSC has also been validated for use with specific ethnic subgroups within the US, including Mexicans (Leiner et al., 2007) and Mexican-Americans (Jutte et al., 2003; Murphy et al., 1996) seen in low-income US communities. The caveat about the need to establish an appropriate cut-off score was particularly important for the former study where optimum sensitivity and specificity were found using a cut-off score of 12. The latter study broke new ground by validating the use of a pictorial form of the PSC.
Studies on use of the PSC as a marker for psychosocial dysfunction
Previous studies using a variety of measures have consistently shown that the prevalence of psychosocial impairment varies considerably based on a number of socio-demographic risk factors, and research with the PSC has paralleled many of these findings. For example, low socioeconomic status (Jensen et al., 1990), living with a single parent (Rutter, 1981), parental mental illness (Orvaschel et al., 1981; Pfeffer et al., 1986), family discord (Earls and Jung, 1987), child's temperamental characteristics, and male sex (Earls & Jung, 1987; Kashani et al., 1987) have all been shown to increase the probability of psychosocial dysfunction. Consistent with these findings, studies using the PSC have shown the prevalence of child psychosocial dysfunction to be two to three times higher in children from low income (Murphy et al., 1992a), single-parent (Murphy et al., 1992a) and/or mentally ill parent families (Jellinek et al., 1991).
By the mid 1990’s, the PSC had been validated in traditional pediatric practices serving both white middle class (Jellinek et al., 1986; Jellinek et al., 1988) and minority/low income samples (Jellinek and Murphy, 1988; Murphy et al., 1992a; Murphy et al., 1992b) Beginning in the mid 1990’s, the first papers from a national sample that used the PSC began to come out. With funding from the National Institute of Mental Health, Kelley Kelleher and his colleagues launched the Child Behavior Study (CBS) in a nationally representative sample of more than 20,000 children seen in US pediatric and family practice offices. This study has already resulted in more than a dozen published papers (Gardner et al., 1999; Gardner et al., 2001; Gardner et al., 2002a; Gardner et al., 2003; Gardner et al., 2007; Kelleher & Wolraich, 1996; Kelleher et al., 1998; Kelleher et al., 1999; Kelleher et al., 2000; Palermo et al., 2002; Walders et al., 2003; Wasserman et al., 1999; Wren et al., 2003); most of which explore pediatrician recognition of psychosocial problems in large national samples of pediatric and family practice visits. In general, these studies confirmed previous estimates of about 12% prevalence of psychosocial problems and that these problems were under-recognized and under-referred. These studies also examined the impact of insurance, gender, race, and other factors on patterns of recognition and referral and the relationship between PSC and related measures like family functioning and physician attitudes.
A by-product of the CBS study is that Gardner and Kelleher used the power of the large number of cases in their sample (20,000+) to develop a briefer version of the PSC. Their studies have provided both statistical (Gardner et al., 1999) and clinical (Gardner, 2007) validation for a 17 item form of the PSC. Other authors have used this form as well (Borowsky et al., 2003; Duke et al., 2005; Miller et al., 2007; Ohene et al., 2006).
In addition to the Kelleher et al. CBS studies cited above, another dozen studies have used the PSC as a marker to explore the prevalence of psychosocial problems in other pediatric settings like HMO’s (Bernal et al., 2000; Navon et al., 2001), low income practices billing Medicaid (Murphy et al., 1996), State Children’s Health Insurance Programs (SCHIP) (Brickman et al., 2002), and well-child vs. sick visits (Borowsky et al., 2003). These studies have generally confirmed a higher than average prevalence rate of psychosocial problems among poor children and/or under-recognition and under-referral (Murphy et al., 2008).
Another group of studies have used the PSC to explore the prevalence of psychosocial problems among children with specific pediatric problems like dermatological conditions (Hansen, 1997), neurological problems (Anderson et al., 1999), thalesemia (Louthrenoo et al., 2002; Saini et al., 2007), mental retardation (Harrison et al., 2006), hunger (Miller et al., 2007), pediatric gastroenterology (Reed-Knight et al., 2011), and prenatal substance exposure (Whitaker at al., 2011). These studies have generally shown that PSC positive rates are two to three times more prevalent in children with chronic conditions.
Studies on use the PSC as an outcome measure for interventions
The PSC is increasingly being used as a pre-post measure to assess the impact of interventions. Studies have shown significant decreases in PSC scores for children who receive mental health interventions in a child psychiatry clinic that are comparable to those found with other standardized measures (Murphy et al., 2007; Murphy, 2007; Murphy et al., 2008; Murphy et al., 2011; Murphy et al., 2012) and YET another has used the PSC as an outcome measure to assess the impact of a school based mental health intervention (Stein et al., 2003). The PSC has also been used to assess the effects of school breakfast programs (Kleinman et al., 2002; Murphy et al., 1998), with significant improvements in functioning demonstrated in all studies.
Studies on use of the PSC as a clinical measure for integrated healthcare
The PSC has been used as clinical measure in a number of large public health initiatives. The Cambridge Health Alliance has mandated psychosocial screening with the PSC in a number of its outpatient facilities for the past five years and research already published (Hacker et al., 2006) has shown the PSC can be utilized for routine use and combined with other innovative elements in an integrated approach to health and behavioral health in a large, urban pediatric system of care. Research focused on examining what factors contribute to change in PSC score (Hacker et al 2006) found that there were statistically signiﬁcant relationships between a positive PSC score and being in counseling, parental/personal concern, having public insurance, and living in an area with median household incomes of less than $50,000.
The Cambridge Health Alliance Pediatric Mental Health Screening and Intervention project was funded by the U.S. Maternal and Child Health Bureau (MCHB) as one of six sites selected to implement an innovative model as part of its Integrated Health and Behavioral Healthcare Initiative. All six implementation sites chose the PSC as the measure to use for the screening component of their integrated health and behavioral health initiatives (Murphy & Wieneke, 2006). The Cambridge Alliance project was originally funded through the Healthy Tomorrows program, another project of MCHB (and the American Academy of Pediatrics [AAP]) that similarly advocated for increased coordination of services in pediatrics.
The PSC is mentioned prominently in Bright Futures Mental Health (BFMH) (Jellinek et al., 2002a; Jellinek et al., 2002b), another joint project of MCHB and AAP that advocates for more integrated health and behavioral healthcare. BFMH has achieved worldwide dissemination and made psychosocial screening more accessible to pediatric practices by including measures like the PSC that can be photocopied and distributed, as well as instructions for their use and recommendations for clinical practices to encourage more integrated health and behavioral healthcare.