Pediatric Symptom Checklist

Research Recent work with the PSC

National Quality Forum: The PSC was endorsed by the National Quality Forum (NQF) as an outcome measure to improve the quality of care provided to children. The NQF is a voluntary and non-profit consensus standards-setting organization whose mission is to improve the quality of health care delivered in the United States. To that end, the NQF embodies a three-part mission -- to set goals for performance improvement, to endorse standards for measuring and reporting on performance, and to promote educational and outreach programs (www.qualityforum.org). In January 2011, the PSC received provisional approval from the NQF as a performance measure in child behavioral healthcare. In doing so, the PSC became one of only fifteen endorsed child health measures, and the only one to focus exclusively on child mental health. NQF gave the PSC its full endorsement from April 12, 2013 to May 2015. Currently, the PSC is under review for measure maintenance and re-endorsement.

Studies exploring the validity of the PSC

In a number of validity studies, PSC case classifications agreed with case classifications on the Child Behavior Checklist (CBCL), Children's Global Assessment Scale (CGAS) ratings of impairment, and the presence of psychiatric disorder in a variety of pediatric and subspecialty settings representing diverse socioeconomic backgrounds (Jellinek et al., 1988; Jellinek et al., 1991; Rauch et al., 1991; Murphy et al., 1992a; Murphy et al., 1996). When compared to the CGAS in both middle and lower income samples, the PSC has shown high rates of overall agreement (79%; 92%), sensitivity (95%; 88%) and specificity (68%; 100%), respectively (Jellinek, 1986; Jellinek et al., 1988; Murphy et al., 1992a). Studies using the PSC have found prevalence rates of psychosocial impairment in middle class or general settings (~12%) that are quite comparable to national estimates of the prevalence of psychosocial problems (Jellinek et al., 1988; Jellinek et al., 1991; Little et al., 1994; Murphy et al., 1992a; Murphy et al., 1996; Rauch et al., 1991).

Factor analysis has found that the measure loads significantly onto three brief subscales for use in identification of attentional, internalizing (depression/anxiety), and conduct problems (Gardner at al., 1999), which have relatively high rates of agreement were found with specific diagnoses on a structured psychiatric rating scale (the K-SADS-PL) and with some of the most commonly used outcome measures in child psychiatry, including the Children’s Depression Inventory (CDI) for depression, the Screen for Child Anxiety Related Emotional Disorder (SCARED) for anxiety, and the ADHD scale of the Child Behavior Checklist for attention problems. Additionally, a youth self report version of the measure has been developed and validated for use (Gall et al., 2000; Pagano et al., 2000).

One reason that the PSC continues to be listed as a recommended instrument is that virtually all of the studies that have assessed it have found the measure to be valid and reliable. With one important caveat, all two dozen or so studies that assessed the validity of the PSC in various heterogeneous or homogeneous subpopulations have confirmed such. This one caveat concerns the specific cut-off score to use to indicate dysfunction and/or the need for further evaluation in any given population. Although two of the largest and most diverse samples studied to date (Bernal et al., 2000; Kelleher et al., 1998) found that the originally recommended cut-off score of 28 for school aged children appeaed to be optimal and that this cut-off appeared to be correct for most of the many subgroups studied (Jellinek et al., 1999), a number of other studies have suggested the need to determine the optimal cut-off score when new populations are studied. These findings will be reviewed below. Although the PSC was well-validated in the 1980’s and 1990’s, new studies assessing the validity of the PSC have been necessitated as it has continued to be used with new populations, in both ethnic subgroups in the US and in non-US samples. Over the past decade, the PSC has been translated into many languages and used in studies in more than a dozen countries on all of the major continents. Published studies have documented the validity of the PSC in Japan (Ishizaki et al., 1997; Ishizaki et al., 2000; Ishizaki et al., 2002; Ishizaki et al., 2005a; Ishizaki et al., 2005b; Ishizaki et al., 2006; Oyama et al., 2002), Holland (Reijneveld et al., 2006), Austria (Herzog et al., 2007),Chile (de la Barra et al., 2005), the Philippines (Cancecko-Llego, 2009), and Botswana (Lowenthal et al., 2011). Unpublished studies have also been conducted in the Philippines, India, Israel, Korea, Tanzania, and Australia.

The PSC has also been validated for use with specific ethnic subgroups within the US, including Mexicans (Leiner et al., 2007) and Mexican-Americans (Jutte et al., 2003; Murphy et al., 1996) seen in low-income US communities. The caveat about the need to establish an appropriate cut-off score was particularly important for the former study where optimum sensitivity and specificity were found using a cut-off score of 12. The latter study broke new ground by validating the use of a pictorial form of the PSC.

Studies on use of the PSC as a marker for psychosocial dysfunction

Previous studies using a variety of measures have consistently shown that the prevalence of psychosocial impairment varies considerably based on a number of socio-demographic risk factors, and research with the PSC has paralleled many of these findings. For example, low socioeconomic status (Jensen et al., 1990), living with a single parent (Rutter, 1981), parental mental illness (Orvaschel et al., 1981; Pfeffer et al., 1986), family discord (Earls and Jung, 1987), child's temperamental characteristics, and male sex (Earls & Jung, 1987; Kashani et al., 1987) have all been shown to increase the probability of psychosocial dysfunction. Consistent with these findings, studies using the PSC have shown the prevalence of child psychosocial dysfunction to be two to three times higher in children from low income (Murphy et al., 1992a), single-parent (Murphy et al., 1992a) and/or mentally ill parent families (Jellinek et al., 1991).

By the mid 1990’s, the PSC had been validated in traditional pediatric practices serving both white middle class (Jellinek et al., 1986; Jellinek et al., 1988) and minority/low income samples (Jellinek and Murphy, 1988; Murphy et al., 1992a; Murphy et al., 1992b) Beginning in the mid 1990’s, the first papers from a national sample that used the PSC began to come out. With funding from the National Institute of Mental Health, Kelley Kelleher and his colleagues launched the Child Behavior Study (CBS) in a nationally representative sample of more than 20,000 children seen in US pediatric and family practice offices. This study has already resulted in more than a dozen published papers (Gardner et al., 1999; Gardner et al., 2001; Gardner et al., 2002a; Gardner et al., 2003; Gardner et al., 2007; Kelleher & Wolraich, 1996; Kelleher et al., 1998; Kelleher et al., 1999; Kelleher et al., 2000; Palermo et al., 2002; Walders et al., 2003; Wasserman et al., 1999; Wren et al., 2003); most of which explore pediatrician recognition of psychosocial problems in large national samples of pediatric and family practice visits. In general, these studies confirmed previous estimates of about 12% prevalence of psychosocial problems and that these problems were under-recognized and under-referred. These studies also examined the impact of insurance, gender, race, and other factors on patterns of recognition and referral and the relationship between PSC and related measures like family functioning and physician attitudes.

A by-product of the CBS study is that Gardner and Kelleher used the power of the large number of cases in their sample (20,000+) to develop a briefer version of the PSC. Their studies have provided both statistical (Gardner et al., 1999) and clinical (Gardner, 2007) validation for a 17 item form of the PSC. Other authors have used this form as well (Borowsky et al., 2003; Duke et al., 2005; Miller et al., 2007; Ohene et al., 2006).

In addition to the Kelleher et al. CBS studies cited above, another dozen studies have used the PSC as a marker to explore the prevalence of psychosocial problems in other pediatric settings like HMO’s (Bernal et al., 2000; Navon et al., 2001), low income practices billing Medicaid (Murphy et al., 1996), State Children’s Health Insurance Programs (SCHIP) (Brickman et al., 2002), and well-child vs. sick visits (Borowsky et al., 2003). These studies have generally confirmed a higher than average prevalence rate of psychosocial problems among poor children and/or under-recognition and under-referral (Murphy et al., 2008).

Another group of studies have used the PSC to explore the prevalence of psychosocial problems among children with specific pediatric problems like dermatological conditions (Hansen, 1997), neurological problems (Anderson et al., 1999), thalesemia (Louthrenoo et al., 2002; Saini et al., 2007), mental retardation (Harrison et al., 2006), hunger (Miller et al., 2007), pediatric gastroenterology (Reed-Knight et al., 2011), and prenatal substance exposure (Whitaker at al., 2011). These studies have generally shown that PSC positive rates are two to three times more prevalent in children with chronic conditions.

Studies on use the PSC as an outcome measure for interventions

The PSC is increasingly being used as a pre-post measure to assess the impact of interventions. Studies have shown significant decreases in PSC scores for children who receive mental health interventions in a child psychiatry clinic that are comparable to those found with other standardized measures (Murphy et al., 2007; Murphy, 2007; Murphy et al., 2008; Murphy et al., 2011; Murphy et al., 2012) and YET another has used the PSC as an outcome measure to assess the impact of a school based mental health intervention (Stein et al., 2003). The PSC has also been used to assess the effects of school breakfast programs (Kleinman et al., 2002; Murphy et al., 1998), with significant improvements in functioning demonstrated in all studies.

Studies on use of the PSC as a clinical measure for integrated healthcare

The PSC has been used as clinical measure in a number of large public health initiatives. The Cambridge Health Alliance has mandated psychosocial screening with the PSC in a number of its outpatient facilities for the past five years and research already published (Hacker et al., 2006) has shown the PSC can be utilized for routine use and combined with other innovative elements in an integrated approach to health and behavioral health in a large, urban pediatric system of care. Research focused on examining what factors contribute to change in PSC score (Hacker et al 2006) found that there were statistically signi´Čücant relationships between a positive PSC score and being in counseling, parental/personal concern, having public insurance, and living in an area with median household incomes of less than $50,000.

The Cambridge Health Alliance Pediatric Mental Health Screening and Intervention project was funded by the U.S. Maternal and Child Health Bureau (MCHB) as one of six sites selected to implement an innovative model as part of its Integrated Health and Behavioral Healthcare Initiative. All six implementation sites chose the PSC as the measure to use for the screening component of their integrated health and behavioral health initiatives (Murphy & Wieneke, 2006). The Cambridge Alliance project was originally funded through the Healthy Tomorrows program, another project of MCHB (and the American Academy of Pediatrics [AAP]) that similarly advocated for increased coordination of services in pediatrics.

The PSC is mentioned prominently in Bright Futures Mental Health (BFMH) (Jellinek et al., 2002a; Jellinek et al., 2002b), another joint project of MCHB and AAP that advocates for more integrated health and behavioral healthcare. BFMH has achieved worldwide dissemination and made psychosocial screening more accessible to pediatric practices by including measures like the PSC that can be photocopied and distributed, as well as instructions for their use and recommendations for clinical practices to encourage more integrated health and behavioral healthcare.

About the PSC

The Pediatric Symptom Checklist is a one-page questionnaire listing a broad range of children's emotional and behavioral problems that reflects parents' impressions of their child's psychosocial functioning. The screen is intended to facilitate the recognition of emotional and behavioral problems so that appropriate interventions can be initiated as early as possible.

The PSC consists of 35 items that are rated as “Never” “Sometimes,” or “Often” present and scored 0, 1, and 2, respectively. The total score is calculated by adding together the score for each of the 35 items. Cutoff scores for pre-school and school-age children indicating clinical levels of dysfunction have been empirically derived using Receiver Operator Characteristic analyses in studies comparing the performance of the PSC to other validated questionnaires and clinicians' assessments of children's overall functioning (Jellinek, 1986; Jellinek et al., 1986; Little et al., 1994). Positive screens are those with scores above 27 for ages 6-18 and scores of 24 and higher for children ages 4 and 5. For thePSC-Y, the cutoff is 30 and higher and for the PSC-17, the cutoff score is 15. More detailed instructions on scoring the PSC can be found in the Scoring the PSC.

Factor analysis of the full set of 35 items has also led to the validation of three subscales for use in identification of attentional, internalizing, and externalizing problems. A description of the items in each of these scales and scoring instructions can also be found in Scoring the PSC.

Background

The philosophy of routine psychosocial screening using the Pediatric Symptom Checklist is congruent with the evolving concept of a “medical home” for children, which is a key facet of the redesigned healthcare system that is being put into place in the US in 2012 (Deloitte Center for Health Solutions, 2008). The pediatric medical home is well described on the AAP website (http://pediatrics.aappublications.org/content/110/1/184.full.pdf). The concept of the medical home encourages a stronger focus on the integration of emotional and behavioral issues into routine pediatric care, especially through the earlier identification of and intervention for children who have or are at risk for emotional, social, or behavioral disorders.

The current focus on mental health as a key aspect of physical health builds on many efforts over the past decade. Early mental health screening was included as a named goal by the President's 2003 New Freedom Commission on Mental Health. Early Periodic Screening, Diagnosis, and Treatment (EPSDT), the regulations governing well child care for children with Medicaid health insurance, includes a clear requirement for routine psychosocial screening. The Surgeon General’s national health goals (Healthy People 2010) included the specific recommendation that pediatricians and other primary care physicians include routine screening for mental health problems as a part of primary care. In light of the growing recognition that mental health is a critical component of overall wellbeing and quality of life in children and adults, the American Academy of Pediatrics (AAP) recently issued a call for pediatricians to consider mental health during all well child visits using validated, reliable measures like the PSC (AAP, 2012). This recommendation followed the AAP’s endorsement of an expanded focus on psychosocial functioning through the publications of its Bright Futures and Bright Futures Mental Health guidebooks which date back more than two decades (http://brightfutures.aap.org/3rd_Edition_Guidelines_and_Pocket_Guide.html).

Because of the screen’s utility in facilitating the recognition of mental health problems, some investigators have recommended that the PSC be considered "basic office equipment" in pediatrics for several decades (Sturner, 1991). Others have urged that the PSC become a mandated part of all well-child visits in managed care settings or large programs like Medicaid EPSDT (Murphy et al., 1996; Pagano et al., 1996). Several states (e.g. Arizona, Massachusetts, Tennessee) now recommend the PSC or other brief questionnaires for psychosocial screening during EPSDT visits, and a number of HMO’s (Kaiser of Northern California, Neighborhood Health Plan of Massachusetts) use the PSC as a routine part of well-child visits. The PSC is also being used as a part of annual screenings in a variety of non-health care settings like Ventura County, California Head Start (Jellinek et al., 2005). As of this writing, the PSC has been used in dozens of studies and is listed as a recommended screening instrument in reviews posted on the websites of at least a half dozen states, a number of school districts, and several children’s advocacy organizations including the Section of Developmental and Behavioral Pediatrics of the American Academy of Pediatrics and the Bazelon Center for Mental Health Law (Bazelon, 1997. From April 12, 2013 to May 2015 the PSC was endorsed as a national standard by the National Quality Forum (http://www.qualityforum.org), one of the two main standard setting organizations in the US. More information on the widespread use of the PSC can be found in PSC Research.

A series of reviews conducted by the Bazelon Institute (Bazelon, 1997; Semansky et al., 2003) have found evidence that, although they are inconsistent in the rigor of their implementation and follow through, most states do require mental health screening as part of their EPSDT programs. In the Bazelon reviews, the PSC was found to be the most commonly recommended measure for general psychosocial screening for school-aged children. Other instruments are recommended for screening younger children or teenagers or to screen for specific problems such as substance use disorder or depression. The PSC is currently recommended for psychosocial screening in pediatrics on the websites of at least a half dozen states: Arizona (Pinal/Gila Long Term Care, 2007), Massachusetts (Massachusetts Behavioral Health Partnership, 2004), Minnesota (Minnesota Department of Health, 2007), North Carolina (North Carolina Department of Mental Health and Human Services, 2007), New York (New York State Office of Mental Health, 2006), and Tennessee (Tennessee Department of Mental Health and Developmental Disabilities, 2007).

As a result of expanded reimbursement policies by the state's major insurers, the practice of routine screening for psychosocial problems in pediatrics took two major steps forward in Massachusetts in recent years. First of all, the state’s largest insurers, including Blue Cross Blue Shield, now reimburse pediatricians for this kind of screening at rates ranging from $15 to $20 per screen (Massachusetts Chapter of the American Academy of Pediatrics, 2007). Similar policies and a reimbursement of about $10 are also mandated for patients with Medicaid health insurance (Goldberg, 2007).

Although the results of these new policies in Massachusetts have not been fully evaluated, a recently published study by Kuhlthau and colleagues (2011) demonstrated that a funded, court-ordered mandate to screen for mental health during Medicaid well-child visits in Massachusetts appears to have resulted in a significant increase in the number of screened children as well as the number of children found to be at risk for emotional or behavioral problems. Another interesting line of questioning is whether, as hoped for by expert opinion like Healthy People 2010, implementing routine psychosocial screening will have a measurable impact on children’s mental health or overall health and the costs associated with these conditions. This will likely be an area for fruitful research in the future. In the meantime, it is clear that the PSC is one of the most frequently recommended instruments for children with both Medicaid and commercial health insurance and that the vast majority of studies over the past decade have shown the PSC’s usefulness as a marker for psychosocial dysfunction. The use of the PSC as a screener and its more recent application as an outcome measure for intervention studies, when added to the number of studies assessing the validity of the measure, provide a simple way of describing the types of studies which have been done with the PSC over the past decade. A review of each can be found in the section on PSC Research.

Why Screen?

The under-detection of child mental health problems in pediatric practice has been well-documented (Costello, 1986; Costello, Edelbrock, & Costello, 1988; Navon et al., 2001). Psychosocial dysfunction has been found to contribute substantially to morbidity among children and adolescents (Prince, 2007) and early detection and treatment of psychosocial problems may lead to considerable health benefits (Reijneveld, 2006) including improved prognosis (Nelson et al., 2003). Screening for psychosocial dysfunction using a brief screening questionnaire among children and adolescents seeking routine medical care can greatly facilitate this process (Eisert, Sturner & Mabe 1991, Sturner 1991).

Over the past decade, the PSC has served as an easy-to-use tool that is broadly valid and reliable and can meet a growing array of therapeutic, research, and administrative assessment needs in a wide range of clinical, educational, and public health settings in the US and other countries. We expect continuing use of the PSC in pediatric settings now that it and other similar measures are mandated in Massachusetts and other states for all children enrolled in Medicaid. The fact that almost all major insurers are reimbursing pediatricians in this and other states for psychosocial screening is also expected to lead to increasing utilization of the PSC. Many offices use implementation of psychosocial and behavioral screening to review broader areas of communication with families. For example, having an introductory letter that explains the pediatrician’s policies and philosophy can be used to alert parents to the practice’s screening procedures, best books or AAP websites for developmental and psychosocial information, policies related to confidentiality, etc. As pediatricians start screening they might want to think through what information they have available in the office on key developmental and psychosocial topics.

Using the PSC

In the field of mental health, the PSC has been used as an outcome measure in an outpatient child psychiatry clinic and has demonstrated preliminary validity and utility for both global and subscale scores (Murphy et al., 2011; 2012). Educational uses of the PSC have expanded from a screen and a marker in some Head Start and school breakfast programs to one of the cornerstones of a national screening program that has grown to more than 50,000 students per year in Chile (JUNAEB, 2007). The nationwide Chilean program incorporates a comprehensive intervention and prevention program in conjunction with routine screening using the PSC in schools. Published results from Chile suggest that psychosocial problems as measured by the PSC have a strong and statistically significant relationship with standardized academic achievement test scores (Guzman et al., 2011). With datasets in public health, psychiatric, and educational settings growing, we expect to work more extensively with subscales of the measure. For example, the Chilean group (JUNAEB, 2007) has added a subscale that measures resilience to the subscales of attention, conduct, and anxiety/depression problems that have been used in English language studies. Another paper assessing the mental health functioning of school children in Cambodia used these same subscales to examine the precise areas where children in an impoverished and socially stressed nation were experiencing dysfunction (Dany et al., 2012).

We also expect to use the PSC and its subscales more as a way to track functioning during pediatric or psychiatric treatment, educational initiatives, or public health interventions. In a large, hospital-based outpatient child psychiatry clinic at Massachusetts General Hospital, the PSC is currently used in an integrated system that inputs data directly into patient electronic medical records through the use of digital pens (Murphy et al., 2011; 2012). Using this system, PSC scores are obtained every three months and clinicians are required to use changes in these scores to guide treatment planning. The PSC is also one of the most commonly used measures in several different state-of-the-art software systems that physicians can use to assess and monitor changes in patients’ functioning and performance. In the CHADIS child health and development interactive system, the PSC is emailed to parents to complete before well child visits, filled out at home, and scored instantly so that any positive screens can become part of the follow up plan. In the CNS Vital Signs system of neurocognitive and behavioral assessment, the PSC can be entered on line, with scored reports emailed back instantaneously to the clinician, parent, or program. The Commonwealth of Massachusetts has adopted an innovative approach to reimbursement for the PSC and other brief psychosocial screens. By requiring a U1/U2 modifier to be included along with the CPT code of 96110, the data system is alerted to the presence of a positive screen. This in turn permits system wide monitoring to insure that children who are screened positive receive follow up visits.

Conclusion

It is important to keep in mind that the goal of the PSC is not rapid diagnosis or an express conduit to the use of medications. Instead, the PSC is a brief assessment of dysfunction in major areas of a child’s daily life. As an administrative or research tool, it is merely a snapshot. As a clinical measure, it is the first step in what should be a multistage assessment procedure. Positive screens with the PSC deserve careful, considerate, intelligent follow up by trained and trustworthy clinicians who will work with parents to understand each child within the totality of a unique individual, family, and community context and then hopefully find ways to provide more of the help that many parents of dysfunctional children so desperately desire.

Back to Top