Evaluation of Faculty
Residents evaluate faculty and seminars in an ongoing manner. Each clinical supervisor and service chief is evaluated midway through every rotation and at the conclusion of the service. Each seminar and its leader are evaluated at the conclusion of the course. The faculty evaluations are completed anonymously and returned to the Training Director.
Clinical faculty members are evaluated as to whether they have adequately taught and demonstrated each of the six core competencies in their particular service or treatment. They, too, are graded as satisfactory or needing improvement. There is ample room on the evaluation forms for comments. It is an expectation that if a resident grades a faculty member as needing improvement, the exact nature of a weakness is described in detail.
Faculty seminar leaders are evaluated as to the quality of seminar presentation and content. There are also questions as to whether the seminar was useful in helping residents in each of the six core competencies. The seminar is evaluated as to whether it has met its goals and objectives, whether the leader has attended all sessions and whether the readings and discussions were useful for the resident. The seminar is given an overall rating. Residents are asked whether the seminar should remain in the required core curriculum and whether the number of sessions is appropriate. Ample space is allowed for narrative comments.
It is expected that residents give faculty members feedback, in both clinical and didactic areas, as to how helpful they are in transmitting knowledge, skills and attitudes relevant to child and adolescent psychiatry. Residents give the Training Director a verbal evaluation of each faculty member in the formal biannual individual review meetings with the Training Director and in an annual mid-year Retreat (see below).
At the end of each year, the Training Director sends each faculty member an annual review of his or her teaching and supervision, based on the anonymous resident evaluations, the individual discussions with each resident and the written report of the annual resident retreat, as well as from other sources, e.g., support staff, other faculty members.
Evaluation of Clinical Services
Residents provide the Training Director an anonymous evaluation of each clinical service at the conclusion of the rotation. This includes an assessment of the patient population seen, in terms of number and variety of cases, the ability of the service to teach the six core competencies, the effectiveness of teaching specific clinical skills and an assessment of the administration of the service. Each resident is also asked whether the rotation should be required, whether its length is adequate and whether the supervision is satisfactory. The resident is asked to produce a formulation about the rotation, including its strengths and weaknesses. Each clinical service evaluation is sent to each of the service chiefs at the end of the year.
Resident Annual Retreat
Once each year, both classes of residents meet with the Training Director for a half-day Residents’ Retreat. During the retreat, every component of the Training Program is reviewed: clinical rotations, supervision, didactic seminars, on-call duties and program administration. The residents write up a comprehensive Retreat Report, which is given to the Training Committee as well as to Drs. Gottlieb, Rosenbaum and Cohen. Each faculty member’s review is placed in his or her file and is included in the annual summary evaluation sent by the Training Director. The Report is kept confidential to all faculty members except members of the Training Committee and Administration.
Both residents and faculty are required to provide evaluations of the overall Training Program. Residents fill out a confidential evaluation form biannually. The form asks residents to assess the clinical, didactic and administrative components of the Training Program. It also asks if the educational goals and objectives of the Training Program have been met. In addition, they evaluate the program in their annual Resident Retreat report.
Each faculty member is required to fill out a Faculty Program Evaluation Form on an annual basis. The faculty evaluates the Program in terms of the resources available to the Program, the contribution of each institution participating in the Program, the financial and administrative support of the Program, the volume and variety of patients available to residents for educational purposes, the performance of members of the teaching staff and the quality of supervision of residents.
The accumulated clinical service, seminar, faculty summary and Retreat Report evaluations are a part of the Training Committee’s annual review of the entire Program. The evaluations and the direct input of the residents to the Training Committee help it determine what modifications in the teaching program are necessary for improvement in the coming year.
Faculty: Expectations and Supervisory Lines of Responsibility
The faculty plays a fundamental role in the Training Program. Faculty members are responsible for the comprehensive professional development of the residents. Faculty members are responsible for resident training in areas of education, clinical service and supervision (direct and indirect), research and scholarship and evaluation.
Definitions of Supervision:
- Direct Supervision (in person on site)
- Indirect Supervision with Direct Supervision Immediately Available
- Indirect Supervision with Direct Supervision Available Each Service defines the nature of its supervision depending on the basis of the following factors: 1) year of training; 2) competency level of the resident 3) age, acuity and/or clinical need of patient and family 4) particular and unique needs of the service at the time of intervention. (Please see section VI. On Policies and Procedures)
As educators each faculty member must:
- Teach the six core competencies through supervision, didactic seminars, and role modeling on clinical services
- Teach the art and science of child and adolescent psychiatry including the biological, psychological, and socio-cultural aspects of the field, according to his or her areas of expertise
- Serve as an academic mentor for residents who express clinical or academic interests in his or her area of expertise. In role of mentor, review the resident’s portfiolio prior to submission to the training director.
- Help teach the residents how to become educators and supervisors of lay and professional populations
Clinical Service and Supervision
As clinicians and supervisors each faculty member must:
- Provide clinical supervision for cases presented in a range of settings (Direct or Indirect as indicated by the factors noted above)
- Be available for emergency supervision of clinical cases (Direct or Indirect as appropriate)
- Provide clinical supervision in areas of his or her areas of clinical expertise
- Review clinical reports of assessments and treatment of supervised cases. Perform Clinical Skills Verification Examinations (CSVs) and Chart Stimulated Reviews of as many residents as possible. See competency assessments below.
- Be responsible for the clinical care of patients under supervision, serving as an attending of record
Research and Scholarship
As members of an academic Training Program, each faculty member must have one or more areas of academic expertise. Each faculty member, through supervision, didactic seminars or individual tutorials must:
- Serve as a mentor for resident research or scholarship in his or her area of expertise
- Instill the means for residents to pursue career interests in an area of research or scholarship if asked by an interested resident
- Demonstrate continuing professional education and development in his or her area of interests through publications, lecturing, leading workshops, attending faculty development programs (or performing them online), attend and document attendance at local, regional and national professional meetings.
Evaluation of Residents and the Training Program
As members of the Training Program, the faculty is crucial in providing ongoing evaluation and feedback of resident performance in the six core competencies and in specialized areas of interest. In addition, the faculty is essential in the continued development of the Training Program. Each faculty member must:
- Provide ongoing verbal and written evaluations of the residents performance in the six core competencies using standard evaluations forms and performing specific evaluations, e.g. CSVs, Chart Stimulated Reviews, Portfolio reviews, 360 evaluations, etc.
- Provide ongoing verbal and written evaluations of the residents in specialized areas of academic, clinical and administrative performance
- Provide ongoing verbal and written evaluations of the Program, including utilization of resources available to the program, the financial and administrative support of the program, the volume and variety of patients available to the program for educational purposes, the performance of members of the teaching staff and quality of supervision of the residents (peer review)
- Demonstrate self-assessment of knowledge, skills and attitude in the process of life-long learning, and personal quality assurance
Administration of the Training Program and Departmental Duties
As members of the Training Program, faculty members are expected if called upon to:
- Serve on clinical or educational committees, such as Training Committee, Selection Committee or others
- Interview applicants to the Training Program
- Teach in didactic seminars or provide clinical supervision for General Psychiatry Residents rotating through Child and Adolescent Psychiatry
- Teach medical students rotating in the Program
- Provide clinical and academic consultation in areas of expertise
- Represent the Training Program in local, regional and national meetings
Policies and Proecedures
- All child and adolescent residents will be in full compliance with the ACGME duty hours requirements. They have no in-house overnight call, but when on call, they are responsible for covering the MGH Acute Psychiatry Service (Emergency Department), Pediatric Wards, Inpatient and Acute Residential Services at Franciscan Children’s Hospital and McLean Hospital. Residents all have a minimum of 10 hours off duty for nights on call. Should a rare instance occur in which a resident is needed to come into a service, the 10 hour rule will prevail.
- It should be noted that there are no PGY1s in the Child and Adolescent Residency, hence duty hours apply to senior residents. All residents are PGY 4 or higher.
- Each resident has one day in 7 free averaged over 4 weeks free of all clinical responsibilities including beeper call. A Beeper Swap Policy is implemented each year in which residents partner with each other to cover beepers.
- The formal policy sent to the chief residents each year is as follows:
- Each resident would be paired with another resident in the same year. The training director could assign these pairing or allow residents to self-identify their partners. Residents could be paired across years if needed. Using an alternating weekend schedule, residents would sign-out their pages from Friday 8pm until Monday 8AM.
- For example, residents A and B are first year child fellows. Resident A will sign out her pager, using the partners paging directory signout feature, to Resident B the weekends of January 1st and 14th. Resident B will sign out his pager to Resident A during the weekends of January 7th and 21st. Although an alternating weekend schedule is encouraged, there is flexibility in the schedule to allow for vacations and on-call weekends. Residents are aware that they are required to have 2 call free weekends within every 4 week period to ensure compliance with the 1 in 7 call-free day (averaged over 4 weeks) requirement.
- The chief residents then make a year-long schedule involving pairs of residents, including their beeper numbers, and other means of contact. As an example, the 2010-11 schedule is as follows:
Beeper Swap, Sign-out Policy Effective July 1, 2011
List of residents pairs: AY 11/12
|Pager #||Pager #|
|Elizabeth Pinsky||12835||Uche Okoli||17593|
|Rohit Chandra||11789||Soonjo Hwang||16306|
|John Tyson||19726||David Lovas||14835|
|Kate Knutson||14786||Nora Friedman||14788|
|Kate Nyquist||14790||Andrea Spencer||16549|
|Peter Adams||20981||Emily Adams||20983|
|Bradford Lewis||18583||Daniel Greene||20984|
|Sophia Maurasse||20988||Corey Meyer||20989|
|Mai Uchida||20990||Peter Klinger||20985|
Each resident would be paired with another resident in the same year or across years. Using an alternating weekend schedule, residents would sign-out their pages from Friday 8pm until Monday 8AM.
Fatigue and Sleep Deprivation Training
- All residents will have the following training in impact of fatigue and sleep deprivation on patient care and self-care:
- A 2-hour course in July is led by Dr. Steve Schlozman, who selects faculty from the MGH administration to lead a seminar on this topic.
- Dr. Laura Prager, Director of Child Emergency Services teaches principles of sleep deprivation and fatigue in the Acute Psychiatry Service for residents on call.
- There is always a workshop at the Partners GME Intern and Chief Resident Retreat. Our first year fellows are required to attend one or the other retreat.
- Dr. Beresin addresses this issue in his monthly seminar with the residents: Administration, Ethics and Management (the 4th Tuesday of the month) as specific situations arise.
Guidelines for Circumstances and Events in Which Residents/Fellows Must Communicate with Appropriate Supervising Faculty Members, Such as Transfer of a Patient to an Intensive Care Unit, Transfer to Higher Level of Care, or in End of Life Situations:
- Franciscan Children’s Inpatient Unit (FCIU): Residents must communicate with supervising faculty regarding care of inpatients if there is an unexpected discharge, transfer to a medical unit, or significant adverse event (elopement, severe injury), either while on call or with patients they are covering. A faculty supervisor is on duty 24 hours daily.
- McLean Adolescent Residential Treatment Unit (ART): Residents see attendings daily and twice weekly at rounds. Faculty must be contacted for abuse, neglect, patients who are not appropriate for ART level of care, due to acuity (safety such as harm to self or others; detox; medical instability; refusal to participate) complex family dynamics, elopements, allegations that could involve mandated reporting (patients reporting sexual activities with other caregivers, allegations about staff, etc.).
- Consultation Service to Pediatrics: On the Service, each new consultation must be discussed with the covering attending 7 days a week. Every patient is seen by an attending in addition to the resident Monday through Friday. This includes discussing the recommendations and disposition planning for all patients.
- Acute Psychiatry Service (APS – Emergency Department): Every patient seen in the APS must be reviewed with Dr. Prager, Chief of Service, or an attending on the backup list (24 hours a day). There are no exceptions to the rule for calling about each case evaluated for a psychiatric emergency.
- Outpatient Clinic: Every resident has three general outpatient supervisors who are attendings of record. In addition, they have supervisors for psychopharmacology, family therapy, cognitive behavior therapy, and behavioral medicine. Each supervisor meets weekly with each resident. Residents are mandated to call their supervisors for psychiatric emergencies, mandated reporting situations, need for hospitalization, serious medical or legal matters, or serious family crises at any time 24 hours a day as needed. When an attending is away she/he is covered by another attending at all times.
Guidelines for Appropriate Levels of Supervision
- Franciscan Childrens Inpatient Unit:
Residents in their 2nd year rotate for 4 months on the Unit, and have Direct Supervision on a daily basis, and Indirect Supervision with Direct Supervision immediately available at all times. Attendings are available to come on to the unit 24 hours a day 7 days a week.
Trainees in their 1st and 2nd year of fellowship (PGY-4 and PGY-5 or above) take call on the inpatient unit. Attendings are available for indirect supervision with direct supervision available. Such supervision is available from the “Attending Back-Up on Call”. That individual should be contacted if there is unmanageable volume, acuity or the resident becomes ill while on duty or experiences fatigue interfering with performance, if there are not available back-up options within the residency program (e.g. if the resident is sick before call and arranges to trade call with a fellow resident).
- Adolescent Residential Treatment Unit: Residents have daily Direct Supervision and Indirect Supervision with Direct Supervision immediately available from one of the 4 attendings on the Unit (24 hours a day, 7 days a week).
- Consultation Service to Pediatrics: Direct supervision or Indirect Supervision immediately available is provided to residents on duty in year 1 (6 months during the day) and for first and second year fellows on call 24 hours a day 7 days a week.
- Acute Psychiatry Service: Every resident must call a child psych attending to discuss every case. Occasionally, the resident sees the patient with a child psych attending on site in the APS (Direct Supervision). Indirect Supervision, immediately available to come in to the APS is provided 24 hours a day, 7 days a week, however the need for this is rare. Case that require attending presence include those in which we must commit a child under the age of 16 years against the will of the parents, any child under age 10, and every child or teen who has alleged sexual or physical abuse.
- Outpatient Clinic: All cases in the Outpatient Clinic must have an attending of record. Supervision by the attendings of record are in general Indirect with Direct Supervision Available. Most attendings will be available by beeper 24 hours, 7 days a week, and if not present, the resident may always have Indirect supervision immediately available by paging Drs. Beresin, Schlozman, Masek or Wilens who will provide an attending to see the case with the resident.
Hand offs on all services are described in detail at the beginning of the rotation. Residents use a variety of measures to report and document hand offs, including written sign outs, HIPPA secured email notes, verbal reports, and the electronic medical record.
The medical director and/or attendings of record of each service are responsible for documenting the effectiveness of the hand off system. These reports are presented at the Executive Training Committee on a regular basis.
All facilities have electronic medical records, and these are increasingly being utlilized for documenting all hand offs, and disposition of cases.
The specifics for each major rotation follows:
- Franciscan Childrens Hospital:
- Prior to the weekend or any vacation, there is a written sign out for each patient to be used by the resident on call for the weekend (or covering physician when resident on vacation) (typically the attending back-up).
- At the end of each rotation preliminary discharge summary or end of service note is prepared. Such notes are reviewed /cosigned by attendings to monitor effectiveness of handoffs.
- Newly installed EMR system (Meditech) is available to assist in structured handoff.
- Adolescent Residential Treatment Unit: Residents inform their supervising attending (Direct supervisor) of any acute issues with their patients when they leave (whether it is midday or end of the day) and attendings are ultimately responsible for all of their patients. All evening/weekend issues are managed by one of the staff psychiatrists at the ART who are always on call each day/night. Occasionally residents are paged about these e.g. patient wants to leave and more info is needed about the case, discrepancy that arises about recent history, etc. Moonlighting residents see any admissions that come to the ART after 5pm on Fridays or on Sat/Sun/holidays (generally not common).
- Consultation Service to Pediatrics: Every Friday afternoon, the residents on the consult service sign out their inpatient pediatric consults to the covering resident and the attending who is back up to the pediatric unit. This is done by HIPPA protected email. Additional phone or face to face conversations occur if there is a complicated situation or issue that needs to be addressed. During the week, there are no handoffs on the Consultation Service to Pediatrics, so it is only a weekend call situation.
- Outpatient Clinic: All residents carry their patients the entire year. There are no hand offs. At the end of the year, residents three months prior to departure are instructed in how to transfer their patients by the Chief of Service. The chief residents are responsible for making up a master list of all resident cases needing transfer, and the recipients, incoming residents and faculty on site are included. They are notified by email, and directed to the electronic medical record in which a complete transfer summary is written. There is a template for this transfer. The Intake Team that meets weekly (Triage director, Chief of Service, Training Directors, and chief residents) review each transfer case the effectiveness of the disposition.
Documentation of Effective Handoffs:
All residents must provide documentation of at least one successful handoff on every clinical service. The service chiefs will be responsible along with the residents to document the efficiency, effectiveness and outcome of handoffs with spot checks at various intervals during their rotations by using the Handoff Assessment Form.
Quality Improvement Requirements
Every resident MUST be included in an annual quality improvement project including both classes of residents. This usually involves improving one or more clinical services in the training program.
- At present there is an outpatient study of the use and monitoring of atypical antipsychotics. Every resident is required to submit each patient (anonymous) with demographics including age, weight, waist circumference, laboratory studies (blood glucose, prolactin, cholesterol) and the name and dose of the medication. In addition they indicate the indications and any adverse effects of the medications. The study is under the leadership of Drs. Hazen, Russell, and Masek, with statistical consultation from Dr, Lee Baer. This is not a study of effectiveness, but rather a study of metabolic adverse effects of the medication used. The data analyzed is fed back to the treating residents, such that any modifications may be made should the patient have an adverse effect.
- Every resident and faculty member in the Outpatient clinic is required starting in July, 2011 to utilize and record the CRAFFT (a screening tool for substance abuse in children and adolescents). This is recorded in the electronic medical record.
Use of the Electronic Medical Record
All residents and faculty are trained in the use of the electronic medical record (Longitudinal Medical Record aka LMR). The LMR has a number of important templates that must be filled out by residents. Due to the sensitivity of some material in child and adolescent psychiatry, there is a special “Restricted Notes” section that only the treating clinician or designated approved clinicians may view. Dr. Masek, Clinical Director of the Outpatient Clinic receives a monthly report of any inappropriate viewing of the restricted notes in the LMR.
The residents are routinely given “Chart Stimulated Reviews,” an MGH Child Psychiatry Competency Tool (see below) with two sections: 1) a compliance section about documentation in concert with national regulatory guidelines and 2) a section in which the attending asks the resident to present a comprehensive case report. This exercise attempts to evaluate the ability of a resident to present an efficient, effective and complete case report.