Volunteer or Nominate For Exam-Writing Committees/Subboards

Bottom-Line-Exam-Writing-Committee-Nom-ToolAlmost every week, our offices in Chapel Hill are buzzing as we host meetings of test writing committees and subboards of the American Board of Pediatrics. The work of the ABP would not go forth without the volunteer time and effort of more than 250 pediatricians who govern the organization, develop and approve policies, write exam questions, set passing scores, establish standards of knowledge and do many other critical tasks that are the foundation of board certification. We have excellent volunteers on our 14 subboards and general pediatric examination committee and we’re grateful for their work!

As ABP seeks to continuously improve its ability to set standards of excellence for certification in pediatrics, our exams and maintenance of certification activities must reflect authentic day to day practice. Part of this effort involves making sure that the subboards and test writing committees continue to be made up of pediatricians from diverse backgrounds who represent modern practice and who know the critical issues affecting children’s health.

To insure an open process of joining an ABP subboard or question-writing committee, we’re now reaching out to all of our diplomates, asking YOU to consider volunteering for a position, or nominating a colleague you think would add important perspective and/or expertise to a particular subboard or to a general pediatrics test writing committee. Check out our new nominating tool.

We want candidates who represent the diversity of pediatric practice – everything from rural, private practices to medical centers in major metropolitan areas. And we want people from a variety of practice situations, including well-seasoned pediatricians, new practitioners, part-time providers and others who reflect today’s trends in pediatric practice.

Volunteers typically serve on a test writing committee or subboard for six years. The terms are staggered, so we have a limited number of vacancies each year. Members are expected to attend meetings as well as complete time-sensitive work assignments.

Who are the current volunteers who help develop our exams? We publish a list of all subboard and general pediatrics examination committees members in our annual report to recognize their outstanding work. Many Board of Directors members have served on one of the ABP’s question-writing committees or subboards, and all volunteers are required to be board certified and meet maintenance of certification requirements[1].

Your participation on one of the ABP’s subboards or committees would help us to help board-certified pediatricians everywhere, but you would get something back. I know about that from personal experience. Before I became the ABP President and CEO, I served for six years on the Pediatric Critical Care Medicine subboard. My time on that subboard was one of the most rewarding experiences in my career. I learned how to convert the important concepts and evidence in our field into valid and reliable exam questions. The real bonus came from the interactions and discussions with the other members of the subboard who were all trying to answer the question: “What would a parent expect us to know as they entrusted their child to our care?”

I hope you’ll give serious consideration to nominating yourself or someone else to work with us.

[1] The exceptions are non-pediatrician members serving on the Board of Directors to represent the public.

Responding to the Mental Health Crisis among America’s Children

Responding to the Mental Health Crisis among  America's ChildrenThanks to all of you who responded to my last post about the mental health crisis among American children. Your comments confirm the impression I’ve received from pediatricians I’ve spoken with from around the country – that mental health diagnoses are increasing among our children and we’re not prepared to deal effectively with the crisis.

The big question is, what should we be doing about it?

The “we” here is inclusive – the ABP, general pediatricians, subspecialists and others.  Your replies offered descriptions of several innovative models and best practices. I feel it’s worth highlighting a few representative ones.

A thoughtful post from Dr. Ellen Perrin sums up many of your concerns. She is Tufts University professor and director of research at the Center for Children with Special Needs – and incidentally, was the first chair of the ABP’s Developmental and Behavioral Pediatrics subboard. She makes these recommendations (view her entire post for additional details):

  • Focus on evidence-based prevention and early intervention,whichwouldincorporate:
    • parenting support and education
    • routine screening and referral for early evidence of emotional/behavioral difficulties
    • systematic liaisons with other professional and non-professional providers
    • collaboration with one or more mental health clinicians present in the practice setting
  • Support social programs, such as job training, quality childcare, early preschool and nutritional support.
  • Encourage off-site psychiatric consultation programs (such as the Massachusetts Child Psychiatry Access Project (MCPAP), replicated in more than 20 states) which can support pediatricians in office-based diagnostic and therapeutic mental health efforts.
  • Re-evaluate residency training. (Primary care pediatricians report that they spend more than half their professional time dealing with developmental and mental health issues, and yet 3-year residency programs provide about one month of developmental-behavioral pediatrics.)
  • Use the opportunities available through the Affordable Care Act and other incentives to develop better payment arrangements that don’t depend on fee-for-service coding (which offer inadequate support for co-located mental health professionals).

Several discussed networks that are providing education and support to general pediatricians in both rural and urban areas.

Dr. David Kaye wrote about his organization in New York State – Child and Adolescent Psychiatry for Primary Care (CAP PC), funded by the state’s Office of Mental Health, which provides education and consultation through both phone and in-person visits to pediatricians across the state. His program is affiliated with The Reach Institute, mentioned by several of you, which provides a 2.5-day intensive training program, followed by phone conferences over a six-month period, and ongoing support.

“To be clear,” Dr. Kaye writes, “we aim to assist and support pediatrics in addressing mild to moderate mental health problems, with an ultimate goal of preventing more serious deterioration. To have pediatrics face this squarely and take it into mainstream practice is a welcome step in de-stigmatizing mental health problems, promoting access to care, and making the kind of strides we need to make in order to assure treatment for all, and prevent disability and reduce risk.”

I couldn’t agree more.

Insurance is a major issue for all physicians, but especially for pediatricians dealing with mental health issues.

Dr. Linda Cozzens of Southboro, MA, writes: “I currently am reimbursed $350 for ablating a wart, and $25 for a mental health diagnosis. Until the system reimburses pediatricians and mental health professionals adequately for their time and training, we will continue to see an increasing number of mental health providers who accept self-pay patients only, and see a limited number of people choosing child psychiatry or counseling as a career path.”

Dr. Kevin P. Marks, a general pediatrician in Eugene, OR, suggests that the ABP consider offering Maintenance of Certification (MOC) credit for activities like team-based training, learning to better implement evidence-based developmental-behavioral screening tools, or demonstrating the quality improvement value of co-locating with an early childhood developmental specialist – all ideas we will consider.

I’m glad to hear positive feedback, even tinged with criticism, as we got from Dr. Neal Spears of Smithville, TX. He writes of one activity approved for MOC credit: “It was tedious, expensive, and time consuming but it completely changed my approach to ADHD assessment and treatment. Although it was a fairly painful experience, I do feel like it has significantly enhanced the accuracy of my ADHD diagnosis as well as helped identify problems with treatment. I continue to use the program daily to keep track of my patients’ Total Symptom Scores and to keep in touch with teachers during the school year.”

While we hope to take the pain out of MOC activities, I am heartened to hear about benefits. We welcome other suggestions on topics we should pursue, both for self-assessment and for quality improvement activities.

Keep sending your comments. The ABP is committed to staying engaged in this issue with you to improve the lives of children with behavioral and mental health conditions.

Mental Health Crisis among America’s Children — What Should We Do?

mentalhealthAs I travel around the country, I have the opportunity to meet many pediatricians who share with me information about the focus of their practice. Many report seeing a swelling tide of children with developmental, behavioral and mental health problems. I use the swelling tide metaphor not just because of the numbers of patients, but also in the sense of feeling overwhelmed by a subject area for which there was inadequate preparation during residency.  The American Academy of Pediatrics (AAP) has issued several reports drawing attention to this problem. Are the readers of this blog also seeing developmental, behavioral and mental health disorders as the major chronic disease in their primary care practices? (If that’s not what you are seeing, then please share what are the most prevalent problems in your practice.) Do you feel prepared to meet the needs of patients with these problems? All of this has got me thinking how ABP’s responsibilities apply if mental health is as important in primary care as my conversation partners have suggested. So let me offer a few opinions and pose some dilemmas.

  • ABP will need to encourage training programs to enhance the preparation of the graduates in mental health. This involves dialogue and collaboration with training programs and the Accreditation Council for Graduate Medical Education, but also a declared intent by ABP to increase the emphasis on these topics in the initial certifying examination.
  • Pediatricians must identify the child with mental health needs and then coordinate care with early intervention services, mental health professionals, schools, and various agencies. The complexity and variability of coordinating this type of care is reflected in the complexity of designing exam question scenarios that involve decision-making skills, judgment and complex analysis on this topic. For instance, one could imagine a question on the indications for referral of a child with anxiety or depression to a mental health specialist. This is a critical judgment call, where either under-referral or over-referral can harm the child.  Yet the boundaries of what can be handled by a pediatrician vs. a mental/behavioral health specialist may vary depending on specialist availability, insurance plans, Medicaid waivers, school programs, etc.
  • One thing the pediatrician can control is an accurate self-assessment of her or his knowledge base in the management of these disorders. ABP-approved MOC part 2 self-assessment activities and/or a part 4 quality improvement activities can help. View a list of these activities. The AAP also offers excellent educational and assessment tools on this topic.
  • Mental health disorders are chronic diseases and as such should benefit from the patient-centered medical home model with extensive care coordination, involvement with the school, and practice quality improvement. How can ABP promote this model?
  • The ABP and AAP are also actively engaged in an Institute of Medicine effort to address children’s mental health needs. If you have developed an innovative model of caring for these children, we would like to hear from you.

Since the days of typhoid fever, pediatricians have always stepped forward to answer urgent child health needs when no one else could or would. Today’s pediatricians carry forward that tradition in extending themselves to tackle the mental health crisis among the young. This is what I think. If you would like to let me know what you think, leave your comment below.

One of Our Own Remembered for Courage, Conviction, and Commitment to Excellence



So began a tweet from the US embassy in Kabul 1:54 AM – 24 Apr 2014.

Dr. Jerry Umanos, diplomate of the American Board of Pediatrics, was one of those Americans killed when a security guard opened fire as he approached the hospital gate.

A Chicago CBS Affiliate, Dr. Umanos’ hometown, gave this brief background report on the day of his death:


Dr. Jerry Umanos

“CURE Hospital is a Christian charity hospital set up by Americans: …but there are very few Americans among the doctors. The facility specializes in maternal and pediatric care. CURE is the first hospital recommended on the U.S. Embassy website for those seeking medical treatment in Afghanistan.

“This loss is a great loss for his family, for those of us he worked with, as well as for the people of Afghanistan. He was a loving, caring physician who served all of his patients with the utmost respect,” said Bruce Rowell, chief clinical officer at the Lawndale Christian Health Center.

Umanos completed medical school training at Wayne State University School of Medicine and residency at the Children’s Hospital of Michigan. He is board certified by the American Board of Pediatrics and practices medicine in both the United States and Afghanistan. He also practices pediatric medicine in Kabul at a community health center and at the hospital – the only two training programs for Afghan doctors in the country.”

This was national news for a day before the country went on to other issues. Dr. Umanos’ death was senseless. But perhaps those of us who are diplomates of the American Board of Pediatrics might pause longer to reflect on the meaning of this remarkable life. With courage, conviction, and commitment to excellence, he went far beyond the requirements of a job and even beyond the high expectations of a profession and gave his all to serve his patients.


Future of Testing: Can You Draft a Question Requiring Skills Beyond Recall?

BottomLineopenwebtestOver the past year, I have had the opportunity to hear about advances and issues facing our profession from pediatricians from all across the country. Your thoughtful comments to my last post provide an even greater wealth of insights into Maintenance of Certification in general and testing in particular.

Many agreed that an “open-Internet” approach to testing is important to consider, and many recognized the complexity of such a change. An overarching theme in the responses was the purpose of the test, which is to assess knowledge as an integral component of medical practice. Many of the comments echoed my earlier assertion that technology-aided retrieval of information has become reality – arguably a necessity – in everyday practice. Conversely, several of you pointed out there is not enough time to look up everything in daily practice and worried whether today’s residents were becoming too dependent on the smartphone.

Your comments raised other pertinent questions:

  • If we allowed access to the Internet during tests, could we agree on selected resources and websites that should be accessible (such as Red Book, ePocrates, UpToDate)?
  • Is the use of the Internet appropriate for initial certification as well as Maintenance of Certification?
  • Could we allow an open-Internet test if we could not guarantee that test-takers are prevented from copying and emailing test questions or entering chat rooms?

Right now, the leading test delivery providers are researching the best ways to allow for open-Internet testing while still ensuring the integrity of the examination process, but we’re not there yet.

Assuming that the security challenges can be resolved, an even more important issue is the kinds of questions that would be appropriate for a section of the exam that included Internet access to certain resources. We would want questions that do not simply test recall, but also focus on analytic thinking, data synthesis, evidence assessment, judgment and decision-making.

Our subboards and question writing committees are made up of board-certified pediatricians who see the same kinds of patients as any practicing pediatrician. They work extremely hard at writing fair and valid questions that require higher-level thinking and critical reasoning skills. For now, we will continue to use closed-book, multiple-choice examinations, which offer the most objective methods for scoring. If, at some point in the future, we incorporate Internet-access during examinations, we would need an expanded pool of questions that examine critical reasoning skills.

We invite you to join the effort by drafting sample questions that you think would assess the higher order thinking skills that define a good pediatrician — even if he or she can look up information before making decisions about management or treatment. Our shared goal should be an examination that mirrors to the greatest degree possible the increasingly complex world in which we practice. And as one commenter suggested, physicians don’t practice in a “closed-book world.”

Before you get started, I strongly encourage you to look at some of the resources used by our question writing committees: one is an overview of Bloom’s taxonomy; the other is an excellent tutorial developed by the National Board of Medical Examiners (NBME) on how to write good questions.

You may also want to beta test your question(s) with your colleagues before you submit them. We look forward to seeing the kinds of questions you come up with.

SubmitOpenWebQuestionBloomsTaxWe will share your submissions with our subboards and question writing committees as they wrestle with the same challenges. You shouldn’t expect to recognize questions that you submit on a future examination. All questions undergo multiple rounds of thorough review and careful editing.

In fact, we have always accepted questions from diplomates, and they all go through the same process. You can find our item writing portal through your portfolio on the ABP website. For now, and for ease of use, we created a simpler method for you to share your submissions, accessed through the teal button.

Realistically, developing new questions and designing new exams takes time. We have begun a rigorous discussion about testing, and we are reviewing the best thinking from testing experts as well as medical educators and expert clinical practitioners. In the meantime, we welcome your input as we explore this important issue.SubmitOpenWebQuestion

So give question-writing a try. As you do, I hope you will also continue to share your ideas about what makes a good question. I invite you to use the comment section of this blog to contribute to the discussion. As always, I look forward to reading your insights and thoughtful suggestions.

Certification Exams: Are We Testing Recall or Judgment?


I have a recurring nightmare: I’m a sophomore in college, taking an engineering exam on differential equations that I don’t understand at all. It’s an open book test, and I still have no idea what to do! Somehow, in my dream, I miraculously pass the test, but know with certainty that I should never take another engineering course, just for fun, ever again. So, I became a doctor.

Fast-forward 30 years and I am making rounds with residents and fellows. The computer on wheels (COW) accesses orders, EMR entries, and PubMed. The residents use tablets and smart phones to log in to sites that provide practice guidelines, differential diagnoses, and more. ABP exams have evolved from oral exams in front of a daunting panel of learned practitioners, to unproctored exams on floppy computer disks to today’s proctored, computer-based exams. Real time internet access for clinical care prompts the question: Should my old college open-book exam be reincarnated as an ABP open-internet exam?

Unfortunately, there is precious little research to guide answers to this question. Overall performance on open-book/internet exams does not seem to change much, with those who benefit from the approach offsetting a population of examinees who become more confused by it.  Historically, data on the effects of open-book exams reveal conflicting results on whether the open-book (and presumably the open-internet) exam encourages or hinders learning.

There are also philosophical disagreements on the role of recall in testing. A closed-book exam requires test-takers to recall facts, which are then used toquote212-10 address the diagnosis, treatment, and management of a given clinical scenario. Proponents of the closed-book exam argue that the practice of medicine requires a certain amount of “pocket knowledge” that every practitioner must have to deliver excellent care. Opponents of the closed-book exam chafe at this requirement for fact recall precisely because technology-aided retrieval of information has become reality – arguably a necessity – in everyday practice. It is increasingly unrealistic to expect physicians to recall information the way they did in the past as the body of knowledge in medicine expands exponentially. Regardless of one’s philosophical perspectives, the current closed-book exam design has produced ample psychometric data in support of its fairness, validity, and reliability.

quote12-10If the ABP and other certifying boards were to shift to an open-internet approach, a substantial amount of preparatory work and pilot testing would be needed. Exam questions would presume access to facts and focus on assessing critical thinking, effective search strategies, decision skills, and complex data analysis. Such questions might have more than one correct answer, and the correct answers might be weighted differently such that one of the correct answers might be considered “more correct” than another. Thorough psychometric analysis would be required to document the validity and reliability of such an approach before it could become the basis for certification.

An open-internet exam would still be timed. So, just as in the clinic, there would be insufficient time to look up every imaginable fact. Some examinees might even feel greater pressure with the combination of access to resources and time limits.

Predictably, open access to the internet carries a host of security concerns, which would have to be solved before this approach becomes viable. The June 3, 2012 issue of The Chronicle of Higher Education (“Online Classes See Cheating Go High-Tech” by Jeffrey R. Young, documents the technological arms race between innovators in online education and those intent on using holes in the online systems to cheat. Robust exam security will be necessary to ensure that everyone taking the exam has an equally fair chance at passage.

Finally, economic concerns have been raised. Although many or most pediatricians have already purchased subscriptions to online databases and information resources, this is not true for all. The ABP would want to avoid any notion that purchasing such a subscription was “required” in order to pass the test.

The ABP is thinking hard about these issues and will conduct the necessary research to evaluate open-internet testing. In the meantime, I encourage you to let us know what you think. Please leave a comment below.

Parents, Sick Children, and the Network

BottomLineJustin Vandergrift and his wife’s lives changed abruptly in August of 2011, when they rushed their daughter Kathryn to a North Carolina pediatrician.

Eleven-year-old Kathryn had just been diagnosed with Crohn Disease.

At a follow-up appointment a week later, the Vandergrifts were offered the opportunity to join ImproveCareNow Network.

The ImproveCareNow collaborative, uniting 50 teams, focuses on two specific efforts: to develop a Collaborative Chronic Care Network, and to support a data collection system to help drive improvements in care for children with Crohn disease and ulcerative colitis.

Kathryn Vandergrift

Kathryn Vandergrift

Learn more about the ImproveCareNow Network.

A nurse practitioner explained to Kathryn’s parents that information about how she was doing could be shared anonymously with others participating in the ImproveCareNow Network so everyone in the network could help identify best practices to improve and standardize care.

“I signed that consent form,” Justin said. “I knew they were doing things behind the scenes.”

Justin later traveled with his local site team to an ImproveCareNow learning session, which brought together teams from around the country involved in the network.  Each team included physicians, nursing staff, researchers, and parents – all sitting at the table together, sharing their invaluable perspectives.

“I really came back from the learning session knowing that the physicians actually cared about what our opinion was,” Justin said.

Dr. Ashish Shah, Arizona Pediatric Cardiology Consultants

Dr. Ashish Shah, Arizona Pediatric Cardiology Consultants

Dr. Ashish B. Shah, an ABP-certified pediatric cardiologist at Arizona Pediatric Cardiology Consultants, said the importance of patient involvement is one of the most influential lessons he’s learned since becoming active in his network, the National Pediatric Cardiology Quality Improvement Collaborative.

“Incorporating a diverse group of families enhances the collaborative process,” Dr. Shah said. “These are the people you are caring for. And if you’re not communicating effectively with that group, then you cannot expect to be successful in caring for the patient.”

The National Pediatric Cardiology Quality Improvement Collaborative focuses on improving outcomes for children with cardiovascular disease.

Learn more about the National Pediatric Cardiology Quality Improvement Collaborative, including participating sites.

Having completed a dual medical/business degree while in medical school, Dr. Shah recognizes the value in standardizing care.

“What if the way you’ve been doing it all along isn’t the best way?” Shah said. “In a collaborative, what can you learn from other places, what can you learn from your own internal failures to do better? Maybe we’re doing something really good and we don’t know it, but now we can find out because we can share our data, ideas, and processes.”

This is particularly valuable when the number of specific cases is limited. By uniting different sites throughout the country, networks are able to “experience” more cases, share more data, and learn how to improve the care they deliver. This knowledge is then diffused throughout the participating network sites.

Justin, like other parents and caretakers, also sees how the work of the collaborative impacts Kathryn directly.

“All of these people at all these care centers are inputting data into this database, and it all comes down to a point,” Justin said. “You’re getting this huge funnel of information and knowledge sent directly down to my child when she needs it.”

“That funnel of knowledge, I think, is remarkable,” Justin said. “I get all this knowledge poured through the data down to my child. It’s remarkable to see that standard of care.”

Graphic courtesy of http://www.karrcreative.com.

This is only achieved by each team’s willingness to remain transparent in sharing both their failures and successes.

“You learn that the process you may be accustomed to is not the best,” Dr. Shah said. “However, by taking bits and pieces of successful processes, you can improve upon your existing system.”

Teams must also be willing to resist the urge to hoard knowledge and expertise, recognizing that the ultimate goal is to improving the care and treatment of children in a more collaborative manner. It is a cultural shift taking place in what has always been an extremely competitive field.

The third and possibly most influential necessity is the parent-patient involvement.

“Parent involvement is huge,” Dr. Shah said. “You can say anything as the doctor, but if the family cannot do it or doesn’t buy into it, you achieve nothing.”

Collaborative networks help not only by bringing the family to the table in a conversation with the medical team but also by offering a sense of community for the patients and caretakers.

Since his daughter’s diagnosis, Justin has taken an active, engaged role in Kathryn’s care. And he has encouraged other parents in his situation to seek out and become involved in these networks.

“The sharing of data to standardize care is so important. I need to know as much as possible,” Justin said. “The only way to do that is to get involved in some organizations like the collaborative.”

Dr. Shah shares his passion for understanding the value of these networks.

“Your baby is part of a network of similar babies across the country,” Shah said. “We’d like you to participate because what happens for you and your baby becomes a learning experience for us so we can…improve quality of life.”

The American Board of Pediatrics (ABP) helped support the development and launch of the ImproveCareNow and National Pediatric Cardiology Quality Improvement Collaborative Networks, as well as other similar learning networks.

Through these learning networks, diplomates of the ABP can engage in collaborative quality-improvement efforts while also earning MOC credit.

If you are a diplomate of the ABP interested in earning Part 4 activity points through participation in collaborative networks, please browse the multiple individual and team-based collaborative network activities available by visiting your ABP Portfolio and using the Part 2 and Part 4 Activity Search.

This is what I think. If you would like to let me know what you think, please leave a comment below.