24/7 Preparation: ACGME Site Visits

Your residents sometimes procrastinate with regard to lab notes. You say you’re teaching fellows to critically evaluate the literature, but where is the documentation? This level of detail is exactly what is involved in preparing—continuously preparing—for site visits from the Accreditation Council for Graduate Medical Education (ACGME). The potential costs of not preparing well? Citations, probation, recruiting problems, loss of funds—and even the survival of the program.

Dean Taylor, M.D., Colonel (retired), Director of the Duke University Sports Medicine Fellowship Program, is thrilled to have just emerged from a terrifically successful site visit. He states, “We had no citations, and have been accredited by the ACGME for five years, a rare event these days. To a great extent I credit Amy Tingen, our Sports Medicine Fellowship Program Coordinator, for staying on top of the extraordinary amount of work required to get us through this process successfully.”

And the elements of their winning “formula?” Dr. Taylor: “You must first have a concrete vision for your program. Once you have established that you need to ensure that you have the right team to carry out that vision. For example, the vision for our program is to train orthopedic sports medicine fellows to be outstanding clinicians, educators, and researchers. Given that, we have focused on creating a team and an environment that fosters leadership development. Caution, however…your vision has to fit in with the ACGME requirements regarding the training of residents and fellows to be clinically competent. The ACGME has six core competencies that must be used as guidelines in crafting your vision and establishing your objectives.”

The aforementioned heroine of Duke’s recent coup, Amy Tingen, stresses the ongoing nature of the preparation required for site visits. “You must keep building on what you already have. At Duke we have an internal review process, which is helpful in keeping us on track. The most challenging task is keeping the Program Information File (PIF)—the document provided by the ACGME—up to date with information on caseloads, meeting documentation (faculty meetings and meetings between faculty and trainees), etc."

"We must remain in contact with graduates to ensure that anything they have published—or anything of significance that they have achieved—is part of our documentation. In addition, the reviewers are looking to see how well you have documented daily activities, including labs, journal clubs, sports medicine conferences, etc. Truly, there is no way to avoid the fact that this is far from a last minute process.”

Delaying preparation, or not understanding how exactly to prepare, can result in serious consequences for a program. Dr. Taylor, “If a program receives citations, then that means additional work for an already busy staff. Also, citations can lead to shorter periods of accreditation, which leads to more frequent site visits, which leads to more work. Lastly, too many citations can result in probation of your program or other sanctions. This, of course, affects the reputation of the program, the ability to recruit good residents for fellows, funding for the program, and possibly the survival of the program.”

With regard to funding, a constant nemesis in any program, Amy Tingen says, “Reviewers look at whether you have sufficient funding to support the number of fellows you are training. They want to ensure that the funding source is reliable/stable…if not, then they won’t approve you for the number of fellows you requested. For example, funding for fellowships is different than funding for residency programs that are sponsored by hospital revenues. Fellowship funding often is a year-to-year process with corporate sponsors, third-party grants, endowments, etc. Due to the new rules by the AdvaMed Code of Ethics on interactions with health care professionals, funding has become so scrutinized that it sometimes can affect the funding amount for the number of trainees in a program or the program itself. There is a real need for stable fellowship funding whereby the programs can be ongoing without the worry of obtaining funding on a year-to-year basis.”

As for where programs can go astray, Dr. Taylor says,

If your vision is along the lines of, ‘I am doing great clinical work and these residents/fellows can just learn from me as I’m operating, ’ then you have fallen into the trap of using people to help with your clinical load.

"If your program leans more toward clinical work then you have to ensure that trainees are not spending more time working than they are learning. I have seen situations where the program director says, ‘An important part of our program is the cadaver lab; residents go once a week and learn how to do arthroscopy, ’ but then the reviewer finds that not only don’t they go to the lab regularly, but the lab is in disarray. Note that you can forever say, ‘We do such and such, ’ but if it’s not documented, then it doesn’t count. This should drive home the point that constant documentation is required in preparing for these visits…there is no waiting until a week before the reviewers come to fill in the paperwork.”

Michael Connors/morgueFileThe waiting, as it were, comes later. Keith Kenter, M.D., Director of the Orthopaedic Surgery Residency Program at the University of Cincinnati is in just that position. His program underwent an ACGME site visit in May 2010 and will have the results in January 2011. Dr. Kenter states, “Because a site review is so complex, I work closely with the Graduate Medical Education (GME) office, discussing programmatic development as it relates to faculty and residents. This is part of our internal review process, and includes a chance to address issues that were previously noted by the ACGME. We also have other departments review our department; for example, our internal review committee includes representatives from pulmonary medicine, plastic surgery, and emergency medicine. This is an enormous amount of work, but it allows us to examine our program in an objective manner.”

Shining such a bright light may indeed elucidate what needs changing…and what can’t be altered before the reviewers show up at your door. Dr. Kenter: “Thorough preparation lets you know your problem areas so that you can do your best to address them before the visit. Sometimes, however, you can’t make changes without a long-term effort. For example, we struggle with our highly overbooked university clinic, which is where many indigent patients are seen. It must be a balance between service and the residents’ educational process, with the hospital requiring that residents treat those individuals, and the GME office saying, ‘That’s too many patients for residents.’ We are in the process of trying to develop a game plan to see how we can maximize the educational process and still provide service to these patients.”

Dealing with a citation may be as simple as filling out a piece of paper, says Dr. Kenter. Or it may require developing a new program entirely. “Upon receiving a report from the ACGME your first question should be, ‘Is this an issue that we can address this year?’ If the site reviewer says that residents need to complete operative logs in a more timely fashion then you just need to meet with residents on a regular basis so that they have a systematic way to do this paperwork. On the other hand, if the reviewer says that you need a sports medicine specialist on your faculty, then it is a much more involved issue. You must approach the administration and say, ‘Our residents have zero opportunity to do arthroscopy because we don’t have a sports medicine specialist on staff.’ In this situation, one option is to work with other local hospitals to develop a program whereby residents rotate elsewhere and get exposure to sports medicine.”

So when do many program directors reach for an Ibuprofen? When it comes to the research arena. Dr. Kenter says,

“Getting one’s research program in line with the expectations of the ACGME is a common area of struggle across the country. There are numerous issues, including the fact that many programs don’t have animal research, and that the residents’ research projects don’t get published in peer-reviewed publications."

Five years ago the ACGME indicated that we needed to place more emphasis on research. In response, I developed a faculty research committee where I enlisted the Ph.D.s on campus who were doing orthopedic research and drew up a strategic plan. We established a staged approach, namely that first-year residents would find a research mentor and develop a research question, second-year residents would derive a hypothesis and study design, etc. By the time they reached their fifth year they would be presenting their research. This has led to much improved research with our residents now presenting their work regionally and at the American Academy of Orthopaedic Surgeons.”

Long hours and detailed documents are part of the picture, to be sure. But also, says Dr. Kenter, it is that some things are just difficult to document. “A major issue is that documentation requirements continue to increase. This is really challenging for those program directors who are full-time faculty with regular practices, the demands of education, and the requirement that they bring in clinical dollars. Another challenge is that it is hard to document how one teaches and assesses professionalism. As opposed to assessing medical knowledge, where you can look at the Orthopaedic In-Training Examination and the board exams, there is no validated way of assessing professionalism.”

But Dr. Kenter and his fellow program directors are no longer wading through these issues alone. “Some of my colleagues and I, along with the American Orthopaedic Association, have founded the Council of Orthopaedic Residency Directors. We meet biannually, hold symposia, examine best practices, and communicate about programmatic problems. I have seen some program directors, particularly those on the young side, who feel overwhelmed and helpless because they have a problem and no idea how to deal with it. Now there is help.”

Dr. Kenter’s final advice?

Be as honest/objective as possible when it comes to your program. It doesn’t do any party any good if your view of the program is skewed. You will miss opportunities to address problems and in the end, will garner more citations than if you had removed the rose colored glasses.