Download AMA Survey Comments on Proposed Changes to AMA/ACCME Alignment and Simplification and more Study notes Communication in PDF only on Docsity!
AMA Call for Comment Results
Background and methodology
The American Medical Association Council on Medical Education was interested in obtaining detailed feedback from the CME community at large regarding proposed changes to the AMA PRA credit system standards, so an online survey was crafted to solicit information.
On April 27, 2017, the AMA survey was sent electronically to 11,646 individuals from the AMA’s MedEd Update newsletter subscription list as well as other stakeholders identified by the AMA’s medical education team (e.g., state medical boards and ABMS specialty boards). Targeted state medical societies—those that are neither ACCME recognized accreditors nor ACCME accredited CME providers— were also included. The questions provided detailed information on proposed changes to the AMA PRA credit system standards, including deletions and edits as well as what remains unchanged. A total of 664 individuals responded to the survey.
In the subsequent pages, you will find the following documents: Summary of responses to the closed-end questions prepared by AMA Market Research. (^) List of the open-end responses to each question, segregated by question. The responses are included verbatim.
Results of AMA Call for Comment on AMA/ACCME Alignment and Simplification Page 1 of 47
Respondent Profile
1
Organization
Self78%
Are
you
responding
for
an
organization
or
for
yourself?
*Please
tell
us
which
of
the
following
describes
you
or
your
organization?
(Note:
Multiple
responses
were
allowed,
so
total
is
more
than
Other
ACCME
‐defined
commercial
interest
State
medical
licensing
board
State
medical
society
Specialty
certification
board
ACCME
member
organization
State
‐accredited
provider
ACCME
‐accredited
Physician^ provider
*Note:
Categories
were
self
‐selected
by
survey
participants,
and
some
potential
inaccuracies
were
noted. However,
recalculations
were
tested
in
some
groups,
by
eliminating
all
that
probably
did
not
belong
in
the
group,
and
did
not
yield
a
significant
change
in
scores. Therefore,
all
demographic
categories
remain
as
submitted
by
participants.
Results of AMA Call for Comment on AMA/ACCME Alignment and Simplification Page 2 of 47
3
64% 31%^ 5%
Agree
(Top
3 Box
‐^
8, 9,
Neutral
(Middle
4 Box
‐^
4 ‐
Disagree
(Bottom
3
Box
‐^
1, 2,
Format–specific
requirements
for
certifying
activities
for
AMA
PRA
Category
Credit
2. Enduring
materials
This
format
specific
requirement
would
remain
the
same:
access
to
appropriate
bibliographic
sources
to
allow
for
further
study.
These
format
specific
requirements
would
be
deleted:
•^
Provide
clear
instructions
to
the
learner
on
how
to
successfully
complete
the
activity.
(This
requirement
will
be
subsumed
into
core
requirement
so
will
not
be
listed
separately
as
a format
specific
requirement.)
•^
Provide
an
assessment
of
the
learner
that
measures
achievement
of
the
educational
purpose
and/or
objective(s)
of
the
activity
with
an
established
minimum
performance
level;
examples
include,
but
are
not
limited
to,
patient
management
case
studies,
a
post
‐test,
and/
or
application
of
new
concepts
in
response
to
simulated
problems.
•^
Communicate
to
the
participants
the
minimum
performance
level
that
must
be
demonstrated
in
the
assessment
in
order
to
successfully
complete
the
activity
for
AMA
PRA
Category
Credit
Q2.
These
changes
to
enduring
material
requirements
should
be
made
‐pt
Scale:
Strongly
Disagree,
Strongly
Agree)
(n=628)
Results of AMA Call for Comment on AMA/ACCME Alignment and Simplification Page 4 of 47
4
63% 30%^ 7%
Agree
(Top
3 Box
‐^
8, 9,
Neutral
(Middle
4 Box
‐^
4 ‐
Disagree
(Bottom
3
Box
‐^
1, 2,
Format–specific
requirements
for
certifying
activities
for
AMA
PRA
Category
Credit
3. Journal
‐based
CME
This
format
specific
requirement
would
remain
the
same:
•^
Be
a
peer
‐reviewed
article.
These
format
specific
requirements
would
be
deleted:
•^
Provide
an
assessment
of
the
learner
that
measures
achievement
of
the
educational
purpose
and/or
objective(s)
of
the
activity
with
an
established
minimum
performance
level;
this
may
include,
but
is
not
limited
to,
patient
management
case
studies,
a
post
‐test
and/
or
application
of
new
concepts
in
response
to
simulated
problems.
•^
Communicate
to
the
participants
the
minimum
performance
level
that
must
be
demonstrated
in
the
assessment
in
order
to
successfully
complete
the
activity
for
AMA
PRA
Category
Credit
Q3.
These
changes
to
journal
‐based
CME
requirements
should
be
made
‐pt
Scale:
Strongly
Disagree,
Strongly
Agree)
(n=620)
Results of AMA Call for Comment on AMA/ACCME Alignment and Simplification Page 5 of 47
6
61% 33%^ 6%
Agree
(Top
3 Box
‐^
8, 9,
Neutral
(Middle
4 Box
‐^
4 ‐
Disagree
(Bottom
3
Box
‐^
1, 2,
Format–specific
requirements
for
certifying
activities
for
AMA
PRA
Category
Credit
5. Manuscript
review
(for
journals)
All
format
specific
requirements
would
be
deleted:
-^
Involve
a
review
of
an
article
that
has
been
submitted
for
publication
in
a
journal
that
is
included
in
the
MEDLINE
bibliographic
database.
-^
Involve
a
review
of
a
manuscript
that
is
an
original
contribution
to
the
medical
literature
that
requires
multiple
reviewers.
-^
Provide
clear
instructions
to
the
physician
on
how
to
successfully
complete
the
activity.
(This
requirement
will
be
subsumed
into
core
requirement
so
will
not
be
listed
separately
as
a
format
specific
requirement.)
-^
Be
at
a
depth
and
scope
that
require
a
review
of
the
literature
and
a
knowledge
of
the
evidence
base
for
the
manuscript
reviewed.
(This
requirement
will
be
subsumed
into
core
requirement
so
will
not
be
listed
separately
as
a
format
specific
requirement.)
-^
Have
an
oversight
mechanism
to
evaluate
the
quality
of
reviews
submitted.
Q5.
These
changes
to
manuscript
review
requirements
should
be
made
‐pt
Scale:
Strongly
Disagree,
Strongly
Agree)
(n=598)
Results of AMA Call for Comment on AMA/ACCME Alignment and Simplification Page 7 of 47
7
61% 34%^ 5%
Agree
(Top
3 Box
‐^
8, 9,
Neutral
(Middle
4 Box
‐^
4 ‐
Disagree
(Bottom
3
Box
‐^
1, 2,
Format–specific
requirements
for
certifying
activities
for
AMA
PRA
Category
Credit
6. Performance
Improvement
Continuing
Medical
Education
(PI
CME)
These
format
specific
requirements
would
remain
the
same:
•^
Have
an
oversight
mechanism
that
assures
content
integrity
of
the
selected
performance
measures.
These
measures
must
be
evidence
based
and
well
designed
(e.g.,
clearly
specify
required
data
elements,
ensure
that
data
collection
is
feasible).
•^
Provide
clear
instructions
to
the
physicians
that
define
the
educational
process
of
the
PI
CME
activity
(documentation,
timelines,
etc.).
•^
Provide
adequate
background
information
so
that
physicians
can
identify
and
understand
the
performance
measures
that
will
guide
their
PI
CME
activity,
and
the
evidence
base
behind
those
measures.
•^
Validate
the
depth
of
physician
participation
by
a review
of
submitted
PI
CME
activity
documentation.
•^
Consist
of
the
following
three
stages:
Stage
A:
Learning
from
current
practice
performance
assessment
Assess
current
practice
using
the
identified
performance
measures,
either
through
chart
reviews
or
some
other
appropriate
mechanism.
Stage
B:
Learning
from
the
application
of
PI
to
patient
care
Implement
the
intervention(s)
based
on
the
results
of
the
analysis,
using
suitable
tracking
tools.
Participating
physicians
should
receive
guidance
on
appropriate
parameters
for
applying
the
intervention(s).
Q6.
These
changes
to
PI
CME
requirements
should
be
made
‐pt
Scale:
Strongly
Disagree,
Strongly
Agree)
Stage
C:
Learning
from
the
evaluation
of
the
PI
CME
effort
Re
‐assess
and
reflect
on
performance
in
practice
measured
after
the
implementation
of
the
intervention(s),
by
comparing
to
the
assessment
and
using
the
same
performance
measures.
Summarize
any
practice,
process
and/or
outcome
changes
that
resulted
from
conducting
the
PI
CME
activity.
This
format
specific
requirement
related
to
Stage
A
would
be
deleted:
Participating
physicians
must
be
actively
involved
in
the
analysis
of
the
collected
data
to
determine
the
causes
of
variations
from
any
desired
performance
and
identify
appropriate
intervention(s)
to
address
these.
(n=582)
Results of AMA Call for Comment on AMA/ACCME Alignment and Simplification Page 8 of 47
Results by Organization Type
*Note: 9
This question
allowed
multiple
responses
so
some
respondents
are
included
in
more
than
one
category.
n^
%^
n^
%^
n^
%^
n^
%^
n^
%^
n^
%^
n^
%^
n^
%^
n^
%
n^
%
Top 3 Box (8-10)
127
68.6%
49
66.2%
2
100.0%
19
65.5%
204
61.1%
14
63.6%
11
55.0%
15
65.2%
73
68.2%
438
66.0%
Middle 4 Box (4-7)
47
25.4%
23
31.1%
0
0.0%
9
31.0%
109
32.6%
5
22.7%
7
35.0%
7
30.4%
30
28.0%
193
29.1%
Bottom 3 Box (1-3)
11
5.9%
2
2.7%
0
0.0%
1
3.4%
21
6.3%
3
13.6%
2
10.0%
1
4.3%
4
3.7%
33
5.0%
Total
185
100.0%
74
100.0%
2
100.0%
29
100.0%
334
100.0%
22
100.0%
20
100.0%
23
100.0%
107
100.0%
664
100.0%
Top 3 Box (8-10)
117
65.0%
53
73.6%
1
50.0%
14
51.9%
201
64.6%
14
73.7%
6
33.3%
11
50.0%
56
55.4%
400
63.7%
Middle 4 Box (4-7)
54
30.0%
18
25.0%
1
50.0%
12
44.4%
93
29.9%
4
21.1%
12
66.7%
9
40.9%
41
40.6%
197
31.4%
Bottom 3 Box (1-3)
9
5.0%
1
1.4%
0
0.0%
1
3.7%
17
5.5%
1
5.3%
0
0.0%
2
9.1%
4
4.0%
31
4.9%
Total
180
100.0%
72
100.0%
2
100.0%
27
100.0%
311
100.0%
19
100.0%
18
100.0%
22
100.0%
101
100.0%
628
100.0%
Top 3 Box (8-10)
113
63.5%
48
67.6%
1
50.0%
15
55.6%
194
63.4%
13
68.4%
6
33.3%
13
59.1%
62
61.4%
392
63.2%
Middle 4 Box (4-7)
52
29.2%
22
31.0%
1
50.0%
12
44.4%
89
29.1%
4
21.1%
11
61.1%
7
31.8%
35
34.7%
187
30.2%
Bottom 3 Box (1-3)
13
7.3%
1
1.4%
0
0.0%
0
0.0%
23
7.5%
2
10.5%
1
5.6%
2
9.1%
4
4.0%
41
6.6%
Total
178
100.0%
71
100.0%
2
100.0%
27
100.0%
306
100.0%
19
100.0%
18
100.0%
22
100.0%
101
100.0%
620
100.0%
Top 3 Box (8-10)
104
60.1%
45
63.4%
2
100.0%
12
46.2%
184
61.1%
13
72.2%
7
43.8%
12
54.5%
52
53.6%
363
60.0%
Middle 4 Box (4-7)
58
33.5%
22
31.0%
0
0.0%
12
46.2%
92
30.6%
2
11.1%
8
50.0%
5
22.7%
38
39.2%
199
32.9%
Bottom 3 Box (1-3)
11
6.4%
4
5.6%
0
0.0%
2
7.7%
25
8.3%
3
16.7%
1
6.3%
5
22.7%
7
7.2%
43
7.1%
Total
173
100.0%
71
100.0%
2
100.0%
26
100.0%
301
100.0%
18
100.0%
16
100.0%
22
100.0%
97
100.0%
605
100.0%
Top 3 Box (8-10)
108
62.8%
43
60.6%
2
100.0%
14
53.8%
179
60.5%
11
57.9%
6
37.5%
14
63.6%
51
53.1%
362
60.5%
Middle 4 Box (4-7)
53
30.8%
26
36.6%
0
0.0%
11
42.3%
97
32.8%
5
26.3%
9
56.3%
5
22.7%
37
38.5%
197
32.9%
Bottom 3 Box (1-3)
11
6.4%
2
2.8%
0
0.0%
1
3.8%
20
6.8%
3
15.8%
1
6.3%
3
13.6%
8
8.3%
39
6.5%
Total
172
100.0%
71
100.0%
2
100.0%
26
100.0%
296
100.0%
19
100.0%
16
100.0%
22
100.0%
96
100.0%
598
100.0%
Top 3 Box (8-10)
107
64.5%
49
70.0%
2
100.0%
13
50.0%
162
56.1%
13
68.4%
8
50.0%
10
45.5%
59
63.4%
357
61.3%
Middle 4 Box (4-7)
54
32.5%
17
24.3%
0
0.0%
11
42.3%
113
39.1%
5
26.3%
8
50.0%
10
45.5%
31
33.3%
199
34.2%
Bottom 3 Box (1-3)
5
3.0%
4
5.7%
0
0.0%
2
7.7%
14
4.8%
1
5.3%
0
0.0%
2
9.1%
3
3.2%
26
4.5%
Total
166
100.0%
70
100.0%
2
100.0%
26
100.0%
289
100.0%
19
100.0%
16
100.0%
22
100.0%
93
100.0%
582
100.0%
Top 3 Box (8-10)
98
59.0%
42
60.9%
2
100.0%
13
50.0%
151
52.6%
12
63.2%
6
37.5%
11
50.0%
51
55.4%
330
57.0%
Middle 4 Box (4-7)
58
34.9%
25
36.2%
0
0.0%
10
38.5%
120
41.8%
5
26.3%
8
50.0%
8
36.4%
34
37.0%
213
36.8%
Bottom 3 Box (1-3)
10
6.0%
2
2.9%
0
0.0%
3
11.5%
16
5.6%
2
10.5%
2
12.5%
3
13.6%
7
7.6%
36
6.2%
Total
166
100.0%
69
100.0%
2
100.0%
26
100.0%
287
100.0%
19
100.0%
16
100.0%
22
100.0%
92
100.0%
579
100.0%
Q1 These changesto the corerequirements shouldbe made.Q^2
These changes to enduring materialrequirements shouldbe made.Q^3
These changes to journal-basedCME requirementsshould be made. Q^4
These changes to test item writingrequirements shouldbe made.Q^5
These changes to manuscript reviewrequirements shouldbe made.Q^6
These changes to PI CMErequirements shouldbe made.Q^7
These changes to internet point-of-care requirementsshould be made.
Other (specify)
Total
*Please tell us which of the following describes you or your organization?
ACCME-accreditedprovider
State-accreditedprovider (provideraccredited by an
ACCME Recognized StateMedical Society
ACCME-defined
commercialinterest
ACCME member organization
Physician
State medical
society
State medicallicensing board
Specialtycertificationboard
Results of AMA Call for Comment on AMA/ACCME Alignment and Simplification Page 10 of 47
NP – ACCME accredited CME provider (national provider); SP – State accredited CME provider; CI – Commercial Interest; MO – Member Organization; P – Physician; SMS – State Medical Society; MLB – Medical Licensing Board; SCB – Specialty Certification Board; O - Other
AMA Survey: Comments on Proposed Changes
Introduction
This document shows comments received on the AMA survey. Out of 664 surveys, a total of 282 individuals provided
750 written comments.
The comments, segregated by question, are listed verbatim in order by date/time received; the number in the left
column is a unique identifier for each respondent.
After each comment, the self-selected designation(s) of each respondent is in parentheses, with the following abbreviations (Note: Some respondents selected more than one category):
NP ACCME accredited CME provider (national provider) SP State accredited CME provider CI ACCME-Defined Commercial Interest MO ACCME-Member Organization P Physician SMS State Medical Society MLB Medical Licensing Board SCB Specialty Certification Board O Other
The total number of comments by question is as follows:
Question on Proposed Changes Related to Number of comments
- Core Requirements 141
- Enduring Materials 74
- Journal-Based CME 66
- Test-Item Writing 67
- Manuscript Review 47
- PI CME 54
- Internet Point-of-Care 56
- Other Comments on the Proposed Changes 135
- Comments on the Glossary 110 Total: 750
Results of AMA Call for Comment on AMA/ACCME Alignment and Simplification Page 11 of 47
AMA Survey Comments on Proposed Changes Page 3 of 37
NP – ACCME accredited CME provider (national provider); SP – State accredited CME provider; CI – Commercial Interest; MO – Member Organization; P – Physician; SMS – State Medical Society; MLB – Medical Licensing Board; SCB – Specialty Certification Board; O - Other
Q1 – Proposed Core Requirement Changes
Comments Related to Core Requirement 7 1. #7 could be modified to "provide an assessment that measures the achievements towards attaining the educational objective and or purpose of the activity (O) 8 2. It's not always clear that the best or only evaluation of an activity should be an assessment of the individual learner. For example, a CME activity based around improving team performance in patient safety might be best evaluated by assessment of the team or by changes in safety reporting in the system. Focusing exclusively on learner assessment may be overly limiting. (NP) 16 3. Clarifies the requirements and expectations with standards referenced (P) 19 4. Item # 7 seems to move the outcome from achieving the goal to assessment of the individual learner. This could inadvertently limit activities that are intended to improve teamwork. I prefer, "Evaluate the effectiveness in achieving the educational purpose and/or objective of the activity." This might include patient outcome measures, system improvements, reflection by the participant at a point in time distance from the activity, other tools such as 360-degree evaluations, tests, or observation. Learner assessment sounds like a test which is at best, Kirkpatrick Level 2 evidence of educational efficacy. (NP, P, O) 22 5. Point 7, would NOT require an assessment. This will thwart such effective POS CME as provided by Up To Date, one of the most effective real time methods of learning in critical , time sensitive situations. (P) 26 6. Providing an assessment of the learner will require substantial new effort and resources. Learners are already reluctant to complete even simple forms that assess the value of the CME content. (P) 27 7. Why fix if it is not broken. A few of the changes are good, others are not needed. Do we need to routinely test adult learners to confirm they were participating? (P) 42 8. The proposed changes provide clarification and direction in some core requirements, while allowing more flexibility and creativity for the CME provider in others. These modifications continue to protect accredited CME from commercial influence and seem to be more manageable for both physicians and CME providers. (SP) 46 9. Language largely makes more explicit and clarifies what we do now. I'm not sure all of the assessment noted in 7 should be of the learner, however. (NP, MO, P) 48 10. My concerns relate to proposed changes in Item 7. I am unsure what is meant by the phrase "provide an assessment". I am also unsure how such assessments would be implemented and managed. (NP, P) 52 11. Wayyyyy to much over-thinking and over-regulation of an already cumbersome process. (P) 59 12. Would like to know the rationale for changes (O) 60 13. It is difficult enough for CME providers (in this case) to follow the regulations - without changing them. If you make any changes, it should only be to simplify by omission. We must constantly be aware of physician burnout and its causes, which include excessive regulations that have very limited impact on patient care and learning. (NP, P) 63 14. Increasingly we are required to take accredited CME related to institutional or governmental priorities, for example, opioids, safety, etc. I would note that this does not mean it is a gap for each of the individual learners as implied with #2 above. / Why is "should" in item 3? (NP, P, O) 66 15. Costs of CME especially hands on courses are thousands of dollars for each in neurosurgery. These costs are not covered by most practices, the VA, Government positions and military positions. And they are not tax deductible unless supported by the employer. More industry support for the meetings aned courses is needed to keep up with current information, procedures, and new drug policies. (P) 73 16. excellent 78 17. #3 does not look good. It seems to imply that goals and/or objectives are optional (P) 84 18. #7 "Assessment of the learner..." might add "in a manner consistent with the nature of the activity" since different formats of learning will call for different assessment tactics. (O) 87 19. CONSIDERO QUE EL PROFESIONAL ,QUE HACE LAS COSAS CON CORRECCION, ESTUDIANDO Y EVALUANDO TODO EL TIEMPO, CADA UNO DEBE HACERSE RESPONSABLE DE SU RENDIMIENTO Y ESTUDIO , A MAS RENDIMIENTO Y SAPIENZA PREMIAR CON MAS ACREDITACION (MO, P, O) 88 20. I think these are god changes. I like taking AMA out but I think non-promotional was a good thing. (NP, SP, SCB)
Results of AMA Call for Comment on AMA/ACCME Alignment and Simplification Page 13 of 47
AMA Survey Comments on Proposed Changes Page 4 of 37
NP – ACCME accredited CME provider (national provider); SP – State accredited CME provider; CI – Commercial Interest; MO – Member Organization; P – Physician; SMS – State Medical Society; MLB – Medical Licensing Board; SCB – Specialty Certification Board; O - Other
89 21. Learning outcome should NOT be individual focused, but rather health focused. Measure how the learning is APPLIED not that it has occurred. (NP, SP, MO, O) 93 22. The requirements are already incredibly cumbersome and nit-picky. They should be simplified, NOT made more complicated. Too much time is wasted in "compliance" paperwork... and the CME is NOT measurably any better. (NP, MO, P) 106 23. It would be helpful if an example of what you are trying to accomplish is attached. (SP) 111 24. #7 will in most cases remain superficial and will fail to achieve any reliable measurement. This should be a goal but NOT a requirement. (NP) 112 25. Under what circumstances would it not be appropriate to communicate an educational purpose? (P) 121 26. we should be jargon free, the scene to rely on bureaucratic teas and the passive voice. (NP, SP, MO, P) 130 27. nicely cleaned up (P) 131 28. The narrow definition down to individual learner's is concerning for all formats. How does a CME provider do this in large symposium settings? Or even at a local level if outside participants attend a general Grand Rounds. You seem to be indicating we need to be able to measure each person who attends and produce specific outcome data. It is simply not possible for all formats. #2 and #7. / Telling physicians what they need to do to complete an activity such as a didactic lecture is overly burdensome to the provider. This should not apply to all formats. #3 / #2 "that activity's learners". / #3 "provide clear instructions on how to successfully complete the activity" / #7 "provide an assessment of the learner that measures" (SMS) 133 29. #3 should not be left to the discretion of the course; they should always be expected to identify purpose and/or objectives for CME credit. #4 is a welcome change, as the AMA does not hold authority on learning methodologies. (P) 136 30. These are good aspirations, but I wonder how well they can be implemented in practice. (P) 137 31. This will decrease confusion among planners and faculty and move CME further in the direction of not just providing knowledge, but of being an agent for both learning and change. (NP) 138 32. I think it is a bit naive to assume that there is a single way to address a gap. Sometimes it is just a piece of knowledge and that is easy. But if there is a lack of ability to synthesize, unidirectional CME might not be the way to address it. Second, I think most learners can figure out what is being taught without having to express it specifically. (P) 142 33. I think these changes are clearer than old version. Especially #2. (O) 145 34. 2 - no problem / 3 - the accredited provider should communicate the goals and instructions, preferably in advance so I know whether or not I want to go through the activity. And when wouldn't it be appropriate for the accredited provider to communicate? / 4. the AMA should provide some level of oversight over the learning formats / 6. why wouldn't we want activities to be in line with CEJA opinions? and activities must be nonpromotional in nature. we should not be letting pharma and device manufacturers slide in under the guise of education / 7 no problem with this one (P) 147 35. assessment of the learner is not necessary (P) 152 36. I would like to see some clarification of number 7. (SP) 161 37. I strongly agree with all except 7. I don't think we should have to take a test/assessment after a live CME activity, such as a lecture/small group learn session at a meeting. This is OK for SAMs but not CME. It is OK for the learner to complete an evaluation as to how much was learned and how it will be used as we do now. (P) 163 38. these seem superficially reasonable but i think we would be better off with less regulation of cme not more and to that end i would prefer the ama leave cme alone (P) 169 39. Of course CME activities should be non-promotional (NP) 179 40. Number 7. may require some thought as to the "assessment". These can be included in the lecture with feedback? At the end? I find this a bit of a problem however well meaning. (P) 182 41. I am not sure if I am understanding #3 strike out of the objectives being submitted in advance of a program. Is this to say the objectives are not required before the presentation? If so I would disagree due to needing to advertise the programs with objectives included. (SP) 186 42. How does effectively measure "Acheivement of the Educationjal Purpose"? (P) 190 43. The assessment req is difficult to ascertain unless there would be a follow-up months later. (NP. P) 198 44. Love the creativity but I think clarification on what qualifies as an assessment of the learner needs clarification and some definition of what learning methodologies are acceptable needs to be enhanced. When appropriate to the activity learning objectives should be communicated. Again, I think this needs some further clarifications. (NP)
Results of AMA Call for Comment on AMA/ACCME Alignment and Simplification Page 14 of 47
AMA Survey Comments on Proposed Changes Page 6 of 37
NP – ACCME accredited CME provider (national provider); SP – State accredited CME provider; CI – Commercial Interest; MO – Member Organization; P – Physician; SMS – State Medical Society; MLB – Medical Licensing Board; SCB – Specialty Certification Board; O - Other
294 70. The new wording aligns more with ACCME criteria for providers. (SP) 296 71. I'm not sure what is meant by "provide clear instructions on how to successfully complete the activity" since many CME programs are presentations (NP, P) 302 72. 2. presupposes "practice gaps," where "practice" would have to be conceived of broadly, and the "gaps," if any, may be different for different learners. 7. appropriately focuses on the learner rather than the activity, but there may be no valid way to attribute achievement of the educational purpose to this particular activity. Still, these are overall improvements. (SMS) 303 73. Challenging to provide an assessment tool other than MCQ that truly measures achievement. (P) 305 74. What happened to single stream education with the AMA - AOA? Is this going to change and then another change? (SMS) 309 75. we were 3 years and the students could not cope with the changing environment in the 80's and were forced to shift to 4 years , how soon do we forget......... (P) 314 76. Seems to explain well what is required with the changes. (O) 315 77. #3 seems a bit much to me.. We already describe the educational purpose more than that to suggests there is a clear way to successfully complete the activity cannot be done simply. / #7 suggests that there should be a pre-post exam for each activity. As one who is responsible for a weekly CME I can assure you that my speakers will not write pre-post exams and neither will I. (P) 327 78. Agree generally with the increased flexibility of these requirements. However, I don't understand why #3 is being edited with the addition of the clause, "When appropriate to the activity and the learners..." When is it ever appropriate to not identify and educational purpose and/or objectives? Agree on the addition of the clause about provision of clear instructions. (NP. P) 330 79. ACCME standards are can be complex, and often not well communicated. / To have the AMA move toward ACCME standards is likely a step in the wrong direction to help better simplify and make readily understandable expectations for CME standards for physicians. (SP, SMS) 331 80. These seem straightforward. (NP, P) 335 81. I don't understand the significance of these changes. On a superficial level, they seem appropriate. (P) 340 82. Not sure about how to define and who defines educational need (#2) and what is implied about assessing the learner (#7) (P) 342 83. Does item 7 mean that you have to take a test at the end of each activity and if you do not respond correctly to a certain percentage of the answers you will not get the CME credit? (P) 346 84. too little too late (P, SMS) 348 85. If AMA wants to maintain value of CME then stop this menace of MOC (P) 361 86. tailoring credit to an individual's needs is doomed to fail. tailoring to a groups needs is unmeasurable. (MLB) 369 87. I don't have any problem with the current requirements, but the changes above make sense. (P) 372 88. As a physician who has been director for CME conferences, number 2 can consume a lot of time in order to show that an educational need exists for a given topic for a CME conference. As long as all other conditions are met for a CME conference, I do not favor additional specifications on what constitutes educational need, in fact I would prefer that number 2 be deleted. It just provides more paperwork for my CME department to require of me. (P) 373 89. To change many programs dropping the CME certification, making it more simple and exact. (MO, P) 379 90. Most are reasonable, however it will be very difficult to accomplish #7 as revised. (SP, P) 392 91. I strongly support these changes to allow CME providers to innovate and meet the evolving needs of physician- learners. (NP) 393 92. Provides more explicit methods and standards by which CME activities must be structured and implemented. (P, O) 400 93. very well written to provide flexibility in programming (O)
Results of AMA Call for Comment on AMA/ACCME Alignment and Simplification Page 16 of 47
AMA Survey Comments on Proposed Changes Page 7 of 37
NP – ACCME accredited CME provider (national provider); SP – State accredited CME provider; CI – Commercial Interest; MO – Member Organization; P – Physician; SMS – State Medical Society; MLB – Medical Licensing Board; SCB – Specialty Certification Board; O - Other
408 94. Overall, AAFP supports AMA and ACCME’s efforts to align their requirements, simplify the process and reduce administrative burdens for CME provider organizations as we believe this will have a positive impact on physicians and the patients they serve. / / Core Requirement 6-The AAFP does not support AMA’s removal of reference to the CEJA opinions. AAFP believes it is important for CME provider organizations to be aware of those requirements, to ensure they do not put physicians in a position to jeopardize their compliance with CEJA opinions. The AAFP believes the CEJA opinions are so important they are included in our own Credit System Requirements and the AAFP’s Bylaws. / / The AAFP does not support removal of the phrase “and be non-promotional in nature.” This phrase ensures that all certified CME is free from promotion, beyond the protection against commercial influence. There are no other regulations protecting the education and thereby the learners from promotion of non-commercial interests. The AAFP believes this requirement should be retained. / / Core Requirement 7-The proposed language change will significantly change core requirement seven from evaluating the activity to assessing the learners’ achievement of the educational purpose of the activity. / (NP, O) 409 95. I have heard the CEO of the ACCME make several inspirational presentations challenging his organization and organized medicine to overall the current process of accreditation for CME credit. The changes above appear to be taking hearing and words missing rather than a fresh look at a new process. If this represents the beginning of bigger changes yet to come, so be it. If on the other hand these represent the compromise between change agents and traditionalists, then we will all be disappointed. (NP) 410 96. My favorite part about these changes are a change to include independence in CME activities. Providing assessment is important but can in someways be somewhat burdensome to the one giving the CME activity but I think it is needed overall. (P) 414 97. Most of these are already being implemented by my institution (P) 427 98. This brings some sense into developing educational objectives, style of delivery of CME and evaluation. The education experts have made this a mess. (P) 428 99. Please keep "in advance of participation in the activity". (NP) 430 100. It looks as though the 7 core requirements will help providers to have clearer direction. The one area of concern is requirement # 7. As long as the assessment is not rigidly defined to measure whether an accredited educational event met its purpose and objective it is acceptable. (NP) 447 101. On 3 above - If planned a live activity, does the flyer have to give instructions on how to successfully complete the activity or is this just meant for enduring materials? (SP) 448 102. should all 6 of the competencies be mentioned in #2? (NP, MO, P) 452 103. Needed updating! (P) 456 104. non-promotional... (P) 460 105. Wordsmithing and neither add nor subtract actual value (P) 468 106. The flexibility can lead to innovation. (NP, P) 470 107. "underlies the professional practice gaps of that activity’s learners" is too negative. CME can be obtained to stay up to date, which does not necessarily indicate a "gap". I think this statement needs revision, or deletion of the last phrase from the sentence. the rest is fine (NP, P) 473 108. I don't like #3. Why state "when appropriate" at the beginning - isn't communicating the learning objectives always appropriate? (SP) 479 109. Seem reasonable (P) 480 110. The Changes do not appear to be that substantial (P, O) 481 111. Changes better align with ethical CME (O) 482 112. The rules and regulations as they stand are so burdensome and arbitrary and unresponsive to different specialties that just changing this language is of no help. This process has become so over reaching that steps may need to be taken to determine other entities to evaluate appropriate education. (P) 483 113. The demonstrated educational need in point 2 may not be based solely on an identified practice gap of an individual but an opportunity to enhance the practice for all physicians. I think that point 7 is important but there is a need to clarify if the measurement of achievement is in the educational setting alone. (P, O) 485 114. I like the idea of incorporating the learning goals into the presentation. (P) 487 115. need to check competence not sit in a lecture after a heavy day of work (P)
Results of AMA Call for Comment on AMA/ACCME Alignment and Simplification Page 17 of 47
AMA Survey Comments on Proposed Changes Page 9 of 37
NP – ACCME accredited CME provider (national provider); SP – State accredited CME provider; CI – Commercial Interest; MO – Member Organization; P – Physician; SMS – State Medical Society; MLB – Medical Licensing Board; SCB – Specialty Certification Board; O - Other
533 127. We support 2, 3,4 and 67, but we do not support AMA’s removal of reference to the AMA Council on Ethical and Judicial Affairs (CEJA) Code of Medical Ethics (Code). CAFP believes it is important for CME provider organizations to be aware of those requirements; to ensure they do not put physicians in a position to jeopardize their compliance with the Code’s opinions. Our national organization, the AAFP believes the AMA Code of Medical Ethics is so significant that they are included in the AAFP Bylaws(Article XIV - Ethics) and Credit System Requirements. CAFP does not support removal of the phrase “and be non-promotional in nature.” This phrase ensures that all certified/accredited CME is free from promotion, beyond the protection against commercial influence (ACCME Standards for Commercial Support). No other regulations protect the education and learners from promotion of non-commercial interests. We believe this requirement should be retained. / (NP, O) 540 128. Changes to #4 provide more flexibility, but are ambiguous and do not align with the intent of #5 being at an "appropriate depth and scope". We would suggest adding bibliographic references related to learning methodologies and educational design. (NP) 543 129. RE: Item 6 - Are CEJA Opinions equivalent to the AMA Code of Medical Ethics? I am concerned that the SCS do not explicitly cover 9.6.2 Gifts to Physicians from Industry, item (d). / RE: Item 7 - Changing "Evaluate the effectiveness..." to "Provide an assessment..." if taken literally, would mean that CME providers will need to implement learning assessments (aka: tests) into their live activities. / I am NOT in favor of this change. / Alternative wording could be, "Measure achievement of the educational purpose and/or objective(s) of the CME activity." / (NP) 551 130. Improved standards may lessen meaningless paper work in presentation (NP) 564 131. They add clarification. (SP) 591 132. #6, why only 1 organization can implement? (P) 602 133. I don't believe providers should be held responsible for the performance of the participants. Providing an assessment is fine, but it should be stated that the purpose of this assessment should be directed to the participant more than the provider. (NP) 651 134. Looks good. (NP, MO, P) 659 135. Clarity is needed on how to assess the learner to measure achievement of educational purpose. Does learner self assessment count? (P, O) 697 136. We only need one set of requirements, not ACCME AND AMA! (NP) 701 137. Clearer wording that more closely parallels wording used by ACCME, which helps people new to CME administration more quickly understand intent. However, no change is made in the substance of expectations/requirements, which were understood reasonably well previously. (NP) 713 138. Uncertain why the non-promotional was struck? (P) 769 139. Not sure how one would reliably measure or demonstrate #2 (P) 877 140. is 'non-promotional in nature' being removed because it is a given? (SP) 891 141. This is a ridiculous question (P)
Results of AMA Call for Comment on AMA/ACCME Alignment and Simplification Page 19 of 47
AMA Survey Comments on Proposed Changes Page 10 of 37
NP – ACCME accredited CME provider (national provider); SP – State accredited CME provider; CI – Commercial Interest; MO – Member Organization; P – Physician; SMS – State Medical Society; MLB – Medical Licensing Board; SCB – Specialty Certification Board; O - Other
Q2 – Proposed Enduring Material Changes
Comments Related to Enduring Materials 19 1. I agree with deleting the three bulleted format requirements. (NP P O) 22 2. Again, would modify post test methodology so that time critical learning can take place (P) 42 3. We don't use Enduring Material so do not have a strong opinion on this. (SP) 46 4. No assessment? Or is that also assumed by the previous changes? (NP MO P) 48 5. Reference earlier comments. (NP P) 52 6. Make it simpler! (P) 62 7. Deleting a minimum performance level seems like a step backwards. Some organizations pass everyone, so you can pass without even reading/learning anything. Think you should keep the 2nd and 3rd bullet requirements. (NP) 63 8. why do away with minimum performance level quiz?? / Maybe I am missing the point. I believe we often learn by being asked questions, and thinking about them. I know I do. The new American Board format of frequent questions to be answered off-line, with on-line discussion among participants has proven popular. (NP P O) 66 9. Still very costly in time and money. (P) 78 10. Why would you no longer require an assessment or goals and objectives? (P) 87 11. EN LO QUE RESPETA A MI EXPERIENCIA ME HA SERVIDO ENORMEMENTE, ESTUDIAR CONSECUENTEMENTE CON ESTA METODOLOGIA , VARIOS ANOS EMPLEO ESTA METODOLOGIA CON EXITO. (MO P O) 89 12. Focus on application of learning not completing the materials (NP SP MO O) 105 13. Not appropriate to give CME credit for just having bibliographic materials (P)
106 14. Provide examples of evaluation forms that would meet the suggested goals. (SP)
111 15. Since most live activities fail to measure improvement, why should enduring materials have a higher standard? Makes no sense. (NP)
115 16. There should be some assessment of the learner that measures achievement of the educational purpose and/or objective(s) of the activity. (NP)
130 17. again, nicely edited (P)
131 18. Leave these applicable to this format. (SMS) 133 19. no changes noted (P)
138 20. Less verbiage is usually good. Not clear why the change is being proposed. (P)
139 21. Second and third bullet are worth retaining, even if redundant to what is covered elsewhere. (P) 142 22. Why delete the minimum performance level? People should know expectations and most doctors are over-achievers, not under achievers. (O)
163 23. again, sounds reasonable, but more important than my opinion would be the opinion of cme providers - carlat psychiatry and psychiatric times / (P)
164 24. Bullets 2 and 3 convey important principles. (P) 167 25. suggest keeping minimum performance level if the activity has a post-test (P SCB)
172 26. This is reasonable; I don't I don't think there should be a double standard (NP SP P)
181 27. Minimum performance good idea (P) 191 28. Sorry I don't quite understand why we are deleting these three requirements? Are they no long required? (SP)
193 29. So there will no longer be a POST test? Hmmmm... (SP) 198 30. again, we need clarification of assessment of the learner. Should all live activities have some assessment other than the evaluation. Do learners need to pass or does the provider just do the assessment (NP)
210 31. second and third bullet should be retained (P)
Results of AMA Call for Comment on AMA/ACCME Alignment and Simplification Page 20 of 47
