I would first like to point out
the purpose, similarities and differences of the two techniques.
First the
Delphi Method’s intended purpose is:
“The Delphi method is an
iterative process used to collect and
distill the judgments of experts using a series of questionnaires
interspersed with feedback. The questionnaires are designed to focus on
problems, opportunities, solutions, or forecasts. Each subsequent questionnaire
is developed based on the results of the previous questionnaire. The process
stops when the research question is answered: for example, when consensus is
reached, theoretical saturation is achieved, or when sufficient information has
been exchanged.”
The
purpose of the Modified Nominal Group Technique is:
“The modified nominal group technique (NGT) is a useful and practical course evaluation tool that complements existing
methods such as evaluation forms, surveys, pretests and posttests, focus
groups, and interviews. The NGT’s unique contribution to the evaluation process
is the semi-quantitative, rank-ordered feedback data obtained on learners’ perceptions
of a course’s strengths and weaknesses. In this paper, we demonstrate through a
worked example how to use a modified NGT as a course evaluation tool in medical
education.”
Although similar,
not identical and it seems that the NGT can actually augment and compliment the
Delphi method. I participated in a NGT event for the purposes of a Team
Building and Strategy session. The end result was surprisingly one of superior
quality. Although the time it took to get to the end state was a significant
investment, the result was well worth it. During the course of the session the
organizational leadership followed the process below. Although the objective
was not to review a course, the intent of the exercise was to identify areas of
strength and improvement within the organization to facilitate focused
improvements, speed the maturation process and develop an organizational
strategy that had 100% leadership endorsement. The results were amazing. I am
normally for anonymity which is certainly a benefit of the Delphi method
however, I think something’s require open dialog to achieve resolution and this
was certainly one of them. With that said I believe each of them has their own
merits and one may be more beneficial than another in a particular scenario.
Step 1. Present evaluation questions to the large
group of learners
• What
were the strengths/highlights of the course?
• What
were the weaknesses/suggestions for improvement?
Step 2. Silent phase
• Form
small groups of four to eight participants, each with a flip chart.
• Assign
a faculty facilitator, or elect a scribe for each group.
• Issue
five pink and five yellow “stickies” to each participant.
•
Without conferring or group discussion, participants record one-strength on
each pink sticky and one weakness/suggestion for improvement
on each
yellow sticky.
Step 3. Round-robin phase
•
Participants stick one pink sticky in turn on the flip chart without comment or
discussion until all ideas are exhausted.
• The
facilitator or scribe groups similar comments together.
• Repeat
the process using yellow stickies for suggestions for improvement.
Step 4. Discussion/item clarification
• The
group clarifies unclear items and edits the grouped items into themes.
• The
facilitator or scribe lists and letters items in order of popularity.
Step 5. Voting phase
•
Participants rank their top five suggestions in each list from 1 to 5.
•
Participants award 5 points to their top item, 4 to the second, and so on.
• The
facilitator or scribe collects these lists for data gathering.
Step 6. Small-group data gathering
•
Scribes or facilitators add the total points for each lettered item to produce
a rank-ordered, weighted list of the groups’ opinions of the strengths
and
weaknesses of the course.
•
Scribes or facilitators write this list (with weightings) on the flip chart to
present to the large group.
Step 7. Large-group data combining
• Reconvene
the large group and examine the results from the small groups.
•
Combine the small-group scores. (Small groups in this exercise usually produce very
similar factors and this can be done with minimal discussion).
•
Present the cohort’s ranked, weighted opinions of the strengths of the course
and suggestions for improvement.
Step 8. Large-group discussion around dominant
themes
• Record or take notes on the rich discussion that
now ensues.
Delphi Source: