Delphi Method vs. the Nominal Group Technique

I would first like to point out the purpose, similarities and differences of the two techniques.

First the Delphi Method’s intended purpose is:
“The Delphi method is an iterative process used to collect and distill the judgments of experts using a series of questionnaires interspersed with feedback. The questionnaires are designed to focus on problems, opportunities, solutions, or forecasts. Each subsequent questionnaire is developed based on the results of the previous questionnaire. The process stops when the research question is answered: for example, when consensus is reached, theoretical saturation is achieved, or when sufficient information has been exchanged.”

The purpose of the Modified Nominal Group Technique is:

“The modified nominal group technique (NGT) is a useful and practical course evaluation tool that complements existing methods such as evaluation forms, surveys, pretests and posttests, focus groups, and interviews. The NGT’s unique contribution to the evaluation process is the semi-quantitative, rank-ordered feedback data obtained on learners’ perceptions of a course’s strengths and weaknesses. In this paper, we demonstrate through a worked example how to use a modified NGT as a course evaluation tool in medical education.”

Although similar, not identical and it seems that the NGT can actually augment and compliment the Delphi method. I participated in a NGT event for the purposes of a Team Building and Strategy session. The end result was surprisingly one of superior quality. Although the time it took to get to the end state was a significant investment, the result was well worth it. During the course of the session the organizational leadership followed the process below. Although the objective was not to review a course, the intent of the exercise was to identify areas of strength and improvement within the organization to facilitate focused improvements, speed the maturation process and develop an organizational strategy that had 100% leadership endorsement. The results were amazing. I am normally for anonymity which is certainly a benefit of the Delphi method however, I think something’s require open dialog to achieve resolution and this was certainly one of them. With that said I believe each of them has their own merits and one may be more beneficial than another in a particular scenario.   

Step 1. Present evaluation questions to the large group of learners

• What were the strengths/highlights of the course?

• What were the weaknesses/suggestions for improvement?

Step 2. Silent phase

• Form small groups of four to eight participants, each with a flip chart.

• Assign a faculty facilitator, or elect a scribe for each group.

• Issue five pink and five yellow “stickies” to each participant.

• Without conferring or group discussion, participants record one-strength on each pink sticky and one weakness/suggestion for improvement

on each yellow sticky.

Step 3. Round-robin phase

• Participants stick one pink sticky in turn on the flip chart without comment or discussion until all ideas are exhausted.

• The facilitator or scribe groups similar comments together.

• Repeat the process using yellow stickies for suggestions for improvement.

Step 4. Discussion/item clarification

• The group clarifies unclear items and edits the grouped items into themes.

• The facilitator or scribe lists and letters items in order of popularity.

Step 5. Voting phase

• Participants rank their top five suggestions in each list from 1 to 5.

• Participants award 5 points to their top item, 4 to the second, and so on.

• The facilitator or scribe collects these lists for data gathering.

Step 6. Small-group data gathering

• Scribes or facilitators add the total points for each lettered item to produce a rank-ordered, weighted list of the groups’ opinions of the strengths

and weaknesses of the course.

• Scribes or facilitators write this list (with weightings) on the flip chart to present to the large group.

Step 7. Large-group data combining

• Reconvene the large group and examine the results from the small groups.

• Combine the small-group scores. (Small groups in this exercise usually produce very similar factors and this can be done with minimal discussion).

• Present the cohort’s ranked, weighted opinions of the strengths of the course and suggestions for improvement.

Step 8. Large-group discussion around dominant themes

• Record or take notes on the rich discussion that now ensues.

NGT Source: http://www.stfm.org/fmhub/fm2004/June/Alison402.pdf

Delphi Source:

http://jite.org/documents/Vol6/JITEv6p001-021Skulmoski212.pdf