Step 4: Scoring the options
Scoring range
The default range from rank 1 (very bad) to rank 7 (very good) was used by only 4 study sites (Chile, China, Greece - Crete and Spain); many used 1-5 (Botswana, Morocco, both Russian and both Turkish sites) and others 1-10 (Cape Verde, Portugal and Tunisia). The rank 0, which indicates a killer criterion, was actually applied by the stakeholders in Botswana, Spain and Tunisia.
Scoring methodology
Most study sites made the scoring in the recommended stakeholder groups
Table 6: Number and type of stakeholder groups formed for the scoring step
| Study site | # of groups | Type of participants |
| Spain | 2 | Farmers, others |
| Portugal | 2 | Gois, Macao (the two subsites) |
| Greece - Crete | - | Individual scoring discussed in plenary (?) |
| Turkey - Karapinar | 3 | 2 farmers groups, 1 experts group |
| Turkey - Eskisehir | 4 | 2 farmers groups, 2 experts group |
| Morocco | - | Discussion in plenary with predominant role of farmers |
| Tunisia | 3 | Transect groups |
| Russia - Djanybek | 4 | Administration, practitioners, farmers, external experts |
| Russia - Novy | 4 | Administration, practitioners, farmers, external experts |
| China | 2 | Local, external |
| Botswana | 2 | Village leaders, other land users |
| Chile | 2 | mixed |
| Cape Verde | 2 | Scientific and political stakeholders, local (land users organized in associations) |
To conduct the scoring exercise in a practical and easy way, the use of a 'scoring ladder' was recommended, on which small cards representing the different options (each with a photograph) can be placed on the score range for one criteria at a time (see guidelines p. 39). Telling from what information the workshop reports provide, only Portugal has used this visual 'scoring ladder' tool (see figure below). Spain has worked with cards of options, but on a big scoring matrix on the wall (which is the same as a separate ladder for each criterion). Most other sites have directly worked with a matrix table, either on small sheets or on a board.
Valuing the options concerning the criteria 'maintenance costs' with the help of the scoring tool in Portugal (Photo: Gudrun Schwilch)
Group scoring outcome and consensus finding
After finishing the scoring process in groups, the result and its variation was assessed by the moderators. A way had to be found to determine a single value, either through consensus or average. The two sites of Russia and Tunisia kept the groups separate till the end of the decision process, only comparing the results after the final analysis. Table 7 shows differences found in the assessments made by different groups, and how they were dealt with.
Table 7: Differences in scoring between the stakeholder groups
| Study site | Differences in scoring |
| Spain | Averages made, graphs were provided illustrating different scoring by farmers and non-farmers. |
| Portugal | Scoring results were presented in detail, highlighting the most divergent assessments in order to discuss and find consensus among the stakeholders. It was observed that in some cases different interpretations of the criteria occurred. |
| Greece - Crete | Differences found in scoring, depending on the participants' age, field experience, and disciplinary background. Final score defined by making a compromise among all participants. |
| Turkey - Karapinar | No significant differences between the groups observed, averages made. |
| Turkey - Eskisehir | Farmer groups scored very close to each other, while the experts groups significantly diverged from each other (due to differences in their professional background) and from the farmers groups. Finally, averages were made. |
| Morocco | The phase of scoring allowed discussions about the validity of every option with regard to the various criteria. Multiple cases of differences of scores were recorded, which sometimes required a long discussion. Every score was only maintained if it was justified and accepted by all participants. The farmers got a predominant role. Differences mainly concerned the economic dimension. |
| Tunisia | Several criteria assessed similarly. No merging of scores, 3 separate matrix tables kept for further analysis. |
| Russia - Djanybek | The preferences given through voting were quite similar. Specialists explained advantages and disadvantages from their point of view, while practitioners and farmers brought in their visions and implementation realities. No merging of scores, 4 separate matrix tables kept for further analysis. |
| Russia - Novy | Same as in Djanybek |
| China | No info on differences. Averages of scores made. |
| Botswana | There were major differences, but the tables and the scoring were discussed to come up with negotiated and agreed scores. |
| Chile | Similar scoring, except one criterion, which was understood differently by the two groups. |
| Cape Verde | There were some differences between local and external stakeholders, which were discussed in plenary. To find a consensus, they took either the average score, or accepted the score of one group, or agreed on a new value. |
Result of step 4
The expected result of exercise 4 was:
- all options are assessed for the different criteria;
- major differences in the assessment made by different stakeholder groups are made transparent.
From the workshop reports it can be concluded, that result 1 was achieved at all sites, and result 2 probably at most of the sites. However, regarding the achievement of the second expected result no conclusive answer can be given, as available information is not detailed enough. Negotiating the various perceptions and understanding regarding the criteria different stakeholders may have, and the scoring of options is an important step in the process, which can only be made if differences are made transparent. Those sites which have continued working with separate matrices have probably skipped these discussions (Russia, Tunisia).
