Step 6: Analysis and interpretation
Use of the Facilitator software
To run the analysis according to the suggestions in the workshop guidelines, the Facilitator software is needed. In reality, not all study sites used the software. In Morocco, it was only used by the researchers and some technicians during a preparatory workshop, but not during the stakeholder workshop due to lack of time (1 day only). Also in Chile it was not used due to the one-day workshop. In Greece - Nestos the software was not used as they applied a completely different approach (training seminar). All other study sites did use the Facilitator software.
The Facilitator software produces graphs that visually display the relative merits of each option. Each option is represented by a bar, showing the range of overall scores for that option, specific for the socio-cultural, economic and ecological categories. The width of the bar and its location relative to the other options need to be interpreted and discussed with the stakeholders to understand the overall performance of each option.
Analysis graphs as presented and adapted from the DESIRE Facilitator software in Spain (Joris de Vente).
In Spain, the moderators adapted the resulting graphs with qualitative range names (from very low to acceptable to very high), which is better understood by the stakeholders than figures. If the options would be ranked alphabetically rather than according to their maximum, it would be even easier to compare the three categories socio-cultural, economic and ecological. However, this sorting was only applied by Botswana and Portugal.
Some study sites struggled with the software, but finally, all applied it successfully. Below some interesting statements from the workshop reports:
"It took several attempts to finally realise success, and display results for participants to review. It needs to be stated however, that once we got the tool right, it came across as powerful and fascinating for the participants, as for the most part it confirmed their scoring." (Botswana)
"The DESIRE Facilitator software was used to objectively present the combined result of all opinions and for all criteria. It should be emphasized, that the moderators always tried to be totally independent and did not participate at any time in the scoring process. So, all graphs obtained here reproduce exclusively the decisions, opinions and desires expressed by the participants in a consensual way. After the presentation of these results, a passionate discussion took place regarding the hierarchy of water and soil measures as presented. The moderators explained that, the opinion of each individual does not necessarily correspond to the overall results which consider the opinion of all participants and all criteria used in the evaluation process. At the end of the clarifications and debate, the majority of the participants understood and agreed with the results." (Cape Verde)
"After the presentation of these results an intense debate was started on the presented hierarchy of conservation measures. At first sight, several of the participants did not believe the results as presented in the graphs. Some participants even suspected that the data were manipulated by the workshop organisers. Therefore, the methodology behind the whole selection process and especially the Facilitator software was explained again step by step. At the end of the explanation and discussion, the vast majority of the participants understood and agreed with the results as presented." (Spain)
"The DESIRE Facilitator software package was actively used for demonstration of voting results by screening to the blackboard by multi-media projector. This procedure was useful for stakeholders by letting them be informed during the meeting time. A discussion following this analysis made the interpretation of the result becoming clear." (Russia - Novy)
In both Russian sites, the stakeholder group results were analysed separately for the 4 groups.
The Tunisian team found it difficult to explain the results from the analysis.
Interpretation
The graphs produced by the Facilitator software need to be interpreted carefully. From the workshop reports, only little information is available on how they were interpreted. Mostly just the resulting graphs were displayed, without further comments.
In Crete (Greece), the analysis and interpretation was first only done for each criterion separately, which is actually just repeating the scoring discussion. After comments on their report, the analysis per category was included, but it was probably not done during the workshop. Interpretation of the graphs remained incomplete, focusing mainly on the ecological and economic criteria. It is not clear why at the end 'no tillage' was interpreted as the preferred option, as it only ranked high in the socio-cultural category (having only 1 criterion compared to many more criteria in the two other categories!).
In Morocco, where the Facilitator software was not used, the interpretation of the scoring phase was still possible due to the extended preparation phase with detailed analysis of each option. It was stated, that this permitted all participants to estimate the advantages and the negative points of each SLM technology. It might be concluded that this incomplete process caused the rather unclear final decision and selection.
In Portugal, interpretation was challenged due to the fact that in each category, a different technology scored best. In the overall analysis this was levelled out, resulting in all options scored more or less equally well. The workshop report proves that the interpretation was done well and correctly.
In Tunisia, the team made a good analysis and interpretation of the resulting graphs (for each transect group). They were also challenged by no option showing high ranks in all three categories, but it was still possible to see which the best option is. The second best option was re-prioritized without giving clear reasons for this. It is unclear, whether e.g. the criteria were wrong and not supported anymore by the stakeholders? 'These results were calibrated and then validated in large part by the farmers. Taking into account the different concerns for the three economic, environmental and social dimensions by farmers, a re-prioritisation was made with farmers.' (Tunisia report).
A profound analysis and interpretation is also presented by the Turkey - Karapinar site.
Result of step 6
The expected results of step 6 are
- that relative merits of different options become clear, and participants are aware of the pros and cons depending on the view of different stakeholders.
- Participants understand which options are most promising in the local context.
We can hardly judge whether these results were really achieved by all sites. It is a core step within the whole methodology, but information is insufficient to analyse the outcome for each site. However, it can be assumed that, where enough time was available, a discussion took place before a final choice was made (see next step). In Spain a long discussion continued following the presentation of the Facilitator output graphs. These were used as a starting point for further informed negotiation.
