Skip to 0 minutes and 10 seconds During the last five weeks, you explored different cases of transdisciplinary research. Such projects offer manifold experiences, results, outputs, and potential impact. How do you evaluate this abundance? How do you define success in this context? And who would be able to decide to what extent a project was successful? The usual evaluation metrics only cover limited aspects of transdisciplinary research projects, and are often limited to scientific output. To broach a rather broad subject, I want to focus on three questions that are specific when it comes to reflecting and evaluating them. What is special about the evaluation of transdisciplinary projects? Who should evaluate transdisciplinary projects? And which approaches are promising when you evaluate a transdisciplinary project?
Skip to 1 minute and 22 seconds It is very likely that transdisciplinary projects do not only have a wide range of possible outputs, but also diverse possibilities for impact. Outputs may, for instance, include publications for different audiences, the organisation of joint events, and shared visions and strategies. Outcomes might be similarly varied. Think of projects that enhance the capacity of the actors to conduct participatory processes or that leverage the importance of certain topics. Impacts could cover new policies, improved health services or water governance. With this rich possible variety, it is important to get the full picture when you evaluate transdisciplinary projects. You need to be aware of the results and impacts that were initially intended, but you need also to be aware of those that are emerging.
Skip to 2 minutes and 34 seconds Typically, many actors are involved in a transdisciplinary project. This is probably also true for its funding.
Skip to 2 minutes and 43 seconds Thus, the question arises: who should evaluate?
Skip to 2 minutes and 48 seconds The answer is: ideally, all perspectives should be included. For the project team, partners, different actors, and funders all play an important role. They may have diverging views of the transdisciplinary project and they may apply different criteria to measure its success. For reflection and evaluation, it is very important to include them all.
Skip to 3 minutes and 22 seconds The standard quantitative signs evaluation metrics are not sufficient to evaluate transdisciplinary projects. But what additional approaches are promising? Formative evaluation is one example that works. It proposes to reflect the project at several moments as it runs and as it ends. The project team and the partners then think about specific questions. Where does the project stand in relation to the goals that were initially decided? What has worked so far? And what has not worked? The answers then help to adjust how to proceed and how to reach the initial goals. At these points, however, the actors may also decide that the process or changing conditions require the goals to be adjusted. They may even introduce new targets.
Skip to 4 minutes and 24 seconds Ideally, evaluating a transdisciplinary project includes several layers. In the team, the project is reflected continuously. And additionally, formative evaluation involves project partners. These ongoing assessments are complemented by external perspectives, like those of funders or stakeholders who have not been part of the process. If you want to evaluate the contribution of a project to a societal challenge, you need to plan enough time to see its impact unfold. Even then, however, it is very difficult to attribute impacts to a specific project. This is especially true for societal systems that are complex.
Skip to 5 minutes and 21 seconds In the water scarcity case, for instance, the involved communes merged into one some years after the project was finished. This merger was suggested in the project. However, there were many other factors that supported the final decision.
Skip to 5 minutes and 43 seconds This all shows that you should not only start reflecting your project as it evolves, you also should assess impact even several years after the project has ended.
Reflecting transdisciplinary research projects
Evaluating transdisciplinary research (TDR) projects is not an easy task. They include several perspectives and a range of possible outcomes, outputs, and impacts. To tackle this challenge, this video proposes to answer three key questions. Welcome to the first step of the final week of this course!
Usually, if you evaluate a research project, you assess key metrics like publications in peer reviewed scientific journals or conference contributions. In the case of a transdisciplinary project, these metrics are not enough to cover and reflect the whole approach. The evaluation, then, becomes rather complex.
In this video, Tobias Buser explores three questions to familiarise you with the broad field of evaluating TDR projects:
- What is special about the evaluation of transdisciplinary projects?
- Who should evaluate transdisciplinary projects?
- Which approaches are promising when you evaluate a transdisciplinary project?
Think of the cases you have seen during the last five weeks. If you were asked to evaluate them and to reflect what they have achieved – how would you do that? How would you organise an evaluation that does them justice?
Educator: Tobias Buser
© University of Basel, Swiss Academies of Arts and Sciences, Swiss TPH, University of Bern, UsysTDLab, Université de Genève, Lucerne University of Applied Sciences and Arts