1998 Cohort Analysis: An impact evaluation of WOTRO-funded research
13 september 2010
WOTRO carried out an impact evaluation of the WOTRO-funded research from 1998. The purpose of this cohort analysis has been to explore the scientific and societal impacts of WOTRO-funded research. The analysis identifies a number of factors that contribute to enhanced social and scientific impacts, which WOTRO has adopted as part of its strategy in recent years. For example, engagement of relevant stakeholders outside the research community in all phases of a programme is now a requirement.
The report presents the findings of a pilot ex-post evaluation, initiated by WOTRO in 2008, of the performance of one cohort of its projects: all WOTRO projects and integrated programmes that received grants in 1998. As most of the 1998 cohort projects presented their final reports between 2001 and 2003, the evaluation covers a time frame of five to seven years after the completion of the projects, a period deemed sufficiently long to assess their impact.
This study adopts a forward-tracking approach, focusing on the research projects or programmes to explore and assess the impacts of completed research over time. The analysis focuses on expected impacts in four broad categories: I) scientific outputs; II) impacts on future research and scientific capacity strengthening; III) societal diffusion, dissemination and collaboration; and VI) impacts on policy and practice. Out of a total of 54 completed research projects funded by WOTRO in 1998, information on 40 projects has been used in this analysis.
I) Scientific outputs. To the end of 2008, the projects produced an average of 5.1 publications, including peer-reviewed articles, book chapters or monographs. The largest number of publications was registered in 2003, but a considerable number of publications appeared after the completion of the project, even up to 2007. The average number of refereed articles was considerably higher for projects in the Earth and life sciences and the medical sciences than those in the humanities and social sciences. This can be explained by the difference in publication cultures (e.g. monographs versus articles), as well as by the fact that researchers in the humanities and social sciences often give priority to finishing their dissertations, while in other disciplines dissertations are often based on already published articles. The 1998 cohort has produced some highly cited articles, including some published in high-impact journals such as Science. Several articles appear to have reached their citation peak in 2008.
II) Future research and capacity strengthening. The majority of projects led to new research within subsequent projects or by other research groups. Practically all respondents indicated that the research had a positive impact on their career development. Most researchers have continued working in the (academic) research sector, in positions ranging from postdoctoral research fellow to full professor. Former researchers from developing countries (DCs) are somewhat more likely to hold positions outside academia, in societal and professional organizations or (international) task forces. The international mobility of both Dutch and DC researchers is considerable, the majority having taken up positions outside their countries of origin, mostly in the ‘North’. Nevertheless, the majority of DC researchers eventually returned to their countries of origin. Collaboration with scientific partners often continued after completion of the project.
III+IV) Societal diffusion and impact. For most of the projects, the research results were eventually used by non-scientific actors. Dissemination activities and collaboration with societal actors appear to have proven effective in facilitating the uptake of research findings. Notwithstanding the small-scale character of most research projects, impact has been achieved, in particular at the local level, but sometimes also at regional or national levels. Projects involving DC researchers have been more successful in establishing collaborations with non-academic actors, and in influencing development policy and practice. There are no indications that a development focus has had negative impacts on the scientific quality of research, or vice versa.
The 1998 cohort analysis has identified a number of factors that contribute to enhanced social and scientific impacts, which WOTRO has adopted as part of its strategy in recent years. For example, engagement of relevant stakeholders outside the research community in all phases of a programme is now a requirement. Also, grant applicants are now required to present a communication plan directed at enhancing the uptake, translation and application of relevant issues and research results among stakeholders. Furthermore, it is mandatory that applications include a co-applicant from a partner organization in a developing country. The present analysis, therefore, confirms the relevance and effectiveness of these measures.
Read the whole report:
1998 Cohort Analysis: An impact evaluation of WOTRO-funded research
