Many public entities are interested in empirical estimation of the overall impact, in terms of benefits to citizens, of publicly funded research. These include:
The ultimate objective of this analysis is to develop reliable measures of such impacts in order to inform decisions about the level of public resources to devote to research, and to evaluate the relative effectiveness of different modes and mechanisms of research support.
Because different types of impacts are fundamentally non-commensurable, it is not possible to derive a single composite metric of all research impacts that would be useful for decision purposes. Further, for some important impacts there may be no meaningful quantitative measure, or there may only exist metrics that are illustrative or indicative of the impact in some approximate way. Any evaluation that excluded those impacts that cannot be directly quantified would be biased.
For these reasons, the approach of this paper is to identify a set of metrics and indicators that broadly covers the major categories of impacts. In each case, metrics are proposed that come as close as possible to direct quantification of the impact of interest, but where indirect or approximate indicators are the only ones available for potentially important impacts, they are included.
The report addresses a number of cross-cutting issues that arise in any attempt to evaluate the long-run impacts of research. These include:
Although these problems cannot be truly solved, the report suggests approaches for dealing with them.
Ministry of Business, Innovation and Employment, GNS Science, BRANZ, Institute of Environmental Science and Research (ESR)