Non-academics with extensive experience of particular sectors and industries can provide unique insights into the potential pathways to impact for new research projects. Drawing on a quasi-natural experiment comparing assessment panels with and without non-academic experts, Junwen Luo discusses the how these skills were perceived by academics and how the inclusion of non-academics might benefit from clearer definitions of impact in applied research grants.
Research funding agencies are increasingly using mixed review panels to facilitate broader discussions on the prospective impact of research proposals. Such mixed panels include traditional peer reviewers from the scholarly community, but also non-academic reviewers. Non-academic reviewers are stakeholders with relevant expertise, experiences, and perspectives in the proposed research areas and come from industry, government, public agencies etc. Research in organisations, juries, and other settings has repeatedly shown that a diversity of backgrounds (discipline, gender, race, and other characteristics) enhances collective decision-making. In short, diverse groups bring different experiences to bear on a problem, with positive effects for legitimacy and creativity.
Perhaps the most studied example of mixed panels is the UK’s REF 2014 that involved over 250 non-academics (accounting for 23% of panel members) in impact panels to discuss retrospectively the use and influence of scientific findings in the real world. In this instance, researchers have found that REF’s academic and non-academic reviewers’ interpretations of the impact criterion varied widely, especially their perceptions of what constituted excellence in impact and in good evidencing of impact.
There are also examples of non-academics’ involvement in ex ante grant evaluations (those that occur prior to research grants being made), many in health research fields and clinical practices. Examples include the United States’ Congressionally Directed Medical Research Program, the Dutch Heart Foundation, and a provincial health care provider in Canada. The Canadian agency, for instance, incorporated ‘user evaluators’ who had experience representing the community, who had been diagnosed with the relevant condition themselves, and/or had been a caregiver of someone diagnosed with the condition. Also, some programmes targeted at industrial innovation have involved industrial experts in the review panels, such as the ‘Industrial Leadership’ pillar of Horizon 2020 Framework Programme and ‘Innovation Projects for the Industrial Sector’ at the Research Council of Norway. Several scholars have suggested that education and public outreach professionals and other user representatives be included in the U.S. National Science Foundation panels.