What role should non-academics have in evaluating the potential impact of new research projects? – London School of Economics Impact Blog (Junwen Luo | August 2021)

Closeup of the dictionary definition of research

Non-academics with extensive experience of particular sectors and industries can provide unique insights into the potential pathways to impact for new research projects. Drawing on a quasi-natural experiment comparing assessment panels with and without non-academic experts, Junwen Luo discusses the how these skills were perceived by academics and how the inclusion of non-academics might benefit from clearer definitions of impact in applied research grants.


Research funding agencies are increasingly using mixed review panels to facilitate broader discussions on the prospective impact of research proposals. Such mixed panels include traditional peer reviewers from the scholarly community, but also non-academic reviewers. Non-academic reviewers are stakeholders with relevant expertise, experiences, and perspectives in the proposed research areas and come from industry, government, public agencies etc. Research in organisations, juries, and other settings has repeatedly shown that a diversity of backgrounds (discipline, gender, race, and other characteristics) enhances collective decision-making. In short, diverse groups bring different experiences to bear on a problem, with positive effects for legitimacy and creativity.

We are seeing a significant progression in the approach to consumers in research.  Being idiosyncratic for a moment, Gary will use Australians who live with a disability as an example.  In the Ninety Nighties people who lived with a disability who participated in research shifted from being described and treated as subjects, to be participants.  Despite some recent missteps (see Gary’s Research Ethics Monthly pieces listed on www.ahrecs.com/blog) the role of consumers and community members in the design and conduct of research (e.g. as co-researchers and on reference groups) has been encouraged and has increased.  This LSE piece discusses the next move in that progression, involving consumer and community members in peer review processes (e.g. to judge the importance and impact of research).  This progression is fantastic! and to be congratulated.

Perhaps the most studied example of mixed panels is the UK’s REF 2014 that involved over 250 non-academics (accounting for 23% of panel members) in impact panels to discuss retrospectively the use and influence of scientific findings in the real world. In this instance, researchers have found that REF’s academic and non-academic reviewers’ interpretations of the impact criterion varied widely, especially their perceptions of what constituted excellence in impact and in good evidencing of impact.

There are also examples of non-academics’ involvement in ex ante grant evaluations (those that occur prior to research grants being made), many in health research fields and clinical practices. Examples include the United States’ Congressionally Directed Medical Research Program, the Dutch Heart Foundation, and a provincial health care provider in Canada. The Canadian agency, for instance, incorporated ‘user evaluators’ who had experience representing the community, who had been diagnosed with the relevant condition themselves, and/or had been a caregiver of someone diagnosed with the condition. Also, some programmes targeted at industrial innovation have involved industrial experts in the review panels, such as the ‘Industrial Leadership’ pillar of Horizon 2020 Framework Programme and ‘Innovation Projects for the Industrial Sector’ at the Research Council of Norway. Several scholars have suggested that education and public outreach professionals and other user representatives be included in the U.S. National Science Foundation panels.

We are seeing a significant progression in the approach to consumers in research.  Being idiosyncratic for a moment, Gary will use Australians who live with a disability as an example.  In the Ninety Nighties people who lived with a disability who participated in research shifted from being described and treated as subjects, to be participants.  Despite some recent missteps (see Gary's Research Ethics Monthly pieces listed on www.ahrecs.com/blog) the role of consumers and community members in the design and conduct of research (e.g. as co-researchers and on reference groups) has been encouraged and has increased.  This LSE piece discusses the next move in that progression, involving consumer and community members in peer review processes (e.g. to judge the importance and impact of research).  This progression is fantastic! and to be congratulated.

Related Reading

Previous

NEXT

Leave a Reply

Other News

Profiled Story

Earlier Stories

Related News Tags