Academia’s wicked problem

What do you do when a crucial knowledge system is under-resourced, highly valued, and is having its integrity undermined? That’s the question facing those working in academic research and publishing. There’s a risk Australians might lose trust in one of the central systems on which we rely for knowledge and innovation.

It’s one of those problems that defies easy solutions, like obesity, terrorism or a tax system to suit an entire population. Academics call these “wicked problems” – meaning they’re resistant to easy solutions, not that they are ‘evil’.

Charles West Churchman, who first coined the term, described them as:

That class of social system problems which are ill-formulated, where the information is confusing, where there are many clients and decision makers with conflicting values and where the ramifications in the whole system are thoroughly confusing.

The wicked problem I face day-to-day is that of research and publication ethics. Though most academics do their best within a highly pressured system I see many issues, which span a continuum, starting with cutting corners in the preparation of manuscripts and ending with outright fraud.

It’s helpful to know whether the problem we are facing is a wicked one or not. It can help us to rethink the problem, understand why conventional problem-solving approaches have failed and encourage novel approaches, even if solutions are not readily available.

Though publication ethics, which considers academic work submitted for publication, has traditionally been considered a problem solely for academic journal editors and publishers, it is necessarily entwined with research ethics – issues related to the actual performance of the academic work. For example, unethical human experimentation may only come to light at the time of publication though it clearly originates much earlier.

Given the pressure editors are under, the system is vulnerable to subversion.

Consider the ethical issues surrounding peer review, the process by which academic experts (peers) assess the work of others.

Though imperfect, formalisation of peer review has become an important mark of quality for a journal. Peer review by experts, mediated by journal editors, usually determines whether a paper is published. Though seemingly simple, there are many points where the system can be gamed or even completely subverted – a major one being in the selection of reviewers.

As the number of academics and submissions to journals increase, editors face a logistical challenge in keeping track of an ever-increasing number of submissions needing review. However, as the rise in submissions has not corresponded with a rise in editors – many of whom are volunteers – these journals are overworked and often don’t have a big enough circle of reviewers to call on for the papers being submitted.

A simple approach to increase the pool of reviewers adopted by a number of journals is to allow authors to suggest reviewers for their paper via the online peer review system. These suggestions can be valuable if overseen by editors who can assess reviewers’ credentials. But they are already overworked and often handling work at the edge of their area of expertise, meaning time is at a premium.

Given the pressure editors are under, the system is vulnerable to subversion. It’s always been a temptation for authors to submit the name of reviewers who they believed would view their work favourably. Recently, a small number took it a step further, suggesting fake reviewer names for their papers.

These fake reviewers (usually organised via a third party) promptly submitted favourable reviews which led to papers inappropriately being accepted for publication. The consequences were severe – papers had to be retracted with consequent potential reputational damage to the journal, editors, authors and their institutions. Note how a ‘simple’ view of a ‘wicked’ problem – under resourced editors can be helped by authors suggesting their reviewers – led to new and worse problems than before.

Removing the ability of authors to suggest reviewers … would be treating a symptom rather than the cause.

But why would some authors go to such extreme ends as submitting fake reviews? The answer takes us into a related problem – the way authors are rewarded for publications.

Manipulating peer review gives authors a higher chance of publication – and academic publications are crucial for being promoted at universities. Promotion often provides higher salary, prestige, perhaps less teaching allocation and other fringe benefits. So for those at the extreme, who lack the necessary skills to publish (or even firm command of academic English), it’s logical to turn to those who understand how to manipulate the system.

We could easily fix the problem of fake reviews by removing an author’s ability to suggest reviewers but this would be treating a symptom rather than the cause. Namely, a perverse reward system for authors.

Removing author suggestions does nothing to help overworked editors deal more easily with the huge amount of submissions they receive. Nor do editors have the power to address the underlying problem – an inappropriate system of academic incentives.

There are no easy solutions but accepting the complexity may at least help to understand what it is that needs to be solved. Could we change the incentive structure to reward authors for more than merely being published in a journal?

There are moves to understand these intertwined problems but all solutions will fail unless we come back to the first requirement for approaching a wicked problem – agreement it’s a problem shared by many. So while the issues are most notable in academic journals, we won’t find all the solutions there.