This week has seen the publication in Nature https://www.nature.com/articles/d41586-018-05414-4 of Four Principles for Synthesizing Evidence – what I see as a key perspective piece. I and a number of others want to put evidence synthesis centre stage in science alongside the key primary research breakthroughs which push the boundaries of our understanding.
I recently read the book “Theory and Reality” by Peter Godfrey-Smith about the philosophy of science and how it changed through the 20th Century. One of the most powerful conclusions from this exploration is just how much our view of reality is moulded by the methods we use to investigate and understand the world around us. This has shifted us in small steps from uncomfortable fundamentalist positions which questioned the nature of reality itself to much more pragmatic views of the world.
Although the former fundamentalist positions still have not gone away, science has progressively developed increasingly convincing methods for describing what is real. Part of this involves the recognition of science as a creative and social process. We still reward individual scientists for their ‘discoveries’ with prizes but in reality progress in science is a mighty aggregation of the efforts of large numbers of people. I believe that some the most significant progress in science in the early parts of the 21st Century come not from individual breakthrough publications but come instead from the synthesis of evidence across many different lines of enquiry.
The advent of online publication and the presence of powerful web-based portals like the Web of Science and search engines are not only the result of this aggregated process of advancing science, but are also the things that will enable more knowledge to be aggregated in future. This is a form of systemic evolution which, if carried out well, could push the benefits of science in society to much greater heights.
The reward structure for scientists is also recognised by philosophers of science as an important component driving this machine for invention and innovation. The scientific establishment has been slow to reward scientists for looking across their disciplines and coming up with new ideas or insights about the world based on gluing together information already in the public domain. Individual scientists are like component manufacturers who have been told to make pieces of a structure without anybody being tasked with joining all of the pieces together.
Indeed, synthesis has been frowned on as secondary or derived information. Synthesis has been disparaged by being confused with ‘reviewing’ or the restatement of old ideas in a new context. We need to transition from talking about scientific review to talking much more clearly about the prospective and inventive process of scientific synthesis. It has been devalued within the reward structure but its potential is vast. In the article in Nature, we want to purge the old idea that merging and analysing the outcome from multiple strands of scientific output somehow lacks importance, and we want to put synthesis in its rightful place as an exciting, intellectually challenging, high-status and respected activity that provides a global public good.
The article has been built on workshops led by the Royal Society and the Academy of Medical Sciences. It recognises that synthesis in science is as valid a pursuit of original knowledge and ideas as any other. The complexity of scientific information these days is such that it takes special skills and procedures to pull out the essential, reliable knowledge from the midst of a huge mass of disparate information. Evaluating the quality of the underlying evidence base and building up new and otherwise unseen pictures of the world are just two of the reasons why scientific synthesis is especially important for areas like policy-making.
The article defines the characteristics of good evidence synthesis to inform policy: it needs to be inclusive, rigorous, transparent and accessible. It also recognises that synthesis can take many different forms and the utility of these forms depends of the audience. Synthesis to support the progress of science itself might be very different from synthesis to support a decision being made by regulators, for example, about whether a particular drug is effective and safe, or by governments about whether a new policy is likely to be effective. Despite this range, the principles should apply to synthesis for all policy purposes and timescales.
We recognise that producing syntheses can be a substantial task, often involving multiple collaborators working together over months or even years. However within the policy environment, sometimes those who need synthesis can require this information in periods of days and perhaps even hours or minutes when involved in fast-moving emergencies. Evidence synthesis for policy represents everything from flying by the seat of the pants when one is being driven by events out of one’s control through to measured and deeply intellectualised representations of the current state of knowledge. The processes involved in manufacturing the syntheses in each case need to be tailored to the circumstances – but they all have the same key features. In essence, they make sense of the vast amount of published data and information and turn it into accessible, usable, knowledge. This can change our view of the world around us in ways which were not predicted and we emerge feeling better off, more educated.
In Defra, we have been grappling with the idea of synthesis for some time. I have wanted all our main policy areas to be informed by “evidence statements”. These are short, authoritative, readable syntheses of the scientific knowledge – including its strengths and weaknesses – in a particular field. For example, we produced one recently on the effects of air quality on semi-natural terrestrial ecosystems. After a lot of careful consideration, we are starting to build what I hope will be a large portfolio of evidence statements produced by applying the principles described in our article. If implemented across the whole of Defra this would amount to a potentially very large number and, perhaps eventually, we will reach a point where we can produce syntheses of the syntheses.
At present, these mini syntheses are scaled so that they can be produced by a PhD student on secondment to Defra for a period of three months. Not only does this time-limit instil discipline in terms of scope but it also means we are exposing our researchers of the future to very practical experience at the coal face where science meets policy. These individuals also walk away with a very tangible output to their names (hence they receive credit which is so important in the social process of science).
Making the scientific effort of the past count more in the present and future has to be a good thing. The art of scientific synthesis will, I predict, be a recognised and increasingly valued part of the scientific effort in future.