I once re-homed a dog which had a pathological hatred of brushes. She had clearly been mentally scarred early in her life by being beaten with a brush. We are all scarred to some extent by our past experiences. One of my mental scars concerns the interpretations placed on historical data suggesting trends through time in natural processes, also known as “time series”. This may seem a somewhat odd aversion to have but let me explain why it has been important to me as a scientist by using a recent example concerning the declines of insects in Germany.
When I began my PhD in the early 1980s the research field I was working in was dominated by a simple idea. This was that as populations of long-lived mammals declined they compensated by beginning to reproduce earlier in life and by producing more offspring. Much of the evidence to support this came from the observation of trends in these reproductive features through time.
But there was a problem. Almost all the trends were going down. I cannot ever recall seeing an upward trend in these data. Everybody thought this was indicative of big problems. For me, as a young researcher, I was worried by this and found myself out of step with the received wisdom. I eschewed including these time series in my studies although at the time I wasn’t sure why.
The observation of these trends was important because it was a dominant force in the arguments being used for the population management of some of our most iconic species such as whales, seals, wolves, bears and elephants. Fortunately, I wasn’t the only person who was worried by this and eventually much of the empirical picture was exposed as an artefact.
Ever since then I’ve been very sceptical whenever anybody presents a time series of data. This includes everything from tree ring data purporting to show trends in climate to trends in the abundance of birds, bees or butterflies and moths across the British countryside. Sometimes I feel I could write a whole book about just how misleading the data about trends can be. And yet, the simple messages they carry mean we lap them up. A wiggly ascending or descending line on a graph carries a lot of beguiling messages. But are they true?
So when I read the recent paper published in PLoS One (http://journals.plos.org/plosone/article?id=10.1371/journal.pone.0185809) showing declines in insect populations in Germany I started from a sceptical viewpoint. I started from a position of knowing what I needed to see in the paper which would convince me that the trend was real. The Guardian had already reported the simple, beguiling message as ‘Warning of “ecological Armageddon” after dramatic plunge in insect numbers’. Did I see all that I needed in that paper to convince me that the simple message was correct? Not quite.
The authors had a sample of 96 data points from 63 sites across Germany. They had trapped insects using a standard method between 1990 and 2016 and then had plotted the total biomes of insects trapped through time. As I read the paper I began to like it. My prejudices were being challenged and weakened.
A strength was that not many sites were sampled on multiple occasions. Some were sampled two or three times. This helps to get rid of a nasty feature of time series data known as autocorrelation. So far, so good.
The authors had also gone to great lengths to describe the data using a robust statistical method. This had helped them to look for relationships with weather and changes in land use, all important for building a picture which might convince a sceptic like me.
The results which emerged were quite startling. In 27 years, on average, there had been a 75% decline in insect biomass. Even if I thought the heavy-weight statistics might have built some form of artefact in to the result, the magnitude of the change was so large that it would be difficult to see this as a statistical artefact. I was convinced that the authors claim of declining insect biomass in the sites they had observed was real.
But when I turned to the Guardian article and saw how this result was being interpreted I started to get worried. Technically, the paper showed declines across 64 sites which had been chosen specifically for their conservation value. These were probably mostly relatively pristine habitats. But people were now saying this was telling us about how insects were declining across the whole countryside. Let me explain why this is misleading.
There are basically two reason. The first concerns what might be called a founder effect and the second concerns how representative the habitats sampled in the paper were of German countryside as a whole.
In Europe the Habitats Directive has encouraged us to scout the country for sites where wildlife seems to be in a relatively natural and abundant state and then to put a protective ring around them. This is a good thing to do but we need to be aware that these sites are not going to stay the same through time. In an environment where there are lots of dynamic processes going on like weather, land use, natural succession and many more, including natural dynamics, change will be the norm. The authors of the paper did a valiant job of trying to recognise this but were very constrained in what they could do to compensate by the lack of control sites chosen at random. This effect, is brought about by the state of these sites when they were founded.
In these circumstances there are only really two likely directions for future change in a set of protected sites which already have high wildlife abundance: they can remain much as they were when they became protected or the abundance of wildlife declines. It is much less likely that abundance will be seen to increase in such a set of sites. It’s hard to make pristine sites more pristine although I acknowledge that management and restoration is an important part of the current philosophy of conservation and could be expected to lead to some increases in measurements like insect biomass.
The second reason concerns representativeness. The sites in Germany were almost certainly a highly biased representation of the German countryside as a whole. A fairer sample would have compensated for these high quality sites by also choosing sites on land which had been cleared of its wildlife. Tracking them in parallel through time using the same methods would almost certainly have produced a very different result.
Can the results in this paper then be used to extrapolate across the whole countryside? I don’t think so. The sites reported in the study are likely to be extremely unrepresentative of German countryside.
Indeed, taking both these problems together, if one was to have thought deeply about this study in advance I suspect that the eventual result would not have been a surprise at all (at least qualitatively if not quantitatively). This is purely because of how the sites were selected.
In my mind, therefore, the idea that there are large changes in insect populations across Germany remains unproven. Of course, it could be correctly reflecting wider trends but this study does not provide that result. Other studies have shown declines but remember what I have said about the multitude of problems with time-series. Were all those other studies fair tests? Almost certainly not and few that I have seen are a fair test. An accumulation of many unfair tests does not amount to a fair test. Indeed, it probably amounts to the creation of an illusion.
We are all victims of our own prejudices. The scars I carry remind me constantly of the dangers of prejudiced interpretations of data. Like everybody else I want to really know what is going on across the countryside but unlike those who uncritically lap up information like that in the German study I also worry terribly about just how blind we are. I cannot bring myself to believe a lot of the data used to track change. The best data we have comes from the BTO, and that shows a mixed picture, but we need to become a lot better at producing synoptic measures of changes which truly capture the total picture. Our work in Defra (including collaborations across the Environment Agency, Natural England, CEFAS and JNCC) on earth observation, when mixed in with the kind of data produced by the BTO, promises much when it comes to putting us on the right track. I want to see us genuinely moving to a new way of systematically measuring and monitoring the environment in ways which can meaningfully track progress.
The lesson from the paper about insect declines in Germany is not about insects at all. Instead, it is about ourselves and whether we want to perceive the real world defined by systematically-gathered, reliable data or whether we prefer to believe our own prejudices and design the data to fit them.