How good are we at detecting ecological change? It depends on where you look!

Nicol S., Roach J., Griffith B. (2013). Spatial heterogeneity in statistical power to detect changes in lake area in Alaskan National Wildlife Refuges. Landscape Ecology 28:507-517.

How well can we detect ecological change? It’s a good question— in the media we often hear about how the climate is changing, but how do we measure change, and does the amount of change that you detect depend on where and how long you look for it? It turns out that measuring change is pretty hard, and that it’s possible to get wildly different answers to the question “is the environment changing?”

A lake in the Yukon Flats NWR, Alaska (Image: May-Le Ng)

With colleagues from the University of Alaska, we played with this question in a paper looking at our ability to detect changes in lake size at different measurement scales in Alaskan National Wildlife Refuges (NWRs). From analysis of satellite imagery, lakes in Alaskan NWRs appear to have been getting smaller over the past 50 years. This sets off alarm bells for managers of the USA’s migratory waterfowl populations, since the NWRs were established to protect the breeding grounds of millions of ducks and geese. There are fears that if the lakes dry up, then there will be less habitat for migrant waterfowl, leading to declines in populations and potentially to reduced waterfowl harvest in the contiguous “lower 48” states, where duck hunting is big business. However, when scientists attempted to predict waterfowl presence based on physical characteristics like lake area, they found that the individual lakes fluctuated wildly in area. These fluctuations made it hard to determine the effect of declining lake size on plants and animals at the lake scale. The problem was so bad that in many lakes, there wasn’t enough data to say whether they were increasing or decreasing in area. If we can’t tell if lakes are shrinking, then how can we predict what the effects on waterfowl will be?

The problem lies in a combination of the sample size and the trend size—in the satellite imagery we can see the behaviour of all the lakes at once, so the average behaviour is powerful enough to ‘smooth out’ the individual variation and see what’s happening to all the lakes put together. However at the individual lake scale, there’s nothing else to compare with, so we’re stuck with the apparently random fluctuations in lake size and whatever trend that individual lake appears to show.  If we only look at a single lake, then we can get completely different conclusions compared to if we look at all the lakes. Two adjacent lakes can have completely different trends, and we’ll have no idea what’s going on!

So, how many lakes do we need to monitor before we can say whether they’re changing? We used a mixed model to investigate how the statistical power to detect average linear trends in lake size of 0.5, 1.0 and 2.0 %/year was affected by the size of the analysis area and the number of years of monitoring in National Wildlife Refuges in Alaska. We estimated power for large (930–4,560 sq km) study areas within refuges and for 1, 5, and 10 sq mile cells nested within study areas over temporal extents of 4–50 years. When we had all the lake data for a refuge, we found that all trends could be detected after 5-15 years. However when we restricted our analysis to have only the data for one of our cells (1,5 and 10 sq mile), trends smaller than 2%/year would take more than 50 years to detect. To put that into context, a lake declining at 2%/year for 50 years would decrease to 36% of its original size. That means you couldn’t even tell it was shrinking (with power >0.8), until it was 36% of its starting size!

There are two morals to the story: (1) even with a lot of data, it can take more than a decade to detect change; and (2) it’s important to consider whether you have enough power to detect the changes that you’re looking for before you start your experiment. In this case, we know that lake area is declining overall, but we can’t use this to predict species presence or absence at the individual lake scale, because we have no confidence that observed trends in our lake-scale data are real.

Want to read more? Check out our paper:

Nicol S., Roach J., Griffith B. (2013). Spatial heterogeneity in statistical power to detect changes in lake area in Alaskan National Wildlife Refuges. Landscape Ecology 28:507-517.

Early morning on the Yukon Flats National Wildlife Refuge, Alaska (Image: May-Le Ng)