How to increase the trustworthiness of Citizen Science data

Excerpted from John Gollan:

“Citizen science occurs when data for scientific research is collected by members of the public in a voluntary capacity. Public participation in environmental projects, in particular, has been described as a global phenomenon.

But there is a stigma associated with these types of projects. The data collected are often labelled untrustworthy and biased. Research in this area continues to show however, that data collected by what is essentially a non-professional workforce, are comparable to those collected by professional scientists.

Provided steps are in place to deal with data integrity, we have much to gain by putting more trust in citizen scientists.

Across the globe thousands of people collect data on everything from counts of stars in distant galaxies to the timing of flowering events. Volunteers have long been collecting data on the health of coral reefs, and ornithologists encourage volunteers to collect data on bird migration.

Citizen science has benefits for scientists – including an inexpensive and potentially large labour force – and citizens, who get knowledge and fulfilment. These schemes expose people to the environment and develop the stewardship ethic.

But what motivates my interest in this area is the potential to create a more scientifically literate society; building the capacity for people to take information they receive in their everyday lives and then being able to make informed choices based on the what they have learned. Those choices could be anything from the products they buy as consumers or the political parties they support.

While citizen science projects vary in their study subjects, the ecosystem of interest and objectives of the research, they all face one important challenge that jeopardises their success, sustainability, integrity and in general, acceptance in the wider scientific community. Many scientists question the quality, reliability and in general, the utility of data.

Evidence for the stigma associated with citizen science projects comes from different sources. One was a long standing community-based program to survey diversity of bird species. Researchers found the estimated numbers of birds changed through time simultaneously with changes in the observers. It was concluded that the trends detected were not likely to represent real changes in bird abundance, but were more likely due to prejudices of the individual observers.

The negative perception of citizen projects is not new. Twenty years ago, the use of volunteer data came into the international spotlight when an amendment was made to prohibit the US National Biological Survey from accepting the work of volunteers. This was supported by two arguments in the House of Representatives declaring that volunteers are incompetent and biased.

Are volunteers’ data that bad?

Questions over data integrity continue to this day. It’s surprising, because a growing body of literature shows that data collected by citizens are comparable to those of professional scientists.

For example, researchers have detected no differences between field samples of aquatic invertebrates that were collected and identified by volunteers and professionals. A similar study showed that data collected by volunteers and scientists agreed 96% of the time.

Both studies concluded that volunteers could collect reliable data and make assessments that were comparable to those made by highly trained professionals.

My own research on vegetation metrics collected as part of ecosystem restoration projects also showed that the degree of agreement of data collected by volunteers can be as good as those recorded by professional scientists.

Results showed that scientists as a collective group collected data that was in closer agreement with “the truth” than those of volunteers. But when data collected by individuals were analysed, some volunteers collected data that were in similar or closer agreement to the truth, than scientists. Both groups’ estimates were in closer agreement for particular attributes than others, also suggesting that some attributes are more difficult to estimate or are more subjective than others.

An important message from these studies is that data-integrity issues can occur. But it’s just a matter of honing in on those particular issues and addressing them if necessary. This can be through training to improve skill sets or calibrating data where possible.

It should not be a case of blaming the citizens. The scientist behind such programs should have checks in place – citizen science project or otherwise!”

Leave A Comment

Your email address will not be published. Required fields are marked *