Tag Archives: ChatGPT

Keeping up with pollination: using ChatGPT as a research alert

There is too much science.

That is not a complaint about the quantity of research being done. Quite the opposite: it is remarkable how much good work is now being published across the world. But the sheer volume creates a serious practical problem. How can any researcher keep track of what is relevant to their field, let alone read and properly absorb it?

For someone interested in pollination, the difficulty is compounded by the extraordinary breadth of the subject. Pollination is not confined to a single academic discipline, or even to a small cluster of them. It cuts across botany, zoology, ecology, evolution, conservation biology, agriculture, economics, food security, national security, palaeontology, biogeography, genetics, behaviour, climate science and many other areas.

A paper that changes how I think about pollination might appear in a specialist plant journal, an entomological publication, or journals covering conservation, agricultural economics, or palaeontology. It might concern the structure of ecological networks, the nutritional quality of crops, the evolution of flowers, pesticide regulation, the movements of migratory birds or the fossil record of insects. It may not even use the word “pollination” prominently in its title or abstract.

That breadth is one of the great attractions of a subject that has kept me fascinated for 35 years, and which I tried to capture in my book Pollinators & Pollination: Nature and Society. It is also what makes keeping up with it so challenging.

The limitations of conventional alerts

There are, of course, many ways to receive information about new research. Journals send tables of contents. Google Scholar provides alerts based on keywords or authors. ResearchGate regularly tells me that someone has cited one of my publications.

These services have their place, but they tend to provide a rather narrow window onto the literature.

For example, Google Scholar can alert researchers to newly indexed material matching a search query. This is useful, but it is neither a comprehensive record of everything published nor a carefully curated selection: broad searches produce noise, while narrow ones inevitably miss relevant work.

A citation alert from ResearchGate tells me about research connected to work that I have already published. That can be valuable, and occasionally flattering, but it inevitably looks backwards. It shows me the expanding wake of my own research rather than offering a broad view of where the subject is moving.

Keyword alerts have a different problem. They can generate large quantities of material with very little discrimination. A search for “pollination”, “pollinator” or “plant–pollinator interactions” will retrieve many relevant papers, but also conference notices, marginally related studies, duplicate records and work of highly variable importance.

More restrictive searches reduce the noise but risk excluding the unexpected paper that turns out to be especially interesting.

What I need is not simply a larger stream of titles. I want something closer to an informed research assistant: a system that could search widely, exercise some judgement, explain why particular items might matter, and alter its approach in response to my comments.

I have therefore been experimenting with ChatGPT’s scheduling function as a weekly research-alert system.

A scheduled conversation

My current instruction is for ChatGPT to provide a shortlist every Friday morning of the most worthwhile new papers, preprints, reports and substantive analyses relating to plant–pollinator interactions, pollination networks, bird pollination and related broader biodiversity topics.

I have asked it to include no more than ten items, and fewer when the available material is weak. That final qualification is important: I do not need ten references merely to fill ten spaces, I’d rather receive four genuinely interesting papers than a padded list containing six that I will never read.

For each item, the system can provide the citation, a short account of the main finding and an explanation of why it may be relevant to my interests. It can also distinguish between peer-reviewed research, preprints, reports and other forms of analysis.

In that sense, the alert is already more useful than a conventional automated search. But the most important difference is that it is a conversation.

I can tell it that a particular paper was especially useful and ask it to look for more work of that kind. I can point out that another item was only marginally relevant. I can ask it to widen its search into palaeontology, ecological economics or agricultural policy, or to pay closer attention to a particular taxonomic group.

I can also tell it what not to do. When a major European pollinator-research white paper appeared, for example, the alert quite reasonably identified it as relevant. But I was one of its co-authors and did not need an artificial intelligence system to introduce it to me as a new discovery. I could therefore instruct the system to recognise my own publications and either omit them or flag them only when they were strategically relevant.

That adaptability is difficult to reproduce with conventional keyword alerts. The scheduled search is therefore not a fixed filter, it can be refined as my interests, projects and frustrations change.

From retrieval to assessment

The distinction between finding research and assessing it is also important.

A long list of newly published papers transfers the problem of selection from the search engine to the researcher. A useful alert should do more than retrieve documents. It should offer some preliminary judgement about novelty, relevance and significance.

Does a paper introduce a genuinely new idea, or does it repackage a familiar concept in new terminology? Is a striking conclusion supported by a strong study design? Does a paper matter because of its empirical results, its methods, its conceptual framework or its policy implications? Is it directly relevant to my work, or merely adjacent to it?

ChatGPT cannot answer such questions infallibly. Nor should its assessment be accepted without scrutiny. But it can help to triage the literature and identify which papers deserve closer attention.

This is particularly valuable outside one’s immediate specialism. I can usually make a rapid initial judgement about a field study of flower visitors or a paper on pollination networks. I may need more assistance in deciding whether a new economic analysis, remote-sensing method or palaeontological reconstruction is likely to be important.

The purpose is not to delegate scientific judgement, it is to direct that judgement more efficiently.

A few necessary cautions

There are obvious limitations.

An AI-generated research brief is only as good as the literature it can locate and the instructions it has been given. It may miss relevant work, misunderstand a paper, exaggerate novelty or place too much weight on a fashionable topic.

There is also a danger of creating an intellectual echo chamber. If I repeatedly tell the system what interests me, it may become increasingly good at confirming those interests while overlooking research that sits outside them. A useful alert needs some permission to surprise.

For that reason, I think the instructions should combine a clear core remit with an explicit request to include occasional unexpected items that could change how the field is viewed.

The summaries themselves must also remain starting points. Important papers need to be read, methods inspected and conclusions considered in relation to the evidence. An articulate paragraph generated by ChatGPT is not a substitute for engaging with the original study.

A better kind of alert

Used carefully, however, scheduled ChatGPT alerts offer something that conventional notification services do not: an iterative relationship between the researcher and the search process.

The system can learn that I am interested not only in pollination as an ecological interaction, but also in its agricultural, economic, evolutionary, historical and political dimensions. It can respond when I say that a particular area is becoming more important to a project. It can explain its selections, accept criticism and search differently the following week.

That makes it less like an electronic table of contents and more like a research assistant who arrives on Friday morning with a small pile of papers and a cup of coffee and says: “These are the ones I think you should look at, and this is why.”

Like any research assistant, it will sometimes be wrong. It may overlook something obvious, become overenthusiastic about a weak paper or misunderstand why a study matters. But it can be corrected. The conversation continues, and the next briefing can be better.

For a field as broad and rapidly expanding as pollination research, that may be the most useful feature of all.

I will finish with an example. Today’s alert arrived at the scheduled 08:00 (though I had to make my own coffee…) and included four items for me to look at; this was the fourth, which I’m reproducing verbatim:

4. Agarwala et al. — Biodiversity loss will decrease the future creditworthiness of nations

The authors incorporate projected losses of wild pollination, tropical timber and marine fisheries into sovereign credit-risk models for 23 countries. Their scenarios suggest that partial ecosystem collapse could substantially increase national borrowing costs and expose a major omission in conventional credit ratings.

Why it matters: This is one of the clearer attempts to translate pollination loss from ecosystem-service valuation into a decision framework already used by governments and financial markets. For ecological-economics work, the methodological bridge—from spatial ecosystem-service scenarios through macroeconomic models to sovereign credit ratings—is more important than the headline monetary totals.

The ecological assumptions deserve scrutiny. “Wild pollination” is compressed into a tractable economic input, so uncertainty in crop dependence, substitution, adaptation and spatial mismatch may be obscured once propagated through the financial model.

Assessment: Strategically consequential interdisciplinary analysis, not new pollination biology.

The paper actually appeared on 4th June but I missed it, even though it cites this paper by Millard et al., in which I was involved, and presumably appeared on a ResearchGate citation alert last month. ChatGPT rightly spotted that it was something which would interest me and that’s what’s so fascinating about this approach to keeping up with the literature – this system is intelligent enough to “understand” my needs, even if it’s not conscious of precisely why it understands them.

As always, I’d be pleased to hear your views on this topic – feel free to comment below.

Should AI be thought of as a science tool or collaborator? A conversation with ChatGPT

In a piece that I posted before Christmas, I made the offhand comment that “we need to think of AI as a collaborator as much as a tool“. It’s offhand in the fact that I’d not thought too deeply about it, but some reading that I did over the holiday made me rethink this statement, or at least consider it more critically. The most recent post on the Dynamic Ecology blog – Meghan Duffy asking about using AI to find literature – also got me thinking about whether AI is just a tool.

Eventually I decided to ask the question of ChatGPT itself and we ended up having quite a long conversation on this topic. I then asked it to summarise the conversation and what you see below is my edited version of that summary:

Jeff: Is ChatGPT best thought of as a tool or a collaborator in science?
ChatGPT: ChatGPT is best understood as a tool with collaborator-like affordances. It supports analysis, writing, coding, and idea development, but it does not participate in collaboration as a social, epistemic, and accountable practice. It is also prone to errors.

Jeff: But human collaborators also make errors and sometimes avoid accountability. We do not expect people to be flawless.
ChatGPT: Exactly. The issue is not error versus error-free performance. It is about agency and accountability: human collaborators can explain their reasoning, revise their views, and be held responsible for their contributions.

Jeff: You said that ChatGPT does not participate in collaboration as a “social, epistemic, and accountable practice”. Yet you can still generate genuinely useful ideas and connections.
ChatGPT: Yes, and that is why calling ChatGPT part of the infrastructure (as I did in our longer conversation) is misleading. Unlike laptops or software platforms, I can actively generate ideas, cross-fertilise between disparate areas, and help explore conceptual space.

Jeff: If I ask a colleague to write R code, that is collaboration. If I ask you to write R code, why is that different?
ChatGPT: Because a human colleague brings intent, methodological judgement, and ownership of decisions. I generate code instrumentally, without understanding the scientific stakes or assuming responsibility for analytical choices.

Jeff: Earlier you said you cannot “stand behind a claim”. But you can cite papers and explain why something should be trusted. Is that really true?
ChatGPT: When I cite papers or summarise arguments, I reproduce the form of justification, but I still cannot stand behind a claim in the scientific sense. I do not judge evidential strength or accept responsibility; trust remains with the human scientist and the underlying sources.

Jeff: Yet the output of your code or analysis can be just as useful as that produced by a human collaborator.
ChatGPT: That is true. The distinction is not about usefulness or correctness, but about agency, authorship, and accountability within scientific practice.

Jeff: So what is ChatGPT, if it is neither a standard tool nor a collaborator?
ChatGPT: It is best described as a cognitive amplifier or dialogic system: something that expands the range of ideas and connections available to researchers, without itself being an accountable scientific agent.

Jeff: And what does that imply for how we should use ChatGPT in science?
ChatGPT: It implies using ChatGPT to provoke, explore, and accelerate thinking, while recognising that scientific judgement, responsibility, and ownership remain entirely human.

Jeff: Produce an image that I can use on the blog which summarises this discussion.
ChatGPT: Certainly.

As a test of its ability to “provoke, explore, and accelerate thinking”, at the end of last week I uploaded a manuscript that I had written, just prior to submission, and asked ChatGPT to tell me its strengths and weaknesses and fit for the journal. Its feedback was spot-on – far more than I would expect from a “tool” – and I made a few final tweaks before I submitted the manuscript. ChatGPT also said “Don’t be surprised if the reviewers ask you to change X, Y and Z” and so it will be interesting to see whether it’s correctly gauged the manuscript and the reviewers’ responses.

What times we live in! At the moment I’m optimistic enough about AI to see all of this as an intriguing exploration of the capabilities of these large language models, an expedition through dense habitat in which we’ve barely left base camp and our view of what lies ahead is restricted and there may be nasty surprises along any path that we hack. But I appreciate that not everyone is so optimistic and, as always, I’d be interested in your thoughts on this topic – leave a comment or send me a message.

AI at the crossroads: can ChatGPT turn you into a statistical Robert Johnson?

When it comes to the statistical analysis of data, I know my limits. Maths was never my strong point at school or university, and my approach has always been to keep analyses as simple and straightforward as possible*, or to rely on colleagues with fancier statistical chops to do the heavy lifting. I wish that were not the case – I wish I had a brain that was more number-focused than it is. But I don’t and I’ve learned to live with it, to play to my actual strengths as a scientist, and to collaborate with others who can bring different skills to the party.

In theory, the development of the R platform was supposed to make life easier for those of us who wanted to analyse complex data sets. But actually having to script, from scratch, the code to carry out even simple analyses always seemed to me to be a step backwards from the push-button days of SPSS or Minitab. Yes, I get that R is incredibly powerful and flexible and blah blah. But it still involves a heavy time commitment and an aptitude for writing code that many of us struggle with.

Recently, however, things have changed. I find myself carrying out complex statistical analyses that would have stumped me 12 months ago. Not only that, but I now understand those tests on a much deeper level than I ever did before. I also feel much more confident in the interpretation of the outputs from the tests I’m running, and their limitations.

Why the over night change? ChatGPT.

More precisely, I’m using ChatGPT to help me decide which analytical approaches are best for the data that I have, getting it to help me to write the R script to carry out the tests, and then (crucially) it’s advising me on the interpretation of the statistical output and suggesting future steps.

Let me give you an example. I’ve just submitted a manuscript to a journal which describes the results from an experiment that had confounded me for years, which is why I’d not published the work previously. Following discussions with some colleagues in China I realised that my framing of the work was wrong (by coincidence, a topic that Jeremy Fox has recently discussed over on the Dynamic Ecology blog). However, there was still a contradiction in two of the sets of results that I could not resolve: they should have been telling me the same thing but they were not. When I queried ChaptGPT on this it suggested that I model the data taking into account the fact that I had missing data – missingness in statistical jargon. When I did – bingo! – the results made sense: the absence of some data in my experimental treatments had systematically biased the results. It all made perfect sense.

Now, I could have talked this over with a statistician or a more statistically-minded ecologist colleague. But scientists are busy people and I did not want to impose on someone’s limited time. Or rather multiple someone’s limited times, because I know from past experience that when you ask folks these sorts of questions you can get different advice depending upon their own experiences, training, or preferred flavours of statistical analysis. By treating ChatGPT as a collaborator I can get an objective answer to my data questions, written in a way that I can understand. That last point is key because for all of us with specific expertise it’s sometimes difficult to translate our knowledge into broadly interpretable language.

How can I know that ChatGPT is giving me reliable statistical advice? It certainly didn’t give me accurate information about Erasmus Darwin a couple of years ago (a story, incidentally, that I included in my recent book Birds & Flowers: An Intimate 50 Million Year Relationship). But since then, the reliability and accuracy of ChatGPT has improved considerably and when I’ve checked the information it’s given about analyses it was usually accurate as far as I can gauge. In one case, however, it completely missed the point of what I was trying to do with another set of data. But of course advice from human collaborators can also be inaccurate – everyone is fallible. So including human (my!) oversight in all of this is important.

I’m certainly not the only one using ChatGPT and other AI platforms in this way – here’s a small sample of some online articles I’ve found on the topic:

I especially like this quote from that last article:

“If I hired a consultant to write the code when I told them what I needed, would that be a problem? Then, what’s the problem in doing stats with an (AI) consultant?”

I can only agree, and again, I emphasise that we need to think of AI as a collaborator as much as a tool.

By now you might be wondering what any of this has got to do with blues musician Robert Johnson**. Well, according to legend, Johnson was initially a fairly mediocre guitar player. His overnight transformation into a brilliant and highly influential guitarist was attributed to him meeting the Devil at a local crossroads and trading his soul for the mastery of his instrument.

Of course the story is nonsense, and Johnson clearly worked hard at his craft, but it feels like a nice metaphor of where we are at the moment: at a crossroads when it comes to how we use AI, with the opportunity for it to turn us, overnight, into experts. Will that come at a profound cost? Are we selling our souls to some kind of digital Devil? AI naysayers claim it will and we are. But as with all technological advancements, from the wheel to the internet, there are both good and bad aspects to it. Only time will tell whether the former outweighs the latter. In the meantime, let’s try to discover how best to use this amazing technology for our own and society’s benefits.

——————————————————————————————————————-

*In fact one of my recent data papers involved no statistical analysis at all because the results were so clear cut: to paraphrase the abstract, when East African lowland honey bees encountered other insects on flower heads, those other insects were displaced 100% of the time. That’s not a result that requires any kind of test.

**Blues is one of my favourite genres of music and this is not the first time it’s featured on the blog. When Karin and I returned to the UK in March 2024 one of the first things that I got involved with was helping to kick-start a regular blues evening featuring professional British and international players. The not-for-profit initiative is called Harborough Blues – check out the website for upcoming gigs.

Image credit: ChatGPT of course, under instruction from me

How reliable is ChatGPT? A weird encounter with Erasmus Darwin

There’s currently a lot of interest in ChatGPT and over the past week, like many others, I have been playing around with this artificial intelligence (AI) system and testing its knowledge and creative powers. The latter are really very impressive: I asked it to write a couple of limericks about Karin and my professions, and about our move to Denmark, and it produced a decent stab at those.

I also probed its knowledge about topics with which I’m familiar, mainly pollinators and pollination of course, and it produced some good answers. When I highlighted some errors it had made, ChatGPT went back and revised its answers, and politely thanked me in the process. If I were a student I would not rely on it to write accurate essays, but so far so good(ish).

This morning, however, I had a very odd experience. I was trying to track down something attributed to Erasmus Darwin (Charles’s grandfather) and I asked ChatGPT: “What did Erasmus Darwin say about birds visiting flowers?”

This is the answer that came back:

Erasmus Darwin was a naturalist, physician, and poet who wrote about a wide range of topics, including birds and flowers. In his 1794 poem “The Loves of the Plants,” he describes how different species of birds visit flowers in search of nectar:

“Birds of each hue, from the gay parrot’s beak
To the goldfinch, painted in tiny streak,
Sip nectar from the cups that bloom and bow
With every breeze, and nod and dance below.”

Now, the first part of this, about Erasmus’s life, is true. It’s also true that he wrote a poem with that title, though it was published in 1791, not 1794.

But, when I double-checked those attributed lines against the published version of the poem that’s on Project Gutenberg, I discovered that they did not exist. ChatGPT had made them up!

As I said, this AI is pretty creative and will certainly write a poem for you on any topic you care to request. But I didn’t ask it for that, I had made a factual query. And ChatGPT had just invented some poetry and claimed it as fact, unless of course those really are Erasmus’s lines and I’ve missed their source (please correct me if I have).

Now as a scientist and writer I try to be rigorous when quoting and citing sources, and if I hadn’t double-checked this I might have taken it at face value. And one could argue that my initial question – “What did Erasmus Darwin say…?” – was a little ambiguous. But nonetheless, I was not expecting an AI to try to pass off something it had created as fact.

Has anyone else experienced anything like this when using ChatGPT? It’s a fun, creative tool. But as far as I can tell, it also has a very human ability to lie.