The red palm weevil (Rhynchophorus ferrugineus) is one of the world’s most destructive palm pests, having spread to more than 60 countries across Asia, the Middle East, Africa, Europe and, more recently, the Americas.
According to FAO, the pest threatens date palms and affects the livelihoods of nearly 50 million farmers in the Near East and North Africa region. The weevil even reaches some of the world’s more remote locations. In 2019, a CABI team led by Dr Arne Witt confirmed its presence on Socotra, an island off Yemen, for the first time.
However, protecting vulnerable palm species requires more than effective monitoring and management. It also depends on access to reliable, up-to-date knowledge.
To help make this knowledge more accessible, CABI recently published the Palm Weevil Knowledge for AI dataset on Hugging Face, a platform for sharing machine learning models, datasets and AI resources. Developed as part of the ‘Generative AI for Agriculture (GAIA)’ project, it adds to a growing collection of AI-ready datasets hosted by CABI, providing trusted information for use in AI tools, chatbots and digital advisory services.
Why is red palm weevil so difficult to manage?
One reason the pest is so difficult to control is that it causes extensive damage inside the palm before visible symptoms appear. Inadequate management practices, such as the indiscriminate use of pesticides, will also negatively impact other insects and organisms.
Early detection and rapid response therefore critical to managing the pest. Researchers, extension workers, policymakers and farmers all need reliable guidance on identifying infestations, monitoring pest spread and choosing pest management strategies.
One organization working to address this challenge is the Consortium for the Red Palm Weevil Control Program (C4RPWC), led by the International Center for Agricultural Research in the Dry Areas (ICARDA). The Consortium is advancing early detection and early warning technologies, including acoustic sensors, remote sensing and AI-powered predictive tools, to support earlier intervention and more effective pest management.
Alongside these efforts, access to reliable information remains important. Much of this knowledge is spread across scientific papers, technical guides, pest datasheets and other specialist resources. Finding and using it quickly can be challenging, particularly as AI-powered advisory tools become more common in agriculture.
Why is CABI’s Palm Weevil Knowledge for AI dataset?
CABI’s Palm Weevil Knowledge for AI dataset brings together specialist information on palm weevils from a range of sources, preparing it for use in AI applications and digital advisory platforms.
“There is already substantial agricultural knowledge available, but finding the right information and making it accessible through digital tools can be a challenge. Preparing this content for AI applications creates new ways for people to access and use that knowledge,” said Katherine Cameron, CABI’s Global Lead for Digital Advisory Services.
The dataset combines information from books, journal articles, CABI Compendium datasheets, farmer factsheets and country-specific resources, creating a single, structured source of knowledge. Where available, it also includes information on registered bioprotection products from the CABI BioProtection Portal.
How can the dataset be used?
The dataset contains around 205 documents and 3.2 million tokens (the basic units of data handled by a language model) of expert-curated content on palm weevils. It covers pest identity and distribution, host plants, symptoms, impacts, prevention and control.
Its structured format supports large language models and retrieval-augmented generation (RAG), which allows AI systems to retrieve relevant information from specialist sources when answering questions. This gives developers access to trusted CABI knowledge when building AI applications.
Potential uses for the dataset include:
- Developing chatbots and digital advisory services that help farmers, extension workers and plant health professionals access trusted guidance on pest identification, monitoring and management.
- Powering knowledge retrieval systems that help users find relevant information across scientific papers, technical guides and other specialist resources.
- Supporting research and analysis of palm weevil biology, distribution and management through access to a curated collection of information.
These applications would need to be developed and tested to ensure they provide relevant and appropriate responses. The dataset gives the specialist content for such tools, rather than a ready-made chatbot or advisory service.
Applying AI-ready knowledge through ChatBug
One example of how AI-ready agricultural knowledge can be used in practice is Chatbug, an AI-powered knowledge and advisory tool developed by researchers at the University of California, Davis (UC Davis), as part of C4RPWC’s work on digital innovation.
ChatBug draws on a range of scientific knowledge resources including CABI content, the Khalifa International Award for Date Palm and Agricultural Innovation, and other publications. It offers dedicated researcher and farmer modes, with responses accompanied by references to their sources.
The integration of CABI’s AI-ready content into ChatBug demonstrates how curated agricultural knowledge can contribute to digital advisory tools alongside other scientific resources. Further evaluation will help assess the platform’s accuracy, relevance and practicality.
“Through C4RPWC, we are bringing together research, innovation and trusted knowledge resources, including those provided by CABI, to explore how AI-powered tools can support researchers, agricultural professionals and farmers.” said Dr Mojeeb Stanikzai, Program Manager of C4RPWC at ICARDA.
Who can use the dataset and benefit from it?
The Palm Weevil Knowledge for AI dataset is available on Hugging Face for researchers and AI developers to explore and use within their work. Through it, they can develop AI applications, including tools that help users find relevant information on red palm weevil from CABI’s scientific and technical resources.
The benefits could extend beyond those working directly with the dataset. For example, extension services and plant health advisers could use tools developed from the content to find relevant guidance when responding to growers’ questions. Meanwhile, government agencies and policymakers could explore applications that support pest surveillance, risk assessment and the planning of management programmes.
Farmers and growers may benefit indirectly if these tools help make reliable information on identifying and managing red palm weevil easier to access. The dataset therefore offers a resource for those developing AI applications, with potential uses for the people and organizations working to manage the pest.
Building AI-ready agricultural knowledge beyond palm weevils
The Palm Weevil Knowledge for AI dataset builds on CABI’s broader Plant Health Knowledge for AI dataset, which covers a wide range of crop pests and diseases.
CABI has also recently published a new Livestock Knowledge for AI dataset, extending this work beyond plant health. This expands the range of CABI’s specialist agricultural knowledge available for developers to use in AI applications, from crop pests and diseases to livestock.
These datasets provide researchers and developers with content they can use to explore new AI applications for agriculture. They represent just a subset of the wider agricultural knowledge available through CABI. The challenge now is to turn access to this specialist knowledge into useful tools that address real information needs, whether in crop protection, livestock management or agricultural advisory services.
Further information
To discuss access to additional CABI datasets, please contact us at:enquiries@cabi.org
Chatbug is available in English and Arabic, and is publicly accessible at https://rpwiki.org/
Featured image: Rhynchophorus ferrugineus (red palm weevil); adult male, from a trunk of Canary island date-palm (Phoenix canariensis). Kefalonia, Greece. August 2016 (©Alan Roberts)
Related news and blogs
Related News & Blogs
Valuing trusted knowledge in the age of AI
As AI systems increasingly use and reshape trusted knowledge, how should publishers respond? This blog explores practical approaches to pricing and licensing content for AI, drawing on insights from the GAIA project and CABI’s wider work on data governance.
7 May 2026
