Deep Learning based modelling of changing agri-food characteristics

Deep Learning based modelling of changing agri-food characteristics


WR-cap AF




Landbouw, Water, Voedsel>Sleuteltechnologieën LWV>Smart Technologies in Agri-Horti-Water-Food






Agri-food product characteristics change with time. Because of ageing, storage or when new varieties of products are introduced. As more companies rely on automation, they depend heavily on decisions made by machine/deep learning (ML/DL) and machine vision. Current static ML/DL models cannot deal with this change and will lead to poor predictions if product characteristics change. This project will design machine learning methods which can adapt to changing product characteristics.

Doel van het project

This project contributes to priority 42 (From sensing to decision support and digital twins) within MMIP ST1 Smart Technologies in Agri-Horti-Water-Food. It supports automated decision making by defining adaptive AI-based models on crop traits and quality information dealing with uncertain and changing product characteristics. Instead of using traditional methods that are limited to processing data that do not vary in time, this project embraces the uncertainty in the data and learns how to deal with this variety. In this way, the models on which decision support is based, can be used even in conditions where only limited amounts of data are available.

The project addresses multiple agri-technological challenges where cutting-edge advances in DL have the potential to transform current state of technology to support even more precision and more adaptive plant production in multiple scenarios (open-field, greenhouses and vertical farming production conditions are part of the project). The knowledge and tools developed in this project will lead to a better understanding of the application of modern DL techniques in different application domains. Such cross-domain application of advances in Sensing and AI fits very well with ambition of the Mission as a whole.

Relatie met missie (Motivatie)

The broad vision of the project is to solve a series of problems which are hindering the progression of agri-food industry towards precision production and more robust and adaptive food processing. Especially, how can we observe and monitor changes in plants, plant-parts or eventual food products to the finest details using machine vision and deep learning technologies. And, beyond monitoring, to make informed decision on product quality, growth stage prediction, early stress detection, yield estimation etc. These are all key questions, and resolving them will bring current automation systems (such as those of the project partners) to the next level of innovation.

The industrial partners have raised challenges with their specific market sector in mind, but the technological innovations needed are transversal across their use-cases. Addressing these situations will make their products frontrunners in the global agri-food industry. They have identified key use-cases in their specific market sectors and foresee advances in AI, specifically deep learning, and computer vision as a path to address them:

Problem leader: Use-cases
Hortikey: Yield estimation and quality assessment for greenhouse settings across multiple tomato cultivars while dealing with seasonal variations and changing illumination conditions, in images from a robotic platform navigating the greenhouse.
Visser: Rapid and large-scale monitoring predictions of plantlet developmental growth and quality across multiple plant types in dense-cropping scenario under varying seasonal and illumination variations in indoor farms, in low resolution drone images.
Tungsram: Holistic monitoring of changing plantlet traits in vertical farms across multiple plant types in dense-cropping scenario under varying seasonal and artificial light intensity variations, based on image and other climate sensors data. Tungsram's main desire is to focus on AI-driven dynamic lighting strategies for uniform and

Geplande acties

Overall project results:

Due to highly applied nature of the project, models, software and hardware are the key results/deliverables in this project. The valorization of these is directly linked to their application in industrial scenarios, which is why the project also has a strong focus on deployment from the very beginning of the project. These products are intended to be deployed on industrial applications under specifications (hardware, APIs, throughput considerations etc.) set by the industrial partners.
- For Visser, Tungsram and Hortikey, a minimal viable software product is the key deliverable, which brings together the models developed over the years and combines them into a software for user-friendly analytics. In particular, the focus will be on visualization of models’ predictions in an intuitive manner
- For Tungsram, a proprietary data acquisition platform suited to their vertical farm settings will be designed.
- For Hortikey, successful models will be integrated into the Hortikey Plantalyzer platform

Pathway to the results:
The output of the project is based on five principle activities:
- Data acquisition: For a data-driven approach like DL, it is essential to collect a significant sufficient amount of “good” data. Such data needs to incorporate examples from a multitude (if not all) variations that might occur in real scenarios. With this mind, each work package leader is committed to acquire a significant amount of data spread over multiple times of the year (across multiple years, for multiple product types, in multiple growth conditions, and in real food processing lines. Good and relevant data is the core of modelling efforts in this project. Data acquisition will be followed by data annotation efforts. A time-consuming but essential step for supervised DL methods to be investigated in this project.
- Model development: Model development is the core of the project. Different challenges across all industrial challenges (divided over WP1-3) necessitate different DL methods to be investigated.
- Hardware and software infrastructure development: Software is the key output of this project. The models developed in the project need to be packaged in a software (with proper APIs defined to incorporate in client processes), incrementally refined (as new data becomes available), replaced (when new/better models become available) and combined (software integration of multiple models). Further software tools are also needed, for instance to compare model performances, intuitive model prediction visualization and end-user friendly user-interfaces. Apart from software, one WP (WP3) also requires development of a new data acquisition hardware for the vertical farm scenario.
- Model and software deployment: Models, software and hardware development is the first step towards deployment in client processes. Deployment will be primarily guided by industry specifications (e.g. throughput requirements, API specs, prediction accuracy, traits to measure etc.). DL models are highly complex (can easily have millions of parameters) and thus are computationally expensive. The final deployed models and software will need regular [re-]design to fall within the specifications.
- Other core activities: Activities such as project coordination, communication, workshops, reporting and dissemination