Quantifying the Crop Load on Apple Trees using Hyperspectral Snapshot Imaging
According to estimates provided by experts, the most important apple types in Germany suffer losses of around 13% on their from harvest to consumer. In addition to the lack of these food products, this results in a high loss of natural resources and inefficient use of labour. Other key factors in terms of costs and environmental impact are high energy consumption with high CO2 emissions and loss of quality during fruit storage.
As part of the research project BigApple, hyperspectral measurements were carried out on test trees at the KOB orchard (Kompetenzzentrum Obstbau-Bodensee) in Bavendorf near Lake Constance, Southern Germany. The focus of the measurements was on two classes, i.e. apple and leaf. The spectral measurement data is the basis for the determination of physiological growth/fruit parameters, which are considered in the form of indices in the modelling and analysis of our project partners and are intended to contribute to improved apple storage.
The hyperspectral measurement was performed in the laboratory under controlled measuring conditions. Furthermore, measurement campaigns were carried out in the orchard of the KOB. The hyperspectral camera S185 with the wavelength range 450 – 950 nm was used for these purposes.
By this, the desired objects could be measured throughout the quasi-continuous spectrum using 125 channels with a bandwidth of 4 nm. Figure 1 shows an example of a measurement setup with the hyperspectral camera S185 and the mobile data processing system at the orchard. The acquisition, processing and storage of the data took place with the in-house software Cubert Utils during the measurement.
Figure 1: Measurement setup shows the hyperspectral camera S185 and the mobile data processing system for acquisition and storage of the measurements.
Automated quantification of apples
Based on the measurements, analyses of the spectral characteristics of the searched classes were performed (Figure 2) and individual differences between the spectral curves of the classes could be determined. Manual quantification of these differences is not appropriate due to the high information density of the hyperspectral measurements.
Therefore, several established supervised classification methods (e.g. k-Nearest-Neighbour, Support Vector Machine, Random Forest, Neural Networks) and their combinations were used within the project for the analysis of the data.
Using the extensive training data obtained from the measurements, it was possible to automatically identify the desired classes.
The results of the classifications show a good visual agreement compared to the panchromatic image (Figure 3). The quantification of the classification result for the searched objects was done by the overall accuracy (OA), the producer’s accuracy (PA) and the user’s accuracy (UA). Figure 4 shows a confusion matrix and the associated accuracies of the individual classes. Due to the confusion between green apple and green leaf, the UA is 75.2 %. The accuracy for the other classes is between 93.4 % and 97.1 %. Also the total accuracy shows a high value of 93.5 % which indicates a very good classification result.
Figure 2: Selected samples for the classes in a measurement example at the KOB. Corresponding spectral/reflectance curves for the respective classes. (apple = red, leaf = blue, background: tree trunk = green, background: black material = pink)
Figure 3: Panchromatic image (left) and classification example of the measurement on 01.08.2018. (apple = light green, leaf = dark green, background: tree trunk = violet, background: black material = brown)
Figure 4: Confusion matrix for k-nearest-neighbour with producer’s accuracy (PA), user’s accuracy (UA), overall accuracy (OA).
Based on the classification results, different growth/fruit parameters could be created. As an example, the parameter leaf-fruit ratio is listed here. This parameter is intended to provide information on the crop load of apple trees, which was previously determined by a manual or destructive method. Based on these results and the measurements taken at different times of growth, the average fruit growth values can be determined.
Within the framework of the project the classification of apples and leaves could be successfully carried out using the hyperspectral camera. In addition, different parameters/indices could be determined from the classification, which are used for the model of our project partners to estimate the optimal storage of apples.
Some error has occured.