A novel multimodal image feature fusion mechanism: Application to rabbit liveweight estimation in commercial farms

In the realm of commercial rabbit farming, assessment of liveweight for each rabbit is crucial for production management. Traditional manual weighing methods are labor-intensive, time-consuming, and can induce stress and disease transmission. To solve this problem, this paper introduced a liveweight estimation method for rabbit utilizing near-infrared and depth image, including a segmentation stage and a weight estimation stage. Specifically, a dual-stream feature fusion mechanism was proposed to effectively integrate information from both near-infrared and depth imaging. The network was trained and validated on a dataset of 1,957 overhead images from 300 rabbits collected from a commercial farm, achieving superior performance with an R-Square (R2) of 0.95, Root Mean Square Error (RMSE) of 194.95 g, Mean Absolute Deviation (MAD) of 155.10 g, Mean Absolute Percentage Error (MAPE) of 4.08 %, and Mean Coefficient of Variation (MCV) of 3.03 %. Ablation experiments revealed that employing the proposed dual-stream feature fusion mechanism led to reductions of 52.87 g, 38.19 g, 1.01 %, and 0.75 % in RMSE, MAD, MAPE, and MCV, respectively, compared to not employing the mechanism. Compared to machine learning method, our proposed approach demonstrates superior performance in RMSE, MAD, MAPE, and MCV, achieving more accurate liveweight estimation for large-scale commercial farms.