Food Volume Estimation Based on Deep Learning View Synthesis from a Single Depth Map
Hamlyn Centre, Department of Surgery and Cancer, Imperial College London, London SW7 2AZ, UK
Hamlyn Centre, Department of Computing, Imperial College London, London SW7 2AZ, UK
Author to whom correspondence should be addressed.
Nutrients 2018, 10(12), 2005; https://doi.org/10.3390/nu10122005
Received: 30 October 2018 / Revised: 10 December 2018 / Accepted: 16 December 2018 / Published: 18 December 2018
(This article belongs to the Special Issue Food Portion Size in Relation to Diet and Health)
An objective dietary assessment system can help users to understand their dietary behavior and enable targeted interventions to address underlying health problems. To accurately quantify dietary intake, measurement of the portion size or food volume is required. For volume estimation, previous research studies mostly focused on using model-based or stereo-based approaches which rely on manual intervention or require users to capture multiple frames from different viewing angles which can be tedious. In this paper, a view synthesis approach based on deep learning is proposed to reconstruct 3D point clouds of food items and estimate the volume from a single depth image. A distinct neural network is designed to use a depth image from one viewing angle to predict another depth image captured from the corresponding opposite viewing angle. The whole 3D point cloud map is then reconstructed by fusing the initial data points with the synthesized points of the object items through the proposed point cloud completion and Iterative Closest Point (ICP) algorithms. Furthermore, a database with depth images of food object items captured from different viewing angles is constructed with image rendering and used to validate the proposed neural network. The methodology is then evaluated by comparing the volume estimated by the synthesized 3D point cloud with the ground truth volume of the object items.