Spatial-frequency domain imaging (SFDI) is a wide-field, noncontact, and label-free imaging modality that is currently being explored as a new means for estimating optical absorption and scattering properties of two-layered turbid materials. The accuracy of SFDI for optical property estimation, however, depends on light transfer model and inverse algorithm. This study was therefore aimed at providing theoretical analyses of the diffusion model and inverse algorithm through numerical simulation, so as to evaluate the potential for estimating optical absorption and reduced scattering coefficients of two-layered horticultural products. The effect of varying optical properties on reflectance prediction was first simulated, which indicated that there is good separation in diffuse reflectance over a large range of spatial frequencies for different reduced scattering values in the top layer, whereas there is less separation in diffuse reflectance for different values of absorption in the top layer, and even less separation for optical properties in the bottom layer. To implement the nonlinear least-square method for extracting the optical properties of two-layered samples from Monte Carlo-generated reflectance, five curve fitting strategies with different constrained parameters were conducted and compared. The results confirmed that estimation accuracy improved as fewer variables were to be estimated each time. A stepwise method was thus suggested for estimating optical properties of two-layered samples. Four factors influencing optical property estimation of the top layer, which is the basis for accurately implementing the stepwise method, were investigated by generating absolute error contour maps. Finally, the relationship between light penetration depth and spatial frequency was studied. The results showed that penetration depth decreased with the increased spatial frequency and also optical properties, suggesting that appropriate selection of spatial frequencies for a stepwise method to estimate optical properties from two-layered samples provides potential for estimation accuracy improvement. This work lays a foundation for improving optical property estimation of two-layered horticultural products using SFDI. Full article