An Appearance Optimisation Method for Projection-Based Spatial Augmented Reality

Projection-based spatial augmented reality (P-SAR) supports appearance-oriented design evaluation by projecting digital materials onto physical mock-ups, but the projected result may deviate from the intended screen-rendered appearance in both colour distribution and normal-induced shading. This paper proposes a screen-referenced, measurement-driven appearance optimisation framework in which a calibrated monitor serves as the visual reference and the projected mock-up as the optimisation target. The workflow combines controlled Unity rendering, colourimetric measurement, D65 CIE Lab analysis, and MATLAB (R2024b)-based iterative update, and separates the problem into albedo appearance optimisation and shading appearance optimisation. The albedo branch uses dominant-colour grouping and CIEDE2000-guided group-wise correction, while the shading branch uses a lightness-contrast descriptor derived from flat and normal-modulated renderings to update the normal-map-driven shading response. Experiments on ten material textures showed that all 23 identified colour groups converged below the adopted $\Delta E_{00}$ threshold of 2.3; the mean texture-level colour difference decreased from 6.24 to 1.36, corresponding to an average reduction of 77.43%. Comparative evaluation showed that the proposed group-wise optimisation outperformed global neutral-grey and global Lab-offset correction baselines. For shading, the mean residual $r_D=|D_{\mathrm{proj}}-D_{\mathrm{ref}}|$ decreased from 1.164 to 0.264 $L^{*}$ units, and all ten normal-map cases satisfied the 1.0 $L^{*}$ tolerance. A comparison with a global luminance-contrast baseline further supported the benefit of material-level normal-map update over image-domain contrast adjustment. Additional analyses examined the sensitivity to the number of dominant colour groups and clarified the rationale and scope of the adopted thresholds. Integrated photographic examples provided qualitative illustrations of the overall appearance tendency after the complete workflow, while the quantitative assessment was based on colourimetric and lightness-domain measurements. The full workflow required approximately 26 min per material case, indicating practical feasibility for controlled or semi-controlled P-SAR material appearance preparation and design-review scenarios.