Abstract
Fully homomorphic encryption (FHE) enables a server to evaluate a neural network on encrypted inputs. The Cheon–Kim–Kim–Song (CKKS) scheme supports approximate arithmetic over packed vectors, making it suitable for the linear computations used in neural networks. This survey examines algorithmic methods that make CKKS-based inference feasible and efficient for convolutional neural networks (CNNs) and Transformers. It organizes the literature around nonlinear approximation, encrypted tensor packing, bootstrapping, and network-level execution planning. Polynomial and iterative approximations balance numerical fidelity against circuit depth, while tensor representations govern slot rotations, layout conversions, and active ciphertext allocations. In parallel, bootstrapping restores the modulus levels required by subsequent stages, and network-level planning coordinates these coupled decisions across layers. We formulate a unified notation for ciphertext states, level feasibility, and evaluation latency to trace these interactions. Published CNN and Transformer cases illustrate how these interactions affect bootstrapping demand, evaluation latency, and task accuracy under the reported execution settings. Future directions include maintaining approximation accuracy across varying input ranges, selecting compatible layouts between successive operators, and integrating bootstrapping primitives and operator fusion into network-level planning.