H²Avatar: Expressive Whole-Body Avatars from Monocular Video via Hierarchical Geometry and Hybrid Rendering

Abstract

Reconstructing photorealistic and animatable whole-body avatars from monocular videos is a hot topic in computer vision and computer graphics. However, existing methods still face challenges due to the limited frequency response of single-scale geometry encodings and the instability of appearance modeling without an explicit surface anchor. In this paper, we present H²Avatar, a real-time framework that builds on a mesh-embedded 3D Gaussian representation guided by SMPL-X and disentangles geometry and appearance into hierarchical and hybrid components. For geometry, we propose a semantic-aware hierarchical encoding based on a multi-scale tri-plane pyramid, where features at different resolutions capture both global structure and high-frequency surface details such as clothing wrinkles. For appearance, we introduce a hybrid rendering strategy that anchors canonical colors using a learnable UV texture map, and complements it with a neural residual color branch conditioned on tri-plane features, pose embedding, and surface normals to model pose- and view-dependent shading variations. This design improves temporal stability and preserves identity details while enhancing photorealism under complex motions. Experiments on the NeuMan dataset demonstrate that H²Avatar consistently outperforms representative baselines across multiple sequences, outperforming ExAvatar by up to 0.66 dB in PSNR and reducing LPIPS by up to 16.3%. These results validate the effectiveness of hierarchical geometry encoding and texture-anchored hybrid appearance modeling.

1. Introduction

The reconstruction of photorealistic, animatable, and real-time digital avatars is a hot topic in pattern recognition and computer graphics, which has many applications such as immersive telepresence [1], virtual reality [2,3] and film production [4]. Traditional methods rely on multi-view capture or expensive scanners, which are difficult to available in daily life. To address this, more and more researchers focus on avatar reconstruction from only a monocular video.

Earlier methods adopt parametric models such as SMPL [5] and SMPLX [6] to offer robust priors for human animation. However, they only recover coarse shape and pose due to the limited expression for 3D human. Some methods [7,8] attempt to address this problem by adding learnable offset of each vertex, and adopt UV texture map to represent the color. These methods can reconstruct more detailed avatars, but the fixed topology limits the expression of the reconstructed avatar, especially for loose clothing. With the development of neural implicit representations like NeRF [9], many methods [10,11,12] introduce this static representation and extend it to model 3D dynamic human. These methods excel at novel view synthesis but often incur prohibitive computational costs [13] and struggle to achieve robust animation results in unseen poses [11]. To address these limitations, recent methods [14,15] adopt 3D Gaussian splatting (3DGS) [16] to represent the 3D avatar, and have achieved impressive performance. By learning a canonical representation using 3DGS, they adopt linear blend skinning to deform the canonical Gaussian Splats to the observation space, and then render the final image. Recently, some methods [17,18,19,20] adopt hybrid representations as the expression to represent avatar. They adopt different representation, and aim to combine their strengths for avatar reconstruction. However, most of them model geometry in a single resolution, which can lead to the loss of fine-grained details. In addition, they adopt implicit or explicit texture encoding, leading to sub-optimal appearance modeling.

The first limitation is due to the inherent spectral bias associated with single-scale geometric representations. Most existing approaches [17,18,19,20] encode geometry using a fixed resolution feature. This design creates an unavoidable trade-off where a coarse grid ensures global structural consistency but fails to present high-frequency details such as dense clothing wrinkles. However, increasing the resolution often increases the computational budget and complicates the optimization process. Besides, the reconstructed avatars can suffer from over-smoothed geometry that lacks the intricate surface variations that are essential for realism.

The second limitation concerns the ambiguity of texture modeling and the mismatch between texture resolution and perceptual importance. Previous methods [14,19] often regress surface color implicitly from spatial features or utilize a single texture map derived from UV parameterization. This uniform treatment of the human body ignores the fact that human perception is highly sensitive to facial details. Furthermore, purely implicit color decoding lacks explicit anchoring to the surface, which often results in temporal texture instability and the loss of high-frequency identity features.

To address these challenges, we present H²Avatar, a novel framework that disentangles geometry and appearance into hierarchical and hybrid representations. To overcome the geometric bottleneck, we propose a hierarchical geometry encoding mechanism based on multi-scale tri-planes. Rather than relying on a single feature level, we construct a feature pyramid that processes geometry across multiple frequency bands. The coarse levels maintain robust body topology, while the fine levels are dedicated to capturing high-frequency surface geometry such as clothing wrinkles. By fusing features from these different scales, our method effectively recovers sharp geometric details without compromising structural integrity. In addition, we model the face and the body separately, which helps to reconstruct more detailed geometry. Moreover, to model fine-grained appearance, we introduce a hybrid appearance modeling strategy to model the texture by combining the explicit and implicit representations. Specifically, we represent the canonical appearance using a single learnable UV texture map, which provides stable and temporally consistent base colors across frames. To account for pose- and view-dependent appearance that cannot be captured by a static texture alone, we further predict a neural residual color term from tri-plane features and deformed surface normals. Then, we add it to the texture-based base color. This hybrid design can preserve the identity details and the fine-grained details. Experimental results conducted on NeuMan dataset [10] outperforms ExAvatar by up to 0.66 dB in PSNR and reducing LPIPS by up to 16.3%, which suggests the effectiveness of H²Avatar.

Our main contributions can be summarized as follows:

• We propose a hierarchical geometry encoding that mitigates geometric smoothing and enables the reconstruction of intricate surface details from monocular video.
• We design a hybrid appearance modeling scheme that combines an explicit UV texture anchor with a neural residual color branch, improving temporal stability and photorealistic rendering under complex motions.
• Our H²Avatar achieves superior performance on human reconstruction tasks in both quantitative and qualitative comparisons, compared to existing baselines.

2. Related Works

2.1. Monocular Animatable Human Avatar Reconstruction

Reconstructing photorealistic and animatable human avatars from monocular videos is a central problem in pattern recognition and computer graphics. A common paradigm is to leverage parametric models to provide deformation priors for pose, and fit the geometry and appearance based on them. These methods enable stable animation, but the fixed topology and limited expressiveness make it difficult to capture realistic 3D avatar. To alleviate this, some mesh-based approaches augment parametric bodies with per-vertex learned displacements, which improves details to some extent but still constrained by the underlying template connectivity and resolution [5,6,7,8]. However, this kind of methods are still difficult to obtain detailed results due to the fixed topology.

In this paper, we adopt a hybrid representations that combines mesh and Gaussian splatting, which can not only achieve robust animation due to the parametric model but can also obtain photo-realistic results due to the effectiveness of Gaussian splatting.

2.2. Neural Implicit Representations for Animatable Avatars

Neural implicit representations have demonstrated strong capability in free-viewpoint rendering by modeling scene appearance as continuous volumetric functions. Neural radiance fields (NeRFs) [9] is a powerful representation that models the density and color of the scene at any point in space. Building upon this foundation, a series of works extend NeRFs to dynamic human capture and avatar modeling. Early efforts [12,21] reconstruct human radiance fields from monocular videos, enabling novel-view synthesis of animatable human. To better handle non-rigid motion, several methods [10,11,22] incorporate explicit deformation or pose conditioning to obtain animatable neural fields. To improve expressiveness, Structured Local Radiance Fields exploit localized representations for human avatar modeling [23]. For scalability to unseen videos, Vid2Avatar [24] proposes human-scene decomposition to separate humans from complex backgrounds and reconstruct 3D avatars from unconstrained inputs.

Despite their quality, avatars reconstructed by NeRF often suffer from heavy training and rendering costs. Recent works therefore focus on efficiency and real-time deployment. InstantAvatar [13] achieve fast avatar learning from a monocular video in 60 s. LiteNeRFAvatar [25] further explores lightweight NeRFs with local feature learning for dynamic avatars. AvatarRex [26] highlight the importance of efficient representations and motion-aware modeling for interactive applications. However, while neural implicit avatars provide high-quality free-viewpoint rendering, their computational overhead and the lack of an explicit surface anchor can complicate stable animation and real-time use cases.

To address these problems, we present a novel framework that disentangles geometry and appearance into hierarchical and hybrid representations. In addition, by introducing the mesh-embedded Gaussian representation, our method is efficient and effective for human avatar reconstruction.

2.3. 3D Gaussian Splatting Based Avatars

Three-dimensional Gaussian splatting (3DGS) [16] replaces costly volumetric integration with differentiable splatting of anisotropic Gaussians, which delivers a strong quality and real-time rendering. This representation has been rapidly adopted for animatable avatar modeling. Many methods learn a canonical Gaussian splats and deform them to observed poses using linear blend skinning, thus supporting reposing and reenactment [14,20,27]. Subsequent works improve deformation stability [19], incorporate tri-plane style feature encoders [17], and introduce hybrid explicit components [18] to enhance identity fidelity and temporal consistency [28,29,30,31]. Hybrid avatar systems that combine Gaussians with additional explicit structures have also demonstrated favorable robustness and realism under challenging motions [17,18,19,20]. However, all these methods adopt a single scale feature encoding, which is difficult to model different scale details. In addition, these methods struggle to obtain fine-grained textures due to implicit color encoding or naive texture mapping, leading to sub-optimal results.

As shown in

Table 1. Comparison of representative animatable avatar reconstruction methods. H²Avatar is the first to combine hierarchical geometry encoding with hybrid appearance modeling.

Method	Geometry	Appearance	Hierarchical	Hybrid	Key Limitations
LoGAvarar [18]	3DGS	Explicit (UV Texture)	No	No	Single-scale geometry, limited details
NeuMan [10]	NeRF	Implicit (MLP)	No	No	Slow rendering, limited details
iHuman [20]	Mesh+GS	Implicit (MLP)	No	No	Single-scale geometry, unstable texture
ExAvatar [17]	Mesh+GS+Tri-plane	Implicit (Tri-plane+MLP)	No	No	Single-scale geometry, implicit color only
NEGSAvatar [32]	Mesh+GS	Implicit (MLP)	No	No	No multi-scale structure
H2Avatar (Ours)	Mesh+GS+Multi-scale Tri-plane	Explicit + Implicit	Yes	Yes	Addresses above limitations

, most existing methods adopt a single-scale geometry encoding and a single texture representation, which limits the reconstruction quality. In this paper, we introduce hierarchical geometry and hybrid rendering, which can model different level of details and achieve high-quality rendering.

3. Method

Given a short monocular video $\mathcal{V}={\left\{{\mathbf{I}}_t\right\}}_{t=1}^T$, we aim to reconstruct an expressive, animatable avatar that supports free-viewpoint rendering and pose-driven animation in real time. We follow mesh-guided Gaussian avatar pipelines and build the canonical representation on top of SMPL-X topology. Specifically, we upsample the SMPL-X template mesh and attach one 3D Gaussian to each upsampled vertex, which provides a stable manifold structure and consistent correspondence across time. In contrast to prior works that use a single-scale canonical encoding and largely implicit appearance, we redesign (i) the geometry feature backbone via a multi-scale tri-plane pyramid, and (ii) the appearance model via a hybrid rendering strategy that combines an explicit UV texture anchor with a neural residual color branch. Figure 1 shows the framework of our proposed method.

3.1. Notation and Definitions

We denote the input video as $\mathcal{V}={\left\{{\mathbf{I}}_t\right\}}_{t=1}^T$ with camera intrinsics $\mathbf{K}$ and per-frame extrinsics $({\mathbf{R}}_t,{\mathbf{t}}_t)$. We assume per-frame SMPL-X parameters, including shape $\beta$ (shared across frames), pose ${\theta }_t$, and expression ${\psi }_t$ are available from current estimator.

Let ${\mathcal{M}}_0$ be the SMPL-X template mesh in a canonical pose with $N_0$ vertices. We apply a fixed upsampling operator $\mathcal{U}(·)$ (e.g., subdivision) to obtain the upsampled canonical mesh ${\mathcal{M}}_{can}=\mathcal{U}\left({\mathcal{M}}_0\right)$ with N vertices ${\left\{{\mathbf{v}}_i\right\}}_{i=1}^N$. We attach one 3D Gaussian ${\mathcal{G}}_i$ to each vertex ${\mathbf{v}}_i$; hence, the number of Gaussians equals N.

We use the subscript “can” for canonical quantities and “t” for posed quantities at frame t. ⊕ denotes concatenation along channel dimension and ⊙ means element-wise multiplication. $\mathrm{Sample}(·)$ denotes bilinear sampling on 2D planes or texture maps. $\mathrm{LBS}(·)$ denotes SMPL-X linear blend skinning with per-vertex weights $\left\{{\mathbf{w}}_i\right\}$. $\mathrm{GS}(·)$ denotes differentiable Gaussian splatting rendering.

We denote the UV coordinate of vertex i as ${\mathbf{u}}_i=(u_i,v_i)\in {[0,1]}^2$. We denote the posed normal at frame t by ${\mathbf{n}}_{i,t}$, computed from the deformed mesh by averaging the normals of incident faces.

3.2. Canonical Representation

To model dynamic human, following previous works [14,17], we first construct the canonical avatar. Then, given the pose and expression parameters, we can deform the canonical avatar to the goal pose, i.e., observation space, and render it to obtain the final image. Here, we given the details of our canonical representation.

We adopt the SMPL-X as the initial template mesh. Each Gaussian ${\mathcal{G}}_i$ attached to vertex ${\mathbf{v}}_i$ is parameterized at frame t by its mean ${\mu }_{i,t}\in {\mathbb{R}}^3$, an isotropic scale ${\sigma }_{i,t}\in {\mathbb{R}}_{+}$, a fixed opacity ${\alpha }_i=1$, and an RGB color ${\mathbf{c}}_{i,t}\in {(0,1)}^3$.

Canonical identity on SMPL-X manifold. We represent the canonical Gaussian center as a residual on top of the SMPL-X manifold:

$${\mu }_{i,can}={\mathbf{v}}_i+\Delta {\mathbf{v}}_{i,can},$$

(1)

where $\Delta {\mathbf{v}}_{i,can}\in {\mathbb{R}}^3$ is a learned canonical offset.

Isotropic constraint and fixed opacity. For better generalization and optimization stability, we restrict Gaussians to be isotropic (a single scalar scale in canonical space) and fix the Gaussian rotation to identity. We also fix all opacities to ${\alpha }_i=1$ throughout training and rendering.

Pose-conditioned residual refinement. At frame t, we additionally predict pose-conditioned residual offsets $\Delta {\mathbf{v}}_{i,t}$ and scale residuals $\Delta {\sigma }_{i,t}$. The posed Gaussian means and scales are:

$${\mu }_{i,t}=\mathrm{LBS}\left({\mu }_{i,can}+\Delta {\mathbf{v}}_{i,t};{\theta }_t,{\mathbf{w}}_i\right),{\sigma }_{i,t}=\mathrm{Softplus}\left({\sigma }_{i,can}+\Delta {\sigma }_{i,t}\right).$$

(2)

Appearance assignment. The per-Gaussian color ${\mathbf{c}}_{i,t}$ is computed by our hybrid appearance module (Section 3.5), where a single learnable UV texture map provides the base color, and a neural residual term models dynamic shading variations.

Given the deformed Gaussians ${\left\{{\mathcal{G}}_{i,t}\right\}}_{i=1}^N$, the rendered image is:

$${\widehat{\mathbf{I}}}_t=\mathrm{GS}\left({\left\{{\mathcal{G}}_{i,t}\right\}}_{i=1}^N;\mathbf{K},{\mathbf{R}}_t,{\mathbf{t}}_t\right).$$

(3)

3.3. Semantic-Aware Hierarchical Geometry Encoding

Previous works have shown that the geometry of a human body can be well-captured by a triangular mesh, which is a natural choice for 3D reconstruction. However, the geometry of a human body is highly non-linear and complex, which makes it difficult to represent with a single mesh. To address this, we propose a hierarchical geometry encoding that decomposes the geometry into multiple levels of detail.

Canonical coordinate normalization. We normalize canonical points into a bounded cube for stable tri-plane sampling. Let $\mathbf{x}\in {\mathbb{R}}^3$ be a point in canonical space; we compute $\tilde{\mathbf{x}}\in {[-1,1]}^3$ by:

$$\tilde{\mathbf{x}}=2·{\displaystyle \frac{\mathbf{x}-{\mathbf{b}}_{min}}{{\mathbf{b}}_{max}-{\mathbf{b}}_{min}}}-1,$$

(4)

where ${\mathbf{b}}_{min},{\mathbf{b}}_{max}\in {\mathbb{R}}^3$ define a fixed bounding box estimated from ${\mathcal{M}}_{can}$.

Multi-scale tri-plane pyramid. We construct a pyramid of tri-planes $\mathcal{T}={\left\{{\mathbf{T}}^{\left(l\right)}\right\}}_{l=1}^L$. Each level ${\mathbf{T}}^{\left(l\right)}$ contains three orthogonal planes ${\mathbf{T}}_{xy}^{\left(l\right)},{\mathbf{T}}_{xz}^{\left(l\right)},{\mathbf{T}}_{yz}^{\left(l\right)}\in {\mathbb{R}}^{C_l\times H_l\times W_l}$. For $\tilde{\mathbf{x}}=(\tilde{x},\tilde{y},\tilde{z})$, we first project it to 2D coordinates ${\pi }_{xy}\left(\tilde{\mathbf{x}}\right)=(\tilde{x},\tilde{y})$, ${\pi }_{xz}\left(\tilde{\mathbf{x}}\right)=(\tilde{x},\tilde{z})$, ${\pi }_{yz}\left(\tilde{\mathbf{x}}\right)=(\tilde{y},\tilde{z})$. We then bilinearly sample features from each plane and concatenate them as:

$${\mathbf{f}}^{\left(l\right)}\left(\mathbf{x}\right)=\mathrm{Sample}\left({\mathbf{T}}_{xy}^{\left(l\right)},{\pi }_{xy}\left(\tilde{\mathbf{x}}\right)\right)\oplus \mathrm{Sample}\left({\mathbf{T}}_{xz}^{\left(l\right)},{\pi }_{xz}\left(\tilde{\mathbf{x}}\right)\right)\oplus \mathrm{Sample}\left({\mathbf{T}}_{yz}^{\left(l\right)},{\pi }_{yz}\left(\tilde{\mathbf{x}}\right)\right).$$

(5)

We then concatenate features across scales and fuse them using an MLP:

$${\mathbf{f}}_{ms}\left(\mathbf{x}\right)=\underset{l=1}{\overset{L}{⨁}}{\mathbf{f}}^{\left(l\right)}\left(\mathbf{x}\right),\mathbf{f}\left(\mathbf{x}\right)={\mathrm{MLP}}_{fuse}\left({\mathbf{f}}_{ms}\left(\mathbf{x}\right)\right).$$

(6)

We use a single tri-plane pyramid $\mathcal{T}$ to extract geometry features for all vertices and denote the fused feature as ${\mathbf{f}}_{geo,i}=\mathbf{f}\left({\mathbf{v}}_i\right)$.

Canonical geometry decoding. We decode canonical offsets and scales per vertex:

$$\Delta {\mathbf{v}}_{i,can}={\mathrm{MLP}}_{can}\left({\mathbf{f}}_{geo,i}\right)\in {\mathbb{R}}^3,{\sigma }_{i,can}=\mathrm{Softplus}\left({\mathrm{MLP}}_s\left({\mathbf{f}}_{geo,i}\right)\right)\in {\mathbb{R}}_{+}.$$

(7)

In practice, we adopt two pyramids to capture the body and the face geometry features, respectively, which results in better face details. We opt for a dual-pyramid design rather than a single high-resolution pyramid. This choice reduces memory consumption (two 256 × 256 pyramids vs. one 512 × 512 pyramid) and allows the face pyramid to specialize on high-frequency facial details, which we found to be crucial for expressive avatars.

3.4. Pose-Conditioned Refinement

To capture the dynamic details, we additionally added a pose-conditioned residual refinement module for both geometry and appearance. Here, we describe the details of our pose-conditioned residual refinement module for geometry.

Pose embedding. We embed the SMPL-X pose parameters using a linear layer:

$${\mathbf{e}}_t={\mathbf{W}}_{\theta }{\theta }_t+{\mathbf{b}}_{\theta }.$$

(8)

Pose-conditioned residuals. Conditioned on the fused geometry feature ${\mathbf{f}}_{geo,i}$ and pose embedding ${\mathbf{e}}_t$, we predict pose-specific residuals:

$$\Delta {\mathbf{v}}_{i,t}={\mathrm{MLP}}_{pose}\left({\mathbf{f}}_{geo,i}\oplus {\mathbf{e}}_t\right)\in {\mathbb{R}}^3,\Delta {\sigma }_{i,t}={\mathrm{MLP}}_{scale}\left({\mathbf{f}}_{geo,i}\oplus {\mathbf{e}}_t\right)\in \mathbb{R}.$$

(9)

With this pose-conditioned residual refinement, our model has ability to capture the pose-related dynamic details, which can help to improve the reconstruction quality.

3.5. Hybrid Appearance Modeling

Previous methods often rely on purely implicit color decoding or a single texture map, which can lead to temporal instability and loss of fine details. Therefore, we propose a hybrid appearance modeling strategy that combines an explicit UV texture anchor with a implicit neural residual color branch.

UV texture anchor. We maintain a single learnable texture map $\mathcal{I}\in {\mathbb{R}}^{H\times W\times 3}$ defined on the SMPL-X UV parameterization. For each vertex i, we obtain a pose-invariant base color by bilinear sampling:

$${\mathbf{c}}_{base,i}=\mathrm{Sample}(\mathcal{I},{\mathbf{u}}_i).$$

(10)

This explicit texture provides a stable appearance anchor and improves temporal consistency compared to purely implicit color decoding.

Neural residual color. To model pose/view-dependent shading and subtle dynamic appearance changes, we predict a residual color term:

$$\Delta {\mathbf{c}}_{i,t}={\mathrm{MLP}}_{rgb}\left({\mathbf{f}}_{geo,i}\oplus {\mathbf{e}}_t\oplus {\mathbf{n}}_{i,t}\right)\in {\mathbb{R}}^3,$$

(11)

where ${\mathbf{n}}_{i,t}$ is the posed normal computed from the deformed mesh. Note that we also introduce the pose-conditioned geometry feature as the input to the neural residual color branch, which can help to model dynamic appearance variations. The final RGB color is:

$${\mathbf{c}}_{i,t}=\mathrm{clamp}\left({\mathbf{c}}_{base,i}+\Delta {\mathbf{c}}_{i,t},0,1\right).$$

(12)

3.6. Animation and Rendering

Given the canonical representation and the target pose parameters, we can deform the canonical Gaussian avatar to the observation space due to the binding of Gaussian center and the surface vertex of SMPL-X.

For each frame t, we deform Gaussians via Equation (1) and (2), set opacity ${\alpha }_i=1$ and rotation as identity under isotropic constraint, and render the image via differentiable splatting:

$${\widehat{\mathbf{I}}}_t=\mathrm{GS}\left({\{{\mu }_{i,t},{\sigma }_{i,t},{\alpha }_i,{\mathbf{c}}_{i,t}\}}_{i=1}^N;\mathbf{K},{\mathbf{R}}_t,{\mathbf{t}}_t\right).$$

(13)

3.7. Loss Functions

To train our model, we adopt several loss functions following previous works [14,17].

Image reconstruction loss. We supervise rendered images using a standard reconstruction objective combining ${\ell }_1$, SSIM, and LPIPS:

$${\mathcal{L}}_{img}={\lambda }_1{\mathcal{L}}_1+{\lambda }_{ssim}{\mathcal{L}}_{SSIM}+{\lambda }_{lpips}{\mathcal{L}}_{LPIPS}.$$

(14)

Regularization. We apply a Laplacian regularizer on the canonical mesh and penalize excessive residual deformations:

$${\mathcal{L}}_{reg}={\lambda }_{lap}{\mathcal{L}}_{lap}+{\lambda }_s\sum _{i=1}^N{\sigma }_{i,can}+{\lambda }_{\Delta }\sum _{i=1}^N{\parallel \Delta {\mathbf{v}}_{i,t}\parallel }_2^2.$$

(15)

Overall, the full loss can be written as:

$$\mathcal{L}={\mathcal{L}}_{img}+{\lambda }_{reg}{\mathcal{L}}_{reg}.$$

(16)

3.8. Implementation Details

H²Avatar is implemented using PyTorch 2.6.0 [33] and optimized using Adam [34]. The learning rate starts from ${10}^{-3}$ and is decayed at $75\%$ and $90\%$ of the total iterations. We conducted experiments on an NVIDIA RTX 4090D GPU with 24 GB memory. We set the tri-plane pyramid level $L=3$ and per-level resolutions as ${\left\{(H_l,W_l,C_l)\right\}}_{l=1}^L$. A summary of key architectural and optimization hyper-parameters is provided in

Table 2. Implementation summary of the proposed H²Avatar. Numbers are to be filled after finalizing training configuration.

Component	Setting
Tri-plane levels L	3
Per-level resolution $\left\{(H_l,W_l)\right\}$	{(64, 64), (128, 128), (256, 256)}
Per-level channels $\left\{C_l\right\}$	{32, 32, 32}
Fusion MLP ${\mathrm{MLP}}_{fuse}$	2 layers, 192
Texture resolution $(H,W)$	(512, 512)
Optimizer	Adam
Learning rate	$1\times {10}^{-3}$
LR schedule	$\times 0.1$ at 75% and 90% iters
Training iterations	25 epochs

. The whole training time is about 2–6 h depending on the duration of the input video. After training, our model can render images at 30 FPS on a single GPU.

4. Experimental Results

4.1. Evaluation Setting

Dataset. To evaluate the performance of the proposed method, we conduct experiments on NeuMan dataset [10]. It consists of several monocular videos that a single human walks about 15 s in in-the-wild scenarios. Following previous works [17], we adopt four sequences for the evaluation, including bike, citron, jogging and Seattle.

Metrics. To evaluate the results, we adopt three metrics following previous works.

• PSNR: We adopt Peak Signal-to-Noise Ratio to measure the similarity between the rendered image and the ground truth.
• SSIM: We also adopt Structural Similarity Index evaluates the perceptual similarity by considering luminance, contrast, and structure.
• LPIPS: To measure the perceptual similarity that better align with human perception, we use Learned Perceptual Image Patch Similarity as the evaluation metric.

Compared Methods. We compare our method with several baseline methods, including:

• NeuMan [10] introduces a dual-NeRF framework to reconstruct a human and scene from a single video by warping the human to a canonical space via SMPL estimates.
• iHuman [20] represents the human avatar as a mesh-embedded 3DGS framework.
• ExAvatar [17] represents the human avatar as a mesh-embedded tri-plane feature field, which is deformed by SMPL-based LBS and rendered by 3D Gaussian splatting.
• NEGSAvatar [32] introduces surface normal to 3DGS and proposes a locality-aware density control strategy.

4.2. Comparison Results

Quantitative comparison. 

Table 3. Comparison on NeuMan dataset [10].

Subject	Seattle			Citron
Method	PSNR	SSIM	LPIPS	PSNR	SSIM	LPIPS
NeuMan [10]	29.34	0.9794	0.0142	27.15	0.9727	0.0193
iHuman [20]	29.64	0.9820	0.0174	28.90	0.9761	0.0232
ExAvatar [17]	36.77	0.9918	0.0054	35.18	0.9873	0.0078
NEGSAvatar [32]	33.07	0.9842	0.0095	32.38	0.9760	0.0163
Ours	37.02	0.9923	0.0051	35.64	0.9874	0.0070
Subject	Bike			Jogging
Method	PSNR	SSIM	LPIPS	PSNR	SSIM	LPIPS
NeuMan [10]	26.24	0.9637	0.0304	26.56	0.9646	0.0255
iHuman [20]	27.62	0.9660	0.0347	26.86	0.9638	0.0291
ExAvatar [17]	32.71	0.9816	0.0115	32.88	0.9778	0.0135
NEGSAvatar [32]	30.63	0.9606	0.0263	30.10	0.9638	0.0217
Ours	33.37	0.9831	0.0101	32.61	0.9779	0.0129

summarizes the quantitative comparison results on NeuMan dataset. Our method consistently outperforms all baselines across the most sequences and metrics, which suggests the effectiveness of hierarchical geometry and hybrid appearance in improving reconstruction quality. Compared with ExAvatar, our method improves perceptual quality and temporal stability by anchoring the base appearance with an explicit UV texture map, while using a residual color branch to model dynamic shading variations. Compared with normal 3DGS-based methods, i.e., iHuman, our method significantly improves the visual quality by modeling high-frequency details and providing a more stable texture-anchored appearance.

Qualitative comparison. Figure 2 shows the qualitative comparison results on the NeuMan dataset. Overall, our results show clearer local details. ExAvatar produces plausible full-body reconstruction, but fine details are often weakened. In the first row, our method captures sharper facial details and is more consistent with the ground truth. In the second row, our method shows more accurate lighting and clearer patterns and wrinkles. In the third row, our method preserves richer texture patterns, which suggests the effectiveness of our hierarchical geometry encoding and hybrid appearance modeling. These visual observations match the quantitative gains in Table 3.

4.3. Ablation Study

We conduct ablation studies on the NeuMan dataset to validate the effectiveness of our proposed modules. Each variant removes one key component of our full model while keeping the remaining modules unchanged. Quantitative results are reported in

Table 4. Ablation study on NeuMan dataset [10].

Subject	Seattle			Citron
Method	PSNR	SSIM	LPIPS	PSNR	SSIM	LPIPS
w/o Hierarchy	36.10	0.9910	0.0063	34.70	0.9858	0.0085
w/o Color Residual	36.60	0.9917	0.0058	35.10	0.9866	0.0078
w/o UV Texture	36.77	0.9918	0.0054	35.18	0.9870	0.0078
Full	37.02	0.9923	0.0051	35.64	0.9874	0.0070
Subject	Bike			Jogging
Method	PSNR	SSIM	LPIPS	PSNR	SSIM	LPIPS
w/o Hierarchy	32.20	0.9805	0.0126	31.55	0.9754	0.0158
w/o Color Residual	32.90	0.9821	0.0111	32.18	0.9769	0.0140
w/o UV Texture	32.71	0.9816	0.0115	32.32	0.9772	0.0135
Full	33.37	0.9831	0.0101	32.61	0.9779	0.0129

.

Effect of Hierarchical Geometry Encoding. To validate the effectiveness of our hierarchical geometry encoding, we replace the multi-scale tri-plane pyramid with a single-scale tri-plane field. From Table 4, we can see that the hierarchical encoding consistently improves reconstruction quality across all sequences and metrics. This indicates that a single-scale representation is insufficient to capture both global structure and high-frequency surface details. Therefore, it is crucial to preserve the multi-scale structure for capturing fine details.

Effect of Neural Residual Color. To evaluate the effectiveness of our neural residual color branch, we remove the residual color branch and render using the UV texture only. We can see that the performance degrades significantly without the residual color branch, especially in terms of perceptual metrics. This indicates that the residual branch is important for modeling dynamic appearances and improving photorealism.

Effect of UV Texture Anchor. To evaluate the effectiveness of our UV texture anchor, we remove the UV texture and predict color solely from the neural residual branch. This variant is similar to a pure implicit color decoding strategy. We can see that the performance degrades without the UV texture anchor, especially in terms of perceptual metrics. This indicates that the explicit UV texture anchor is important for modeling dynamic appearances and improving photorealism.

4.4. Limitations and Future Work

While H²Avatar achieves state-of-the-art performance, it has certain limitations. First, H²Avatar assumes reliable per-frame SMPL-X parameters from off-the-shelf estimators. Inaccuracies in these estimates can propagate through our pipeline, causing artifacts such as misaligned geometry or blurred textures. While we partially mitigate this by optimizing SMPL-X parameters jointly during training, significant errors remain challenging. Future work can optimize the SMPL-X parameters during avatar reconstruction. Second, our hierarchical tri-plane pyramid (three levels up to 256 × 256 resolution) increases memory usage and training complexity compared to single-scale alternatives. This design was intentionally chosen to capture both global structure and high-frequency surface details, and the performance gains reported in our ablation study justify this trade-off for the current setting. However, this increased footprint may limit scalability to longer video sequences or higher-resolution avatars. Future work could explore more efficient multi-scale representations, such as sparse or adaptive feature grids, to reduce memory usage while preserving geometric detail. Third, our evaluation is currently limited to the NeuMan dataset, which contains short monocular videos of a single person walking in relatively simple scenes with static backgrounds. Though there are different sequences with different backgrounds, appearances, and lighting conditions, they are not representative of real-world scenarios. Future work should evaluate H²Avatar on more diverse and challenging datasets to thoroughly test its robustness and generalizability across different scenarios. Last but not least, we do not explicitly model the light, which limits the generalization of our method to novel lighting conditions. Future work can explore incorporating an explicit lighting model to improve the generalization ability of our method.

5. Conclusions

In this paper, we present H²Avatar, a novel framework for animatable avatar reconstruction from monocular video. To model detailed human geometry, we introduce a hierarchical geometry encoding strategy that leverages a multi-scale tri-plane pyramid to capture both global structure and fine surface details. In addition, we propose a hybrid appearance modeling strategy that leverages explicit and implicit texture information to capture both static and dynamic appearances. Compared with previous works, our method on the NeuMan dataset demonstrate that our H²Avatar achieves superior performance compared to other methods. Besides, ablation studies demonstrate the importance of our hierarchical geometry encoding and hybrid appearance modeling. In the future, we will continue to improve our method by incorporating more advanced deep learning techniques and exploring more challenging scenarios.