An Improved Eye Auxiliary Line Considering Boundary Margin, Iris, and Pupil for Portrait Drawing Learning Assistance System

Abstract

Portrait drawing is effective in cultivating artistic skills and visual understanding for a lot of people. Since drawing facial features with proper proportions, structures, and spatial relationships is hard for novices with no professional guidance, we developed the Portrait-Drawing Learning Assistance System, which offers auxiliary lines to assist in the drawing of portraits by acting as references; these auxiliary lines are extracted by applying OpenPose and OpenCV to a facial image. The drawing exactness assessment method using the Localized Normalized Cross-Correlation algorithm is also implemented to evaluate drawing accuracy. Unfortunately, previous experiments involving the use of the Portrait-Drawing Learning Assistance System by novices found that Localized Normalized Cross-Correlation scores for eyes are relatively low compared with others because of two drawbacks: (1) eye boundaries are often blurred by eyelids and eyelashes, which can make eyes bigger, and (2) the iris and pupil are not considered in auxiliary lines. In this paper, we propose an improved eye auxiliary line for the Portrait-Drawing Learning Assistance System to address these concerns. For the first drawback, we shift the lines for eye boundaries outside and add hatching (a series of short line segments) to them. For the second one, we expand region of interest of an eye and apply the Canny edge detector to capture its iris and pupil. In a preliminary within-subject evaluation, eight graduate students at Okayama University used the previous and proposed eye auxiliary lines in Procreate on an iPad. The average localized NCC scores increased for both eye regions under the tested conditions. These results suggest that the proposed auxiliary line may improve immediate eye-region reproduction accuracy; however, they do not constitute definitive validation of drawing-skill learning.

1. Introduction

Nowadays, portrait drawing is widely used to cultivate artistic skills and visual understanding of people. However, beginners often find difficulty in drawing facial features correctly if they cannot receive professional guidance to understand proper facial proportions, structures, and spatial relationships.

To support portrait-drawing learning by novices, we developed the Portrait-Drawing Learning Assistance System (PDLAS). PDLAS offers auxiliary lines to represent boundaries of facial features by applying OpenPose (version 1.7.0, CMU Perceptual Computing Lab, Pittsburgh, PA, USA) and OpenCV (opencv-python version 4.5.5.62, OpenCV team) programs to a target facial image. Learners are expected to draw portraits by referencing auxiliary lines, using the Procreate (version 5.4.11, Savage Interactive Pty Ltd., Hobart, Tasmania, Australia) drawing tool on an iPad. Since the lines are displayed on a separate layer on the tool, they can be removed instantly to generate the final portrait. In addition, the drawing exactness assessment method using the Localized Normalized Cross-Correlation (NCC) algorithm is implemented to evaluate the drawing accuracy for feedback.

In portrait drawing, capturing eyes accurately is sometimes the most difficult task for learners. The anatomical boundaries of eyes are rarely sharp. Instead, they are often blurred by eyelids, eyelashes, and shadows. As a result, previous experiments on the use of PDLAS by novices found that localized NCC scores for eyes are relatively low compared with the others because of two drawbacks: (1) eye boundaries are often blurred by eyelids and eyelashes, which can make eyes bigger, and (2) the iris and pupil are not considered in auxiliary lines. Drawback (1) led to an unintended result whereby users drew eyes smaller than their actual size. Drawback (2) caused mistakes in the drawing of irises and pupils inside eyes. Both of them decreased localized NCC scores for eyes.

Based on these observations, this study addresses the following research question: Can an eye-specific auxiliary line that represents the both boundary range and internal eye details improve novice learners’ eye-drawing accuracy in PDLAS? The objective of this study is to design such an improved eye auxiliary line and preliminarily evaluate its effectiveness using localized Normalized Cross-Correlation scores.

In this paper, we propose an improved eye auxiliary line for PDLAS to address the two drawbacks described above. Methodologically, the proposed method uses OpenPose keypoints to construct open eyelid curves and applies OpenCV-based image processing to extract iris and pupil contours from the eye region. For the first drawback, we shift the lines for eye boundaries outside and add hatching (a series of short, outward-pointing line segments) to them. These interventionswill help learners perceive the depth and volume of an eye more naturally, preventing them from just tracing a thin line. For the second one, we expand the Region of Interest (ROI) of an eye and apply the Canny edge detector to capture its iris and pupil. By combining flexible hatching boundaries with the Canny edge detector, the proposal can provide a further complete reference for learners.

For evaluation, we applied the proposal using the Procreate drawing tool on an iPad, used by eight graduate students at Okayama University, Japan, showing an increase in the average localized NCC scores for both eyes under this preliminary evaluation setting. Specifically, the average localized NCC score for a left eye increased from 0.41 to 0.56, and that for a right eye increased from 0.42 to 0.60. These results provide preliminary evidence that the improved eye auxiliary line may help users draw eyes more accurately in terms of size, shape, and internal details.

The rest of this paper is organized as follows: Section 2 reviews related work in the literature. Section 3 reviews the PDLAS framework. Section 4 presents the improved eye auxiliary line. Section 5 analyzes user results for evaluation. Finally, Section 6 concludes the paper and discusses future work.

These sections are organized to first identify the limitations of the previous system; then present the proposed method; and, finally, evaluate whether the proposed eye auxiliary line answers the research question described above.

2. Related Works in the Literature

In this section, we review related works on artistic image generation, line drawing, sketch processing, and portrait-related image analysis and clarify the position of the proposed method. Several studies have investigated automatic generation or transformation of artistic images. In [1], Mellor et al. proposed a framework for unsupervised learning of artistic styles by combining a painting agent with a neural style-transfer objective. In [2], Song et al. proposed a photo-to-sketch generation method using cycle consistency, without requiring paired photo–sketch training data. In [3], Chen et al. developed SketchyGAN, which converts simple sketches into realistic images. In [4], Li et al. proposed a universal style-transfer method based on feature transforms. These studies are useful for automatically generating or transforming artistic images. However, their main objective is for the system to produce an output image. In contrast, the Portrait-Drawing Learning Assistance System (PDLAS) does not aim to automatically generate a completed portrait. Instead, it provides auxiliary lines that learners can refer to while drawing by themselves. Line representation and stroke-based rendering are also related to this study. In [5], Kalnins et al. proposed coherent stylized silhouettes for 3D models, focusing on the temporal and spatial stability of generated outline strokes. In [6], Hertzmann reviewed stroke-based rendering methods that generate artistic images by placing individual strokes. These studies show the importance of line representation in non-photorealistic rendering and drawing-related applications. However, they mainly focus on visual rendering or automatic stroke placement. The proposed method uses line representation in a different way: the generated lines are not final artistic strokes but auxiliary information for novice learners. In particular, the open-spline eyelid curves and hatching segments are designed to guide learners in understanding the boundary range and structure of the eye region. Some studies have focused on extracting, cleaning, or simplifying structural lines from images. In [7], Xu et al. introduced an edge-preserving image deblurring method that removes unnecessary image variations while preserving important structures. In [8], Simo-Serra et al. developed a learning-based method for the cleaning of rough sketches into clearer line drawings. In [9], Fan et al. proposed user-guided line-drawing simplification, allowing users to control the level of detail in line drawings. These works are closely related to the present study because they address how visual structures can be represented by lines. However, their goals are different from ours. They mainly aim to generate, clean, or simplify line drawings, whereas our method constructs localized auxiliary lines for drawing practice. The proposed eye auxiliary line is designed to help users understand eye boundaries, eyelid volume, and internal eye details during portrait drawing. Controllable sketch generation and portrait image processing are also relevant to this study. In [10], Zhang et al. introduced a controllable sketch-generation framework that allows users to adjust parameters such as stroke thickness and detail levels. In [11], Wang et al. proposed a deep portrait completion method that reconstructs missing facial regions based on learned facial structures. These studies show the importance of controllability and facial-structure understanding in image-based drawing and portrait processing. However, they do not directly address how to provide facial-feature-level guidance for novice learners. In PDLAS, the system extracts facial keypoints and image features from a reference portrait and converts them into interpretable auxiliary lines for drawing support.

This work is also closely related to several drawing-guidance and portrait-sketch assistance systems. ShadowDraw [12] provides real-time drawing guidance by showing shadow-like suggestions retrieved from a large image database. Its strength is that it can interactively support freehand drawing without requiring users to follow a fixed drawing order. However, its guidance is mainly based on visually similar shadow images, and it does not explicitly analyze or evaluate specific facial parts, such as the eyes, nose, or mouth. Therefore, it is effective as general drawing support but less suitable for identifying and improving localized errors in portrait learning.

iCanDraw [13] supports users in drawing faces by providing step-by-step guidance and feedback. Its advantage is that it is designed for novice users and gives structured instructions during the drawing process. However, the guidance is mainly based on a predefined face-drawing procedure, and it does not focus on detailed auxiliary-line generation for a particular facial feature. In contrast, PDLAS generates auxiliary lines from the reference portrait using facial keypoints and image features, and it allows the drawing accuracy of each facial part to be evaluated quantitatively.

DualFace [14] provides portrait-drawing assistance by combining global and local guidance. Its strength is that it considers both the overall face structure and local facial components, which is useful for portrait-drawing support. However, its main objective is to assist in the whole portrait-drawing process, whereas the present study focuses on a more specific problem observed in previous PDLAS experiments: the low drawing accuracy of eye regions. Therefore, the proposed method is not a replacement of DualFace or other general portrait-guidance systems but a localized refinement of PDLAS to improve eye-drawing accuracy.

Compared with these existing systems, PDLAS is distinguished by two main characteristics. First, it generates interpretable auxiliary lines for each facial part from the reference portrait, so learners can use them as visual guides while drawing by themselves. Second, it evaluates drawing accuracy using localized NCC scores, which makes it possible to quantitatively identify weak facial regions. On the other hand, a limitation of the previous PDLAS was that the eye auxiliary line was not sufficiently detailed. The eye boundary, eyelid volume, iris, and pupil were not clearly represented, which could make it difficult for novice learners to draw the eye region accurately. To address this limitation, the present study proposes an eye-specific auxiliary line using open-spline eyelid curves, outward hatching segments, and Canny-based iris and pupil contours. Thus, the contribution of this paper is positioned as a localized drawing-support approach for improving eye-drawing accuracy within the PDLAS framework.

Compared with the above studies, the contribution of this paper is the design of an eye-specific auxiliary line for portrait-drawing learning. Previous image generation and sketch processing methods mainly focus on producing or modifying images automatically. In contrast, the proposed method focuses on supporting the learner’s drawing process. Specifically, it improves the previous PDLAS eye guide by using open-spline eyelid curves, outward hatching segments, and Canny-based iris and pupil contours. These components are introduced to address two specific problems observed in previous PDLAS experiments: ambiguous eye boundaries and the lack of internal eye details. Therefore, the proposed method is positioned as a localized drawing-support approach rather than a general sketch-generation or style-transfer method.

3. Review of Portrait-Drawing Learning Assistance System

In this section, we revisit the framework of the Portrait-Drawing Learning Assistance System (PDLAS) [15].

3.1. Software Architecture

PDLAS is designed to aid novices in sketching human faces using digital platforms—specifically, those featuring stylus support and illustration software. The prototype environment was built on an iPad Air (6th generation, Apple Inc., Cupertino, CA, USA) with Apple Pencil Pro (Apple Inc., Cupertino, CA, USA) and Procreate (version 5.4.11, Savage Interactive Pty Ltd., Hobart, Tasmania, Australia), as illustrated in Figure 1.

Figure 1. Procreate and Apple Pencil used in PDLAS. The screenshot was prepared by the authors using Procreate on an iPad for research documentation.

To facilitate the process of sketching a face, first, the algorithmic module produces auxiliary lines describing facial features. By integrating OpenPose and OpenCV, this module detects and traces essential facial anatomy—including the eyes, brows, nose, and lips—based on an input face photograph. Specifically, OpenPose can detect the spatial coordinates of structural keypoints that map out the human face. Our system transfers these markers into auxiliary lines for lips, eyes, nasal structure, jawline, and hair. Concurrently, library functions in OpenCV are utilized to identify secondary attributes like glasses, crown hair, and eyebrow boundaries, as shown in Figure 2. Then, the auxiliary lines are imported into Procreate in an independent layer, distinct from the user’s active drawing canvas.

Figure 2. The 70 facial keypoints identified by OpenPose [16].

3.2. Operational Workflow

Figure 3 illustrates the operational workflow of PDLAS.

Figure 3. Operational workflow of PDLAS. The arrows indicate the sequence of the drawing and evaluation process. The flowchart was created by the authors to summarize the drawing and evaluation process.

The drawing and evaluation process within PDLAS follows a structured, iterative loop designed to seamlessly integrate with the user’s creative workflow.

• Image Initialization: A user begins the drawing process by loading a facial photograph onto an iPad, setting the foundation.
• Algorithmic Generation: The algorithmic tool is executed to automatically extract and synthesize the structural auxiliary lines from the target image.
• Canvas Setup: The generated image of the auxiliary lines is then seamlessly imported into Procreate in a foundational background layer to anchor the sketch.
• Artistic Construction: Guided by the underlying auxiliary lines, the user freely draws a portrait, fully utilizing Procreate’s versatile brush engines and advanced layer capabilities.
• Artwork Export: Upon completion of the drawing, the user exports the final digital canvas for the subsequent evaluation.
• System Import: The exported artwork is fed into the drawing exactness assessment method.
• Precision Scoring: The system meticulously computes localized NCC scores for facial features, providing targeted quantitative feedback on the hair, eyebrows, lips, and nasal area.
• Targeted Refinement: Responding to this feedback, the user returns to the canvas to iteratively correct and redraw specific sections that yielded lower correlation scores.
• Re-evaluation Loop: The updated sketch is re-submitted to the drawing exactness assessment method, establishing a continuous cycle of targeted practice and measurable improvement until satisfactory accuracy is achieved.

3.3. Auxiliary Line Generation

The purpose of auxiliary lines is to guide users to better grasp the facial proportions and to assist them in precise localizations of individual traits during the painting process. The set of generated auxiliary lines comprises the following:

• A triangle, as frequently adopted in classical sketching pedagogies [17];
• Outlines defining the lips and eye regions, alongside the jawline contour;
• A set of three circular guides to characterize the nasal region; and
• Contour strokes delineating the boundaries of scalp hair, eyebrows, and eyeglasses.

The first three components are derived from specific keypoints located by OpenPose, whereas the latter elements are constructed utilizing the OpenCV library. Table 1 details the specific index mapping between the keypoints and their corresponding geometric guides.

Table 1. Correlation between auxiliary lines and OpenPose keypoints.

Guideline Classification	Associated Keypoint Data
Midline	Mean x-value of the 27th~30th nodes
Eye-alignment axis	y-values of the 36th, 39th, 42nd, and 45th nodes; x-values of the 36th, and 45th nodes
Horizontal guidelines placed above and below the ears	Mean y-values of the 37th, 38th, 43rd, and 44th nodes; x-values of the 3rd and 13th nodes
Primary structural inverted triangle	Vertex endpoints: 17th and 26th nodes; base midpoint: average of the 60th and 64th nodes
Eye regions	36th~41st and 42nd~47th nodes
Lips	60th~67th nodes
Jawline boundary	0th~16th nodes
	Radius: ${\displaystyle \frac{d}{4}}$ (where d denotes the gap between the 31st and 35th nodes)
Tri-circle nasal framework	Shared x-coordinate for all circle centers: identical to the midline axis;
	Central circle’s y-coordinate: ${\displaystyle \frac{31\mathrm{st}+35\mathrm{th}}{2}}$;
	Lateral circles’ y-coordinates: displaced by one radius length on respective sides
Cranial dome (upper semi-circle)	Origin point: 27th node; radius limit: linear distance spanning the 27th and 16th nodes

Table 1 was generated based on the 70-facial-keypoint definition of OpenPose and the auxiliary line-generation rules implemented in PDLAS. Each row summarizes which OpenPose keypoints are used by the algorithm to construct a specific auxiliary guide, such as the eye regions, lips, jawline, nasal framework, and cranial dome.

Figure 4 and Figure 5 illustrate a sample portrait photo and its auxiliary line image, respectively.

Figure 4. AI-generated portrait reference image obtained from Generated Photos [18]. The image does not represent a real person.

Figure 5. Auxiliary-line image generated by the authors from the AI-generated portrait reference image using OpenPose and OpenCV.

3.4. Drawing Exactness Assessment Method

To quantify sketch accuracy, we integrated the drawing exactness assessment method within PDLAS, using the localized NCC algorithm [19].

3.5. Normalized Cross-Correlation

This assessment framework employs Normalized Cross-Correlation to evaluate the drafted portrait against the original photograph. Localized NCC has been widely used for several visual-computing tasks. For example, it has been applied to template matching and image registration [20,21], stereo matching [22], face localization [23], and face matching [24]. These studies support the use of localized NCC as a pixel-level similarity metric for comparing corresponding image regions. Owing to its computational efficiency and robustness, correlation remains a widely used method for addressing matching inconsistencies [25]. From a technical perspective, the algorithm determines the optimal alignment by locating peak correlation values across the corresponding image matrices [26,27].

In this study, the localized NCC score is interpreted as a quantitative measure of pixel-level structural similarity between the reference eye region and the corresponding drawing region. It can reflect immediate differences in local shape, size, position, and grayscale structure under the tested conditions. However, localized NCC does not fully evaluate artistic quality, perceptual correctness, drawing style, or the learner’s overall drawing skill and learning outcome.

The mathematical expression of the localized NCC calculation is defined as follows [28]:

$$\mathrm{NCC}={\displaystyle \frac{{\sum }_{x,y}\left[T(x,y)·I(x+x^{\prime },y+y^{\prime })\right]}{\sqrt{{\sum }_{x,y}T{(x,y)}^2·{\sum }_{x,y}I{(x+x^{\prime },y+y^{\prime })}^2}}}$$

3.5.1. Feature-Specific Localized NCC Analysis

To provide detailed and accurate feedback, our approach computes the localized NCC score for specific facial components, such as the eyes, eyebrows, nose, and lips. For this purpose, OpenPose is utilized to identify 70 keypoints, which help generate rectangular bounding boxes around the features. The localized NCC metric is then calculated by comparing the original face photograph and the drawn image within each bounding box.

The procedure for determining the localized NCC score for each facial region is detailed below:

• Area Selection: We locate the specific section within the image that matches the target component under evaluation.
• Vector Extraction: Pixel data is gathered from this chosen area to build a feature vector. A pre-processing step is also included to reduce noise and random variations.
• Vector Normalization: To decrease the influence of different lighting or exposure conditions, the vector is adjusted to ensure it achieves a zero mean and a unit variance.
• Dot-Product Calculation: We evaluate the similarity by calculating the dot product between the two adjusted vectors.
• Score Normalization: The localized NCC metric is generated by dividing this dot product by the multiplied magnitudes of both normalized vectors.

3.5.2. Defining Boundaries for Facial Elements

To implement the region-specific evaluation method detailed above, we establish distinct rectangular zones around primary facial components. By isolating these areas, we ensure that the independent localized NCC analysis for a given part remains unaffected by neighboring image information.

The physical dimensions of these enclosures are directly determined by the associated OpenPose landmarks. To construct a proper rectangle around a specific facial structure, we extract its absolute highest and lowest positional values across both the horizontal (x) and vertical (y) axes.

Table 2 was generated based on the localized evaluation rules implemented in the drawing exactness assessment method. The OpenPose keypoints surrounding each facial region were selected, and their minimum or maximum x- and y-coordinates were used to define a rectangular bounding box. These bounding boxes were then used to extract the same local regions from the reference portrait and the user’s drawing for the Normalized Cross-Correlation calculation.

Table 2. Keypoint indices for localized bounding boxes.

Facial Region	Minimum x-Boundary	Maximum x-Boundary	Maximum y-Boundary	Minimum y-Boundary
Left eye	36th	39th	17th	28th
Right eye	42nd	45th	26th	28th
Left brow	17th	21st	Peak among 28th, 29th, 30th	Lowest among 17th, 22nd
Right brow	22nd	26th	Peak among 23rd, 24th, 25th	Lowest among 22nd, 26th
Lips	48th	54th	33rd	5th
Nasal area	39th	42nd	29th	33rd
Forehead hair	0th	16th	Top of image	Highest among 17th∼26th

Figure 6 provides a visual representation of the generated assessment boundaries overlaid on a subject.

Figure 6. Visual sample of localized bounding boxes generated by the authors using OpenPose keypoints for feature-specific Normalized Cross-Correlation evaluation. The blue box indicates the forehead hair region, while the other boxes indicate the localized facial feature regions used for evaluation.

3.6. Hair Texture Detection

To analyze the hair region, we apply texture feature analysis. This process uses specific algorithms to pull detailed surface patterns from both the reference photo and the user’s sketch. After isolating these traits, we measure how closely they match by applying the localized NCC technique.

We introduce a specific evaluation step for the hair area. Because a hair contains highly complex patterns, it remains one of the hardest parts of the portrait to score accurately. To address this challenge, we apply texture feature analysis to extract surface details from both the reference photo and the user’s sketch. These extracted patterns are then compared by computing the localized NCC. The step-by-step workflow for this texture analysis is outlined below:

• Grayscale Transformation: The original color portrait is first changed into a grayscale format to simplify the visual data.
• Boundary Detection: We apply the Canny edge detection algorithm to find the structural outlines within the image, which helps in locating the hair zone.
• Morphological Adjustments: Dilation and erosion techniques are used to clean up and improve the accuracy of the detected edges.
• Main Contour Extraction: The system identifies the biggest outline, assuming it represents the bulk of the hair, and generates a corresponding mask based on it.
• Hair Area Isolation: This mask is then merged with the grayscale picture to separate the hair from the rest of the image background.
• Texture Pattern Recognition: The cornerEigenValsAndVecs function is employed to calculate the eigenvalues of different image blocks, which effectively captures the structural texture of the hair.
• Localized NCC Similarity Calculation: Finally, we compute the localized NCC between the extracted texture maps of the original photo and the user’s artwork to determine their exact likeness.

Figure 7 and Figure 8 illustrate the visual results of this texture extraction process for the reference photograph and the user’s drawing respectively.

Figure 7. Processed reference image generated by the authors using OpenCV-based hair texture extraction.

Figure 8. Processed user sketch generated by the authors using OpenCV-based hair texture extraction.

Ultimately, the localized NCC score derived from the hair texture is combined with the matching scores of the other facial components. This total result is then presented to the user as comprehensive drawing feedback.

3.7. Low Normalized Cross-Correlation Score for Eyes

In previous experiments, we observed that users consistently achieved lower localized NCC scores in eye regions. This indicates a low similarity between the users’ drawings and the original portraits. Through analysis, we identified two main reasons for this result:

• Simplified Auxiliary Shapes: The previous auxiliary lines consisted of only simple linear strokes that formed a basic outline. This lack of structural guidance often misled users, causing them to draw the overall size of the eyes smaller than the actual proportions in the reference image.
• Absence of Internal Details: The original lines did not include anatomical details, such as the pupils or the thickness of the eyelids. Without these cues, users may struggle to capture these necessary features that define the eye’s character, leading to a significant loss of visual information.

To address these limitations, in the following section, we will propose an improved set of eye auxiliary lines. By expanding the guiding range and incorporating finer details, the new lines will assist users in capturing more realistic and accurate facial features.

4. Proposal of Improved Eye Auxiliary Line

In this section, we present an improved eye auxiliary line as a solution to the two drawbacks pointed out in the previous section.

4.1. Solutions to Two Drawbacks

The improvements of the eye auxiliary line consist of the following two proposals:

• Improvements of the auxiliary lines for eye boundaries by shifting them outside and adding hatching or a series of short line segments;
• Additions of auxiliary lines for irises and pupils by defining an ROI for eyes and applying the Canny edge detector to capture them.

For clarity, the baseline and proposed conditions are defined as follows. In the baseline condition, participants used the original PDLAS eye auxiliary lines generated by the previous algorithm. These lines represented the eye region mainly using simple eye-boundary guides based on OpenPose eye keypoints and did not include spline-based eyelid curves, outward hatching segments, or iris and pupil contours extracted from the original image.

In the proposed condition, the same PDLAS workflow was used, but the eye auxiliary lines were replaced with the proposed improved ones, which consist of open Catmull–Rom eyelid curves, outward hatching segments for boundary-range guidance, and Canny-based internal contours for the iris and pupil. The other facial auxiliary lines, drawing device, software, and evaluation procedure were kept unchanged between the two conditions.

Figure 9 compares the baseline and proposed eye auxiliary lines for the same face image. The baseline uses the original eye-boundary auxiliary lines generated by PDLAS, while the proposed method adds hatching around the eyelid boundaries and iris/pupil contours.

Figure 9. Comparison between the baseline and proposed eye auxiliary lines for the same AI-generated portrait reference image. Both auxiliary-line images were generated by the authors using OpenPose and OpenCV.

4.2. Improvements of Auxiliary Lines for Eye Boundaries

First, we present improvements of the auxiliary lines for eye boundaries, with a specific focus on the ocular region. The generation process of the improved lines consists of three components to accurately capture the structural and volumetric features of the eyes.

4.2.1. Generation of Eyelid Contour Curves

The human eye possesses a distinct anatomical structure characterized by sharp transitions at the inner and outer canthi (eye corners). To accurately capture this morphology, we extract facial keypoints using OpenPose and construct the eyelid contours using cubic interpolating splines.

For example, the boundary of a left eye comprises six discrete keypoints ($P_{36}$ to $P_{41}$ in Figure 2). The boundary of an eye should not be modeled as a single continuous, closed curve. This representation enforces smooth transitions and may obscure the sharp corners of the canthi. Therefore, the eye can be divided into two independent open curves. Figure 10 and Figure 11 illustrate the closed curve and open curve respectively.

Figure 10. Closed-curve representation generated by the authors to illustrate eyelid-curve construction. Points P1–P6 indicate the ordered control points used to construct the closed eyelid curve.

Figure 11. Open-curve representation generated by the authors to illustrate eyelid-curve construction. Points P1–P6 indicate the ordered control points used to construct the upper and lower eyelid curves separately.

Based on facial landmarks generated by OpenPose, each eye can be represented by six sequential keypoints. For the left eye, these keypoints are indexed from $P_{36}$ to $P_{41}$ in Figure 12. $P_{36}$ represents the outer canthus, and $P_{39}$ represents the inner canthus (tear-duct area). To faithfully reconstruct the overlapping structural nature of human eyelids instead of drawing a simplified geometric loop, we divide these keypoints into two distinct structural segments:

Figure 12. Eye-related OpenPose keypoints used for the construction of upper and lower eyelid curves. The numbers indicate the OpenPose facial-keypoint indices, and the colors indicate different facial feature groups in the OpenPose facial-keypoint definition. Only the eye-related region is shown. The illustration was prepared by the authors based on OpenPose’s 70-facial-keypoint definition.

• The upper eyelid is composed of four keypoints: $P_{36}-P_{37}-P_{38}-P_{39}$.
• The lower eyelid is composed of four keypoints: $P_{39}-P_{40}-P_{41}-P_{36}$.

To generate a smooth curve between any two adjacent keypoints denoted as $P_i$ and $P_{i+1}$, we employ a piecewise Catmull–Rom spline with a sliding window of four control points ($P_{i-1},P_i,P_{i+1}$, and $P_{i+2}$). The $C\left(t\right)$ coordinate on the curve at parameter $t\in [0,1]$ is calculated as follows:

$$\begin{array}{cc}\hfill C\left(t\right)={\displaystyle \frac{1}{2}}& [2P_i+(-P_{i-1}+P_{i+1})t\hfill \\ \hfill & +(2P_{i-1}-5P_i+4P_{i+1}-P_{i+2})t^2\hfill \\ \hfill & +(-P_{i-1}+3P_i-3P_{i+1}+P_{i+2})t^3.]\hfill \end{array}$$

(1)

The endpoints (e.g., $P_{36}$ and $P_{39}$) are carefully handled in our process. By utilizing the open-curve strategy, we apply an endpoint duplication method. For example, to draw the segment originating from the outer canthus ($P_{36}\to P_{37}$), the control-point array is padded by $(P_{36},P_{36},P_{37},P_{38})$. This mathematically ensures that the initial tangent vector points strictly towards the subsequent points, perfectly preserving the anatomical sharpness (edges) of the eye corners, thereby avoiding the blunt artifacts typically caused by closed-loop interpolation.

4.2.2. Generation of Normal Hatching Lines for Volumetric Representation

While the standard eyelid contour can provide a basic outline, it often leads learners to merely copy the exact lines instead of deeply understanding the drawing structure. To address this issue, we create an auxiliary boundary range as a part of the improved eye auxiliary line. By giving a drawing area instead of a single line, it is expected to effectively prevent learners from simply tracing auxiliary lines. To select a set of uniformly sampled points along the generated Catmull–Rom spline ($C\left(t\right)$), we determine the tangent vector ($T\left(t\right)$) by calculating the first derivative of the spline equation:

$$\begin{array}{cc}\hfill T\left(t\right)=C^{\prime }\left(t\right)={\displaystyle \frac{1}{2}}& \left[\right(-P_{i-1}+P_{i+1})\hfill \\ \hfill & +2(2P_{i-1}-5P_i+4P_{i+1}-P_{i+2})t\hfill \\ \hfill & +3(-P_{i-1}+3P_i-3P_{i+1}+P_{i+2})t^2.]\hfill \end{array}$$

(2)

As discussed before, although the basic eyelid contour can represent a simple 2D shape of an eye, providing single lines only might cause learners to simply trace them without deeply learning eye drawing [12,13]. To prevent this mindless copying, we introduce a drawing range to the auxiliary lines. This forms a guided area where learners can choose to draw their eye-boundary lines.

To generate this drawing range, we take a set of evenly spaced points along the Catmull–Rom spline ($C\left(t\right)$) [29] and calculate the tangent vector ($T\left(t\right)$) by taking the first derivative of the spline equation:

$$T\left(t\right)={\displaystyle \frac{d}{dt}}C\left(t\right).$$

(3)

Let a normalized tangent vector be $\widehat{T}=(T_x,T_y)$. The corresponding normal vector ($\widehat{N}$) is calculated to be perpendicular to the tangent. To ensure the guide lines are placed correctly, we apply an algorithmic constraint so that $\widehat{N}$ strictly points outward, which is away from the eye center, avoiding any incorrect inward directions.

Using this outward-pointing normal vector ($\widehat{N}$), we generate short auxiliary line segments originating from $C\left(t\right)$ and extending outwards with a fixed length (L) to define the drawing range:

$$P_{range}=C\left(t\right)+L·\widehat{N}.$$

(4)

In the implementation, the Catmull–Rom spline was sampled with 24 points for each segment between adjacent eye keypoints. For each upper or lower eyelid curve, eight hatching line segments were generated at uniformly sampled positions along the curve, excluding six sampled points near both ends to avoid unstable directions around the eye corners. The hatching length (L) was fixed at 7 pixels, and the gap ratio between the eyelid curve and the starting point of the hatching segment was set to 0.7. The eyelid contour curve was drawn with a thickness of 2 pixels, while the hatching segments were drawn with a thickness of 1 pixel.

The outward normal direction was determined using the eye center. First, the tangent vector was computed by the central difference of neighboring sampled points on the spline. Then, a normal vector was obtained by rotating the tangent vector by 90 degrees. If the dot product between the normal vector and the vector from the eye center to the sampled point indicated that the normal was directed inward, the normal direction was reversed. To avoid abrupt changes in hatching directions, an exponential moving-average smoothing factor of 0.75 and a maximum angle change of 15 degrees between adjacent normals were applied. In addition, a vertical direction bias of 0.55 was used so that the hatching segments were visually stable and naturally extended upward for the upper eyelid and downward for the lower eyelid. These parameter values were fixed before the evaluation and were used for all participants and all images.

Collectively, these outward-extending short auxiliary line segments build a specific drawing area. Instead of just showing a single path, the range will help learners understand the acceptable boundary for drawing eye boundaries. This approach can effectively break a habit of simple tracing and encourage them to practice their own line controls within a guided area.

Figure 13 shows the overlay comparison between the previous auxiliary lines and the proposed open eyelid curve, together with the outward hatching directions.

Figure 13. Overlay comparison of eyelid auxiliary lines generated by the authors using Python (version 3.7.8, Python Software Foundation, Wilmington, DE, USA). The arrows and text annotations were added using Microsoft PowerPoint for Microsoft 365 MSO (version 2606, Build 16.0.20131.20086, Microsoft Corporation, Redmond, WA, USA). The numbers 36–41 indicate the OpenPose facial-keypoint indices around the eye. The gray dashed line represents the previous auxiliary lines, while the blue solid curve represents the proposed open eyelid curve. The short blue segments indicate the outward hatching direction, and the gray center point is used to determine the outward normal direction.

4.3. Auxiliary Lines for the Iris and Pupil

Second, we present improvements of the auxiliary lines to draw irises and pupils inside eyes.

Region of Interest for Canny Edge Detection

Keypoints only cover limited points along the boundary of an eye. They completely miss important internal features, such as pupils and irises. For our previous eye auxiliary lines, we did not include them by extracting the specific details. This lack of guidance may lead to lower localized NCC scores in our experiments. Learners could not accurately draw the complete eye structure.

To solve this drawback, we present a new method of selecting the bounding box for an eye as the ROI [30]. First, we find the smallest box that covers the six eye keypoints: $(x_{min},y_{min},x_{max},y_{max})$. Then, we apply an uneven padding method to expand this ROI. This expansion ensures that the internal details, such as irises and pupils, are captured, along with the surrounding features [31]:

$$\begin{array}{cc}\hfill x_{min}^{\prime }& =x_{min}-{\mathrm{pad}}_x\hfill \end{array}$$

(5)

$$\begin{array}{cc}\hfill x_{max}^{\prime }& =x_{max}+{\mathrm{pad}}_x\hfill \end{array}$$

(6)

$$\begin{array}{cc}\hfill y_{min}^{\prime }& =y_{min}-{\mathrm{pad}}_{top}\hfill \end{array}$$

(7)

$$\begin{array}{cc}\hfill y_{max}^{\prime }& =y_{max}+{\mathrm{pad}}_{bottom}.\hfill \end{array}$$

(8)

Based on our tests, we set ${\mathrm{pad}}_x=60$ and ${\mathrm{pad}}_{top}=60$. We also made the bottom expansion much larger, at ${\mathrm{pad}}_{bottom}=120$ pixels. This large bottom expansion is specially designed to make sure we include under-eye areas.

The expanded ROI was clipped to the image boundaries before edge detection to avoid invalid extraction near image borders.

Before applying the Canny edge detector, the ROI was converted to grayscale, blurred with a 6 × 6 filter, and dilated using a 4 × 4 kernel. The Canny thresholds were fixed at 100 and 200. Detected edges were converted into black auxiliary lines on a white background and placed back into the original image. The same preprocessing steps, thresholds, and ROI padding values were used for all images.

The padding values (60 pixels for left/right, 60 pixels for the top, and 120 pixels for the bottom) were empirically selected to include the eyelid, eye boundary, and under-eye regions.

Finally, the Canny edge detector [32] is applied to the inside of this ROI to clearly extract the pupils and irises. By combining the mathematical splines, the short segment lines for the drawing range, and the internal details using the Canny edge detector, a more complete and highly accurate reference map for drawing is made as the eye auxiliary line, as shown in Figure 14.

Figure 14. Improved eye auxiliary line generated by the authors from an AI-generated portrait reference image using OpenPose and OpenCV.

These parameter values were empirically selected based on preliminary implementation tests, and their robustness across different parameter settings was not systematically evaluated in this study.

5. Evaluation

The evaluation consisted of two separate parts. The main experiment was a preliminary within-subject comparison involving eight participants who drew eyes using both the previous and proposed auxiliary lines. This experiment was used to compare the localized NCC scores between the two conditions. A separate experiment was conducted with four additional participants. These participants used only the proposed auxiliary line and did not complete the previous auxiliary-line condition. Therefore, their results are reported only as descriptive evidence of applicability and are not included in the before/after comparison.

All portrait reference images used in this experiment were generated by artificial intelligence and did not contain real human identities [18]. The participants drew portraits based on these AI-generated reference images using PDLAS with the proposed auxiliary lines.

The participants were graduate students at Okayama University who voluntarily agreed to participate in the experiment. They were not fine-arts students and had not received a long-term professional art education. However, detailed information regarding their prior casual drawing experience, digital drawing experience, and familiarity with Procreate was not systematically collected. Therefore, possible differences in these backgrounds should be considered when interpreting the results.

The main comparison was designed as a preliminary within-subject experiment. The same eight participants completed two drawing tasks: one using the previous eye auxiliary lines and the other using the proposed improved eye auxiliary lines. In both conditions, the same AI-generated face images were used as reference images so that the difference in localized NCC scores could be mainly attributed to the difference in the auxiliary lines rather than to differences in face images.

The same hardware, software, canvas size, brush type, and drawing procedure were used throughout the experiment. Participants were asked to draw the portrait while referring to the auxiliary lines, then export the completed drawing for evaluation.

Although the same participants were used in both conditions, the order of the conditions was not fully counterbalanced in this preliminary experiment. Therefore, possible learning effects, fatigue, and familiarity with Procreate cannot be completely excluded. Moreover, the participant group was small and homogeneous because all eight participants were graduate students from the same university. Accordingly, the observed score increases should be interpreted as preliminary within-sample evidence rather than definitive validation or generalizable evidence of effectiveness.

5.1. Drawing Process

The drawing process followed the workflow described in the previous section. After the participants completed their artworks on the iPad, their drawing results were exported to a PC to apply the drawing exactness assessment method. This evaluation aims to preliminarily examine whether the combined guidance using mathematical spline curves, outward hatching lines, and eye details extracted by the Canny edge detector is associated with higher drawing-similarity scores than the previous auxiliary lines. The present experiment evaluates these three components as one combined design and does not isolate the individual contribution of each component.

5.2. Normalized Cross-Correlation Scores of Drawing Results

In this subsection, we discuss the quantitative comparison of drawing results. The localized NCC score was used as the primary metric to evaluate the similarity between the drawing result and the original photograph. The primary goal of this evaluation is to investigate whether the improved eye auxiliary line helps improve the drawing result by focusing on eye regions.

Table 3 presents the results of the main within-subject comparison involving the eight participants who completed both the previous and proposed auxiliary-line conditions. For each participant, it provides a side-by-side comparison of scores before and after applying the proposal. This comparison allows us to examine the observed differences in eye-drawing similarity between the previous and proposed auxiliary-line conditions.

Table 3. Comparison of localized NCC scores before and after the proposal.

User	Left Eye			Right Eye
	Before	After	Improvement	Before	After	Improvement
No. 1	0.34	0.51	+0.17	0.49	0.54	+0.05
No. 2	0.33	0.54	+0.21	0.26	0.50	+0.24
No. 3	0.57	0.53	-0.04	0.54	0.66	+0.12
No. 4	0.54	0.57	+0.03	0.53	0.60	+0.07
No. 5	0.37	0.57	+0.20	0.32	0.45	+0.13
No. 6	0.27	0.60	+0.33	0.28	0.60	+0.32
No. 7	0.42	0.69	+0.27	0.47	0.71	+0.24
No. 8	0.43	0.46	+0.03	0.50	0.72	+0.22
Average	0.41	0.56	+0.15	0.42	0.60	+0.17
SD	0.10	0.07	0.13	0.12	0.10	0.10
Participants Improved	7/8 (87.5%)			8/8 (100%)

In addition to the main within-subject comparison with eight participants, we conducted a separate supplementary applicability check with four new participants as shown in Table 4. These four participants used only the proposed eye auxiliary line from the beginning and did not perform the baseline condition using the previous auxiliary lines. Therefore, no before/after comparison was available for them. Their localized NCC scores are presented only as descriptive results to examine whether the proposed system could be applied to additional users. These results are not used as evidence of improvement over the previous auxiliary lines.

Table 4. Descriptive localized NCC scores for the four additional participants who used only the proposed auxiliary lines.

User	Left Eye	Right Eye	Average of Both Eyes
No. 9	0.65	0.71	0.68
No. 10	0.49	0.49	0.49
No. 11	0.48	0.56	0.52
No. 12	0.54	0.59	0.57
Average	0.54	0.59	0.57

5.3. Result Analysis

In the main within-subject comparison, seven of the eight participants showed an increase in the left-eye localized NCC score, while all eight participants showed an increase in the right-eye score.

The four additional participants were analyzed separately because they used only the proposed system. Their scores provide a supplementary descriptive check of applicability but cannot demonstrate improvement over the previous auxiliary lines.

As shown in Table 3, the localized NCC scores for eyes increased for most participants after adopting the improved eye auxiliary line. Before the proposal, the average localized NCC score was 0.41 for the left eye and 0.42 for the right eye. After the proposal, the average localized NCC score increased to 0.56 for the left eye and 0.60 for the right eye.

The relatively low scores before the proposal indicate that participants had difficulty in capturing the correct proportions and shapes of the eyes when they were guided by simple, thin auxiliary lines, where they tended to draw the eyes smaller than the actual ones.

However, after the proposal, higher average localized NCC scores were observed. One possible interpretation is that providing a flexible boundary range rather than a single fixed line may have helped participants perceive the approximate spatial extent of the eye area. This guidance may have contributed to drawings with more appropriate eye sizes and positions. Nevertheless, the present experimental design cannot isolate the effect of the hatching component from the other proposed components or from possible order and practice effects.

These observed increases suggest that the proposed auxiliary line may help learners draw eyes with more appropriate sizes, positions, and internal details. However, the localized NCC results indicate only increased localized image similarity and immediate drawing accuracy under the tested conditions. They should not be interpreted as evidence of improved artistic quality, general drawing skill, or long-term learning. Also, the current experiment evaluated the open-spline eyelid curves, outward hatching, and Canny-derived iris/pupil contours together as one combined auxiliary-line design. Therefore, the observed changes in the localized NCC scores cannot be attributed to any single component. The present results should be interpreted as an evaluation of the complete proposed design rather than of the individual effects of the three components. Since the main within-subject comparison involved only eight participants, the results should be regarded as preliminary evidence rather than a general conclusion. A larger-scale experiment with more participants and an ablation study will be necessary to further examine the proposed method.

Furthermore, while the overall trend shows an improvement, the degree of improvement varies among participants. In the following section, we will discuss specific cases to understand how the proposed eye auxiliary line influenced drawing behaviors in detail.

It should be emphasized that in this work, the educational goal of PDLAS is to assist novice learners with auxiliary information about facial proportions, eye structure, and local details during drawing practice. By helping users produce drawings that are more similar to the reference image, the system may support their drawing practice and sense of accomplishment. However, these possible educational benefits were not directly evaluated in the present experiment. The present evaluation is concerned only with immediate guided drawing performance and does not assess learning retention, transfer to unguided drawing, or long-term skill development.

5.4. Representative Examples

To further interpret the quantitative localized NCC results, we visually analyzed representative participants with the highest and lowest improvements. This analysis was used to illustrate typical changes in drawing results and was not intended as a formal case-study research strategy.

5.4.1. User No. 6: Representative Example with the Highest Improvement

In this case study, we visually analyzed the drawing result of User No. 6, showing the highest improvement of the localized NCC scores. The score for the left eye was increased from 0.27 to 0.60, and the score for the right eye was increased from 0.28 to 0.60.

Figure 15 shows the drawing results by User No. 6 before and after the proposal. In the before image in Figure 15a, the eyes are drawn significantly smaller than the actual ones. The shapes of the eyes look flat, lacking the natural volume and depth of the eyelids. Furthermore, important internal details like pupils and irises are completely missing.

Figure 15. Drawing results by User No. 6 with the highest improvement. The drawings were produced by the participant using Procreate on an iPad based on AI-generated portrait reference images [18] and are shown in anonymized form with informed consent.

In contrast, in the after image in Figure 15b, a clear visual improvement was observed. Guided by the hatching auxiliary lines, the eyes were drawn with more accurate sizes and proportions. The newly introduce hatching successfully encouraged this learner to recognize the spatial range of eye boundaries, not as single lines. In addition, thanks to the Canny edge detector, the details of pupils and irises were clearly drawn. The eyes in the after image give a better volume, a more realistic structure, and complete inner features. This visual comparison is consistent with the quantitative increase in the localized NCC scores for this participant and illustrates how the proposed guidance may have supported the drawing of eye size and internal details. However, this individual example should be interpreted descriptively and does not, by itself, establish the effectiveness of the proposal.

To further visualize the local drawing errors, Figure 16 shows a heatmap-based error visualization of the eye regions for User No. 6. The heatmaps were generated from the pixel-wise differences between the reference image and the user’s drawing result based on the grayscale values used in the localized NCC calculation. Regions with stronger red intensity indicate larger differences relative to the reference image, while white or pale regions indicate smaller differences.

Figure 16. Heatmap-based error visualization of the eye regions for User No. 6. Regions with stronger red intensity indicate larger pixel-wise differences relative to the reference image, while white or pale regions indicate smaller differences.

As shown in Figure 16a, before using the proposed method, relatively strong red regions appeared around the eyelid boundaries and the internal eye structures, especially around the areas of the iris and pupil. This indicates that the user had difficulty capturing both the correct eye size and the internal details. In contrast, Figure 16b shows that the red error regions became smaller and lighter after using the proposed eye auxiliary lines. This visual result supports the quantitative improvement in the localized NCC scores and suggests that the proposed method helped reduce local drawing errors in the eye regions.

5.4.2. User No. 3: Representative Example with Lowest Improvement

The drawing results by User No. 3 showed the lowest average score increase. Notably, the localized NCC score for the left eye was slightly decreased from 0.57 to 0.53.

Figure 17 displays the drawing results by User No. 3. Unlike the ones by User No. 6, User No. 3 already demonstrated a relatively good grasp of eye proportion and volume in the before image in Figure 17a. The drawn eyes were not undersized, and the baseline scores were already significantly higher than the average.

Figure 17. Drawing results by User No. 3. The drawings were produced by the participant using Procreate on an iPad based on AI-generated portrait reference images [18] and are shown in anonymized form with informed consent.

In the after image in Figure 17b, the overall drawing style becomes softer. The left eye, in particular, seems slightly less clarified compared to that in the before image, which can explain this minor drop in the localized NCC score.

This case may suggest a “ceiling effect” of the proposal. Hatching-based auxiliary lines can be highly effective for beginners who struggle to perceive the depth and volume of facial features. However, for learners who already possess some drawing skills and a good spatial understanding, the hatching will not provide additional benefits. This guide might slightly disrupt their established drawing habits. Therefore, we conclude that the improved eye auxiliary line is beneficial for novice learners who tend to draw facial features too small or flat.

6. Conclusions

This paper presented an improved eye auxiliary line for the Portrait-Drawing Learning Assistance System (PDLAS) to support novice learners during eye-drawing practice. The proposal combines clear mathematical splines for eye outlines, additional special hatching lines for drawing range, and auxiliary lines for pupils and irises inside eyes.

The main preliminary within-subject comparison involved eight participants who completed both the previous and proposed auxiliary-line conditions. In this comparison, the average localized NCC score increased from 0.41 to 0.56 for the left eye and from 0.42 to 0.60 for the right eye. These results suggest that the proposed auxiliary line may improve immediate eye-drawing accuracy under the tested conditions. However, the experiment did not evaluate whether this improvement was retained over time or transferred to drawings produced without auxiliary lines. A separate supplementary applicability check was also conducted with four additional participants who used only the proposed system. Because they did not complete the baseline condition, their results are interpreted only descriptively and are not considered evidence of improvement over the previous auxiliary lines.

This study bridges portrait-drawing education, computer vision, image processing, and educational technology by converting facial features into interpretable auxiliary lines for novice learners. Key challenges included how to represent ambiguous eye boundaries, extract iris and pupil details, and evaluate drawing accuracy with localized NCC-based measures. Regarding the possible target users of the proposal, beyond novice art learners, it can be useful for children, adult self-learners, and persons at some drawing-skill levels. In addition, the proposed system may facilitate accessibility-oriented drawing practice for users with mild visuospatial or perceptual-processing difficulties who have difficulty in face-proportion estimation, spatial alignment, or translating visual information into drawing actions and may be applied to repeated drawing practice for seniors who have cognitive difficulties to enhance their sense of achievement and enrich daily lives. These potential applications were not examined in the present study and should be investigated in future work with appropriate participant groups and evaluation settings.

However, this evaluation has several limitations. The number of participants was small, and all participants were graduate students from the same university. In addition, learning effects and familiarity with Procreate were not fully controlled. The current experiment evaluated open-spline eyelid curves, outward hatching, and Canny-derived iris and pupil contours as one combined design. Therefore, the individual contribution of each component could not be determined. Furthermore, the evaluation mainly measured immediate drawing similarity using localized NCC scores and did not examine long-term learning retention, unguided drawing performance, or qualitative evaluations by drawing experts. In addition, the robustness of the empirically selected parameters, including the spline sampling, hatching length, ROI padding, and Canny thresholds, was not systematically evaluated.

Therefore, the results should be regarded as preliminary, which will require future work that involves a larger and more diverse participant group with a more rigorous experimental design, as well as a study to evaluate the effects of the spline curves, hatching segments, and iris/pupil contours individually. We will also improve the nose auxiliary lines where the localized NCC score is also lower than others due to the three-dimensional feature of a nose.