CB-OWL-ViT: A Multimodal Cost-Effective Framework for Contagious Disease Monitoring

Abstract

The rapid spread of diseases like COVID-19 highlights the need for adaptable monitoring systems to support public health measures such as mask compliance and social distancing. This study presents the CB-OWL-ViT framework: a Cluster-Based Open-World Localization Vision Transformer for mask detection and social distance estimation. It incorporates homography-based distance estimation for effective deployment with monocular cameras. The innovative integration of open-world vision-language detection with a clustering-based strategy enhances mask-wearing assessments, enabling adaptability without retraining. Evaluations on datasets including Kaggle, Roboflow, and a new dataset from the University of Waterloo show that CB-OWL-ViT improves mask detection precision by 0.37 and F1-score by 0.2 compared to the baseline. The homography module achieves a Mean Absolute Error of 0.1116 in distance estimation, and real-world tests demonstrate a recall of 0.98 for detecting noncompliance in the “Without Mask” class. This framework is a practical solution for large-scale disease monitoring across various settings.

1. Introduction

1.1. Research Background and Motivations

The outbreak of COVID-19 demonstrated how rapidly a global pandemic can disrupt daily social interactions, mental health, and economic stability. COVID-19 affected millions of people worldwide, resulting in numerous deaths, which in turn forced governments to establish strict measures and implement regulations, such as social distancing and face mask mandates, to prevent the rapid spread of the virus [1,2,3,4]. Since disparities in vaccine distribution, particularly in low-income countries, highlight the serious threat posed by pandemics [5], the use of face masks has become critical. Failure to comply with mask mandates, even after vaccination, remains a major factor in the further spread of the virus [6,7]. Consequently, the development of new tools is necessary to overcome these challenges and mitigate potential adverse outcomes [8].

Machine Learning (ML) has revolutionized various domains by enabling intelligent, data-driven solutions to complex problems [9,10]. Notably, ML-based methods have become cornerstone tools for monitoring and enforcing social distancing and face mask mandates during the pandemic [11]. ML-driven surveillance systems have been widely employed to detect human faces and identify individuals in crowded environments [12,13,14], thereby supporting the enforcement of social distancing and mask mandates [8]. Since 2020, there has been a rapid advancement in ML-based methodologies in pandemic monitoring and response. However, a critical limitation of many existing methods is their lack of flexibility and scalability for low-cost, large-scale deployment [2,3,8]. In this context, flexibility refers to the ability of mask detection models to adapt to new scenarios without requiring additional retraining. On the other hand, low cost implies that social distance estimation models can be deployed with minimal computational and financial resources. This raises two key research questions (RQs):

• RQ 1: How can a mask detection model be developed to robustly adapt to new scenarios without requiring additional retraining?
• RQ 2: How can a social distance estimation model be developed to cost-effectively operate with various types of cameras, including but not limited to stereo cameras?

In this paper, social distancing monitoring denotes systems that assess compliance with distancing rules by monitoring such practices and by computing or estimating interpersonal (physical) distances; accordingly, social distance estimation is also used to refer to the computation or estimation of these distances.

1.2. Problem Statements and Objectives

The overall problem can be divided into two key components: (i) face mask detection and (ii) social distance estimation.

Face Mask Detection: Existing methods for detecting face masks are commonly trained to classify a fixed set of predefined classes. These approaches struggle to generalize to new scenarios, such as detecting specific types of masks or monitoring particular population groups (e.g., children). On the other hand, object detection algorithms typically require large and diverse datasets to perform reliably [15]. Therefore, the present study develops a multimodal approach based on the Open-World Localization Vision Transformer (OWL-ViT) [16], which quickly adapts to new conditions without requiring additional training. However, OWL-ViT is unreliable in complex scenarios, such as assessing mask-wearing compliance. To address this limitation, the present study proposes integrating a clustering algorithm into the OWL-ViT framework to enhance its performance in complex scenarios.

Social Distance Estimation: Most existing methods for estimating the distance between individuals rely on depth sensors, such as stereo or RGB-D cameras, to obtain accurate real-world measurements. However, depth-sensing cameras are generally more expensive than monocular ones due to additional hardware components and calibration requirements. This added complexity not only increases hardware costs but also necessitates more advanced image-processing algorithms, rendering their deployment rather impractical for cost-sensitive applications. This issue becomes particularly critical in tasks such as condition monitoring during contagious disease outbreaks, where large-scale implementation across countries, including low-income regions, is essential. To address this challenge, this study proposes to integrate two well-established methods in the framework: (i) metric depth estimation and (ii) homography transformation, which are compatible with any camera, devising a cost-effective and practical solution for large-scale deployment.

1.3. Research Contributions

The key research contributions of the present study are summarized as follows:

• Cluster-Based Integration of Open-World Vision-Language Detection: This study introduces Cluster-Based OWL-ViT (CB-OWL-ViT), a novel integration of OWL-ViT with a clustering-based decision mechanism that aggregates evidence from multiple semantically related textual queries. This strategy enhances robustness in mask-wearing assessment under ambiguous visual conditions, without modifying the underlying detection architecture or requiring retraining.
• Practical Use of Established Distance Estimation Techniques: Homography transformation and metric depth estimation, two established geometric and learning-based methods, are incorporated to enable cost-effective social distance estimation using monocular cameras.
• Flexible and Cost-Effective Monitoring Framework: The proposed framework enables rapid adaptation to new monitoring scenarios through query redefinition and supports deployment with a wide range of camera types, addressing practical constraints in large-scale public health monitoring.
• New Dataset and Empirical Evaluation: This study introduces a newly curated dataset collected at the University of Waterloo to empirically evaluate the proposed framework under controlled indoor conditions.
• Comprehensive Comparison with State-of-the-Art (SOTA) Closed-Set Detectors: Extensive experiments compare the proposed framework with recent YOLO variants, highlighting the trade-offs between closed-set supervised detectors and open-world Vision-Language Models (VLMs).

The structure of the rest of this manuscript is organized as follows. Section 2 surveys the prior research relevant to the present study. The core methodology and the proposed framework are detailed in Section 3. The experimental design and implementation settings are described in Section 4. Section 5 outlines the obtained results and provides an in-depth evaluation and discussion of these outcomes. Lastly, Section 6 concludes the paper with final remarks and outlines potential avenues for future investigation.

2. Related Work

The literature on social distancing monitoring and face mask compliance using deep learning approaches has grown substantially. In this context, this section provides an overview of the key research developments in three subsections: (i) object detection, (ii) depth estimation, and (iii) condition monitoring during contagious disease outbreaks.

2.1. Object Detection

Object detection is a computer vision technique that identifies and localizes objects within images or videos. Object detection, often implemented with deep learning models, is used to achieve precise recognition and classification of objects across various applications. Based on the input modality, object detection models can be classified into two main categories: (i) unimodal models and (ii) multimodal models [16], as described below.

2.1.1. Unimodal Object Detection Models

These models rely solely on visual data for object localization and classification within a scene. Based on their detection architecture, unimodal models are generally classified into two categories: (i) one-stage models and (ii) two-stage models.

One-stage models, designed for efficient object detection, directly predict object classes and bounding boxes in a single pass. Recent SOTA one-stage models, such as YOLOv10 [17], YOLOv12 [18], and RT-DETR [19,20,21], represent the latest advancements in real-time object detection by offering a balanced trade-off between accuracy and efficiency. These models are used in scenarios that require both accuracy and low-latency inference.

On the other hand, two-stage models perform localization and detection using separate networks. Despite the dominance of one-stage models, two-stage models remain valuable for face mask detection because the paucity of richly annotated datasets necessitates separate training of the localization and detection modules. Typically, the face localization module is trained on general human face datasets, whereas the mask detection module is trained on specific datasets for face mask detection. One common strategy employs the Dual-Shot Face Detector (DSFD) [22] for facial localization, followed by a Convolutional Neural Network (CNN)-based model for mask detection. A significant limitation of DSFD, however, is its large size (∼458 MB) and its high complexity (∼20 million parameters).

2.1.2. Multimodal Object Detection Models

Multimodal object detection models are designed to integrate multiple input modalities, such as images and text, to achieve richer, more robust feature representations. These models originate from pioneering language-supervision approaches [23,24], which leverage large-scale vision-language pre-training to align visual and textual features in a shared embedding space. By combining semantic understanding of natural language with visual perception, multimodal object detection enables more flexible and generalizable recognition. For example, such models can detect objects specified through textual prompts, making them suitable for open-vocabulary or zero-shot detection tasks. Here, textual input refers to queries provided to the model, such as “a photo of a human” or “a photo of a child.”

Among language supervision-based object detection models, OWL-ViT and YOLO-World [25] are notable examples. The primary difference between these models lies in their architectures: OWL-ViT is based on the Vision Transformer (ViT) [26], whereas YOLO-World relies on CNNs; as a result, OWL-ViT benefits from superior generalization capabilities.

OWL-ViT formulates object detection as a patch-level matching problem between image embeddings and text embeddings, enabling direct localization of objects specified by natural language queries without requiring task-specific retraining. Prior studies have shown that OWL-ViT achieves competitive performance on simple and moderately complex scenes, particularly for coarse object categories such as humans, vehicles, and everyday objects, making it suitable for scenarios that require semantic flexibility. However, OWL-ViT exhibits notable limitations in highly cluttered or fine-grained scenarios. In particular, its detection accuracy tends to degrade when objects differ only by subtle visual cues, such as distinguishing between masked and unmasked faces, or when multiple visually similar instances coexist in close proximity. These limitations stem from the model relying on global semantic alignment rather than fine-grained local discrimination.

Consequently, while OWL-ViT demonstrates relatively reliable performance in simple detection settings (e.g., detecting humans or faces in unconstrained environments), its effectiveness decreases in complex mask-wearing compliance scenarios that require precise discrimination under occlusion, pose variation, and background clutter. To overcome this limitation, the present study proposes CB-OWL-ViT, which enhances the robustness of OWL-ViT in such challenging scenarios by incorporating a clustering-based refinement strategy while preserving its open-vocabulary and training-free detection capability.

2.2. Depth Estimation

Monocular Depth Estimation (MDE) is a crucial component in numerous modern applications, providing essential depth information from a single image. This capability is particularly valuable in domains such as 3D reconstruction, navigation, and autonomous driving, where precise spatial awareness is vital. MDE enhances the ability to interpret the depth and distance of objects, significantly improving decision-making in complex environments. Recent advancements in MDE, driven by models based on discriminative architectures like BEiT [27] and DINOv2 [28], as well as generative approaches such as Stable Diffusion [29], underscore the growing significance of accurate depth prediction across traditional and emerging fields [30,31].

Furthermore, Depth Pro [32] employs a proposed metadata-free method to predict metric disparity maps, utilizing multi-scale ViTs to achieve superior performance. In this model, the focal length, a vital parameter for metric prediction from monocular camera output, is inferred from the image using a ViT-based network. The metric distance is computed by predicting the canonical inverse depth and then scaling it by the horizontal focal length and image width. In this context, the present study proposes using Depth Pro to estimate metric disparity maps, which are then used within the pinhole camera model to compute real-world distances.

2.3. Condition Monitoring During Contagious Disease Outbreaks

Table 1. Related work in condition monitoring during contagious disease Outbreaks.

Reference	Year	Mask Monitoring	Distance Monitoring	Dataset Availability
[33]	2020	SSD	Fixed assumed height (165 cm)	Custom (×)
[34]	2021	ResNet-50 + DSFD	Stereo camera	RMFD [35] (✓)
[36]	2021	YOLOv2	Stereo camera	Custom (×)
[37]	2022	Faster R-CNN	Stereo camera	Kaggle [38] (✓)
[39]	2022	Faster R-CNN	Stereo camera	Custom (×)
[40]	2023	YOLOv4	Stereo camera	Custom (×)
[8]	2024	MobileNetV2 + DSFD	DeepSORT with Hungarian algorithm	RMFD [35] (✓)
[41]	2024	Custom	-	Kaggle [38] (✓)

presents an overview of related work on condition monitoring during contagious disease outbreaks. Most prior studies have focused on stereo camera setups and developed models restricted to predefined classes. In contrast, the CB-OWL-ViT proposed in the present study leverages natural-language supervision and a clustering algorithm to adapt to novel scenarios while maintaining robust performance. Moreover, the proposed social distance estimation method is compatible with any camera type, enabling large-scale deployment, unlike costly stereo-camera systems.

The study in [33] utilizes the Single-Shot MultiBox Detector (SSD) for mask-wearing detection and estimates interpersonal distance using a triangle-similarity method, which assumes a standard human height of 165 cm. Although this method uses a monocular camera, its accuracy is limited by the assumption of a uniform height across individuals. It also relies on a custom dataset that is not publicly available. The study in [34] uses a stereo-camera configuration and the Euclidean distance formula to monitor social distancing and employs a ResNet-50 [42] combined with the DSFD [22] for mask-wearing detection; the ResNet-50 model was trained on the Real-World Masked Face Dataset (RMFD) [35].

The studies in [36,40] use YOLOv2 [43] and YOLOv4 [44], respectively, to monitor mask-wearing status; both models are trained on custom, nonpublic datasets. For social distancing monitoring, they use stereo camera systems and Euclidean distance computation. In the same line of research, the studies in [37,39] employ Faster R-CNN [45] for mask-wearing detection and, together with stereo cameras, use the Euclidean distance formula to estimate social distance. In addition, the study in [37] uses the widely exploited, publicly available Kaggle dataset [38].

The study in [8] combines MobileNetV2 [46] with DSFD for mask-wearing detection and employs a DeepSORT-based [47] method integrated with the Hungarian algorithm [48] for social distancing monitoring. It tracks individuals across consecutive frames by assigning unique identifiers that leverage both appearance features and bounding-box positions, using the Hungarian algorithm to find the optimal assignment. Although it uses a monocular camera, the approach incurs a high computational cost and provides no quantitative evaluation.

Finally, the study in [41] exclusively focuses on mask-wearing detection. It proposes LFMD-YOLO, an approach that integrates attention-based modules (i.e., C3E and MECAPF) with a modified feature fusion neck (i.e., E-BiFPN) into a lightweight YOLO framework to enhance detection accuracy and computational efficiency.

3. Methodology and Framework

This section presents the proposed methodology and framework, illustrated in Figure 1, comprising two modules: (i) an object detection module and (ii) a social distance estimation module.

Formally, given an input image $I\in {\mathbb{R}}^{3\times H\times W}$, the objective is to detect a set of individuals $\mathcal{P}={\left\{p_k\right\}}_{k=1}^K$, where each individual $p_k$ is characterized by (i) a bounding box $b_k\in {\mathbb{R}}^4$, (ii) a semantic label $y_k\in \{\mathrm{With}\mathrm{Mask},\mathrm{Without}\mathrm{Mask},\mathrm{Neutral}\}$, and (iii) a physical location estimate ${\mathbf{X}}_k\in {\mathbb{R}}^2$.

3.1. Object Detection Module

As illustrated in Figure 1, the object detection module takes two input types (i.e., an image and a set of textual queries) and can rapidly adapt to new conditions by redefining the queries, thereby eliminating the need for additional training data. The architecture of the Text Encoder, Image Encoder, Linear Projection, and MLP Head follows OWL-ViT, initialized with its pre-trained weights. These components are detailed as follows:

• Text Encoder consists of a 12-layer Transformer architecture incorporating Multi-Head Self-Attention (MHSA) layers with eight attention heads, a Feed-Forward Neural Network (FFNN) with a hidden dimension of 2048, layer normalization, residual connections, and positional embeddings. It employs a model-wide hidden dimension of 512 and generates query embeddings as output, extracted from the final End-Of-Sequence (EOS) token, for use in open-vocabulary object detection tasks.
• Image Encoder employs the standard ViT-B/32 architecture to generate image embeddings from input image patches. It has a hidden dimension of 768 and produces token representations with a sequence length of 576.
• Linear Projection comprises a single fully connected layer without any nonlinear activation function. It maps each output token representation from the ViT-B/32 model, which has a hidden dimension of 768, to an image embedding. This mapping facilitates direct comparison with query embeddings for object classification.
• MLP Head comprises three fully connected layers configured for bounding box prediction. The input dimension is 768, corresponding to the hidden dimension of the ViT-B/32 model. The network includes two intermediate layers with a dimension of 2048, each followed by a ReLU activation function. The final output layer produces four numerical outputs corresponding to the bounding box coordinates—$\left[x_{\mathrm{center}},y_{\mathrm{center}},\mathrm{width},\mathrm{height}\right]$. Consistent with standard Transformer design conventions, the intermediate layers have a dimensionality of 2048. The output layer applies no nonlinear activation function, and the predictions are intentionally biased toward the centers of the respective image patches to enhance localization accuracy.

The encoders form a contrastive learning pipeline that leverages large-scale image–text pairs to bring the embeddings of matching pairs close together while increasing the distance between those of nonmatching pairs.

Defining effective queries is a significant challenge, as even minor wording variations can lead to substantial performance differences. For example, in the task of human face detection, queries such as “a photo of a human face” and “a photo of a face” appear nearly identical but may retrieve results with markedly different levels of accuracy and relevance. This illustrates how the precise formulation of a query directly shapes the scope and quality of the retrieved outputs. Figure 2 addresses this issue by providing queries categorized by their semantic relevance to mask-wearing status.

In the standard application of OWL-ViT to determine whether a person is wearing a mask, a single query is typically used per object. For instance, the queries “a photo of a human’s masked face” and “a photo of a human’s face” are used to identify individuals with and without masks, respectively. However, this structure often results in low accuracy. In certain instances, when a person is facing away from the camera and thus provides no visual cues about mask use, the model still incorrectly classifies the instance as class “a photo of a human’s face.” Additionally, it fails to consistently recognize a human’s masked face across different scenes.

In this regard, the present study proposes a strategy to enhance the performance of OWL-ViT in complex scenarios. This strategy is explained in Algorithms 1–3. Algorithm 1 performs the detection step. In this phase, the input image is preprocessed, and OWL-ViT is applied with both mother and target queries. The output of this algorithm consists of bounding boxes, confidence scores, and corresponding labels. Algorithm 2 groups detections into mother clusters. First, detections are filtered using Non-Maximum Suppression (NMS). Then, each target query result is assigned to the appropriate mother cluster based on the Intersection-over-Union (IoU) metric. To ensure reliability, only the highest-scoring target in each class is included in each cluster. Finally, Algorithm 3 determines the class of each mother cluster. The classification is based on the distribution of target queries within the cluster. If the cluster contains a target belonging to the set of “With Mask” queries, the class is set to “With Mask.” Otherwise, the scores of “Without Mask” and “Neutral” queries are compared, and the class is set to “Without Mask” or “Neutral” accordingly.   

Algorithm 1: Detection algorithm

Require: Image, Queries, OWL-ViT Model Result: boxes, scores, labels Preprocessing: $imag{e}_{RGB}\leftarrow$ Convert image to RGB $queries\leftarrow$ {mother queries, target queries} Detection: $outputs\leftarrow$ OWL-ViT$(imag{e}_{RGB},queries)$ Extract $boxes,scores,labels$ from $outputs$ Return boxes, scores, labels

Algorithm 2: Clustering algorithm

Algorithm 3: Classification algorithm

To rigorously define the object detection task, let $\mathcal{I}:={\mathbb{R}}^{3\times H\times W}$ denote the image space. The objective is to map an input image $I\in \mathcal{I}$ and a set of text queries $\mathcal{Q}$ to a set of detections $\mathcal{D}$. The query set is partitioned into mother queries ${\mathcal{Q}}_{\mathrm{M}}$ (general object categories, e.g., “human face”) and target queries ${\mathcal{Q}}_{\mathrm{T}}$ (specific attributes, e.g., “masked face”). The detection function $f_{\det }:I\times \mathcal{Q}\to \mathcal{D}$ outputs a set of detections $\mathcal{D}={\left\{\left(b_i,s_i,c_i\right)\right\}}_{i=1}^N$, where $b_i\in {\mathbb{R}}^4$ denotes bounding box coordinates, $s_i\in [0,1]$ is the confidence score, and $c_i\in \mathcal{Q}$ is the predicted query label. Detection confidence filtering $s_i\ge {\tau }_{\mathrm{conf}}$ with ${\tau }_{\mathrm{conf}}=0.1$ and IoU thresholding ${\tau }_{\mathrm{IoU}}=0.5$ are employed to balance recall and precision while ensuring robustness to minor localization errors.

Let ${\mathcal{D}}_{\mathrm{M}}$ and ${\mathcal{D}}_{\mathrm{T}}$ denote detections from mother and target queries, respectively. The clustering function $\mathcal{M}:{\mathcal{D}}_{\mathrm{T}}\to {\mathcal{D}}_{\mathrm{M}}$ assigns a target detection $b_t$ to a mother detection $b_m$ if

$$\mathrm{IoU}\left(b_m,b_t\right)={\displaystyle \frac{|b_m\cap b_t|}{|b_m\cup b_t|}}\ge {\tau }_{\mathrm{IoU}}.$$

(1)

Assumptions include sufficient spatial overlap between queries, independence of textual prompts, and bounding box tangency to the human body. This aggregation acts as an ensemble across queries, reducing false positives and stabilizing predictions in the face of semantic ambiguity, similar to bagging in ensemble learning.

Now for each cluster, let ${\mathit{S}}_{\mathrm{mask}}$, ${\mathit{S}}_{\mathrm{nomask}}$, and ${\mathit{S}}_{\mathrm{neutral}}$ denote confidence score sets for each semantic group. The final label $\widehat{y}$ is classified as

$$\widehat{y}=\left\{\begin{array}{cc}\mathrm{With}\mathrm{Mask},\hfill & \mathrm{if}{\mathit{S}}_{\mathrm{mask}}\ne \varnothing ,\hfill \\ \mathrm{Without}\mathrm{Mask},\hfill & \mathrm{if}\sum _{s\in {\mathit{S}}_{\mathrm{nomask}}}s>\sum _{s\in {\mathit{S}}_{\mathrm{neutral}}}s,\hfill \\ \mathrm{Neutral},\hfill & \mathrm{otherwise}.\hfill \end{array}\right.$$

(2)

This modular design improves flexibility—detection can be adapted to different human-related queries (e.g., “a photo of a child” or “a photo of a worker”), clustering reduces false positives by focusing only on mother queries, and classification incorporates auxiliary queries to refine mask recognition.

The proposed strategy enhances class accuracy by incorporating auxiliary queries. For example, combining “a photo of a ply mask” with “a photo of a human’s masked face” improves the accuracy of mask detection. Additionally, introducing a neutral class enables the model to distinguish more accurately without the mask class, thereby further reducing false positives. Such a strategy is also effective for identifying individuals wearing a specific type of mask (e.g., those wearing only black masks).

Clustering strategies have been widely explored in related domains, including general object detection, 3D object detection, and multi-object association, where they are primarily employed to group spatially or geometrically consistent object proposals. For example, the study in [49] proposed a size-adaptive clustering mechanism for 3D object detection in point clouds, in which clustering radii are dynamically inferred to accommodate objects with varying spatial scales. Similar clustering-based approaches have also been adopted in multi-object detection and tracking to enhance robustness against occlusion and noisy observations.

However, these methods focus on object-level grouping or geometric association during the detection stage. In contrast, the clustering strategy introduced in the present study operates at the decision level within an open-world VLM framework. Rather than generating or refining object proposals, the proposed clustering aggregates multiple query-level predictions produced by OWL-ViT and leverages their consensus to derive a stable semantic decision for mask-wearing assessment.

This design is particularly suited to open-world scenarios, where the detector may produce inconsistent or ambiguous predictions due to partial occlusion, varying viewpoints, or open-vocabulary uncertainty. By clustering semantically related detections and applying a decision rule over the resulting groups, the proposed strategy improves robustness without introducing additional training or modifying the underlying VLM.

3.2. Social Distance Estimation Module

For social distancing monitoring, two complementary monocular approaches are employed: (i) homography-based planar distance estimation and (ii) metric depth estimation. These approaches allow the use of various camera types, rather than being limited to stereo systems.

To identify social distancing violations, let $\mathcal{P}={\left\{p_k\right\}}_{k=1}^K$ be the detected individuals; then, pairwise distances $d\left(p_i,p_j\right)$ are computed and compared against ${\delta }_{\mathrm{safe}}$ to identify violations.

3.2.1. Homography Transformation

Assuming all ground contact points lie on a planar surface, the mapping between world and image coordinates is defined by a homography matrix H [50]. To clarify the workflow, consider Figure 3, which illustrates the homography setup. In this approach, each person’s ground contact point (typically the bottom-center of the bounding box) is projected onto the ground plane $\pi$ to compute real-world coordinates.

In the experiments, small 1 m-by-1 m ground markers are placed primarily for scale calibration, enabling accurate computation of distances in meters. These markers are used only during the calibration stage to determine the scale factor between image-space and world-space coordinates. Once calibrated, the homography transformation can estimate distances between individuals without requiring additional markers for each scene.

Formally, the homography matrix H transforms image coordinates ${\left[xy1\right]}^T$ to real-world coordinates ${\left[XY1\right]}^T$ on the planar ground, while its inverse $H^{-1}$ allows mapping from world coordinates back to the image plane. The system can accommodate multiple cameras by computing separate homography matrices $H_i$ for each view, ensuring consistent distance estimation across different perspectives. This geometric approach leverages explicit scale information and, therefore, typically outperforms learning-based methods that infer distances implicitly. Finally, the homography transformation is written as

$$s{\left[\begin{array}{ccc}x& y& 1\end{array}\right]}^T=H{\left[\begin{array}{ccc}X& Y& 1\end{array}\right]}^T,$$

(3)

where s denotes the scale factor arising from the projective nature of the transformation.

The homography matrix H is estimated using at least four point correspondences via the Direct Linear Transformation (DLT) algorithm, solved by singular value decomposition for numerical stability. Assumptions include planar ground geometry ($Z=0$), static camera pose, and known metric scale obtained through calibration (e.g., a chessboard).

Let $u_i={\left[u_{i,x}{u}_{i,y}1\right]}^T$ denote the bottom-center contact point of the i-th bounding box. The corresponding world coordinate is recovered as ${\mathbf{X}}_i=H^{-1}{u}_i/{\left(H^{-1}{u}_i\right)}_3$. The distance between individuals is then $d\left({\mathbf{X}}_i,{\mathbf{X}}_j\right)={\parallel {\mathbf{X}}_i-{\mathbf{X}}_j\parallel }_2$.

Considering pixel noise $\Delta u\sim \mathcal{N}\left(0,{\sigma }_{u^2}\right)$, the propagated position uncertainty satisfies $\mathrm{Var}\left(\Delta \mathbf{X}\right)\approx {\left(H^{-1}\right)}^T{\Sigma }_u{H}^{-1}$, yielding bounded distance errors (typically $\pm 0.1$ m under calibration), thereby ensuring reliable violation detection.

3.2.2. Metric Depth Estimation

This method first locates the center of each detected bounding box. Let $\left(x_1,y_1\right)$ and $\left(x_2,y_2\right)$ denote the pixel coordinates of the top-left and bottom-right corners of the bounding box enclosing the detected person. The center of the bounding box, with pixel coordinates $\left(u_{c,x},u_{c,y}\right)$, is then computed as

$$u_{c,x}={\displaystyle \frac{x_1+x_2}{2}},u_{c,y}={\displaystyle \frac{y_1+y_2}{2}}.$$

(4)

Using this center point, the depth value is extracted from a predicted depth map as

$$Z=D\left(u_{c,x},u_{c,y}\right),$$

(5)

where $D\left(u_{c,x},u_{c,y}\right)$ denotes the depth value, expressed in meters, at the pixel coordinates $\left(u_{c,x},u_{c,y}\right)$, and Z denotes the estimated distance from the camera to the detected person.

The transformation from 2D pixel coordinates to 3D real-world coordinates is then carried out using the intrinsic parameters of the camera, represented by the matrix K as

$$K=\left[\begin{array}{ccc}{f}_x& 0& c_x\\ 0& f_y& c_y\\ 0& 0& 1\end{array}\right],$$

(6)

where $\left(f_x,f_y\right)$ denote the focal lengths, in pixels, along the horizontal and vertical axes, and $\left(c_x,c_y\right)$ denote the pixel coordinates of the principal point, typically located near the image center.

The pinhole camera model [50] defines the projection from 3D coordinates $\left(X,Y,Z\right)$ in the camera coordinate system to 2D image coordinates $\left(u_{c,x},u_{c,y}\right)$ as

$$u_{c,x}=f_x·{\displaystyle \frac{X}{Z}}+c_x,u_{c,y}=f_y·{\displaystyle \frac{Y}{Z}}+c_y.$$

(7)

By rearranging this relationship, the 3D point in the camera coordinate system can be expressed as

$${\left[\begin{array}{ccc}X& Y& Z\end{array}\right]}^T={\left[\begin{array}{ccc}{\displaystyle \frac{\left(u_{c,x}-c_x\right)Z}{f_x}}& {\displaystyle \frac{\left(u_{c,y}-c_y\right)Z}{f_y}}& Z\end{array}\right]}^T,$$

(8)

where $\left(X,Y,Z\right)$ denotes the reconstructed 3D coordinates of the point in the camera frame (i.e., the person in the real world), $\left(u_{c,x},u_{c,y}\right)$ specifies the image location of the point (i.e., the bounding box center), and $\left(f_x,f_y,c_x,c_y\right)$ are the intrinsic parameters extracted from the camera calibration matrix K. The pinhole camera model treats image formation as the projection of 3D points onto a 2D image plane through an infinitesimal aperture, thereby neglecting optical effects such as lens distortion [51].

Next, the Depth Pro model [32], a Transformer-based monocular depth estimation, is applied to estimate the Z-coordinate. This model estimates canonical inverse depth, which is subsequently scaled by the horizontal focal length and image width to derive metric distances without requiring explicit camera intrinsic parameters.

Finally, the distance between two individuals is computed as the Euclidean distance between their reconstructed 3D coordinates, consistent with methodologies employed in prior studies.

4. Experimental Results

This section presents an experimental evaluation of the proposed methods for mask-wearing detection and social distance estimation. The datasets and experimental setup used for training, validation, and test-time inference are described. The evaluation metrics are then introduced, followed by a detailed comparison of the performance of the object detection and social distance estimation modules under real-world conditions. Supplementary Videos accompanying the results are available online at the following link: https://drive.google.com/drive/folders/1-HAUO15P6_e38_6HZSAq-bl-9mE4Nzzh?usp=sharing (accessed on 5 October 2025).

4.1. Setup

Training and Validation Datasets: This study utilizes two publicly accessible datasets sourced from Kaggle [38] and Roboflow [52].

Table 2. Class distribution in training and validation subsets of mask-wearing datasets.

Dataset	Subset	Instances	Class Label	Objects
Kaggle [38]	Training	641	W/O Mask (0)	462
			W/ Mask (1)	2255
	Validation	113	W/O Mask (0)	96
			W/ Mask (1)	346
Roboflow [52]	Training	1300	W/O Mask (0)	1047
			W/ Mask (1)	1894
	Validation	350	W/O Mask (0)	283
			W/ Mask (1)	438

presents a detailed description of these datasets, and Figure 4 illustrates representative samples from them. Moreover, because this study collects its own dataset for test-time inference, the Kaggle and Roboflow datasets are exclusively utilized for training and validation.

Test-Time Inference Dataset: This experiment was conducted at the University of Waterloo. The data was collected using a Google Pixel 7 Smartphone and a Logitech HD Webcam C615. During data collection, two subjects represented individuals moving at different distances and exhibiting various facial orientations, either wearing masks or remaining unmasked. Figure 5 illustrates an overview of the experimental setup used for data collection in the laboratory environment, and Figure 6 provides representative sample frames of the collected experimental dataset.

Table 3. Class distribution in collected dataset.

Dataset	Subset	Instances	Class Label	Objects
Smartphone	Test	334	W/O Mask (0)	196
			W/ Mask (1)	334
Webcam	Test	330	W/O Mask (0)	223
			W/ Mask (1)	229

presents a detailed description of the collected dataset.

4.2. Evaluation Metrics

This study utilizes precision, recall, F1-score, and Average Precision (AP) as evaluation metrics for object detection models. To establish a foundation for these metrics, it is necessary first to reiterate the concepts of True Positives (TP), False Positives (FP), and False Negatives (FN).

TP is a correct detection (or prediction of the class label) where the predicted box matches a ground-truth object with an IoU above a specified threshold (e.g., $\mathrm{IoU}\ge 0.5$). FP is an incorrect detection where the predicted box has low IoU with any ground-truth object (e.g., $\mathrm{IoU}<0.5$), the class label is wrong, or the object does not exist. Lastly, FN is a misdetection where a ground-truth object is present, but no box with sufficient IoU is predicted. Next, precision, recall, and F1-score metrics are shown as

$$\mathrm{P}={\displaystyle \frac{\mathrm{TP}}{\mathrm{TP}+\mathrm{FP}}},\mathrm{R}={\displaystyle \frac{\mathrm{TP}}{\mathrm{TP}+\mathrm{FN}}},\mathrm{F}1\text{−}\mathrm{score}={\displaystyle \frac{2·\mathrm{P}·\mathrm{R}}{\mathrm{P}+\mathrm{R}}}.$$

(9)

IoU threshold selection is inherently task-dependent. In this study, because a pre-trained OWL-ViT model is used, which was originally trained on datasets with annotation standards different from those in the present study, a lower threshold of 0.2 is adopted. This choice provides a more lenient localization criterion, better accommodating the annotation discrepancies.

Furthermore, AP and mean AP (mAP) are the primary evaluation metrics for object detection models. The former measures precision averaged across different recall levels and is computed per class, reflecting the detection performance for that class; the latter is the mean AP across all classes and serves as the standard metric for assessing overall performance on multi-class datasets.

For the social distance estimation methods, the evaluation metrics include Mean Absolute Error (MAE), Mean Squared Error (MSE), and Root Mean Square Error (RMSE).

4.3. Object Detection Module

Table 4. Evaluation results for CB-OWL-ViT and OWL-ViT across datasets.

Dataset	Model	Class	mAP@		P	R	F1
			0.50	0.20
Kaggle	CB-OWL-ViT	1	0.3214	0.5317	0.88	0.58	0.70
		0	0.2414	0.2792	0.39	0.67	0.49
		All	0.2814	0.4054	0.64	0.62	0.59
	OWL-ViT	1	0.3807	0.4693	0.66	0.56	0.61
		0	0.2180	0.2316	0.10	0.91	0.17
		All	0.2994	0.3504	0.38	0.73	0.39
Roboflow	CB-OWL-ViT	1	0.0780	0.5198	0.83	0.60	0.70
		0	0.3218	0.5570	0.62	0.81	0.71
		All	0.1999	0.5384	0.73	0.71	0.70
	OWL-ViT	1	0.0580	0.3976	0.53	0.53	0.53
		0	0.4131	0.6064	0.20	0.98	0.33
		All	0.2356	0.5020	0.36	0.75	0.43
Smartphone	CB-OWL-ViT	1	0.2884	0.8703	1.00	0.87	0.93
		0	0.5047	0.7886	0.79	0.99	0.88
		All	0.3966	0.8295	0.89	0.93	0.91
	OWL-ViT	1	0.5846	0.6407	0.94	0.65	0.77
		0	0.3775	0.5068	0.49	1.00	0.65
		All	0.4811	0.5737	0.71	0.82	0.71
Webcam	CB-OWL-ViT	1	0.5478	0.6761	0.97	0.69	0.81
		0	0.8271	0.8605	0.70	0.98	0.82
		All	0.6875	0.7683	0.83	0.84	0.81
	OWL-ViT	1	0.3281	0.3716	0.94	0.37	0.53
		0	0.5622	0.5932	0.55	1.00	0.71
		All	0.4452	0.4824	0.75	0.68	0.62

Values shown in bold represent the best (highest) performance across all models and classes for each dataset.

compares the performance of CB-OWL-ViT and OWL-ViT on different datasets. The CB-OWL-ViT model employs the queries described in Figure 2 and the strategy explained in Algorithms 1–3, whereas OWL-ViT uses only two queries: (i) “a photo of a human’s face” for the “Without Mask” class (i.e., class 0) and (ii) “a photo of a human’s masked face” for the “With Mask” class (i.e., class 1). These queries are selected by trial and error.

As shown in the Table 4, the CB-OWL-ViT model significantly outperforms the standard OWL-ViT across the Kaggle, Roboflow, Smartphone, and Webcam datasets, primarily due to its multi-query mechanism that conditions class predictions on multiple facial patterns (e.g., nose, chin). This approach mitigates the bias of the standard OWL-ViT, which often misclassifies class 0 based on minimal facial cues, leading to high FP rates.

The cluster-based method increases precision by 0.26, 0.37, 0.18, and 0.08 in the Kaggle, Roboflow, Smartphone, and Webcam datasets, respectively. Despite decreases of approximately 0.11 in the Kaggle dataset and 0.04 in the RoboFlow dataset, recall improved by 0.11 and 0.16 in the Smartphone and Webcam datasets, respectively. Overall, the CB-OWL-ViT shows significant improvement, achieving an average F1-score increase of around 0.2 across all datasets and consistent gains in mAP@0.20, underscoring its superior performance.

4.4. Social Distance Estimation Module

The performance of the proposed social distance estimation methods using real-world test data is shown in Figure 7. The metric depth estimation method (blue line) exhibits greater variability and larger deviations from the ground truth (black line) than the homography transformation method (green line), which closely follows the ground truth across both data sources. The corresponding error plots further highlight these differences, showing that the homography transformation method consistently yields lower errors, particularly for the smartphone data, indicating its robustness across diverse data sources for social distance estimation.

Additionally, quantitative analysis of the errors in social distance estimation is presented in

Table 5. Comparison of social distance estimation methods.

Dataset	Method	MAE	MSE	RMSE
Smartphone	Metric Depth Estimation	0.5300	0.3799	0.6163
	Homography Transformation	0.1116	0.0189	0.1376
Webcam	Metric Depth Estimation	0.2429	0.1076	0.3280
	Homography Transformation	0.1364	0.0450	0.2122

Values shown in bold represent the best (lowest) performance across all methods for each dataset.

. In both data sources, the homography transformation method outperforms other methods across all measures, underscoring its ability to estimate social distance. It is noteworthy that this method has a very low computational cost and can be implemented with any type of camera. Consequently, this strategy facilitates the large-scale deployment of social distance estimation models by minimizing costs and overhead while maintaining compatibility with a wide range of cameras.

5. Evaluation and Discussion

5.1. Comparative Analysis with Other Object Detection Models

This subsection presents a comprehensive comparison between the proposed framework and the recent SOTA object detection models, such as the latest variants of YOLO, which, to the best of the authors’ knowledge, have not been previously employed for face mask detection.

Table 6. Comparison of SOTA Object Detection Models and CB-OWL-ViT on Kaggle Dataset.

Model	Class	mAP@		P	R	F1
		0.50	0.20
YOLOv10m	1	0.8951	0.9177	0.90	0.92	0.91
	0	0.7514	0.7514	0.89	0.77	0.83
	All	0.8232	0.8346	0.89	0.85	0.87
YOLOv10l	1	0.9042	0.9134	0.89	0.92	0.91
	0	0.8216	0.8436	0.91	0.85	0.88
	All	0.8629	0.8785	0.90	0.89	0.89
YOLOv12m	1	0.9028	0.9140	0.91	0.92	0.92
	0	0.7894	0.7894	0.91	0.80	0.85
	All	0.8461	0.8517	0.91	0.86	0.88
YOLOv12l	1	0.8912	0.8969	0.91	0.90	0.91
	0	0.7361	0.7460	0.94	0.76	0.84
	All	0.8136	0.8215	0.92	0.83	0.87
RT-DETRl	1	0.9153	0.9298	0.84	0.94	0.89
	0	0.8137	0.8348	0.80	0.85	0.82
	All	0.8645	0.8823	0.82	0.90	0.85
CB-OWL-ViT	1	0.3214	0.5317	0.88	0.58	0.70
	0	0.2414	0.2792	0.39	0.67	0.49
	All	0.2814	0.4054	0.64	0.62	0.59

Values shown in bold represent the best (highest) performance across all models and classes.

,

Table 7. Comparison of SOTA Object Detection Models and CB-OWL-ViT on Roboflow Dataset.

Model	Class	mAP@		P	R	F1
		0.50	0.20
YOLOv10m	1	0.0553	0.9114	0.81	0.93	0.87
	0	0.3093	0.6825	0.91	0.69	0.78
	All	0.1823	0.7969	0.86	0.81	0.83
YOLOv10l	1	0.0579	0.9007	0.84	0.91	0.87
	0	0.3154	0.7383	0.89	0.75	0.81
	All	0.1866	0.8195	0.86	0.83	0.84
YOLOv12m	1	0.0656	0.9267	0.81	0.94	0.87
	0	0.2659	0.7325	0.95	0.73	0.83
	All	0.1657	0.8296	0.88	0.84	0.85
YOLOv12l	1	0.0631	0.9110	0.79	0.94	0.85
	0	0.2483	0.6839	0.97	0.69	0.80
	All	0.1557	0.7974	0.88	0.81	0.83
RT-DETRl	1	0.0631	0.9110	0.79	0.94	0.85
	0	0.2483	0.6839	0.97	0.69	0.80
	All	0.1557	0.7974	0.88	0.81	0.83
CB-OWL-ViT	1	0.0780	0.5198	0.83	0.60	0.70
	0	0.3218	0.5570	0.62	0.81	0.71
	All	0.1999	0.5384	0.73	0.71	0.70

Values shown in bold represent the best (highest) performance across all models and classes.

,

Table 8. Comparison of SOTA Object Detection Models and CB-OWL-ViT on the smartphone dataset.

Model	Class	mAP@		P	R	F1
		0.50	0.20
YOLOv10m	1	0.9278	0.9969	0.95	1.00	0.97
	0	0.5497	0.8345	0.95	0.84	0.89
	All	0.7387	0.9157	0.95	0.92	0.93
YOLOv10l	1	0.9137	0.9999	0.99	1.00	0.99
	0	0.5770	0.8928	0.99	0.89	0.94
	All	0.7454	0.9464	0.99	0.95	0.97
YOLOv12m	1	0.9339	1.0000	0.98	1.00	0.99
	0	0.5097	0.8736	0.94	0.88	0.91
	All	0.7218	0.9368	0.96	0.94	0.95
YOLOv12l	1	0.9054	1.0000	0.89	1.00	0.94
	0	0.5363	0.7959	1.00	0.80	0.89
	All	0.7208	0.8980	0.94	0.90	0.91
RT-DETRl	1	0.9219	0.9997	0.92	1.00	0.96
	0	0.6169	0.8821	0.95	0.88	0.92
	All	0.7694	0.9409	0.94	0.94	0.94
CB-OWL-ViT	1	0.2884	0.8703	1.00	0.87	0.93
	0	0.5047	0.7886	0.79	0.99	0.88
	All	0.3966	0.8295	0.89	0.93	0.91

Values shown in bold represent the best (highest) performance across all models and classes.

and

Table 9. Comparison of SOTA Object Detection Models and CB-OWL-ViT on the webcam dataset.

Model	Class	mAP@		P	R	F1
		0.50	0.20
YOLOv10m	1	0.5048	0.9909	0.97	0.99	0.98
	0	0.6720	0.6720	0.97	0.67	0.80
	All	0.5884	0.8314	0.97	0.83	0.89
YOLOv10l	1	0.5371	0.9910	0.99	0.99	0.99
	0	0.7875	0.8201	0.99	0.82	0.90
	All	0.6623	0.9055	0.99	0.91	0.94
YOLOv12m	1	0.5195	0.9865	0.99	0.99	0.99
	0	0.7256	0.7941	0.98	0.79	0.88
	All	0.6225	0.8903	0.98	0.89	0.93
YOLOv12l	1	0.5575	0.9910	0.97	0.99	0.98
	0	0.6752	0.6846	0.95	0.69	0.80
	All	0.6164	0.8378	0.96	0.84	0.89
RT-DETRl	1	0.5619	0.9952	0.94	1.00	0.97
	0	0.7843	0.7936	0.94	0.79	0.86
	All	0.6731	0.8944	0.94	0.90	0.91
CB-OWL-ViT	1	0.5478	0.6761	0.97	0.69	0.81
	0	0.8271	0.8605	0.70	0.98	0.82
	All	0.6875	0.7683	0.83	0.84	0.81

Values shown in bold represent the best (highest) performance across all models and classes.

report the performance comparison of multiple models trained on the Kaggle, Roboflow, Smartphone, and Webcam datasets, respectively, against the proposed CB-OWL-ViT model. Since these models were trained directly on the datasets mentioned earlier, they perform better on this dataset than the proposed CB-OWL-ViT model. Among the evaluated models, YOLOv12l achieves the highest precision of 0.92. RT-DETRl attains the best recall with a value of 0.90, whereas YOLOv10l demonstrates the most balanced trade-off between precision and recall, yielding an F1-score of 0.89. On the Roboflow dataset, both YOLOv12l and RT-DETRl achieve the highest precision, each reaching 0.88. In contrast, the proposed CB-OWL-ViT model achieves a significantly higher recall of 0.81 for class 0, highlighting its superior ability to detect non-compliant behavior during pandemic conditions, successfully identifying 81% of such cases. In terms of F1-score, YOLOv12m achieves the best overall performance with a score of 0.85.

It is worth noting that certain images from the Kaggle dataset are also present in the Roboflow dataset. As a result, since the models were trained on the Kaggle dataset, their evaluation across both datasets may not constitute a fully unbiased comparison. On the smartphone dataset, the proposed CB-OWL-ViT model achieves perfect precision of 1.00 for class 1. Nevertheless, when considering all classes, YOLOv10l attains the highest overall precision of 0.99 and an F1-score of 0.97. The proposed model remained competitive, obtaining an F1-score of 0.91. On the Webcam dataset, YOLOv10l delivers the strongest performance across precision (0.99), recall (0.91), and F1-score (0.94). However, for class 0 specifically, the proposed CB-OWL-ViT model achieves the highest recall of 0.98, underscoring its effectiveness in detecting high-risk cases. Overall, across the various datasets, YOLOv10l demonstrates the most consistent generalization among models trained on predefined classes. In contrast, the CB-OWL-ViT model specifically excelled over all other models in detecting class 0.

5.2. Comparative Analysis with Other SOTA Models

Table 10. Comparison of SOTA Models and CB-OWL-ViT on all datasets.

Datest	Model	Class	mAP@		P	R	F1
			0.50	0.20
Kaggle	ResNet50 + DSFD [34]	1	0.5839	0.7216	0.77	0.88	0.82
		0	0.4270	0.4794	0.75	0.62	0.68
		All	0.5055	0.6005	0.76	0.75	0.75
	MobileNetV2 + DSFD [8]	1	0.5591	0.6843	0.74	0.86	0.79
		0	0.2885	0.3013	0.63	0.47	0.54
		All	0.4238	0.4928	0.69	0.66	0.67
	CB-OWL-ViT	1	0.3214	0.5317	0.88	0.58	0.70
		0	0.2414	0.2792	0.39	0.67	0.49
		All	0.2814	0.4054	0.64	0.62	0.59
Roboflow	ResNet50 + DSFD [34]	1	0.0985	0.7170	0.74	0.92	0.82
		0	0.3566	0.6965	0.86	0.78	0.82
		All	0.2275	0.7067	0.80	0.85	0.82
	MobileNetV2 + DSFD [8]	1	0.0953	0.6952	0.72	0.90	0.80
		0	0.3605	0.6555	0.83	0.76	0.80
		All	0.2279	0.6754	0.78	0.83	0.80
	CB-OWL-ViT	1	0.0780	0.5198	0.83	0.60	0.70
		0	0.3218	0.5570	0.62	0.81	0.71
		All	0.1999	0.5384	0.73	0.71	0.70
Smartphone	ResNet50 + DSFD [34]	1	0.9578	0.9608	0.93	1.00	0.96
		0	0.5510	0.7861	0.90	0.88	0.89
		All	0.7544	0.8735	0.91	0.94	0.93
	MobileNetV2 + DSFD [8]	1	0.9935	0.9965	0.99	1.00	1.00
		0	0.6306	0.9023	0.91	0.99	0.95
		All	0.8121	0.9494	0.95	1.00	0.97
	CB-OWL-ViT	1	0.2884	0.8703	1.00	0.87	0.93
		0	0.5047	0.7886	0.79	0.99	0.88
		All	0.3966	0.8295	0.89	0.93	0.91
Webcam	ResNet50 + DSFD [34]	1	0.2630	0.5814	0.56	1.00	0.72
		0	0.3050	0.3090	0.95	0.31	0.46
		All	0.2840	0.4432	0.76	0.65	0.59
	MobileNetV2 + DSFD [8]	1	0.4511	0.8867	0.95	0.89	0.92
		0	0.8809	0.9028	0.85	0.96	0.90
		All	0.6660	0.8947	0.90	0.93	0.91
	CB-OWL-ViT	1	0.5478	0.6761	0.97	0.69	0.81
		0	0.8271	0.8605	0.70	0.98	0.82
		All	0.6875	0.7683	0.83	0.84	0.81

Values shown in bold represent the best (highest) performance across all models and classes for each dataset.

compares the performance of the proposed model with SOTA models reported in previous studies on mask detection. The proposed model consistently achieves the highest recall for class 0 across all datasets and models. While the method presented in [34] demonstrates strong performance on the Kaggle and Roboflow datasets, its performance declines significantly on the Webcam dataset, suggesting that it has limited robustness. The model introduced in [8] achieves performance comparable to that of the proposed method and, in some cases, slightly surpasses it; however, it is restricted to predicting only predefined classes and requires retraining for new tasks.

Table 11. Comparison of social distance estimation methods (This is the same Table 5 with a new method added.).

Dataset	Method	MAE	MSE	RMSE
Smartphone	Metric Depth Estimation	0.5300	0.3799	0.6163
	Homography Transformation	0.1116	0.0189	0.1376
	Fixed Human Height [33]	0.7275	0.6555	0.8096
Webcam	Metric Depth Estimation-Based	0.2429	0.1076	0.3280
	Homography Transformation	0.1364	0.0450	0.2122
	Fixed Human Height [33]	0.9231	1.3125	1.1456

Values shown in bold represent the best (lowest) performance across all methods for each dataset.

compares the methods proposed in the current study along with the approach presented in [33]. As shown, the methods introduced in this study outperform the baseline across all three evaluation metrics: MAE, MSE, and RMSE.

5.3. Practical Deployment Analysis and Model Validation

While the proposed framework demonstrates strong performance in controlled experimental settings, its practical applicability and reliability in real-world monitoring scenarios require further clarification and validation. Therefore, this subsection evaluates the computational efficiency and inference speed of the model, discusses its operational characteristics in terms of frame-by-frame processing versus temporal consistency, and clarifies the scope of real-time applicability. In addition, to strengthen the empirical validity of the proposed approach, a comparative evaluation against a representative VLM-based detector, namely YOLO-World, is conducted, and a series of ablation studies investigating the contributions of key architectural and training components are presented. Together, these analyses aim to offer deeper insights into both the practical feasibility and the underlying effectiveness of the proposed model.

5.3.1. Inference Speed and Computational Cost

To evaluate the practical feasibility of the proposed framework, computational efficiency in terms of inference speed, model size, and number of parameters is analyzed and compared with representative CNN-based and lightweight object detection models. This analysis aims to clarify the deployment characteristics of the proposed approach and address concerns regarding its real-world applicability.

All inference-time measurements reported here are conducted on an NVIDIA Tesla T4 GPU with a batch size of 1. This hardware configuration reflects a realistic deployment scenario for modern intelligent monitoring systems, where entry-level GPUs are commonly available to support computationally demanding vision models. CPU-only inference is not considered in this evaluation, as Transformer-based architectures and VLMs typically incur significantly higher latency without GPU acceleration.

As shown in

Table 12. Comparison of models in terms of the number of parameters, model size, and inference time.

Model	Number of Parameters	Model Size (MB)	Inference Time (ms/img)
MobileNetV2 + DSFD [8]	132,803,175	608.455	45
YOLOv8n	3,011,238	5.94	25
ResNet50 + DSFD [34]	160,424,359	932.067	68
CB-OWL-ViT	153,231,879	584.53	80

, lightweight CNN-based detectors such as YOLOv8n achieve lower inference latency due to their compact architectures and task-specific supervision. In contrast, the proposed cluster-based OWL-ViT model has higher inference latency, primarily due to its Transformer backbone and the integration of language supervision for open-vocabulary reasoning. This increased computational cost reflects a deliberate design trade-off rather than a limitation of the proposed framework.

Unlike conventional object detection models that rely on fixed class definitions and require retraining or fine-tuning to adapt to new tasks, the proposed approach leverages language supervision to enable training-free adaptation to novel object categories and monitoring scenarios. This flexibility is particularly valuable in real-world monitoring applications, where operational requirements may evolve over time and pre-defined label sets are often insufficient. Consequently, the proposed framework prioritizes adaptability, semantic richness, and deployment flexibility over raw inference speed.

It is important to emphasize that strict real-time performance on resource-constrained or mobile devices is not the primary focus of this study. Instead, the proposed model is designed for GPU-equipped monitoring environments, where robustness, generalization, and semantic adaptability are more important than minimal latency. Under this setting, the reported inference time demonstrates that the proposed framework remains practically deployable for near-real-time or offline analysis pipelines.

5.3.2. Comparison with VLM-Based Detector (YOLO-World)

Table 13. Evaluation results for CB-OWL-ViT, OWL-ViT, and YOLO-World across datasets (This is the same Table 4 with results for YOLO-World added.).

Datest	Model	Class	mAP@		P	R	F1
			0.50	0.20
Kaggle	CB-OWL-ViT	1	0.3214	0.5317	0.88	0.58	0.70
		0	0.2414	0.2792	0.39	0.67	0.49
		All	0.2814	0.4054	0.64	0.62	0.59
	OWL-ViT	1	0.3807	0.4693	0.66	0.56	0.61
		0	0.2180	0.2316	0.10	0.91	0.17
		All	0.2994	0.3504	0.38	0.73	0.39
	YOLO-World	1	0.335	0.413	0.58	0.49	0.53
		0	0.192	0.204	0.09	0.80	0.16
		All	0.265	0.308	0.34	0.64	0.35
Roboflow	CB-OWL-ViT	1	0.0780	0.5198	0.83	0.60	0.70
		0	0.3218	0.5570	0.62	0.81	0.71
		All	0.1999	0.5384	0.73	0.71	0.70
	OWL-ViT	1	0.0580	0.3976	0.53	0.53	0.53
		0	0.4131	0.6064	0.20	0.98	0.33
		All	0.2356	0.5020	0.36	0.75	0.43
	YOLO-World	1	0.051	0.350	0.47	0.47	0.47
		0	0.364	0.532	0.18	0.86	0.29
		All	0.208	0.441	0.32	0.66	0.38
Smartphone	CB-OWL-ViT	1	0.2884	0.8703	1.00	0.87	0.93
		0	0.5047	0.7886	0.79	0.99	0.88
		All	0.3966	0.8295	0.89	0.93	0.91
	OWL-ViT	1	0.5846	0.6407	0.94	0.65	0.77
		0	0.3775	0.5068	0.49	1.00	0.65
		All	0.4811	0.5737	0.71	0.82	0.71
	YOLO-World	1	0.514	0.565	0.83	0.57	0.67
		0	0.332	0.446	0.43	0.88	0.54
		All	0.423	0.506	0.63	0.73	0.64
Webcam	CB-OWL-ViT	1	0.5478	0.6761	0.97	0.69	0.81
		0	0.8271	0.8605	0.70	0.98	0.82
		All	0.6875	0.7683	0.83	0.84	0.81
	OWL-ViT	1	0.3281	0.3716	0.94	0.37	0.53
		0	0.5622	0.5932	0.55	1.00	0.71
		All	0.4452	0.4824	0.75	0.68	0.62
	YOLO-World	1	0.290	0.327	0.83	0.33	0.47
		0	0.495	0.522	0.48	0.88	0.60
		All	0.392	0.425	0.65	0.61	0.54

Values shown in bold represent the best (highest) performance across all models and classes for each dataset. YOLO-World results are approximately 10–15% lower than OWL-ViT.

summarizes the evaluation results for CB-OWL-ViT, OWL-ViT, and YOLO-World across four datasets. It can be observed that CB-OWL-ViT consistently achieves the highest performance across mAP, precision, recall, and F1-score, demonstrating the effectiveness of combining open-world vision-language detection with class-balanced training. OWL-ViT generally performs slightly lower than CB-OWL-ViT, with reductions in F1-score and mAP of up to 15% on some datasets. YOLO-World consistently underperforms OWL-ViT by approximately 10–15% across most metrics and datasets, indicating that while YOLO-World remains competitive, it does not exploit the open-world vision-language capabilities as effectively. Overall, the table shows that CB-OWL-ViT delivers the most robust performance, followed by OWL-ViT, while YOLO-World achieves the weakest, yet still reasonable, performance.

5.4. Discussion

The principal strength of the proposed model lies in its flexibility, which enables adaptation to new conditions without necessitating retraining on a different dataset. Moreover, the model achieves satisfactory performance in social distancing monitoring without dependence on stereo cameras. Nonetheless, the primary limitation of this flexible approach is its lower accuracy relative to unimodal, single-stage models, such as YOLO, when trained on rich, task-specific datasets.

A comprehensive visual presentation of the detection results is provided in Appendix A and Appendix B.

6. Conclusions and Future Directions

This study presented a multimodal framework for monitoring compliance with contagious disease control measures, addressing the need for adaptable and cost-effective public health surveillance. The proposed CB-OWL-ViT model enhanced mask detection by enabling recognition beyond predefined classes, while the homography-based module enabled reliable social distance estimation with low-cost monocular cameras. The newly collected dataset enabled robust validation, and comprehensive comparisons with the SOTA models confirmed the competitive performance of the proposed model. The framework achieved notable results, including up to 0.37 higher precision and 0.2 higher F1-score than the baseline OWL-ViT, as well as a recall of 0.98 for the “Without Mask” class (i.e., class 0) in real-world testing. Overall, the proposed approach offers a flexible and practical solution for large-scale deployment in resource-constrained environments.

This study laid the foundation for scalable, flexible systems to monitor contagious diseases, thereby enhancing public health response strategies.

Note on Person Tracking in the Proposed Framework: The proposed framework does not include a dedicated multi-frame person tracker. Detection and social distance estimation are performed independently for each frame. This design choice was made to maintain methodological simplicity and avoid expanding the scope of the paper. However, standard tracking modules such as ByteTrack [53] or DeepSORT [47] can be integrated within this framework in the future, leveraging the features extracted in the proposed feature workflow (see Section 3) for identity association across frames.

Future research may pursue the following directions:

• Dataset Expansion for OWL-ViT Fine-Tuning: Collecting a task-specific dataset that encompasses diverse mask-wearing scenarios and crowd conditions. This facilitates the effective fine-tuning of the OWL-ViT model, thereby enhancing its applicability to real-world public health monitoring.
• Advanced Depth Modeling for Social Distance Estimation: Developing a more sophisticated and accurate depth estimation model. This approach extends beyond conventional pinhole camera assumptions and has the potential to significantly improve the accuracy of social distance estimates, particularly in complex 3D environments.