Search Results (85)

Long referring expressions create two coupled sources of hallucination in visual grounding. A detector can select an object that matches only part of the instruction, while a structured vision–language model (VLM) branch can hallucinate a target head or an attribute–object binding. We propose DeRecG, a training-free, anchor-preserved framework that addresses both sources. A frozen Grounding DINO detector supplies an anchor, and a frozen Qwen2-VL model decomposes the expression for candidate verification. Confidence-Constrained Disagreement Arbitration (CCDA) replaces the anchor only when a candidate passes head-validity, score-floor, score-margin, and spatial-disagreement gates. Lexical–Visual Candidate Expansion (LVCE) repairs parser-induced head errors before arbitration. Across six RefCOCO-family splits (46,842 expressions), LVCE+CCDA improves weighted accuracy at an intersection-over-union (IoU) threshold of 0.5 from 53.40% to 55.00%, while weighted mean IoU rises from 53.28% to 54.66%. With the same frozen thresholds, Ref-L4 validation accuracy rises from 31.02% to 34.73% and mean IoU from 33.43% to 36.72% over 13,420 long expressions; 646 anchor errors are corrected and 148 are induced ($p<{10}^{-74}$), with no degradation on short-phrase Flickr30k Entities grounding. The complete pipeline requires no training, uses 6.10 GiB peak allocated GPU memory, and processes 1.15 images/s on an RTX 5090 D. Full article

Social media has become an important tool for understanding visitor experiences in museums and cultural heritage sites. This literature review identifies, organizes, and thematically synthesizes existing studies on museum visitor experience based on social media data. It examines 41 studies retrieved from Scopus and Web of Science and published between 2017 and 2026. Furthermore, the review examines the selected studies across six dimensions: social media platforms, types of user-generated data, the role of digital interactions, the museums and cultural heritage sites studied, the analytical methodologies applied, and the main findings on visitor experience. The findings indicate that TripAdvisor is the most frequently used platform for collecting textual reviews and star ratings, whereas Instagram and Flickr are mainly used for visual and spatial data. Most studies rely on computational methods, often combined with quantitative techniques, while qualitative approaches are used less frequently. The identified methods include content analysis, statistical analysis, sentiment analysis, topic modeling, machine learning, image analysis, and spatial analysis. Across the reviewed studies, visitor experience is examined as a multidimensional phenomenon encompassing emotions, service quality, authenticity, historical connection, aesthetics, education, and social participation. Finally, the review identifies recurring themes across the dimensions and synthesizes them into broader research streams. These are brought together in an integrative synthesis framework that organizes existing research, highlights research gaps, and outlines directions for future studies. Full article

The increasing complexity of steganographic techniques used to conceal information in digital content constitutes a major challenge for cybersecurity and digital forensics. Although deep learning has achieved strong performance in binary steganalysis, generalization across embedding algorithms, image formats, and datasets remains limited, highlighting the need for controlled benchmarks that can systematically evaluate such distribution shifts. We propose StegBench, a reference dataset for multi-class steganalysis for JPEG and PNG formats, with public release on Hugging Face planned upon publication. Beyond simply flagging a suspicious file for quarantine, identifying the specific embedding algorithm provides additional information that can support further analysis. In particular, it reduces the number of extraction methods that need to be considered when attempting to recover the hidden payload. Moreover, once the payload is recovered, this information can help associate the analyzed file with known steganographic tools or toolchains, supporting threat intelligence and the correlation of related security incidents. StegBench is constructed from 70,000 source cover images and comprises 525,000 labeled images distributed across four complementary branches, structured on grayscale and color segments for each format: the JPEG branches are derived from the BOSSBase 1.01 and ALASKA2 sets, while the PNG branches are built on the basis of BOSSBase 1.01 and Flickr30k. The JPEG branch uses an eight-class taxonomy comprising unaltered images, four DCT-domain steganographic algorithms (F5, JSteg, Steghide, and OutGuess), and three types of noise-based perturbations. The noise-based categories serve as adversarial foils, allowing us to assess whether the detectors learn genuine embedding signatures rather than merely responding to generic perturbation energy. Accordingly, these categories are included for robustness evaluation and are not intended for steganographic algorithm attribution. The PNG branch integrates LSB steganographic methods, along with the same noise categories, thus enabling direct comparison between the mentioned formats. The evaluation includes EfficientNetV2-L, ConvNeXt-Large, and SwinV2-Large, together with SRNet as a specialized steganalysis baseline; the general-purpose backbones were additionally evaluated with a fixed SRM high-pass front-end to preserve steganalysis-relevant residual information. The ConvNeXt-Large model recorded the best results, reaching an accuracy of 99.60% on the color JPEG dataset. Cross-domain evaluations confirm the expected collapse when transferring between the DCT-domain (JPEG) and spatial-domain (PNG) embedding regimes, and additionally reveal a comparably severe degradation when transferring across color distributions (grayscale vs. color) within the same format, which is a less predictable result indicating that models rely on source- and channel-specific statistics rather than transferable steganographic signatures. These results demonstrate the value of dataset diversity for systematically exposing generalization limitations and provide a controlled benchmark for evaluating future domain-robust steganalysis methods. Full article

Image captioning with pretrained vision and language models often requires substantial model adaptation, while lightweight settings restrict the number of components that can be updated. This study investigates an input-level visual conditioning approach for image captioning using a frozen ResNet-50 image encoder and a frozen Phi-3 Mini language model. A global image representation is first projected into ten continuous visual tokens. A gated local residual module is then introduced to incorporate spatial features from the final ResNet feature map without modifying either pretrained backbone. The framework was evaluated on the MS COCO Karpathy split under a fixed training and decoding protocol. The global visual-conditioning baseline achieved a CIDEr score of 0.8382 on the test split. Adding the gated local residual module increased CIDEr to 0.8693, while adding only 3.95 million stage-specific trainable parameters. Under a near-matched trainable-parameter budget, a rank-10 QKV-only LoRA control achieved a CIDEr score of 0.8439, whereas direct and gated local residual conditioning achieved 0.8611 and 0.8693, respectively. These results are limited to the evaluated frozen ResNet-50–Phi-3 configuration and do not imply a general advantage over LoRA variants. As an additional sequence-level optimization experiment, self-critical sequence training increased CIDEr marginally from 0.8693 to 0.8704 without changing the inference architecture. Caption quality was further examined using a structured LLM-based evaluation over all 5000 images in the Karpathy test split, assessing semantic accuracy, information coverage, fluency, and reference-based unsupported content. A zero-shot evaluation on Flickr30k showed a marked performance decrease, indicating limited cross-dataset generalization. Overall, the experiments provide an empirical analysis of global and local visual conditioning for parameter-efficient caption generation with frozen pretrained components. Full article

Image–Text retrieval requires accurate semantic alignment between visual content and natural language descriptions. Most existing methods primarily rely on region features, which are typically object-centric and may overlook important contextual information. In contrast, grid features provide denser spatial coverage and richer local details, but often lack explicit semantic structure. To better exploit their complementarity, we propose a novel Region–Grid Interaction and Calibration Network (RGICN) for Image–Text retrieval. Specifically, we first design a Global-Guided Feature Interaction Module to promote information exchange between region and grid features under the guidance of global visual semantics, allowing object-level semantics and contextual cues to complement each other. We then introduce a Text-Guided Feature Calibration Module, which leverages auxiliary image descriptions to calibrate visual features by suppressing redundant and text-irrelevant content. Finally, an Adaptive Gating Fusion Module is developed to dynamically integrate multiple visual representations according to the input image, yielding a more comprehensive and discriminative visual embedding. Extensive experiments on the benchmark datasets MS-COCO and Flickr30K demonstrate the effectiveness of RGICN and its competitive performance against recent state-of-the-art methods. Full article

Existing image–text retrieval methods often compute cross-modal similarity using global single-vector representations. Although efficient for coarse semantic alignment, such compressed representations are limited when textual queries involve fine-grained semantics, including objects, attributes, relations, and their compositional structures. This paper focuses on fine-grained text-to-image retrieval and proposes Q-GrAM, a retrieval-oriented adaptation of the BLIP-2 Q-Former. Instead of treating Q-Former queries as a homogeneous set, Q-GrAM partitions a fixed query budget into semantically differentiated groups. A text-guided router assigns token-level semantic demands to query groups, while query conditional initialization modulates each group according to group-level textual summaries. The resulting grouped visual query features are matched with text tokens through a group-aware late interaction scorer, and auxiliary routing balance and inter-group diversity regularization are introduced to stabilize semantic specialization. Experiments on MS-COCO 5K, Flickr30K, and Flickr30K-CFQ show that Q-GrAM achieves strong text-to-image retrieval performance against both global embedding baselines and representative fine-grained image–text matching methods, while maintaining competitive bidirectional retrieval performance. These results demonstrate the effectiveness of structured, text-conditioned Q-Former query specialization for fine-grained text-driven image search. Full article

Image captioning using deep learning bridges computer vision and natural language processing, enabling machines to generate human-like textual descriptions for images. While significant progress has been made in English, in Arabic, the image captioning field remains under-explored due to the language’s morphological complexity, right-to-left script, and scarcity of annotated datasets. This paper addresses this gap by adapting the BLIP-2 (Bootstrapped Language—Image Pre-training) model for Arabic caption generation, leveraging machine-translated datasets, like Flickr 30k, to overcome resource limitations. BLIP-2 combines a vision transformer (ViT) for image encoding and a CamelBERT large language model (LLM) for text generation, enhanced by a lightweight Querying Transformer (Q-Former) for cross-modal alignment. Despite challenges such as translation artifacts and linguistic nuances, our experiments demonstrate promising results in generating coherent Arabic captions. In short, this study highlights the potential of BLIP-2 for multilingual applications while underscoring the need for native Arabic datasets and further optimization. Ultimately, this work contributes to advancing inclusive artificial intelligence technologies for Arabic-speaking communities, with applications in assistive tools, education, and content creation. Full article

This paper presents a new image captioning system which contains facial expression recognition as a way to provide better emotional and contextual comprehension of the captions generated. A combination of affective cues and visual features is made, which enables semantically full and emotionally conscious descriptions. Experiments were carried out on two created datasets, FlickrFace11k and COCOFace15k, with standard benchmarks such as BLEU, METEOR, ROUGE-L, CIDEr, and SPICE to analyze their effectiveness. The suggested model produced better results in all metrics as compared to baselines, like Show-Attend-Tell and Up-Down, remaining consistently better on all the scores. Remarkably, it has reached gains of 2.5 points on CIDEr and 1.0 on SPICE, which means a closer correlation to the prompt captions made by people. A 5-fold cross-validation confirmed the model’s robustness, with minimal standard deviation across folds (<±0.2). Qualitative results further demonstrated its ability to capture fine-grained emotional expressions often missed by conventional models. These findings underscore the model’s potential in affective computing, assistive technologies, and human-centric AI applications. The pipeline is designed for on-prem/edge deployment with lightweight interfaces to IoT middleware (MQTT/OPC UA), enabling smart-factory integration. These characteristics align the method with Industry 4.0 sensor networks and human-centric analytics. Full article

Image captioning, as a representative cross-modal task, faces significant challenges, including high annotation costs and modality alignment difficulties. To address these issues, this paper proposes CMSA, an image captioning framework that does not require paired image-text data. The framework integrates a generator, a discriminator, and a reward module, employing a collaborative multi-module optimization strategy to enhance caption quality. The generator builds multi-level joint feature representations based on a contrastive language-image pretraining model, effectively mitigating the modality alignment problem and guiding the language model to generate text highly consistent with image semantics. The discriminator learns linguistic styles from external corpora and evaluates textual naturalness, providing critical reward signals to the generator. The reward module combines image-text relevance and textual quality metrics, optimizing the generator parameters through reinforcement learning to further improve semantic accuracy and language expressiveness. CMSA adopts a progressive multi-stage training strategy that, combined with joint feature modeling and reinforcement learning mechanisms, significantly reduces reliance on costly annotated data. Experimental results demonstrate that CMSA significantly outperforms existing methods across multiple evaluation metrics on the MSCOCO and Flickr30k datasets, exhibiting superior performance and strong cross-dataset generalization ability. Full article

To resolve the conflict between global structure modeling and local detail preservation in image super-resolution, we propose SwinT-SRGAN, a novel framework integrating Swin Transformer with GAN. Key innovations include: (1) A dual-path generator where Transformer captures long-range dependencies via window attention while CNN extracts high-frequency textures; (2) An end-to-end Detail Recovery Block (DRB) suppressing artifacts through dual-path attention; (3) A triple-branch discriminator enabling hierarchical adversarial supervision; (4) A dynamic loss scheduler adaptively balancing six loss components (pixel/perceptual/high-frequency constraints). Experiments on CelebA-HQ and Flickr2K demonstrate: (1) Very good performance (max gains: 0.71 dB PSNR, 0.83% SSIM, 4.67 LPIPS reduction vs. Swin-IR); (2) Ablation studies validate critical roles of DRB. This work offers a robust solution for high-frequency-sensitive applications. Full article

To address the semantic mismatch between limited textual descriptions in image captioning training datasets and the multi-semantic nature of images, as well as the underutilized external commonsense knowledge, this article proposes a novel image captioning model based on multi-step cross-attention cross-modal alignment and external commonsense knowledge enhancement. The model employs a backbone architecture comprising CLIP’s ViT visual encoder, Faster R-CNN, BERT text encoder, and GPT-2 text decoder. It incorporates two core mechanisms: a multi-step cross-attention mechanism that iteratively aligns image and text features across multiple rounds, progressively enhancing inter-modal semantic consistency for more accurate cross-modal representation fusion. Moreover, the model employs Faster R-CNN to extract region-based object features. These features are mapped to corresponding entities within the dataset through entity probability calculation and entity linking. External commonsense knowledge associated with these entities is then retrieved from the ConceptNet knowledge graph, followed by knowledge embedding via TransE and multi-hop reasoning. Finally, the fused multimodal features are fed into the GPT-2 decoder to steer caption generation, enhancing the lexical richness, factual accuracy, and cognitive plausibility of the generated descriptions. In the experiments, the model achieves CIDEr scores of 142.6 on MSCOCO and 78.4 on Flickr30k. Ablations confirm both modules enhance caption quality. Full article

What if machines could seamlessly translate between the visual richness of images and the semantic depth of language with mathematical precision? This paper presents a theoretical and empirical analysis of five novel cross-modal Wasserstein adversarial translation networks that challenge conventional approaches to cross-modal understanding. Unlike traditional generative models that rely on stochastic noise, our frameworks learn deterministic translation mappings that preserve semantic fidelity across modalities through rigorous mathematical foundations. We systematically examine: (1) cross-modality consistent dual-critical networks; (2) Wasserstein cycle consistency; (3) multi-scale Wasserstein distance; (4) regularization through modality invariance; and (5) Wasserstein information bottleneck. Each approach employs adversarial training with Wasserstein distances to establish theoretically grounded translation functions between heterogeneous data representations. Through mathematical analysis—including information-theoretic frameworks, differential geometry, and convergence guarantees—we establish the theoretical foundations underlying cross-modal translation. Our empirical evaluation across MS-COCO, Flickr30K, and Conceptual Captions datasets, including comparisons with transformer-based baselines, reveals that our proposed multi-scale Wasserstein cycle consistent (MS-WCC) framework achieves remarkable performance gains—12.1% average improvement in FID scores and 8.0% enhancement in cross-modal translation accuracy—compared to state-of-the-art methods, while maintaining superior computational efficiency. These results demonstrate that principled mathematical approaches to cross-modal translation can significantly advance machine understanding of multimodal data, opening new possibilities for applications requiring seamless communication between visual and textual domains. Full article

This paper outlines a methodology to compile geo-referenced observational data of Australian birds acting as pollinators of Strelitzia sp. (Bird of Paradise) flowers and dispersers of their seeds. Given the absence of systematic published records, a crowdsourcing approach was employed, combining data from natural history platforms (e.g., iNaturalist, eBird), image hosting websites (e.g., Flickr) and, in particular, social media. Facebook emerged as the most productive channel, with 61.4% of the 301 usable observations sourced from 43 ornithology-related groups. The strategy included direct solicitation of images and metadata via group posts and follow-up communication. The holistic, snowballing search strategy yielded a unique, behavior-focused dataset suitable for analysis. While the process exposed limitations due to user self-censorship on image quality and completeness, the approach demonstrates the viability of crowdsourced behavioral ecology data and contributes a replicable methodology for similar studies in under-documented ecological contexts. Full article

To address the issues of modeling the relationships between multiple local region objects in images and enhancing local region features, as well as mapping global image semantics to global text semantics and local region image semantics to local text semantics, a novel image captioning method based on CLIP and integrating local feature enhancement and multi-scale semantic guidance is proposed. The model employs ViT as the global visual encoder, Faster R-CNN as the local region visual encoder, BERT as the text encoder, and GPT-2 as the text decoder. By constructing a KNN graph of local image features, the model models the relationships between local region objects and then enhances the local region features using a graph attention network. Additionally, a multi-scale semantic guidance method is utilized to calculate the global and local semantic weights, thereby improving the accuracy of scene description and attribute detail description generated by the GPT-2 decoder. Evaluated on MSCOCO and Flickr30k datasets, the model achieves a significant improvement in the core metric CIDEr over established strong baselines, with 4.7% higher CIDEr than OFA on MSCOCO, and 16.6% higher CIDEr than Unified VLP on Flickr30k. Ablation studies and qualitative analysis validate the effectiveness of each proposed module. Full article

Currently, the phenomenon of deepfakes is becoming increasingly significant, as they enable the creation of extremely realistic images capable of deceiving anyone thanks to deep learning tools based on generative adversarial networks (GANs). These images are used as profile pictures on social media with the intent to sow discord and perpetrate scams on a global scale. In this study, we demonstrate that these images can be identified through various imperfections present in the synthesized eyes, such as the irregular shape of the pupil and the difference between the corneal reflections of the two eyes. These defects result from the absence of physical and physiological constraints in most GAN models. We develop a two-level architecture capable of detecting these fake images. This approach begins with an automatic segmentation method for the pupils to verify their shape, as real image pupils naturally have a regular shape, typically round. Next, for all images where the pupils are not regular, the entire image is analyzed to verify the reflections. This step involves passing the facial image through an architecture that extracts and compares the specular reflections of the corneas of the two eyes, assuming that the eyes of real people observing a light source should reflect the same thing. Our experiments with a large dataset of real images from the Flickr-FacesHQ and CelebA datasets, as well as fake images from StyleGAN2 and ProGAN, show the effectiveness of our method. Our experimental results on the Flickr-Faces-HQ (FFHQ) dataset and images generated by StyleGAN2 demonstrated that our algorithm achieved a remarkable detection accuracy of 0.968 and a sensitivity of 0.911. Additionally, the method had a specificity of 0.907 and a precision of 0.90 for this same dataset. And our experimental results on the CelebA dataset and images generated by ProGAN also demonstrated that our algorithm achieved a detection accuracy of 0.870 and a sensitivity of 0.901. Moreover, the method had a specificity of 0.807 and a precision of 0.88 for this same dataset. Our approach maintains good stability of physiological properties during deep learning, making it as robust as some single-class deepfake detection methods. The results of the tests on the selected datasets demonstrate higher accuracy compared to other methods. Full article