Machine Learning and Knowledge Extraction

Unsupervised Hebbian learning is a biologically inspired algorithm designed to extract representations from input images, which can subsequently support supervised learning. It presents a promising alternative to traditional artificial neural networks (ANNs). Many attempts have focused on enhancing Hebbian learning by incorporating more biologically plausible components. Contrarily, we draw inspiration from recent advances in ANNs to rethink and further improve Hebbian learning in three interconnected aspects. First, we investigate the issue of overfitting in Hebbian learning and emphasize the importance of selecting an optimal number of training epochs, even in unsupervised settings. In addition, we discuss the risks and benefits of anti-Hebbian learning in model performance, and our visualizations reveal that synapses resembling the input images sometimes do not necessarily reflect effective learning. Then, we explore the impact of different activation functions on Hebbian representations, highlighting the benefits of properly utilizing negative values. Furthermore, motivated by the success of large pre-trained language models, we propose a novel approach for leveraging unlabeled data from other datasets. Unlike conventional pre-training in ANNs, experimental results demonstrate that merging trained synapses from different datasets leads to improved performance. Overall, our findings offer fresh perspectives on enhancing the future design of Hebbian learning algorithms.

Amid ongoing efforts to develop extremely large, multimodal models, there is increasing interest in efficient Small Language Models (SLMs) that can operate without reliance on large data-centre infrastructure. However, recent SLMs (e.g., LLaMA or Phi) with up to three billion parameters are predominantly trained in high-resource languages, such as English, which limits their applicability to industries that require robust NLP solutions for less-represented languages and low-resource settings, particularly those requiring low latency and adaptability to evolving label spaces. This paper examines a retrieval-based approach to multi-label text classification (MLC) for a media monitoring dataset, with a particular focus on less-represented languages, such as Slovene. This dataset presents an extreme MLC challenge, with instances labelled using up to twelve thousand categories. The proposed method, which combines retrieval with computationally efficient prediction, effectively addresses challenges related to multilinguality, resource constraints, and frequent label changes. We adopt a model-agnostic approach that does not rely on a specific model architecture or language selection. Our results demonstrate that techniques from the extreme multi-label text classification (XMC) domain outperform traditional Transformer-based encoder models, particularly in handling dynamic label spaces without requiring continuous fine-tuning. Additionally, we highlight the effectiveness of this approach in scenarios involving rare labels, where baseline models struggle with generalisation.

The growing use of machine learning (ML) and artificial intelligence across sectors has shown strong potential to improve decision-making processes. However, the adoption of ML by non-technical professionals remains limited due to the complexity of traditional development workflows, which often require software engineering and data science expertise. In recent years, low-code and no-code platforms have emerged as promising solutions to democratize ML by abstracting many of the technical tasks typically involved in software engineering pipelines. This paper investigates whether these platforms can offer a viable alternative for making ML accessible to non-expert users. Beyond predictive performance, this study also evaluates usability, setup complexity, the transparency of automated workflows, and cost management under realistic “out-of-the-box” conditions. This multidimensional perspective provides insights into the practical viability of LC/NC tools in real-world contexts. The comparative evaluation was conducted using three leading cloud-based tools: Amazon SageMaker Canvas, Google Cloud Vertex AI, and Azure Machine Learning Studio. These tools employ ensemble-based learning algorithms such as Gradient Boosted Trees, XGBoost, and Random Forests. Unlike traditional ML workflows that require extensive software engineering knowledge and manual optimization, these platforms enable domain experts to build predictive models through visual interfaces. The findings show that all platforms achieved high accuracy, with consistent identification of key features. Google Cloud Vertex AI was the most user-friendly, SageMaker Canvas offered a highly visual interface with some setup complexity, and Azure Machine Learning delivered the best model performance with a steeper learning curve. Cost transparency also varied considerably, with Google Cloud and Azure providing clearer safeguards against unexpected charges compared to Sagemaker Canvas.