An Online Collaborative Approach to Developing Ontologies to Study Questions About Behaviour

Abstract

Almost all societal grand challenges, whether concerning the environment, health, well-being, or the development of sustainable economic models, have at their heart a need to understand people’s behaviour. However, uniting data and insights across disparate fields requires an explicit and shared understanding of concepts, variables, and ideas (e.g., how to characterise and differentiate behaviours). Ontologies provide a mechanism for creating this explicit and shared understanding and are starting to be developed and used in the social and behavioural sciences. This paper proposes an online co-design approach to use and develop ontologies of behaviour to specify the characteristics of behaviour (e.g., habitual, changeable, effortless) and studies that investigate behaviour as part of a project designed to understand how behaviours are related. We report on our experience of collaborative co-development of ontologies using real-time interactive tools and reflect on the benefits and challenges of our approach. We also offer a set of recommendations for researchers interested in applying such methods to co-develop ontologies. The work contributes to efforts to understand the characteristics of behaviour and enable these to be used to understand questions about behaviour (e.g., is poor sleep associated with greater engagement in habitual behaviours?).

1. Introduction

The wider project to which this work contributes (ESRC Project reference: ES/T009179/1; hereafter the ‘TURBBO Project’) aims to capitalise on the burgeoning body of data available on the relationships between behaviours (e.g., alcohol consumption, political behaviours, sleep, domestic violence, recycling) to (i) develop our understanding of how different types of behaviour are related and (ii) make that information available to behavioural scientists and key stakeholders in a flexible and easy-to-access format. Although there are individual studies that examine these (and related) questions, to date, it has proved difficult to integrate the information across disparate studies and fields. This is partly because studies may appear to focus on the same behaviour when most would agree that the measures reflect different behaviours. For example, one study might measure physical activity in terms of the number of steps taken a day, while another measures physical activity in terms of minutes of vigorous activity. To move forward, researchers need to specify which behaviours (or sets of behaviours) are sufficiently similar so as to be contextually exchangeable (i.e., interchangeable for a given set of purposes [1]), e.g., because they reflect the same behaviour (e.g., physical activity) or behaviours with the same characteristics or attributes (e.g., are all relatively habitual). Ontologies provide the opportunity to create this shared understanding and to structure the knowledge in a useful way.

1.1. Ontologies and Ontology Design

Ontologies are formal descriptions of concepts, including their attributes and relationships, of which domain experts share a common understanding [2]. Logical rules and constraints [3] are further defined, which enable software to reason about the knowledge represented in the ontology, and discover or derive new knowledge [4]. An ontology provides a standardised representation of the semantics of a domain and so can be used to exchange data and models, encouraging re-use, communication, collaboration, and integration [2]. The ontology design process has been described by Pinto and Martins [5] as containing five main stages:

• Specification: where the scope and requirements of the stakeholders are understood,
• Conceptualisation: where the generic abstract description of the concepts needed to represent a purpose are created [6],
• Formalisation: where the conceptual model is translated,
• Implementation: where the formal model is implemented in standard ontology language,
• Maintenance: where the ontology is evolved and evaluated into a formal model.

With roots in participatory design, co-design is an approach towards the design of computer systems, where people destined to use the system play a critical role in designing it [7]. Co-creation of ontologies refers to a creative continuous involvement of stakeholders in ontology engineering [8], thereby increasing the likelihood of the adoption and acceptability of ontology-driven technologies [4]. In co-designing an ontology, therefore, a close collaboration between domain experts and ontology engineers is required to comprehensively and accurately capture knowledge within a domain [9]. Ontology engineers and domain experts become equal partners and contributors [10] in this process. Incorporating the feedback of expert stakeholders ensures that the ontology is appropriate, comprehensive, meets community needs, and is interoperable with other ontologies and classification systems. Within the broader field of (health) care, ontology co-creation is a key approach toward creating accurate and practical ontologies [10,11]. Traditionally, however, ontology development has only involved users at discrete stages and has not adopted a holistic co-design approach across all stages of development, including ontology maintenance and evaluation (e.g., [8,12]).

In order to ensure continuity and lifetime beyond our project, we aimed to engage stakeholders at all stages of the development as co-owners of the ontology and related tools. The project within which this approach is applied—the ‘Tools for Understanding the Relationship Between Behaviours using Ontologies’ or TURBBO Project—aimed to develop or bring together multiple levels of ontologies pertaining to (human) behaviour, where an upper level ontology (which is an ontology that defines the most general concepts of a domain [13]) will identify the characteristics of (human) ‘behaviour’ (e.g., that some behaviours are relatively habitual) and ‘studies of behaviour’ (e.g., that the study was conducted in the UK). Specific behaviours (such as sleeping, running, or smoking) will be formalised in a middle-level ontology; for example, by integrating The Human Behaviour Ontology [14], which is a part of the Behaviour Change Intervention Ontology [15].

The upper, middle, and domain ontologies will then be used as a framework within which to collate information on the relationship between behaviours. While behaviours are often viewed in isolation, psychologists and behavioural scientists are increasingly interested in whether and how behaviours are related to each other and how they influence or impact others (e.g., [16]). Some relationships between behaviours are intuitive (e.g., higher levels of physical activity may influence the quality of sleep). However, some relationships between behaviours are less obvious (e.g., recycling may be associated with higher levels of physical activity). The TURBBO project requires a set of ontologies that can be used to capture empirical data on the relationship between behaviours. Such ontologies will serve two primary communities: (i) researchers and academics interested in studying behaviours and (ii) practitioners interested in changing behaviours.

1.2. Novelty and Contribution

We will take a co-design approach to all of the stages of Pinto and Martins’ [5] framework; however, this paper focusses specifically on our use of the co-design approach for Stage 1 (specification) and Stage 2 (conceptualisation), for the development of an upper-level ontology. A common practice of developing ontologies has been to employ a collaborative approach in an in-person setting. Standard ontology co-design methodologies typically use face-to-face interactions that involve hands-on tools like sticky notes, drawing boards, card sorting, concept maps, and so on [17]. Recent research on distributed co-design has developed specific tools to support the co-design process [18], albeit not yet applied to the co-design of ontologies. The novelty of our project lies in re-thinking the process of co-designing an ontology in an online distributed format to enable a synchronous co-design process that allows experts to collaborate. Despite the availability of online ontology authoring and synchronous collaboration tools such as Web Protege (which are aimed at ontology engineers and editors), at the time of writing, to our knowledge, research on synchronous online distributed co-design approaches for ontology design with domain experts is limited. Our approach focusses on understanding how existing tools could be used to translate the thinking process that is traditionally associated with designing ontologies to an online interaction format. Subsequent technical validation of the ontology development (i.e., formalisation, implementation, and maintenance stages) is being planned and is not reported further here.

In this paper, we seek to answer the overarching question: how can existing online synchronous tools be used to support the co-design of ontologies? This research question is broken down into the following sub-questions:

• RQ1: How do participants engage with identifying characteristics of (human) behaviour?
• RQ2: How do participants experience a “hands-on” ontology co-design process in an online setting?
• RQ3: What kind of discussions and interactions emerge when co-designing an ontology in an online setting?
• RQ4: What potential application contexts do participants envisage the ontology to support?

The contribution of our work is threefold:

1.

We present our approach and experiences in the specification and conceptualisation stages of co-creating an upper ontology of the characteristics of human behaviour;

2.

To our knowledge, this is one of the first times that a synchronous online distributed approach has been used to support the co-design of an ontologies with domain experts;

3.

Drawing from our experiences, we share practical recommendations for developing ontologies using online co-design processes.

The paper is structured as follows: First, we discuss the related work in collaborative ontology design approaches and review a number of online collaborative tools. In the next section, we discuss our collaborative co-design process, involving multiple workshops. Third, we present our findings and the co-developed ontology, while using the findings to answer our research questions. Finally, we present a set of lessons learnt, based on our experience of co-designing the ontology in an online setting, which might inform researchers when planning online co-design workshops. We conclude the paper with a discussion around our plans for the latter stages of the project.

2. Related Work

2.1. Co-Design in Ontology Development

Previous research in the Human–Computer Interaction (HCI) community has examined the methodology of co-design, the roles of different stakeholders, and the benefits of involving users. Several studies have shown the importance of involving users in designing new IT tools to ensure high quality tools that are acceptable to users [19,20,21]. Norris et al. [9] also emphasise the importance of involving domain experts to enhance ontology development and ensure that the domain is sufficiently and accurately expressed. Previous research also describes the involvement of users and domain experts during the ontology development lifecycle [8,12,15,22], which involves five main stages: specification, conceptualisation, formalisation, implementation, and maintenance [5]. However, user and expert engagement has been typically limited only to the stages (specification and conceptualisation) that involve scoping and identifying requirements [8]. There is a need for a wider adoption of co-design approaches in ontology development, as well as tools and methods to empower domain experts to actively participate in the entire ontology lifecycle [8]. Ref. [9] recommended approaches for incorporating expert feedback during ontology development, including feedback tasks, consensus exercises, discussions, and workshops, that are suitable for both in person and online delivery.

In order to co-design an ontology, collaborative tools should be used; however, there is not one single tool able to support the full ontology co-design cycle. Instead, a different collaborative tool would be necessary for each stage of ontology development. For example, in stage 4 (implementation), a collaborative ontology editor would be necessary that can encode ontologies in standard formats such as OWL. In our project, we also aimed to co-design the ontology online given our need to collaborate with users that are geographically widespread.

In this paper, we focus on whether and how an online collaborative environment facilitates co-design by non ontology developers in a specialised domain. To our knowledge, there is little evidence on ontology co-design approaches that adopt the use of online synchronous collaborative environments. Indeed, there are more instances of ontology co-design exercises taking place in physical face-to-face settings [8,12,22] than in an online environment [15]. However, it might not be easy for a group of experts to physically meet for a workshop [12]. In addition, the Covid-19 pandemic has changed the way people work in organisations, with more people switching to remote working [23], thus creating more barriers to physical workshops. The following subsections therefore discuss different ontology co-design approaches and the different tools available for co-designing ontologies.

2.2. Ontology Co-Design Approaches

Norris et al. [9] recommended engagement with experts through advisory boards and working groups, which involve a select group of domain experts or engagement with a wider audience through expert panels. For engagement with expert panels, this can be achieved through feedback tasks, consensus exercises, discussions, and workshops [9].

Refs. [8,12,22] adopted a participatory design approach for developing ontologies by using a series of workshops for the different development stages. The participants included ontology engineers, domain experts, and potential system users, henceforth referred to as stakeholders. In addition to workshops, Refs. [8,12] used other methods that included participant observations and interviews. Ref. [12] favored interviews over workshops, especially for the design stage (conceptualisation), arguing that the interviews allowed eliciting a broader range of feedback from different stakeholders. Ref. [15] also carried out participatory design for the development of the Upper Level of the Behaviour Change Intervention Ontology (BCIO). Ref. [8] also employed a variety of techniques during different co-design workshops such as hands-on exercises to specify requirements and role-playing. Several tools have also been used in co-design workshops such as (physical) storyboards and idea boards with post-it notes [8,22] and decision trees [8].

2.3. Online Collaboration Tools

Adaptation to remote working has made tools for online collaboration more popular, particularly given the recent transition to online work and learning [24]. Tools such as Miro (https://miro.com/app/, accessed on 20 June 2025), LucidChart (https://www.lucidchart.com/pages/, accessed on 20 June 2025), and (the now retired) Jamboard (https://jamboard.google.com/, accessed on 20 June 2025) provide many features that support collaboration and brainstorming. For example, Miro allows collaborators to visualise their ideas with online whiteboards and work together as if they are in the same room. There is also traditional video conferencing software such as Zoom and Google Meet, that allows people to connect and collaborate [25]. In addition to allowing users to be geographically distanced, these online tools offer other benefits over face-to-face interactions. For example, collaborators can all have copies of mind maps instantly without the need to make physical copies after workshops, and video conferencing sessions can be recorded for future reference (https://zoom.us/, accessed on 20 June 2025). Standardised open access and licensed tools for ontology development do, however, also exist, such as Protégé, Web Protégé [26], WebVOWL [27], and Fluent Editor (Cognitum’s Fluent Editor, https://cognitum.eu/semantics/fluenteditor/, accessed on 20 June 2025). These ontology editing tools support visualisation, which is one of the core functionalities of the online collaboration tools discussed above. Web Protégé and Fluent Editor also have collaborative editing functionality that is useful for co-creating ontologies. Web Protégé allows users with different access levels to be able to interact with and manipulate the same ontology [26]. Other collaborative functionalities in Web Protégé include adding comments, tracking changes, and entity tagging to flag issues that need addressing [26].

Although such editing tools have many useful features, they were primarily built for ontology engineers, and as such, domain experts need to learn how to build ontologies and use the tools. However, the focus of ontology co-creation is not to make domain experts learn how to build an ontology, but rather to ensure that the ontology itself adequately represents the knowledge in the domain [22]. The challenge with existing ontology development tools is that domain experts need to have sufficient knowledge of ontologies and their development to use such tools. Our stakeholders, being domain experts and not ontology engineers, therefore required us to look for alternatives to Web Protégé and instead use a more generic design tool (Miro). The present research investigates whether this challenge might be addressed by integrating some of the collaborative features of popular tools, like Miro, into ontology development tools.

3. Materials and Methods

Figure 1 illustrates our approach toward ontology development based on Pinto and Martins’ [5] framework for ontology design. As mentioned earlier, this paper focusses on the first two stages of Pinto and Martins’ framework, namely, specification and conceptualisation.

3.1. Stage 1: Specification

3.1.1. Approach

The specification stage traditionally encompasses activities intended to identify the aim of the ontology, future users of the ontology, and the requirements that users have for the ontology [28]. In the present research, the specification stage involved two activities to understand the aims and the scope of the ontology:

• A systematic review of how existing ontologies have characterised behaviour [29];
• Identification of relevant stakeholders who could participate in a series of online workshops.

3.1.2. Identifying the Scope of the Ontology

The process of ontology development was informed by a systematic literature review that aimed to identify existing ontologies that have considered human behaviour [29]; the review did not identify any specific ontologies that fulfilled all of the requirements of our project, although it should be noted that the review (and the research described in this paper) was completed prior to publication of the Human Behaviour Ontology [14].

Our systematic literature review included searching four online ontology repositories: BioPortal, Ontology Lookup Service, Open Biological, and Biomedical Ontology Foundry, and ontologies and institutions/groups listed on the Basic Formal Ontology website. Each ontology was manually reviewed for relevance. Our review sought to identify all ontologies that could provide information on human behaviour, defined as ontologies that included at least one concept that represented a behaviour or that could be used to measure or describe a behaviour. Our initial search identified 1626 records, of which 1073 were retained for screening after removing duplicates. After reviewing each ontology, 68 were included for close reading, following which, a number of ontologies and relevant concepts were identified. A majority (40) of the ontologies identified were designed to facilitate knowledge in the health domain (e.g., chronic and neurological diseases, cancer, injury, and rehabilitation). Five ontologies related to specific health behaviours (e.g., sleep, physical activity, diet and nutrition) and two related to mental health. Six ontologies related to environmental influences on behaviour and the impact of adverse childhood experiences on health. Thirteen ontologies focussed on behaviours related to clinical practice and collection of health data, and seven focussed on research activities. We also identified two ontologies that focussed on human emotions. Our analysis of ontology concepts identified several ontologies that had a high number of concepts related to human behaviour. The Gender, Sex, and Sexual Orientation Ontology, Physical Activity Ontology, and BCIO_Behaviour are examples of such ontologies. At this stage, our requirements for the project were to identify or design an ontology that (i) captures the characteristics of behaviour, (ii) correlations between behaviours, and (iii) how behavioural studies report on these correlations and the study settings.

Existing ontologies consider human behaviour in discrete contexts (e.g., the Physical Activity Ontology and the Medical Adherence Behaviour Ontology consider behaviours related to physical activity and medication adherence, respectively), and initiatives such as the IC Behaviour [30] and Human Behaviour Ontology [14] are starting to describe behaviours across domains. Such ontologies could form the basis of a middle-ontology for coding data on the relationship between behaviours. The Human Behaviour ontology has also begun defining behavioural attributes, informed by analyses of the characteristics of health-related behaviours (e.g., [31]) and ‘applied behavioural analysis’ [32]. However, these focus primarily on the level of physical, mental, and cognitive exertion expended on a behaviour, and a number of characteristics of behaviour that have been studied extensively (e.g., the extent to which behaviours are habitual, frequently performed, underpinned by autonomous vs. controlled motivation) were not included. It might also be useful to identify characteristics that are identified by—and thus, relevant to—stakeholders alongside those identified from the published literature. We also did not identify an existing ontology that describes the characteristics of empirical studies that provide evidence on the relationship between different behaviours. Given our specific needs in the project, it was decided to develop a new ontology that captures characteristics of behaviour, correlations between behaviours, and the study characteristics of these correlations.

Taken together, it seemed appropriate to solicit the views of stakeholders on how to characterise behaviour and the properties of studies that investigate the relations between behaviour(s) in order to inform the use and development of ontologies in the TURBBO project. To identify participants for the co-design activities, project researchers interacted during a set of teleconferences to discuss potential end users and stakeholders. During the first teleconference, it was agreed that we would identify potential end users in the following categories:

• Researchers interested in human behaviour across a wide range of disciplines (e.g., psychologists, economists, sociologists, political scientists), for whom the ontology would help them to understand the relationships between behaviours;
• Researchers, practitioners, and policy-makers designing interventions intended to increase or reduce specific behaviours (e.g., physical activity), for whom the ontology would help to (i) identify ‘core’ behaviours (e.g., behaviours that are likely to co-occur with others and potentially influence them [33]) and (ii) estimate the possible effects of any changes on other behaviours;
• Businesses that need to understand how behaviours are related. For example, a supermarket that needs to understand whether and how behaviours like reusing coffee cups are associated with other actions, such as avoiding meat.

After identifying these categories, a purposive snowball sampling methodology [34] was followed, which started with project researchers completing a spreadsheet of contacts belonging to the different categories and then asking those contacts to identify further contacts.

3.2. Step 2: Conceptualisation

3.2.1. Method

The conceptualisation stage was intended to provide a description of the key concepts needed to represent human behaviour [6]. With this in mind, we designed a set of workshops that used interactive tools and specifically designed tasks to support users to conceptualise an ontology of behaviour, choose the right terminology, and agree on the shared representation.

A pilot was run with an internal project researcher and an external academic participant, to evaluate and reflect on the design of the workshop, after which some elements of the workshop were improved (e.g., prepared videos instead of live presentations were used to ensure homogeneity of content and consistency of timings). Following the pilot, four further workshops were organised, with different types of participants (as described in the next subsection).

The length of the workshop was set to 2 h, to ensure participants’ comfort while still allowing sufficient time to capture important information and further discussions.

3.2.2. Participants

Following an ethical approval process (reference number: 037670, approved by the School of Psychology Research Ethics Committee, University of Sheffield), participants were recruited via an initial email that included an information sheet along with a link to an online consent form (implemented using Qualtrics). The information sheet stated that all information collected from participants would be kept strictly confidential and only accessible to members of the research team, and they would not be able to be identified in any reports or publications. The audio and/or video recordings of their activities made during this research would be used only to create an anonymous transcript of the workshop discussion that would be used for analysis and for illustration in conference presentations and lectures. Only participants who consented were invited to the event and sent an email with the pre-workshop activities. The co-design workshops were attended by the following stakeholders:

• Pilot—internal project researcher and external academic stakeholder (n = 2);
• Workshop 1a—external academic stakeholders (n = 6, all academics);
• Workshop 1b—internal project researchers (n = 6, all academics);
• Workshop 1c—practitioners, businesses, and charities interested in behaviour (n = 4, comprising government health agency (1), conservation (1), transport company (1), pet food company (1);
• Workshop 1d—practitioners, businesses, and charities interested in behaviour (n = 4, comprising practitioners from charity (1), local government (1), government health agency (1), academic (1)).

It is important to highlight our (the authors’) position in this research—the TURBBO project involves a multidisciplinary team that includes computer scientists and psychologists. While the computer science team have been involved as observers and facilitators of the workshops, the psychology team were also involved in one workshop as participants. Their contribution as workshop participants is valuable given the highly specialised domain and that the psychology team are likely to be (or at least to represent the interests of) future users of the ontology and tools being developed in the project.

To facilitate discussions and productive sessions [35], the workshops were designed to be in small groups with a maximum of 6 participants. Additionally, facilitators involved in each workshop ensured that the focus of the session was maintained and helped to translate natural language into conceptual terms without influencing conceptual inclusion.

Practitioners and representatives of businesses and charities interested in behaviour were recruited from the personal networks of the researchers involved in the project. It is important to note that there are potential limitations in the recruitment of participants through such means, namely in introducing selection bias and potential representativeness. The diversity of the Psychology team, with a range of different disciplinary interests helped us reach out to a wider more diverse potential set of participants. However, despite the wide range of participants invited, we recognise that the sample was relatively small and drawn from existing networks, which could potentially introduce some bias. Psychologists in the TURBBO project were involved in Workshop 1b, as they would also likely be end users of the ontology and related tools. Therefore, our project participants were not only external stakeholders but also internal project researchers.

3.2.3. Pre-Workshop Activities

To ensure that all participants shared a basic knowledge of what an ontology is before starting the workshop, we created a simple video to explain what an ontology is, using pizza as an example (see Figure 2). The video was distributed to participants in advance of the meeting, as part of an email explaining how the workshop would work.

3.2.4. Facilitation

The workshops were held using an invite-only Google Meet link, and video and audio were recorded after obtaining verbal permission from all participants (in addition to obtaining informed consent prior to the workshop). One member of the research team led the workshop, introducing the videos and the tasks, whilst another member facilitated the participants’ interaction with the tools. During the workshop, both facilitators used a live document to take observation notes, as well as timing information.

3.2.5. Materials and Activities

Following extensive conversations and subsequent trials testing various online tools such as Google Docs, Microsoft 365, and other standard tools, we decided to focus on tools that are used by the HCI and UX (user experience) communities for wireframing, prototyping, and walkthroughs. The majority of these tools offer features relating to co-designing concept maps, and our trials with tools such as Miro, Lucidcharts, and Google Docs provided initial insights into how they could be used in the ontology co-design process. For example, Miro and Lucidcharts could use an interactive whiteboard for participants to add concepts and organise them in a logical manner. Google Docs or Slides could also allow multiple users to synchronously add concepts to a document/slide, which felt more cumbersome than tools like Miro and Lucidchart, which were purpose-built for tasks such as these.

We chose Miro (https://miro.com/, accessed on 20 June 2025), an online whiteboard software that offers interactive features such as collaborative mind mapping, building workflows, flowcharts, and user stories as the online tool for co-design, in part because Miro allowed participants to vote for concepts. Participants were asked to carry out a set of tasks using Miro. Whilst familiar to researchers in HCI (e.g., those involved in interface design, user experience, website design), Miro was an unfamiliar approach for the workshop participants.

Miro provides a visual environment that enables participants to note down concepts and relationships in an easy and intuitive manner, similar to using sticky notes in a face-to-face setting. It offers support for collaborative editing and for voting, which were identified as fundamental tasks for our co-design approach. Miro also allows all participants to contribute to a collaborative work, while observing the contributions of others. These contributions are recorded transparently, allowing post hoc analysis of how concepts emerged and, at the same time, minimising the ability of facilitators or project teams to edit or dismiss contributions without documentation.

Drawing inspiration from [36], the following three tasks were presented to participants:

• Task 1: “Thinking about behaviour, what are the core (high level) concepts that are needed to describe and model it? Can you list some concepts?”
• Task 2: “Which five concepts are the most important ones?”, “How would you characterise the concepts?” (looking at the concepts everyone has listed)

  -

  “Choose some concepts and add comments with the relevant properties (i.e., name, etc.)”

  -

  “Can you connect pairs of concepts with a relation?”

• Task 3: “Imagine you had a system that uses the concepts and relationships to find relevant information from studies of behaviour. What kind of questions would you be interested in answering using such a system?”

Participants were allowed 15–20 min to perform each task, during which they were encouraged to ‘think aloud’, to vocalise their thoughts and decisions [37]. If the participants did not say much, then the facilitator provided prompts to encourage participants to talk through their actions.

3.2.6. Data Collection and Analysis

Data collected during the workshops primarily consisted of three types: audio and video recordings of the workshops, contributions of participants, and observer notes. Participants were encouraged to ‘think-aloud’ while performing the tasks, which were captured in video and audio recordings. The five workshops resulted in a total of 10 h of audio and video recording. As discussed earlier, each workshop involved the use of Miro as the primary tool. Two Miro boards were created for each workshop, one for Tasks 1 and 2 and another for Task 3. Tasks 1 and 2 helped us to develop the ontology, identifying common themes and concepts, while task 3 led to the development of competency questions. Therefore, for five workshops, we collected 10 Miro boards. Observer notes were collected as the workshops progressed by either of the two observers in a shared online document. This included short-hand notes of specific events during the workshop, reference points, key phrases and quotations, as well as important timings. The workshops were grounded in a qualitative constructionist paradigm, recognising that ontology development in interdisciplinary contexts is inherently interpretive [38,39]. The purpose of the workshops was not to eliminate participant subjectivity but to capture and understand it as meaningful data. In the context of the present research, participants’ disciplinary, experiential, and value-based perspectives inevitably shape how they conceptualise and prioritise elements of the ontology. Within this stance, participants’ disciplinary and experiential perspectives are treated as integral to meaning-making, rather than sources of bias. This position informed both the participatory workshops and qualitative analysis: facilitator influence was managed through structured protocols and transparency, while analytic rigour was established through reflexive dialogue and negotiated consensus as a more epistemologically appropriate form of rigour than statistical inter-rater reliability [40].

Analysis of the data collected involved studying the Miro boards and understanding which concepts emerged during the conversations and how participants approached organising the concepts as a collective. We then conducted an analysis of individual concepts emerging in each workshop to identify concepts that were common across multiple workshops. Following the workshops, we employed a thematic analysis approach [41] in studying the conversations and discussions captured while participants were thinking aloud and, using an inductive approach, determined the themes that emerged. Subsequent reflection meetings with project team members reviewed and validated the themes identified.

4. Results

The data collected during the workshops was analysed from a variety of perspectives to answer the research questions: an analysis of the concepts identified by participants, post-workshop analysis of the workshop recordings (on-screen interactions with Miro and conversations among participants), and an analysis of facilitator notes and observations. We present our results and discuss our findings based on our research questions.

4.1. Co-Designed Ontology

In understanding the concepts and relations identified by the participants, we seek to answer the research question:

• RQ1: How do participants engage with identifying characteristics of (human) behaviour?

We focussed our analysis on the different concepts and relations identified by workshop participants to understand whether and how they could be coherently incorporated in a common ontology. Participants were able to vote for concepts identified by others in the workshops, which allowed us collect data that indicate the community’s agreement of the relevance of these concepts. Our analysis identified the most common concepts and relationships discussed in the workshops. Figure 3 shows a word cloud of the 50 most frequent concepts from all the workshops.

Figure 4 (higher resolution image available at: https://drive.google.com/file/d/1YoJ9zVHQ_rmAyLoutk8I-dXCNK9FyxR6/view?usp=sharing, accessed on 20 June 2025) presents the Miro boards that the workshop participants created when asked to think about the concepts that are needed to describe and model behaviour. Each sticky note represents a concept that the participants felt should be represented in an ontology of behaviour. Links between concepts indicate relationships between the concepts. While the concepts are analysed in the following subsections, the figures illustrate the different ways in which ‘behaviour’ was conceptualised. The figures also highlight the different ways in which participants in each workshop used colours, comments, and relations.

Analysis of the Miro boards initially involved an offline task of creating a matrix of concepts, their hierarchies, and the workshops that they were mentioned in. The matrix also helped us to understand which types of concepts were prioritised by specific workshops. Following this, the facilitators (as a group) went through each concept and filtered out concepts that were too domain-specific (for example, reference to locations) to be in an upper ontology (e.g., habitual biting of nails is too detailed to describe the wider concept of characteristics of ‘behaviour’) or concepts that were too similar to be classified as the same concept (e.g., ‘who’, ‘participant’, ‘actor’, ‘human’). After the initial data cleaning, we identified 142 distinct concepts across all workshops. Among these, 48 concepts recurred in more than one workshop, reflecting commonalities across groups. Figure 5 shows a visual representation of the concepts with the workshops as occurrence points. Our overall analysis of the data identified that there were two kinds of concepts, approximately 90% were about behaviour (e.g., characteristics of behaviour, goal, and motivation), and about 10% were concepts that are associated with studies of behaviour (e.g., participants and measures).

4.2. Characteristics of Behaviour

Four of the five workshops (see

Table 1. Different types of concepts were discussed in the workshops. The Miro boards also reflected these concepts. The table illustrates how the workshops conceptualised the behaviour domain, highlighting the different perspectives of the participants.

Workshop and Composition	Concepts Contributed
Pilot
Internal and external (n = 2)	Characteristics of behaviour, material, goals, participants, temporality
Workshop 1a External academic stakeholders
(n = 6)	Characteristics of behaviour, material, goals, participants, temporality, needs and emotions, norms, beliefs, impact and consequence
Workshop 1b
Internal project researchers (n = 6)	Characteristics of behaviour, participants, temporality
Workshop 1c
Practitioners, businesses, and charities interested in behaviour (n = 4)	Goals, context of behaviour, needs and emotions, norms, belief, impact and consequences, motivations and determinants
Workshop 1d
Practitioners, businesses, and charities interested in behaviour (n = 4)	Material, goals, needs and emotions, norms, belief, motivations and determinants

) explored the characteristics of behaviour. Using the ‘mindmap’ metaphor of a Miro board, these characteristics were represented by participants as concepts. Since the central component of a behaviour ontology is the broader concept of behaviour, these characteristics were represented as data properties within the ontology, with a boolean data type to represent true or false. For example, in one workshop, participants added concepts such as ‘habitual behaviour’, ‘behaviour is changeable’, ‘automatic behaviours’, and ‘complex or simple’. Such concepts were described in the ontology as properties of behaviour such as ‘isHabitual’, ‘isChangeable’, and ‘isEffortless’.

We recognise, however, that many of the characteristics of behaviour (e.g., whether they are habitual) are likely continuous (e.g., behaviours vary in the extent to which they are habitual ranging from fully conscious goal-directed actions to completely automatic cue-driven behaviours) and that describing characteristics of behaviour using boolean values is essentially an artificial dichotomisation. This decision was taken to simplify the characterisation of different behaviours, facilitating comparison of behaviours (e.g., users could compare habitual vs. less habitual behaviours), because this was how participants in our workshops referred to behaviours (e.g., a ‘habitual behaviour’). Although it is possible for ontology classes to have continuous properties, it would be difficult to estimate where on the continuum a given behaviour falls (e.g., the extent to which alcohol consumption [BCIO:036092] is habitual), especially given that this likely varies between people and within contexts (e.g., [42]). However, it is probably reasonable to decide whether a behaviour is or is not likely to be habitual (e.g., likely to be performed frequently in a stable context [43]), and indeed, [44] did this in order to compare the effects of changes in intention on habitual vs. less habitual behaviours. Therefore, although artificial, we believe that dichotomising the properties of behaviour means that characterising behaviours becomes feasible and facilitates the kind of comparisons that researchers are likely to be interested in (i.e., comparing different ‘types’ of behaviour). The difference and the gap between domain and ontology experts is well described in the literature [45]. As such, this observation was expected, and for the context of this workshop, it was more important to explore whether an online workshop allowed the community to bring their ideas together through a collaborative exercise.

4.3. Identifying Associated Concepts

A primary reason for engaging with a range of stakeholders was to capture a range of views and perspectives. Our analysis first explores the conversations during the workshops. We observed two types of discussions: participants in some workshops (1a,1b) tended to focus more on theory, while in other workshops, discussions tended to be around aligning experience to concepts (1c,1d). Table 1 highlights the different types of concepts that participants contributed.

All workshops identified knowledge and skill as key concepts when studying behaviour. The goals (of behavior) were also identified by the majority of the workshops (four of five). It was interesting to note that a majority of the concepts identified by the participants in the pilot, 1a, and 1b involved characterising the types of behaviour (i.e., a behaviour is habitual) as opposed to different behaviours (i.e., exercising, recycling). On the other hand, concepts identified by Workshops 1c and 1d aligned more with describing motivations and determinants of behaviour, needs and emotions, norms, and beliefs. It may be that this difference in characterising concepts within behaviour comes from applied experience: whereas practitioners (in Workshops 1c and 1d) described their domains based on their own experiences, researchers (in the pilot and Workshops 1a and 1b) expressed the behaviour domain as a means to define the different characteristics that might define a behaviour. This also highlighted the benefit of engaging a wider set of stakeholders in developing our ontology.

In addition to characterising behaviours, participants identified broader concepts that are related to the concept of behaviour. These concepts centred around four themes: measurement of behaviour (e.g., impact, outcomes), context of behaviour (e.g., barriers, goals, motivations), temporality of behaviour (e.g., going for runs in the morning), and co-occurrence of behaviours (e.g., smoking when drinking). Following further discussion of how concepts could be grouped together into broader categories, we developed a representative ontology that was initially visualised via a Miro board and then developed as a formal ontology using Protégé (see Figure 6) and shared with internal project members.

Our ontology has several key components, labelled A–G. Central to the behaviour ontology is the Behaviour class (labelled as ‘A’), instances of which will represent a specific behaviour. Our workshop participants identified a range of specific behaviours (such as running, cycling, pro-environmental behaviours), which will be developed through a middle-level ontology (categorising the different types of behaviour) together with the community. Behaviours have temporal aspects such as the frequency or the duration of the behaviour. These aspects are labelled as ‘B’. Another core aspect of our study is the relationships between behaviours that have been reported in the literature. There are two ways we represent this in our ontology: (i) two types of object properties; (ii) correlation measures. The two types of object properties are captured as

<Behaviour> correlatesWith <Behaviour> <Behaviour> largeCorrelation <Behaviour> <Behaviour> mediumCorrelation <Behaviour> <Behaviour> smallCorrelation <Behaviour>

where large, medium, and small correlation object properties are subproperties of the ‘correlatesWith’ object property. We expect the ‘correlatesWith’ relation to capture any correlation, while specific object properties are designed to capture different strengths of correlations. These will help in future stages, when we will explore the use of data analytic methods to identify potential correlations between behaviours not previously studied. The subproperties model the degree/strength and type of correlation using Cohen’s criteria [46] for the strength of correlations (i.e., if the correlation coefficient ‘r’ is 0.10, then it represents a small association; if ‘r’ is 0.30, then it is considered a medium correlation; if ‘r’ is 0.50 or more, then the association is large). It is important to note that since our ontology has not yet been aligned with real-world data, our starting point is to use Cohen’s criteria for interpreting the size of correlations. Specifically, r = 0.10 is a ‘small’ correlation, r = 0.30 is a ‘medium’ correlation, and r = 0.5 is a ‘large’ correlation. However, in the future, with exposure to real-world datasets, it is possible that the characterisation of correlation between behaviour could be revisited by the community.

To support more fine-grained analysis, we capture correlation coefficients, correlation types, and details about the measures used in the study (statistics, effect size, etc.), as can be seen in the section labelled ‘C’. Our workshops also centred around the wider contexts of behaviours such as the different motivating factors that can influence the nature of the behaviour (extrinsic factors, psychological factors, etc.), as shown in the area labelled ‘D’. Our participants noted the importance of capturing the outcomes and impacts of behaviours, particularly when these behaviours are aimed at achieving certain goals. For example, interventions that aim to encourage pro-environmental behaviour are eventually aimed at improving environmental outcomes. Hence, we captured this using Outcomes and Impact, as can be seen in the area labelled ‘E’. As the ontology will be used to capture correlations between behaviour reported in the literature, we also aim to record publications that have reported the correlations, as can be noted in the area labelled ‘F’. Finally, our ontology captures details about the actors that have been involved in the study that reported on the behaviour, who can be an organisation, a person or a group, as shown in the area labelled ‘G’.

Our upper level ontology is a work in progress, and the latest version is available at https://github.com/fatibaba/turbbo/blob/main/upperlevel/turbbo_upper_level.owl, accessed on 20 June 2025. The ontology metrics, which we extracted from Protégé, are presented in

Table 2. Ontology metrics showing a breakdown of concepts from the upper level and reused ontologies.

Metrics	Upper Level	FOAF	OM	SEPIO	Time
Axiom	651	70	24	163	88
Logical axiom count	170	15	3	12	12
Declaration axioms count	168	13	6	28	15
Class count	73	4	2	16	6
Object property count	43	2	2	0	5
Data property count	25	6	1	0	1
Individual count	0	0	0	0	0
Annotation property count	31	4	3	14	6
Class axioms
SubClassOf	46	3	0	12	4
EquivalentClasses	0	0	0	0	0
DisjointClasses	1	1	0	0	0
Object property axioms
SubObjectPropertyOf	2	0	0	0	2
ObjectPropertyDomain	41	1	1	0	2
ObjectPropertyRange	39	0	1	0	4
SubPropertyChainOf	0	0	0	0	0
Data property axioms
SubDataPropertyOf	0	0	0	0	0
FunctionalDataProperty	3	3	0	0	0
DataPropertyDomain	18	2	1	0	0
DataPropertyRange	20	5	0	0	0
Annotation axioms
AnnotationAssertion	297	42	15	123	61
AnnotationPropertyDomain	4	0	0	0	0
AnnotationPropertyRangeOf	1	0	0	0	0

. As this paper reports on the initial co-design phase, the metrics in Table 2 should be interpreted as early indicators of structure and coverage; formal benchmarking against mature behavioural ontologies such as BCIO will be undertaken in later iterations, once the ontology has reached full alignment and stability. In this implementation, we did not explicitly adopt an established upper ontology standard such as BFO or DOLCE. However, for subsequent iterations, we will have a stronger alignment with BFO, since some of the ontologies we reused (e.g., IAO) are aligned to BFO. Since many ontologies in the behavioural science domain align with BFO, we recognise both the importance and the practical benefit of aligning our ontology to it. At this time, we postponed committing to a specific upper ontology to avoid constraining participants’ conceptual input during knowledge elicitation. Interoperability risks were mitigated by reusing BFO-aligned modules, such as the Information Artifact Ontology (IAO) and the Scientific Evidence and Provenance Information Ontology (SEPIO), and by maintaining a separate bridge module that allowed later alignment without structural refactoring. Although this initial implementation did not explicitly adopt an upper ontology such as BFO or DOLCE, subsequent iterations of the ontology (available at https://github.com/fatibaba/turbbo/, accessed on 20 June 2025) include a stronger alignment with BFO. In particular, the extended version incorporates the individual human behaviour class from the BCIO_behavior ontology, which is itself BFO-aligned. As many ontologies in the behavioural-science domain follow BFO, this staged approach ensures conceptual flexibility during early co-design while making later integration and interoperability both straightforward and loss-free.

We integrated selected terms—covering agents and person attributes, measurement units, evidence structures, and temporal concepts—from the Friend of a Friend Ontology (FOAF) [47], the Ontology of Units of Measure (OM) [48], the Scientific Evidence and Provenance Information Ontology (SEPIO) [49], and the W3C Time Ontology [50]. During integration, we encountered typical reuse challenges, including (i) semantic overlap (e.g., people/agents and time/duration concepts), (ii) differences in modelling granularity (such as SEPIO’s detailed evidence model versus our simpler annotations), and (iii) occasional property-type inconsistencies. To manage these issues, we applied a lightweight mapping workflow in which a canonical ontology was chosen for each domain (e.g., W3C Time for temporal relations, SEPIO/IAO for evidence and information artefacts, OM for measurement units, and FOAF for agent attributes).

Imported terms were reused as-is, and local IRIs were minted under https://purl.org/turbbo#, accessed on 20 June 2025 with stable human-readable identifiers. Where local concepts refined imported ones, rdfs:subClassOf axioms were asserted (for example, TURBBO:EffectSize and TURBBO:Magnitude were defined as subclasses of IAO:0000027 “data item”), and SEPIO/IAO alignment was achieved through the axiom SEPIO:0000125 (“research study”) → TURBBO:produces → IAO:0000310 (“document”).

W3C Time relations (time:hasDuration, time:Interval) were paired with OM units (seconds, minutes, etc.) to ensure consistent temporal representation. Alignment axioms were maintained in separate import modules: SEPIO-imports.owl, FOAF-imports.owl, om-imports.owl, and time-imports.owl, which were then referenced in the main turbbo_upper_level.owl file, preserving the stability and auditability of the core ontology.

The breakdown of concepts we imported from these ontologies are also presented in Table 2. We describe the ontology in greater detail in [51], and the specifics of the developed ontology are outside the scope of this paper.

4.4. Participant Experiences

We analysed how the Miro boards emerged in the workshops, and through conversations following the discussions, sought to understand the experience of the participants in the online activity. Through this, we answer the research question:

• RQ2: How do participants experience a “hands-on” ontology co-design process in an online setting?

4.4.1. Nature of Online Collaboration

The workshops were organised in January 2021, by which time most organisations had transitioned to fully online working for many months due to the Covid-19 pandemic. As a result, all participants were familiar with online conferencing software such as Google Meet, Zoom, and Microsoft Teams. However, while this meant that discussions and conversations could proceed in a form that was as close to natural as possible, most participants had limited experience with online collaborative co-design tools such as Miro, particularly while simultaneously interacting via an online conferencing tool.

In each of the workshops, we noticed that there was an initial period of silence, where all participants independently added their concepts on the Miro board (e.g., using sticky notes). It is likely that the participants were familiarising themselves with the task and the Miro interface. After several initial concepts appeared on the whiteboard, group discussions started.

When the workshops were too quiet, the facilitators encouraged participants to “think aloud” [52] when contributing to the Miro board. Conversations primarily concerned explaining why certain concepts were added and how the concepts could be connected. Discussions also centred around explaining why concepts were positioned in the whiteboard at specific locations or creating specific clusters. For example, one participant mentioned (pointing to one of the concepts):

• “Where should it go… I would probably leave it there.”

Another participant asked “Is there social norming anywhere?”, checking whether a concept reflecting social norms had been added to an already populated Miro board before adding a new concept. Other discussions involved explaining or asking what specific concepts meant. For example,

• “Can someone remind me what was the value exchange?”,
  which was followed by another participant responding with examples of value exchange, explaining how the concept links with behaviour.

All groups clustered concepts together into themes, and some colour-coded concepts based on themes. The process of moving concepts around the Miro board involved a degree of discussion, where the participants explained what they were doing. In Workshop 1b, one participant reorganised the whiteboard for everyone to keep it tidy. The extent to which the concepts were connected with explicit links also varied among the workshops, such that some ontologies were highly interconnected (e.g., Workshop 1d), whereas other ontologies had limited connections (e.g., pilot, Workshop 1a). One explanation might be that workshops focussing more heavily on concepts resulted in fewer links being expressed in the final ontologies.

4.4.2. User Experience

Following the workshops, we asked participants to share their impressions of using Miro to co-design an ontology. Although the Miro tool is commonly used for co-designing user interfaces, workflows, and mind-maps, our participants did not have prior experience with the tool. Therefore, their impressions of using the tool for co-designing the ontology, were mainly reflections on the tool, rather than the application of the tool for ontology development. Nevertheless, we can derive some valuable insights from what participants felt about the features of the system, in comparison to a face-to-face workshop. Overall, all of the participants expressed their positive impression of the interface. For example, following the workshop, one participant commented:

• “I have to say, I absolutely love this this system!… I find it mesmerising watching people move about.”

To this comment, a participant responded about a feature of Miro (which can also be disabled), which shows every participant’s mouse movement:

• “I like the fact that you can literally see people’s thought process by the movement of their cursor.”

The first participant responded:

• “Yeah. Exactly! I need to stop moving around while I’m thinking”

As such, one of the advantages of the online approach to co-design is the collaboration experience: having an overview of the collaborative process itself provides another level of understanding of the contributions that every other participant makes. That participants can observe the contribution that other participants bring to the collaborative process is a risk for both online and face-to-face workshops. Having these contributions emerging in a transparent manner can potentially cause discomfort and concerns for participants. Nevertheless, Miro boards offer anonymous logins, which reduces this transparency sufficiently enough to help collaborators contribute without the concern of being identified.

4.5. Discussions and Interactions During the Workshops

In each of the workshops, the participants spent the first 10 min discussing the concept of behaviour between themselves using examples, to ensure there was a shared understanding. There was quite a bit of hesitation:

• “I don’t know if I understood what you were getting at”: trying to confirm whether the difficult concepts have been sufficiently well articulated.

• “I just needed some moral support for that” before adding the concept, confirming with other participants. In understanding how the conversations emerged during the workshops, we answer the following research question:

• RQ3: What kind of discussions and interactions emerge when co-designing an ontology in an online setting?

4.5.1. Think-Aloud Strategy

When a sufficient number of concepts were added to the Miro board, the facilitators prompted the participants to share their views as they added the concepts. For example, one of the prompts asked participants to explain why ‘thinking-aloud’ would be helpful:

• “… it would be very useful if you could try to verbalise why you are doing specific things or what you refer to when you create specific concepts so the others may suggest and agree…”

• “So it’s really helpful if we do this a bit like a think-aloud study so we say what we are writing.”
  The prompts initiated a variety of conversations, most of which could be categorised as (i) announcing contributions; (ii) co-working discussions; (iii) seeking opinions; (iv) seeking clarification; or (v) inviting contributions. We identified these types of conversations by manually re-playing video recordings of the sessions and coding the different themes of the discussions using a thematic analysis approach [41]. We describe the different types of conversations together with some examples in the following subsections.

4.5.2. Announcing Contributions

One of the most common themes emerging from the discussions involved participants announcing the specific contributions that they were making as they progressed through the tasks. Participants often announced when they created some concepts that (they believed) required some conversation. For example, pointing to a specific section in the ontology, one participant announced their interpretation of behaviours being individual or collective:

• “I’m making a little cluster here of aspects that involve other people: individual vs. collective.”
  Participants would, additionally, also often provide some rationale for why they created a concept or positioned the concept in a specific location and why they felt that the concept was important. At times, this would also involve questioning the meaning of concepts that others had created. For example, one participant said:

• “…I moved certain concepts near to one another, i.e., habitual behaviour, frequency which is a component of whether behaviour is habitual or not. I initially thought [about] whether a behaviour is biologically driven, thinking of food- and sleep-related behaviours… and somebody else had written ‘need’. Maybe we were thinking on similar lines, perhaps a broader construct? So, I positioned them close to one another.”

Occasionally, participants would explain how their contributions aligned with others. For example, one participant (while referring to a section of the Miro board) said:

• “So, I’ve got this one here: context and shifting priorities. So with that… I mean…maybe in one circumstance, my behaviour would be one thing but in another, in a different circumstance, I don’t know… I mean a different group of people, or in a different geography, my behaviour would change.” In reply to the participant raising the context of the behaviour, another participant explained how their concepts align with the broader topic of the context of the behaviour:

• “Yeah, it also links to this group here (pointing to another section of the ontology). So with compliance, personal desire, wanting to do the behaviour, that might also change with your wanting to do and the motivation of the behaviour.”

These types of conversations were usually followed by further discussions from other participants on how, in their experience, the concept would be applicable. These led to wider discussions on the links between different behaviours and how they had been captured in the Miro board.

4.5.3. Discussions During Collaboration

Often during the course of the workshop, multiple participants worked on a specific area of the Miro board concurrently. This was either due to initial conversations focussing on a specific area or one participant drawing others’ attention to a specific topic of their interest. For example, one participant noted the importance of complexity in behaviour:

• “Just added something about complexity, thinking about something very simple almost reflex-related behaviours vs. much more complex behaviours and how a lot of the drives might interact in different levels depending on the complexity of the behaviour.”
  Another participant, in response, mentioned that their contribution also aligned with the notion of complexity, albeit with a different framing:

• “I put something related to that; so, I put effortfulness, ease, and difficulty and positioned it next to the complex tag.”
  During the task, participants also announced how they were contributing to the co-design of the ontology. For example, in deleting or adding new concepts:

• “It’s there already. I’ll delete that.”

• “I stuck that in, and I just stuck a couple of examples in.”

4.5.4. Asking for Opinions

Some participants asked others about their opinions on their contributions, which initiated further conversations. For example, a participant asked:

• “I’ve just put something on ‘behaviour is changeable’, but I’m now thinking that some behaviour is changeable in some circumstances… I’m just wondering where that might fit?…”

Another participant responded to this question: “So are there behaviours that are changeable then?” A third participant joined the conversation, indicating the notion of the extent of change in behaviour:

• “Behaviours are harder to change, aren’t they? Can we say they are completely unchangeable?”
  The second participant responded, linking to the effort involved in conducting a behaviour:

• “Maybe it links to the effort for (the) most bit…” The first participant, who initiated this discussion, also indicated the role of habit:

• “I think it partly relates to habit and habit formation…”

Some participants also asked for opinions on where to position the concepts and to confirm whether their suggestions were aligned with those of others. For example, one participant asked where the motivators and barriers of behaviour could be positioned in the ontology, announcing that they felt a specific location was the most appropriate:

• “I’m just wondering if short term/long term cost benefits might go somewhere closer to prevention/promotion? Whether these things are positive or negative, whether we are promoting them or preventing them. So just shifted that there…”

4.5.5. Seeking Clarification

Participants, having expertise in multiple domains, often shared why they expressed a particular concept in their ontology. In doing so, they would often explain how they represented the concepts, as well as the location that they felt would be the most appropriate. However, they also sought to find out why others had mentioned specific concepts in order to know more about the context behind their choices. For example, one participant was not sure what the concept ‘world view’ meant to the author of the post:

• “Who wrote that behaviours co-occur? Because I was kind of thinking the same thing.”
  One participant asked:

• “I was thinking about the idea of world view: who put world view in there? what did you mean by world view?”
  Another participant responded:

• “That was me. I guess I was trying to think about a collection of influences about how individuals see the world, that’s often influenced by values, social norms… as a broader umbrella of how we see the world-threats, opportunities…”
  Some participants also wanted to clarify the meaning of existing concepts before adding any new concepts. At times, requests for clarifications invited some further discussions around the topic, where participants shared why the concepts that were added were relevant, based on research or experience in practice.

4.5.6. Inviting Contributions from Others

Another type of conversation that we observed involved the participants posing a specific question or a scenario to others, following which, other participants were invited to collaborate on a specific section of the ontology. For example:

• “I’m making a little cluster here of aspects that involve other people; so, I’ve got individual vs. collective, whether the behaviour benefits others or not, does the behavior necessarily require others. I don’t know if others have other ideas that could be added to this cluster?”

All of the workshops involved these five types of conversations, often centred around different topics or scenarios, while participants were working on the tasks. In summary, the real-time collaborative platform provided a deeper insight into how the participants engaged with the task and with each other.

4.5.7. Participant Choices

During the workshop, participants were asked to provide a set of upper-level concepts and a set of possible relationships between the concepts. The Miro board aimed to facilitate this process. However, it also introduced the possibility that participants needed to make additional choices, e.g., about how to position and cluster concepts. Much of the conversation during the tasks centred around how the participants positioned their sticky notes, thereby creating clusters. From an analysis of the observations, subsequent conversations, and video recordings, we determined that the positions were mostly either based on aligning sticky notes based on their broader themes or individual participants working on specific regions of the Miro board. During the discussions, sticky notes were moved as a part of conversations to their final positions. However, the reasons behind participants’ choice of colours for their sticky notes was not as clear. As Figure 4 shows, participants used a range of different colours. A project team member asked the participants about their use of colours for the sticky notes:

• “…when you click ‘sticky notes’, you get to choose the colour. I noticed that people are creating notes of different colours. I wonder if there is any purpose to their colours?”

To this question, some participants highlighted their prior experience and familiarity with colours, such as:

• “not for me, (team member name)—I used yellow because I am familiar with yellow post-it notes.”
  Another participant (jokingly) added: “I think you are giving us too much credit”.
  A third participant added that their choice of colour was to differentiate their contribution from others: “The blue one’s just me.”

The reasons underlying some of the colour choices were, therefore, unrelated to the content of those sticky notes. However, some participants noted that they used colours to indicate the themes of the concepts, and how they used the colours to differentiate concepts, while at the same time to facilitate the collaborative process:

• “So just thinking, where we’ve got the purply blue colour vs. our pink colour, I feel like … I don’t know there is a distinction between them that I am not going to be able to articulate, but I think for me…maybe the determinants is almost like we’ve got high level determinants like the environment and then we have the proximally distal ones which then map to some of the pink ones…so we could look at the determinants that way? Would others be okay with that?”

4.6. Potential Applications

In understanding the potential application contexts of an upper level ontology of behaviour, we analysed the responses that participants provided for Task 3, where participants were asked to reflect on the questions they would be interested in answering using a system that uses the ontology. We answer the following research question:

• RQ4: What potential application contexts do participants envisage the ontology to support?

Our analysis of participant reflections on how they would benefit from a system based on the ontology highlighted how they noted the value of the ontology as a mechanism for structured querying. Most often, the questions were fact-based, looking for specific studies, sample sizes of studies, or names of behaviours. These questions were collected through a Miro board, as sticky notes. We collated these questions, analysed the broader themes of the questions, and developed competency questions. Eight competency questions were developed as a result, as shown in

Table 3. Competency questions and candidate answers based on participant responses to Task 3.

Competency Question	Candidate Answer
What is the name of the study that measured/observed this behaviour?	Name of the study
What is the sample size of the study that is used to measure this behaviour?	Sample size values
Given a behaviour, what is the name of a correlated behaviour and the correlation coefficient?	Name of the correlated behaviour (e.g., recycling, sleeping), a value of r (e.g., 0.1, 0.5)
Where does ’this behaviour’ occur (location)?	At home, at work, at the park, etc.
If this behaviour requires the use of materials, what are the materials?	Car, Walking stick
What is the weighted correlation between two behaviours that are measured in more than one study?	A calculated weighted correlation (Pearson, Spearman, etc.)
What is the association between two behaviours that have not been measured/compared in the same study	The relationship represented as the list of behaviours that link the two behaviours
Is this behaviour legal?	Yes, No, Unsure

.

As can be seen, the questions revolved around specifics of studies that observed behaviours (such as study names, sample sizes) or specifics of behaviours (legality, use of materials, correlated behaviours). The competency questions were designed to ensure that the ontology covers all the necessary concepts to describe a behaviour in the context of how it was measured and how it relates to another behaviour. These questions are designed to represent the typical needs of the community and how they expect to compare behaviour measured in different studies.

Once we implemented the ontology into an OWL format, we manually collected five papers that measured a range of behaviours. We added the data elements and mapped them to their corresponding classes in our ontology. With the implemented ontology, we could construct SPARQL queries on the collection of the papers. As a short example, the question ‘Given a behaviour, what is the name of a correlated behaviour and the correlation coefficient?’, and as the focus of the behaviour being ‘Moving/Exercising’, we constructed the SPARQL query as follows:

  PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#>

  PREFIX rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#>

  SELECT ?behaviourA ?correlation_r ?behaviourB

  WHERE {

    ?y rdfs:label ?behaviourA.

    ?r rdfs:label "correlation coefficient r".

    ?relates rdfs:label "correlates behaviour".

    ?rValue rdfs:label  "rValue".

    ?rexample rdf:type ?r.

    ?rexample ?relates ?y.

    ?rexample ?rValue ?correlation_r.

    ?rexample ?relates ?x.

    ?x rdfs:label ?behaviourB.

    filter regex(?behaviourA, "Moving/Exercising")

    filter(?y != ?x)

  }

The SPARQL query provided answers based on our dataset as shown in

Table 4. Answers to SPARQL query on behaviour correlations with Moving/Exercising run on a collection of papers.

Behaviour A	Correlation_r	Behaviour B
Moving/Exercising	0.19	Eating Vegetables
Moving/Exercising	0.01	Smoking Cigar(ette)
Moving/Exercising	0.69	Using Alcohol
Moving/Exercising	0.11	Using Alcohol
Moving/Exercising	0.16	Smoking Marijuana
Moving/Exercising	0.125	Smoking Cigar(ette)

.

We used a local installation of Apache Jena Fuseki to run SPARQL queries to evaluate our ontology. The details of the evaluation are out of scope for this paper, but they are described in our GitHub page, available at https://github.com/fatibaba/turbbo/blob/main/evaluation/CompetencyQuestions.md, accessed on 20 June 2025.

5. Discussion

The experience of organising and running the co-design workshops online was informative but, at times, challenging. In this section, we first reflect on the benefits and challenges of using the online platform for ontology co-design tasks. In this, we contrast this experience with our prior experience developing ontologies in face-to-face settings [53].

Several of the challenges that we encountered were similar to the challenges of online meetings. For example, identifying common times that each participant could attend was challenging, particularly considering that participants came from different time zones. Consent forms also needed to be pre-collected in order to ensure that time was effectively utilised. In short, considerable preparation was needed ahead of the online sessions, which at the time of a global pandemic was difficult, considering shifting priorities, the challenging research landscape, and increasing teaching loads, as well as uncertainties in organisations (such as job risks and furloughing).

Challenges also emerged from the mode of delivery. For example, bandwidth across participants and different computer setups varied, which meant that participants experienced the ‘online’ nature of workshops differently. Running a resource-intensive website like Miro (which involved multiple collaborators at the same time) together with video conferencing software in a browser was challenging for participants with devices that have lower specifications. An extension of this challenge is, therefore, the risk that some participants would not be able—or want—to contribute to online workshops.

Despite these challenges, we believe that online co-design workshops offered a wide range of benefits that would not have been possible in a traditional face-to-face setting. From an organisational and participation perspective, a considerable benefit was widening the possible participants in our project—with an online setup, experts from around the world could contribute to the conversations and collaborate in the co-design process, without incurring any travel costs and time. All of the workshops were easily recorded, which captured all of the on-screen interactions between participants as well as conversations, without the need for cumbersome microphone and camera arrangements that would be required for a face-to-face setup. The Miro boards were also easily accessible and archived, as well as shared with the participants and project members for easy exploration at any time. Unlike face-to-face setups, where co-design activities result in several static images, Miro boards can be dynamically explored and interrogated in future (e.g., to re-visit the comments or votes of the participants).

From the perspective of the co-design process, simultaneously viewing the Miro board during the workshop allowed participants and organisers to have an overview of how the co-design process evolves, allowing participants or facilitators to probe different elements of the ontology. In a face-to-face setting, it can be cumbersome if all participants are working on one artefact. The process of every participant concurrently working on different elements of a single artefact was also very helpful, while difficult to achieve in a face-to-face setting. Another benefit of an online setting with a collaborative tool is that participants can see how other participants are contributing. In face-to-face settings, while participants can observe others collaborating on a particular section of the ontology, it could be possible to identify who added topics. In an online setting, participants can choose to be anonymous and, hence, be able to freely add their contributions without concerns of being identified.

The Miro interface offered a range of features which simplified many processes, for example, participants could easily change the colours of their concepts. Participants could also easily edit their contributions. Participants were also able to add comments on any concept as they felt. The tasks required participants to vote for concepts they felt would be the most important in the domain: this was easily achieved through a polling mechanism made available in Miro. The votes are captured for easy analysis after the event. All of these things would be difficult to organise and capture in a face-to-face workshop.

Furthermore, the process of drawing links between concepts (e.g., with just a mouse click and drag gesture) seemed seamless for participants in the online workshops. In a face-to-face setting, this is often performed using physical strings, but it is challenging to re-organise entire layouts of ontologies. Finally, the ontology concepts could be easily moved around by participants without disconnecting the other concepts. The ease of this process meant that participants had flexibility in organising the colours, comments, connections, and arrangement of the Miro board, while also sharing reasons for their decisions.

5.1. Recommendations for Using Online Tools to Support Ontology Co-Design

Finally, we reflect on our experience of running the co-design workshops to provide some practical suggestions for the organisers of future workshops:

5.1.1. The Value of Pre-Recording

Our initial pilot helped us to make some decisions about how to facilitate the workshops with external participants. In particular, owing to the time constraints of online meetings and the availability of experts, we decided to pre-record all of the content that would explain the context of the project, ontology, and activities in the workshop. The recordings also helped maintain consistency across the workshops and helped us adhere to the planned timeline.

5.1.2. Recording Workshop Sessions

One of the benefits of organising co-design activities online is that all interactions happen digitally and, as such, are easily captured. Following obtaining consent, video recording the entire workshop enabled us to revisit the workshop at any point in time and to review the workshop as it happened. Scrolling through the video of the workshop also helped us to identify where interesting discussions took place. We also found it helpful to share the Miro screen within the workshop meeting, so that the video recording could capture the on-screen co-design process.

5.1.3. The Value of Note-Taking

One aspect to highlight is that, during live sessions, videos of all participants are active. However, the recorded sessions only capture the primary speaker’s video. As such, any non-verbal clues of other participants are lost in the recording. It is, therefore, important to also take notes of such observations along with specific timestamps for future analyses. Furthermore, long workshops result in long recordings; revisiting workshop videos can be tedious and time-consuming. Manual notes, recording conversations of interest for future analysis, should also include the specific times of these events.

5.1.4. Multiple Facilitators with Dedicated Roles

We found that the facilitator played a considerable role in ensuring that the workshop proceeded as planned, all participants were engaged, and they felt comfortable sharing their views and inputs. As such, it is probably necessary to have multiple facilitators. In our workshops, one facilitator was engaged with the participants, while the other facilitator took notes, recorded timestamps, and provided support to participants who needed technical help. The defined roles of the facilitators helped to run the workshops smoothly and according to plan. As in a face-to-face setting, having these clear roles divided among multiple facilitators can be helpful in the organisation of the workshops.

5.1.5. Length of Workshops and Importance of Pilot

Our pilot indicated the estimated times for each task, which helped estimate the total length of the workshop. We originally scheduled the workshops for one hour. However, we quickly realised that this time would not provide the depth of insights required for the purpose of the project. We therefore decided to extend subsequent workshops to two hours, thereby allowing sufficient time for each task and the subsequent questions that followed naturally. However, upon reflection, we believe it would have helped to have had a short break in between, due to the relatively intense nature of the tasks.

5.1.6. Participant Numbers and Stakeholder Groups

Our workshops also highlighted the importance of a small number of participants in the co-design tasks. At this early stage, it is important to recognise the variety of concepts emerging from the discussions and co-design work. While we had not conducted a systematic evaluation with different numbers of participants, we quickly recognised that our workshops with six participants had a strong engagement that was also observed in workshops with four participants. However, we anticipate that more than six participants could introduce difficulties in structuring the co-design activity and supporting conversations between participants. As such, with a higher number of total participants, our recommendation would be to introduce more workshops instead of larger participant numbers for a workshop. There could be different combinations of participant types (practitioners, researchers, decision-makers, policy-makers, etc.) that might bring interesting perspectives in different workshops. We also observed that several concepts emerged across multiple workshops, which can suggest that, given a domain, a few workshops might be effective at capturing most of the key concepts. However, the number of the workshops needed might depend on the complexity of the domain. Future work could conduct a more systematic analysis of how many workshops would help capture concepts most effectively.

5.2. Future Directions and Applications

While the focus of this paper is on the process of co-designing the ontology, our future work will involve collaborating with the behaviour research community in developing applications. The first set of activities will involve future workshops to understand how closely the developed ontology meets the requirements of the stakeholders beyond the competency questions. This will require stakeholders to use standard ontology authoring tools (e.g., Web Protégé) to study our ontology and align the ontology concepts with papers reporting on behaviour studies. We will then work with the community to further build on our existing work on identifying overlaps between ontology classes within behavioural research [1]. The next set of activities will use data mining methods to automatically align the ontology concepts to a large repository of behaviour study papers, which will help develop a knowledge graph. We aim to develop a dashboard that can then provide the community with mechanisms to interrogate the knowledge graph using complex querying mechanisms. For example, a researcher who was interested in whether poor sleep is associated with greater engagement in habitual behaviours could use the dashboard to explore links with sleep, selecting behaviours on the basis of habitual property. Finally, we aim to investigate how our design decision of using boolean properties meet the needs of the community through a series of workshops. At this stage, we would explore fuzzy logic, probabilistic, or hybrid approaches to capture uncertainties or continuous properties.

The ontology developed within this project can be accessed via GitHub: https://github.com/fatibaba/turbbo/blob/main/upperlevel/turbbo_upper_level.owl, accessed on 20 June 2025 and has been added to the Behavioural and Social Sciences Ontology (BSSO) Foundry [54]. Although there are no formal mechanisms for maintaining the ontology, the team continue to work in the field (e.g., via funded projects such as Behavioural Research UK www.br-uk.ac.uk, accessed on 20 June 2025 and APRICOT Ontology Tools, https://www.ucl.ac.uk/behaviour-change/research-projects/2025/jul/apricot-project, accessed on 20 June 2025) and explore mechanisms for developing the ontology further, including integrating it with other ontologies [14]. In addition to these efforts, we have also been involved in further activities on annotating datasets in behavioural and social sciences to promote interoperability [55].

6. Conclusions

The research described in this paper was designed to inform the development and use of ontologies of behaviour in a project aiming to help researchers and practitioners understand relations between different behaviours. In this, we adopted an online co-design approach using a standard online collaborative whiteboard platform. Although it is a common practice in the HCI community for user interface development, to our knowledge, the use of online synchronous co-design is limited in ontology development.

We answered our research questions using data collected during the workshops, which primarily comprised video recordings, facilitator notes, and whiteboards co-developed by the community. We summarised the final ontology thereby developed, with an analysis of the concepts identified by the participants, particularly reflecting on how researcher and practitioner workshops focussed on different themes. Based on our conversations and analysis of data, we reflected on how participants experienced the online co-design process. Using thematic analysis of conversations during the workshops, we explained how different types of conversations occurred and how they facilitated the co-design activity.

While we share our experiences with using an online synchronous co-design approach and offer recommendations on using online tools for co-design, we acknowledge that we have not conducted a comparative analysis with traditional face-to-face settings. We believe it would be valuable to conduct comparative evaluations of different settings in co-developing ontologies as an extension of our work in the future.