A Meta-Analysis of Studies on Groupware for Collaborative Work Environments ^†

Abstract

This meta-analysis examines the studies on groupware published between the years 2010 and 2020. Descriptive analysis was conducted to determine the distribution of studies in terms of publication year, time–space matrix category, targeted sectors, research methods, and the academic field of the journals that published these studies. Although groupware played a significant role in communication, collaboration, and coordination of users in various collaborative work conditions and sectors, the majority of studies focused on asynchronous and distributed collaborative work environments in the software engineering field, and the research method preferred was design science.

1. Introduction

Groupware, also known as collaborative software, is defined as “computer-based systems that support groups of people engaged in a common task (goal) and that provide an interface to a shared environment” [1] (p. 40). As a particular aspect of the field of computer-supported cooperative work (CSCW), groupware systems are designed to support geographically and temporally distributed work groups or teams for effective collaboration towards a common goal, solution of a common problem, data sharing, idea generation, conflict resolution, decision making, or system analysis and design [2,3]. According to Ellis et al. [1], for successful design of groupware, social processes and procedures should be delicately balanced with appropriate technology. Since technology might influence people, social interactions, and how and what to work on, these influences should be taken into consideration in the groupware development process [1]. Therefore, it is essential for understanding the diverse needs and characteristics of organizations and sectors in integrating groupware into their work.

A widely referred classification of groupware was created by Johansen (as cited in [3]) as a 2 × 2 matrix of time and space variables, as shown in Figure 1. Each type serves different needs and refers to different technologies. For example, for ongoing coordination of workers separated by both time and space, group writing tools can be used.

As information communication technologies get increasingly integrated into organizational activities, and the need for supporting geographically dispersed teams and organizations soars up, groupware gains a more critical role in organizational productivity and growth [4]. For this reason, understanding the recent research trends and gaps in groupware literature is significant. This study addresses this need and employs a meta-analysis of research studies that develop or examine the use of groupware tools for collaborative work environments between the years 2010–2020. The study answers the following research questions.

• How are the studies distributed into a ten-year timeline?
• Which quadrant of the time–space matrix of groupware is addressed?
• Which sectors or fields do the studies refer to?
• Which research purposes or contexts of use do the studies contribute?
• Which research methods are used?
• What are the fields of the journals in which the studies are published?

2. Methodology and Theoretical Framework

In this paper, the study by Palvia et al. [5] and Cumbie et al. [6] are referenced for the design of the analysis conducted. Similar to Palvia et al. [5], the first phase consists of the accumulation of papers related to collaborative software or groupware, since they are used interchangeably. The second phase involves the development of the taxonomy and classification of articles, based on six classification categories—research types, research methodology, groupware matrix category, field of the study, context of use, and journal published. The final phase consists of the presentation of the results and the conclusions that are derived from them. A figurative description of the three-phased structure can be seen in Figure 2.

The search for the study was completed in two steps. In the first step, the journals were selected as a result of a search from the Web of Science (WoS) database by selecting the keywords as the term ‘groupware’ and the term ‘collaborative software’, and these two phrases were combined with the word ‘or’. The custom year range was limited from 2010 to 2020. The entire WoS core collection citation indices were included. The result yielded 772 documents. Among these 772 documents, only the ‘article’ document types were included in the collection, which reduced the search to 276 articles. In the second step, these 276 articles were reviewed and the heavily technical articles, which are incomprehensible to individuals from other fields, the review articles, the articles that conduct studies with a sample of students in K-12 grade levels, and the articles that were written in a language other than English were omitted. After this elimination process was completed, there were 139 articles remaining for analysis.

The coding of these 139 articles was completed according to the six classification categories as mentioned in Figure 2. Three doctoral students who are also the co-authors of this paper coded the papers during the spring semester of 2020. In order to calculate the concurrency among the coders, 20 papers were randomly selected and the coders were asked to classify them according to the determined criteria for the study. The inter-coder reliability was found to be 95%, indicating a high reliability. Cohen’s Cappa was found to be 88%, indicating that the agreement level among all inter-rater pairs was close to perfect, as percent interval of 81–100 is considered almost perfect for Cohen’s Kappa [7].

3. Analysis and Results

3.1. Publication Year

Within the ten-year period for this study, research interest in groupware studies dramatically increased between 2011 and 2013, reaching its peak in 2013 with 20 studies (Figure 3). Since then, a relative decrease could be seen in the number of studies on groupware. This paper considers only the first couple of months in 2020.

3.2. Groupware Matrix Categories

The groupware matrix classifications were assigned according to the taxonomy mentioned in the introduction section (Figure 1). Studies were distributed in these categories according to the time–space work condition, which the groupware tools mentioned in these studies serve. In some articles, the groupware tools correspond to more than one category. Studies targeting two or three categories were distributed into the categories; while 36 studies that did not target any specific category were classified as ‘generic’, and were evaluated separately.

Results indicate that the majority of the research focused on groupware tools that enable remote interactions for users in geographically distributed areas. Out of 139 studies, 101 studies included groupware that allows remote collaboration of workers. Fifty-nine of these 101 studies focused on groupware that allows users to work asynchronously, while the remaining 42 studies were on groupware providing synchronous interaction of users. Groupware tools that allow collaborative work at the same place were investigated in 19 articles in total. Studies that allowed working at the same place but at different times were limited, with two studies indicating that either there was not much need for this type of groupware usage; or it could be evaluated as a gap in the literature to be filled with future studies (

Table 1. Frequency of groupware matrix categories.

Time/Space	Same Time (Synchronous)	Different Time (Asynchronous)	Total
Same Place (Colocated)	17	2	19
Different Place (Remote)	42	59	101
Total	59	61	120

).

3.3. Distribution of Target Fields/Sectors

The field classification mainly comprised the sector or tangible life science field that the research study covered. The fields were mostly cited within the studies. Out of the 139 studies, 110 studies’ objectives were matched with a specific target field or sector; while the remaining 29 studies contributed to the general use of groupware systems. As can be seen in Figure 4, distribution of the studies with respect to the sector, revealed that the majority of studies (32.37%), were related to the groupware usage in the field of software engineering/computer science and programming. A total of 11.51% of studies were related to the collaborative work of the practitioners from the education sector. This was followed by both the business and healthcare sectors, with 7.19% each. Engineering and energy/environmental sectors both consisted of 3.60% of the total. While 2.88% of the studies addressed the game/entertainment sectors, 2.16% of them contributed to public services. Other sectors consisted of art, human resources, journalism, banking, transportation, linguistics, mechanical engineering, and tourism, with less than 1% share each.

3.4. Context of Use

Another category analyzed in this study was the context of use or aim of the groupware tools developed or investigated in the studies. Since each groupware tool produces a solution for a problem in a collaborative work context, it is significant to detect mostly studied or ignored contexts that the groupware tools serve. Even though previous Information Systems (IS) studies developed certain research topic classification schemes [8,9], there is no context of use classification for specifically groupware or collaborative software research. Fourteen context of use categories emerged in this study, as shown in

Table 2. Frequency of context of use categories.

Context	Total	% Total
Development of CS or CS models	53	38.13%
Knowledge management	13	9.35%
Collaborative learning	12	8.63%
Awareness in groupware	12	8.63%
Investigation of CS within the business, info systems, software development or legal processes	9	6.47%
Development of online CS or CS models	7	5.04%
Medical context	7	5.04%
Decision making	7	5.04%
Investigation of CS with transparency, rumor propagation or team collaboration	5	3.60%
Open-source software development and CS	4	2.88%
Investigation of other CSs	4	2.88%
Investigation of CS with single-display groupware (SDG) and WIKIs	3	2.16%
Project Management	2	1.44%
Collaborative writing and CS	1	0.72%
	139	100.00%

.

At the end of the analysis, it was determined that most studies (38%, that is 53 out of 139), were about the development of collaborative software (CS) or CS models. These 53 studies were not specific to a single context of use, rather they provide solutions for software development issues. The remaining 86 studies were related with the development or use of a groupware tool for a specific context or aim. Among these studies, most frequently targeted contexts of use were knowledge management (9.35%), collaborative learning (8.63%), awareness in groupware (8.63%), and investigation of CS or CS models within different fields of work (6.47%).

3.5. Research Types and Methodologies

The first step in the classification was to determine whether the research was qualitative, quantitative, mixed, theoretical, descriptive, or a combination of any two of those branches. An example of a study that combines two branches might include a design study in the theoretical part and present empirical results of a survey in the quantitative part. Quantitative research type was the most used (29.5%) followed by qualitative (20.9%), theoretical (20.1%), mixed (10.1%), and a combination of theoretical and quantitative (10.1%) (

Table 3. Research types.

Research Type	Total
Descriptive	0.70%
Descriptive & Qualitative	4.30%
Mixed	10.10%
Qualitative	20.90%
Quantitative	29.50%
Theoretical	20.10%
Theoretical & Qualitative	4.30%
Theoretical & Quantitative	10.10%

).

Eleven distinct methodologies utilized by the studies were included in our study. It was found that almost half of the studies used more than one methodology for their study. Namely, 79 (57.6%) studies used one methodology, 45 (32.4%) studies used two, 12 (8.63%) studies used three, and 2 (1.44 %) studies used four distinct methodologies.

To find the proper research methodologies that would be used to classify the 139 studies, previous literature was sourced [5,10,11]. Figure 5 shows the frequencies of research methodologies used in total. Total number of methodologies used in 139 studies—studies were counted more than once in the case of two, three, or four methods—was 214. Design science and conceptual modeling were the most frequently used methods with percentages of 19.63 and 18.22, respectively. These were followed by survey (15.89%), experimental (14.95%), case study (12.62%), interview (7.01%), content analysis (5.14%), and mathematical modeling (4.21%). Three qualitative methods, namely grounded theory, ethnography, and action research were the least used methodologies with percentages of 0.93, 0.93, and 0.47, respectively.

As a result of the close examination of 59 of 139 studies (42%) that included more than one method, we determined that 40 developed a conceptual model or designed a groupware tool and additionally complemented their study with an empirical study—such as case study, survey, or laboratory experiment—to apply or test their models or tools.

3.6. Fields of Journals

The categories of the journals associated with the accumulated studies were looked up from the Master Journal List categories of the Web of Science group website [12], and it was found that 139 articles were distributed among 20 different categories (Figure 6). The main category was Computer Science (53%), followed by Education (9%), Engineering (9%), Social Sciences (5%), Electrical & Electronic Engineering (4%), Psychology (4%), Economics & Business (3%), and Information science (3%). Other fields included art, environmental sciences, instruments and instrumentation, automation and control systems, biochemical research methods, clinical medicine, ergonomics, neurosciences, nuclear science and technology, applied physics, and remote sensing/geosciences, all consisting of less than 1% each.

4. Conclusions and Discussion

Information communication technologies have a significant role in workplaces to facilitate collaboration in between teams and team members, and to increase organizational productivity. In this technological era, groupware tools support team members to achieve their common goals by facilitating communication, coordination, and collaboration among them, in organizational activities. This paper, therefore, aimed to analyze the articles on groupware development and use, between the years of 2010–2020 in Web of Science, to understand the research trends and gaps in literature, as well as to inform future studies.

Between the years of 2010 and 2020, 139 articles were analyzed and found that the most productive period for groupware research was 2012–2015. A decline in the number of papers since then might be associated with a decrease in the number of studies, based on context development. Context development for groupware showed variations over the years. If 2013 was to be compared against 2019, for example, the CS tools and model development context dropped significantly in terms of percentage, however, there was an increase in knowledge management and CS from 0% to 2.2% for the same time period. This indicates that even though in the past the development studies were most popular, there was an inclination towards exploring new topics.

The analysis indicated that asynchronous distributed interactions are the most frequent groupware type (59 out of 120) used in collaborative software tools developed. Since groupware tools allow users to work at different times and from different places, they create opportunities to form teams from widespread geographical locations, and also provide more freedom with respect to efficient working environments. To serve this need, considering the Covid-19 pandemic crisis, it could be asserted that this type of tools would create practical opportunities in various fields.

Most studies (32%) contributed to the software engineering, computer science, and programming sector. This might be because the information communication technologies become widespread in almost every area of people’s lives, including business, medicine, education, communication, etc., which requires development of groupware tools to facilitate digital activities [12]. Accordingly, more than half of the articles (53%) were mostly published in journals related to the field of computer science. Although technological developments require deep consideration for the influences on people and their interactions, fields of journals published as social science category (5%) indicate that studies related to groupware tools from the perspective of social sciences might be included more in future studies.

In addition, most empirical studies were conducted in university settings with students as participants (12%). It is more convenient for people in academia to use and practice groupware tools. On the other hand, there is a need for studies that investigate sector practitioners’ experiences, examine needs of groupware tools, or develop models, or design tools for the diverse needs of various sectors related to computer-supported collaborative work.

Studies on groupware mostly (20%) used design science and conceptual modeling as their research methods. Among empirical methods, survey, and experimental designs were favored by researchers. Design science, conceptual modeling, and mathematical modeling studies were mostly coupled with an empirical method that provided the opportunity to test and apply the models or designs. In addition to survey and experimental designs, case study was also highly preferred by the studies that developed a model or designed a tool.

5. Limitations and Suggestions

The first limitation of this study was regarding the selection of the database. Since Web of Science was used as the only database for review, studies from other databases were intentionally excluded. The study was also limited to a ten-year period between 2010 and 2020, thus year span could be enlarged in further studies to follow how trends changed over the years. The keywords could also include different variations of the combinations of related words in new studies. Many combinations of associated words were tried to come up with the most related results, however, the only two remaining were the ones that were used in this study.

Future studies might be designed to involve a broader range of database, a broader date range, and a new set of keywords to further the investigation.