Learning to and to Learn: An Experiential Approach to Teaching Collaborative Systems Learning to Collaborate and Collaborating to Learn: An Experiential Approach to Teaching Collaborative Systems

This paper describes the use of experiential learning theory in the development of an undergraduate subject in collaborative systems. The purpose of the subject was to introduce students to the design, development and use of collaborative systems in organizational environments. Early in the subject’s development it was decided that in order for students to gain a deeper understanding of the issues involved in the development of collaborative systems, they should collaborate using an e-learning system so that they could experience these problems first hand. The paper provides an overview of the subject structure, the views of students with regard to the learning approach and some of the outcomes that were observed. The findings revealed that the experiential approach was successful in providing students with a good understanding of the issues associated with the design and use of collaborative systems. However, while many achieved deeper learning outcomes than would normally be provided by a traditional didactic approach, there were students who failed to achieve the desired learning levels. The findings not only provide support for Perry’s model of intellectual development and the value of setting unstructured problems, but also demonstrate the need for structure in early experiential events to which students are exposed. students learnt about knowledge management while sharing and creating knowledge in an ICT-supported collaborative process. Outcomes showed that students found the experiential learning to be valuable, and that the online learning system was an appropriate tool to support this. The learning approach and the ICT tools together allowed them to learn first hand the team skills and ICT skills that would be useful to them when working in virtual teams.


Introduction
The use of online learning, or e-learning, tools has experienced a rapid growth in universities across the world. Unfortunately, theoretical advances in pedagogy are often not incorporated into the way these new technologies are utilised and as such the employment of these systems has not always resulted in the learning outcomes that were promised. Like any teaching tool, online learning technologies can be and often are misused by teachers [6]. Regularly, the use of these technologies is limited to the simple replacement of written teaching materials with webpages, mass lecturing via video, or putting lecture notes on the Web. None of these uses has a significant effect on the quality of student learning [5] and reduces the technology to nothing more than a repository of information rather than an active agent in the student's learning and intellectual growth.
In an attempt to address these issues, this paper describes the use of experiential learning theory [8] in the development of an undergraduate subject on collaborative systems. The purpose of the subject was to introduce students to the design, development and use of collaborative systems (CSCW) in organizational environments. Early in the subject's development it was decided that, in order for students to gain a deeper understanding of the issues involved in the development of collaborative systems, the simple provision of content about the topic would not be effective: instead they should collaborate using an e-learning system so that they could experience the problems of collaborating online first hand. This paper provides an overview of the theory underpinning the subject's development, the subject's structure, the views of students with regard to the learning approach, and some of the learning outcomes that were observed as a result, with a view to evaluating the ongoing potential of the approach.

An Experiential Approach to Learning
The fields of self-directed and problem-based learning are very broad, both in terms of research focus and practical application. They have produced and include a number of sub-categories: lifelong learning, self-planned learning, independent study, distance education, learning projects, andragogy, and self-directed learning readiness [3]. The field is also described as encompassing experiential, formal, informal, non-formal, incidental and workplace learning [1].
Applied to either adult or pre-adult learners, the goals of these approaches are quite similar and complementary. In essence, "The ultimate goal of experience based learning involves the learner's own appropriation of something that is to them personally significant or meaningful (sometimes spoken of in terms of the learning being 'true to the lived experience of learners')" [1, p. 227].
Consequently, the differences between traditional and experiential curricula are quite stark, both in philosophy and practice. Traditional didactic approaches support what may be termed an 'object' view of knowledge. The lecturer/teacher is the source of truth regarding the topic and transmits this truth to the student cohort via a combination of lectures and written materials. Knowledge is conceptualized as an object that may be passed from one person to the next in the form of language [14]. By contrast, an experiential approach assumes that the most important learning outcomes occur through the learner's own experiences of action in the 'real world', i.e., they learn by doing not so much by listening [8].
In recent years there have been a number of applications of experiential learning theory to support collaborative learning, and to support learning about collaboration and working in teams. The Kolbe Team Learning Experience is one approach which involves teams working through a structured written simulation: as teams move through the programmed learning experience they progress through the developmental stages of effective team functioning and acquire teamwork skills [7]. In another example e-commerce students explored intra-group and inter-group collaboration by developing research papers in heterogeneous groups, and then giving and receiving feedback about the papers with other groups [2]. The results demonstrated that active involvement in a challenging collaborative process allowed effective learning about collaboration to take place.
Experiments in experiential learning are far less common in online learning environments. One example involved a mixed delivery course involving face-to-face interaction as well as online learning via a web-based learning management system [10]. In this course, students learnt about knowledge management while sharing and creating knowledge in an ICT-supported collaborative process. Outcomes showed that students found the experiential learning to be valuable, and that the online learning system was an appropriate tool to support this. The learning approach and the ICT tools together allowed them to learn first hand the team skills and ICT skills that would be useful to them when working in virtual teams.
Learning to Collaborate and Collaborating to Learn: An Experiential Approach to Teaching Collaborative Systems Robert Kay Laurel Evelyn Dyson

Experiential Learning and Perry's Model of Intellectual Development
One of the key implications to emerge from experiential and other related learning approaches is that traditional lecture methods tend to reinforce what may be described as the dualistic levels of William Perry's model of intellectual development [11]. Perry's model describes student changes in attitude and views toward their environment and authority, and changes in their values -in general, a change from a simple, dualistic view of life and knowledge to a more complex relativistic view [4, p. 221]. Within Perry's dualistic view of the world, the student believes that an absolute right answer can be found for everything. They are unquestioning of the knowledge espoused by authority (in this case teaching staff) and believe that, through the propositional knowledge provided to them by their teachers, the answer to any problem may be found. This position can also be associated with a desire to be spoon-fed information as opposed to the adoption of a level of learning autonomy.
Perry's study, which examined the perceptions of engineering students over a five-year period, found that, after students had been exposed to a number of ill-structured problems where there were no unequivocal answers, the students began to acknowledge diversity and uncertainty to be legitimate states of mind [11]. Furthermore, students became suspicious of the validity of authority opinions, observing that different interpretations and opinions could not be evaluated objectively. At this stage of the model, termed multiplicity by Perry [11], students begin to view all arguments as equally correct or biased, with decisions based predominantly on personal belief or hunches. This position regularly produces a stage of philosophical uncertainty for the student, where they recognize potentially every idea is logically as good as the next, and consequently wonder how it is possible to choose one answer over another [13]. The resulting turmoil produces one of two possible outcomes. The first is that the student will revert to the dualistic position, manifested by reducing problem complexity and avoiding ambiguity. The second outcome is a move to a position of contextual relativism. Here the student holds that truth is dependent on the context in which the study is taking place. Students no longer consider all views as equal and base decisions on evidence rather than the 'gut feel' emerging from the multiplistic position [4]. When considering the learning outcomes for a given curriculum Culver and Hackos observe, "…students failing to reach this [contextual relativistic] level were considered poorly prepared to deal with complex issues in their professional careers." [4, p. 222].
If one accepts Culver and Hackos' position then the process of curriculum development necessarily needs to move away from the transmission of content toward a model where the learning process itself takes precedence in the lecturer's considerations. This is because "…development strategies which do not invite challenge of a person's implicit theories may be seen as comfortable but will not lead to any reappraisal of current theory and practice" [12, p. 4]. The next section will provide a description of the subject that was developed based upon these principles.

Subject Development and Structure
In order to create an experience where students not only thought about collaborative systems but were actually required to use one to complete their project, a scenario was created that it was hoped would be of interest to the students. The assignments were then built around the completion of tasks related to the scenario. The tasks required collaboration between different groups of students.
There were effectively two experiential events within the subject designed to provide students with the opportunity to understand the issues involved with collaboration. Both of these events formed an integral dimension of an assessable item for the subject. The first involved sharing information regarding user needs. The second involved the provision of feedback on use cases developed for the students' proposed systems.
Based upon experience gained from the development of other problem-based subjects (see for example [9]) students were provided with an overview document describing the scenario in which they were to play a part, but only limited information with regard to how they should complete the tasks. The presumption here was to force the students to set their own direction with regard to the resolution of the problem. This decision had significant implications for the experiences of the students, as will be described below.

The Scenario
The students were required to adopt the role of system developers who had been employed by an events management organization to design a collaborative system for the management of a large rock concert. The fictitious rock concert would involve bringing multiple rock bands together from all over the world to play at a concert that would be run over a weekend in Sydney.
As a first step in the design process, students were asked to divide into groups of four to five and adopt responsibility for researching the information needs of one of the following key stakeholders in the rock concert: • Promotions • Transport and Stage crew •

Band Managers
These information needs would form part of the requirements analysis for the system. It is important to note, that due to the large size of the class (232 students) there were eight parallel tutorial sessions operating, each with between 25 and 30 students. Consequently each tutorial had one group dedicated to researching the needs of a particular stakeholder.
The student groups were then required to share the information that they had gathered via the university's online learning system -a customized version of Blackboard, one of the major e-learning systems available on the market -in order to produce reports that covered the information needs of all of the stakeholder groups. As such, although each group had done research on one particular stakeholder, their report required an understanding of all the stakeholders. Therefore they relied on their ability to gain information about the different stakeholders from the other groups through the collaborative system. This process was followed by the development of use cases for the systems that the students would design based upon the user needs analysis. Use case modeling is a way of describing in numbered, plain English statements, supplemented by simple diagrams, how users of a proposed system will interact with it. In their groups, students were asked to share their use cases online and provide detailed feedback to other groups on where they felt these could be improved using a template provided by the lecturing staff.

Ensuring Online Collaboration
The previous experience of lecturing staff while teaching other subjects was that students overwhelmingly preferred to work together face-to-face and as such, where e-learning technologies were provided, the systems were often not used. Such an outcome would of course defeat the purpose of this subject. To facilitate the sharing of the information between the student groups, half the tutorial sessions were run as labs in the Faculty's computer rooms, with class activities based on using the Blackboard e-learning system. In addition, students were encouraged to place information on the Discussion Boards within Blackboard whenever they had the chance by logging into the system outside of class time.
In the online space, students had access to the usual subject materials in the form of course notes they could download. However, more importantly, the other functionalities of Blackboard were brought to the centre of subject design. A large number of different Discussion Boards were established: some were available on general access to all students, while others were for the use of each specific student group. Also chat room facilities and virtual drawing tools were provided (both general access and group access) and file sharing was enabled.
From the beginning of the subject, the subject coordinator made it clear to students that any questions or queries regarding the assignments or exam needed to be posted to specified Discussion Boards, as there would be no answers provided via email or face-to-face. This further encouraged use of the online system and sharing of information.
Furthermore, a requirement of the assignments was that any references used in the group reports had to be drawn from resources that had been placed on a Discussion Board; no other sources would be recognized. As such, a referencing system was devised for the assignments that allowed students only to reference material that had been placed in the collaborative system -effectively making it compulsory to use the system.

Methodology
The data collection that took place to evaluate the subject had two broad objectives: • To gain students' perceptions regarding the value of the approach and where they thought it might be improved • To assess changes in students' understanding resulting from their participation in the subject Four forms of data were collected: • A questionnaire on student perceptions prior to the beginning of the subject

Sample Population
The student cohort consisted of 232 first year undergraduate students drawn from a range of degrees including the mainstream Bachelor of Science in Information Technology, the highly selective Bachelor of Information Technology, Bachelor of Business students taking an IT major, as well as combined degree programs such as the Bachelor of Mathematics and Computing. Students also came from a range of cultural backgrounds, with over half the student cohort coming from a non-English speaking background.

Analysis
Each data set was analyzed separately, with the exception of the initial questionnaire and the exam, which were considered together.

Questionnaire and Reflective Exam
The initial questionnaire and the exam went together as a pair. The questionnaire, filled out at the beginning of the subject, provided a picture of the student's views and knowledge about the topic prior to undertaking it. It also provided the students with a basis for comparison, when it came time to write the reflection at the end of the subject. From the total sample, responses that were graded with High Distinction were selected out, along with those who failed. A thematic analysis was then conducted on these responses.

Online Discussion Board
It is important to note that the online discussion board was not originally expected to yield much in terms of meaningful discussion about the approach taken in the subject and for the most part this was indeed the case. However, following the first experiential event in the subject, and the negative response of some students to the exercise, a thread emerged where students began to debate the positives and negatives of the approach with the subject coordinator. This discussion continued for 13 days and received the highest hit rate of any of the threads in the system.

Subject Evaluation
In the evaluation of the subject 159 students responded. The form they filled out was a standard university issue subject evaluation consisting of eight questions graded on a Likert scale, with two open-ended questions at the bottom. The open-ended questions asked: 1. What did you particularly like about the subject?
2. Please suggest improvements that could be made to this subject.

Limitations of the Research
There are a number of limitations that need to be considered with regard to the findings from this analysis. Firstly, although a range of data sets were collected as part of the process of the subject, the subject has to date only been run in this form once and therefore ongoing evaluations over several semesters would be needed to adequately validate or invalidate the approach.
Secondly, as the reflections formed a part of the assessment of the subject, there is always the possibility that students attempted to provide answers they thought the lecturing staff wanted to read, rather than what they actually thought.
Lastly, although the response rate to the subject evaluation was quite high, so was the non-completion rate in the responses, which of course limits its value.

Student Responses to the Approach
Responses to the teaching approach were received through the subject evaluation, the reflections and significantly via online postings to the discussion board itself. Responses collected through each of these media will be discussed below.

Subject Evaluation
Positive comments in regard to the subject related mostly to the idea of what was being attempted. Examples of these comments include: "A thought provoking approach to learning"; "How different it was, I found it way more interesting than any other subjects I took."; "It made me think from different perspectives and learn to understand the uses and what collaborative systems enable users to do; benefits and disadvantages -evident through hands on experience"; "It made you think a lot more about university in general, and about ways of thinking -I enjoyed group work a lot".
However, students also felt the approach could be improved. There was a considerable amount of consistency regarding this point, with all comments indicating that students wanted to receive a higher level of direction in the task, as can be seen below.

Online Responses
A thread developed in the Assignment Discussion Board and became a key source of student responses, as they actually debated online with the subject coordinator about the approach that he had taken. Some key excerpts illustrate the point.
Initially this thread began as a complaint regarding the first round of information sharing: "I propose (and probably at least 99% people agree with me here) that the 'Interview Round' stuff was a total waste of time and just confused people on the assignment even more than they were already. Anyway, after that, I've figured out pretty much what I have to do. After much brain drain" (student posting). Other students quickly joined the debate, agreeing with the first poster: "This was the perfect forum for a black and white interview protocol. E.G. No discussion, this is what you do and where you go to ask a question, and this is what you do and where you go to answer other groups questions. My cynicism suggests that you left this as a sociological exercise to see if the tutorial members could first agree amongst themselves on a protocol, whereas all the time we were waiting for the tutor to give us some instructions. It's just wasted a lot of time that could otherwise have been spent in improving the overall quality of information needs supplied in our reports." The subject coordinator responded by noting: "The actual process of collaboration is not something that can be taught -as such, you have to experience it. You suggested that the structure was a social experiment to see whether you were able to organize yourselves. Well that's not quite the case, I knew that you would organize yourselves and you have. The value of the process was not in the interviews but the issues that you experienced through the process".
This thread proved to be a turning point in the subject as students engaged with the process and thought more deeply about what the subject was trying to do. One student noted that this thread "has the best quality on this discussion board" and another described it as "a fascinating thread". The debate allowed students to develop their understanding of the subject and the processes involved. The comments that began the thread are much more negative than those which concluded it. One student described her change of thinking: "I have to be honest and say that, before I read your posts, I was probably leaning more towards the 'bitter' view of proceedings of some of my colleagues. But having read your responses I think I have an understanding of what you're trying to teach us-or at least allow us to learn!" The last student contribution to the thread was a whole-hearted endorsement for the learning approach taken by the subject: "Cheers for being the only interesting subject so far and ALLOWING me to think for myself." In general, however, most students' final view of the approach could be summed up along the following lines: "While I appreciate the unstructured aspects of assignment are to help us learn more about collaboration and collaborative systems, I think the assignment would benefit from being just a little clearer and slightly more structured".

Student Reflections
The majority of reflections examined indicated that the students appreciated the learning approach adopted in the subject even if most found it "initially quite unsettling and frustrating." These first frustrations, though, gave way as the subject proceeded and learning took place: "At first I blamed a poorly structured course and inadequate task descriptions, but I gradually realized what the lecturer had been trying to convey from the start: that it was as much about the process of learning as what was learned itself." Students realized that they were "much better off for experiencing the weaknesses of each other and the group dynamic in action" than having "boundaries and a methodology and approach to the task … clearly defined by the co-ordinator". One student found the aim "to stimulate the process of thought in us students … very refreshing as I was expecting this subject to be a waste of time involving the memorizing and regurgitation of a bunch of theories of greater or lesser merit that had little to do with the actual process of trying to really uncover what questions we should be asking, from what perspectives we should be thinking in order to design a good collaborative system, or in fact a good computerized system".
Again, however, even from those who were positive about the approach, there was a consistent call for more structure and direction in terms of how to complete the tasks. "I feel that what is being said here and which has been elucidated in the tutorials time and time again is that the process was not structured and orderly. I have read all the course info and nowhere does it say how we are to go about doing this on [Blackboard]".

Learning Outcomes Observed
The learning outcomes of students were essentially drawn from the reflections they wrote on the subject as part of the exam. Although it is possible to glean some learning development through the online responses, these developments are seen across a group of students and it is not possible to draw out the learning of individuals.

Evidence of Learning from Student Reflections
There was considerable evidence of student learning in the reflections as students realized that their initial view of collaborative systems was "rather simplistic" and "quite superficial". Many came to appreciate that collaborative systems are far more complex than they had imagined. They realized that systems could only support collaboration if they allowed for the dynamic, ever-changing quality of collaborative interactions, which they had experienced themselves throughout the semester. They were surprised to find the stages of developing a system were not the textbook phases of the Systems Development Life Cycle (or waterfall model), but "Most of these stages are concerned with the organization of the group, tasks, dynamics etc." The "'delivery method' rather than the goal" of the final system, the "'process' of developing the system" was for them the important lesson learnt.
Apart from the basic objectives of learning how collaborative systems work and how to design them, students stated that they had also learnt many other things they had not expected at the start. For some it was how to work in a team, for others how to present in front of a group of people. For a significant number of students it was learning a whole new way of thinking. For example in one reflection a student observed: "I suppose that prior to taking this subject I hadn't really thought how the way I think might affect my ability to do my job, but it is fair to say that I am very much the reductionist and always have been. I can appreciate that thinking from a more holistic stance would realize benefits when designing a large computer system, and also that being aware that I do not naturally think that way is the first step towards being able to consciously make an effort to do so." The experiential nature of the course allowed students to move from simple dualistic notions of right and wrong approaches to a more complex, relativistic way of thinking: "I was initially frustrated at what I felt was a lack of clear direction. I now realise that I was simply looking for confirmation that I was heading towards the 'right' answer. I now realise that to approach contextual relativism is to realise that there is no definitive answer and it is instead a question of 'how well does it solve the problem?' … I was able [to] make this intellectual move." The best evidence that students had made this intellectual shift was that they were able in their reflections to critique the theories that they had been studying and interpret them in view of their own experiences. Theories that students evaluated, and sometimes developed further based on their own experience, included Perry's model of intellectual development, Kolb's experiential learning model and Bion's theory of group dynamics.
Although the subject described in this paper has only been run once according to this particular curricular design, it is clear that the nature of the learning outcomes to emerge from the process were quite different from those normally associated with a curriculum based on a didactic approach. Clearly there are both strengths and weaknesses to the approach and these characteristics need to be considered in future iterations of the subject.
In terms of strengths it would appear from the comments of students that the approach achieved its objective in terms of providing students with a well grounded (in terms of student experience) understanding of the issues associated with designing and using collaborative systems. In terms of weaknesses, however, there are questions in terms of how many students achieved the desired level of learning outcomes and, for those who didn't, the question of what exactly did they learn?
Certainly it is unusual for students to discuss learning revelations such as those highlighted above. However, for those who didn't experience such revelations it is difficult to tell whether they learnt anything. In a more traditional subject it would be clear that even the poorer students managed to memorize something about the subject; but here their poorly written reflections essentially contained gut-feel opinions which often did not vary much from what the student had suggested at the beginning of the subject in their initial questionnaire. It should be noted that in the case of some of the High Distinction students instinctive feelings were also prominent: however, the justification for why and analysis of the reasons behind their feelings indicated that they had experienced significant growth with regard to the development of self-directed learning skills and group work.
It is also important to note that although the purposeful lack of structure would appear to be what gave rise to the deeper learning outcomes of the students, it is the aspect of the subject that they most wanted changed. The desire to be spoon-fed remained strong even though many understood the need not to receive it. Perry's model does provide indicators as to why this is the case, with the recognition that students become frustrated and demotivated when confronted with unstructured problems [11], but doesn't provide guidance as to how much structure is enough.
Certainly there was considerable evidence to support Perry's model from the students' responses. Early evidence of dualism included comments indicating that there was a right way and a wrong way to conduct the online interviews that were used to gather information for the collaborative systems they were designing. Moreover, they believed that clear-cut answers and all-knowing Authorities also existed outside university in industry; "I also find it hard to comprehend something like this occurring in industry as well, as surely these sort of 'decisions' or assumptions would be well-defined by any initial consultant or tender for the work?" The students' initial confidence in the existence of an absolute truth gave way to a state of confusion for many, and then, at least for some students, the emergence of a relativistic view of knowledge. This intellectual development was marked by a clear change in tone, with the declarations of certainty of the early contributions giving way to conditional statements, more hesitancy and less surety as the students experienced the transformation: "I think", "perhaps", "could", "would", "I personally feel".
They realized that many views are possible, if not equally valid, and that there are means at their disposal for choosing a position to adopt which include collaboration with other students and "discussing and negotiating issues", rather than direction from the Authorities.

Conclusion
This paper has described an attempt to introduce experiential learning into the use of an e-learning system to learn about collaborative system design. The premise of the approach was that the best way for students to learn about the issues involved in collaborating online was to actually collaborate in order to solve a complex problem. The paper presented the results of an analysis conducted on the four forms of data collected about the subject and the students' responses to it. Although it would appear that in some ways the approach was a success, there was considerable unrest amongst the students and many failed to grasp the deeper learning outcomes that formed the purpose of the subject. The provision of more structure to the first experiential event, in line with the students' comments will be the main change to the subject the next time it runs: this will provide some answers with respect to questions regarding the level of unstructuring needed to trigger change and deeper learning outcomes regarding online collaboration.