Supervision of Undergraduate Final Year Projects in Computing : A Case Study

Final Year Projects (FYPs) play a significant role in undergraduate education in the computing field of study, and most of the related university departments and schools consider them an essential contribution to this study. However, issues such as whether to assign the projects individually or to a group of students, the procedures followed in their assignment, the supervision process and the evaluation of the outcomes have been of concern to many academics in the field. In this case study, we present the methods for activities such as assignment, supervision, and evaluation of FYPs at the University of Kurdistan Hewlêr (UKH) between the years 2009 and 2017. We discuss the development of our approach and the lessons learned during the mentioned period. Furthermore, we present our current way of managing the FYP module. The aim is to develop a platform for interested and involved academics to discuss the topic further. Sharing the experiences on managing FYPs would not only help in making the module more attractive and beneficial to the students but also in the consolidation of the guidelines and indicators of the proper supervision and a fair evaluation of this significant undergraduate study endeavor.


Introduction
This article is a case study on our experiences and practices regarding the definition, supervision, coordination, assessment, and evaluation of undergraduate Final Year Projects (FYPs) in computing at the University of Kurdistan Hewlêr (UKH) from 2009 to 2017, during which we engaged in the supervision and assessment of more than 250 FYPs (To be precise, two authors were involved in the addressed activities for the whole mentioned duration and the others for 3 to 6 years.)The projects were accomplished by our students in partial fulfillment of different degree programs including Information Technology, Applied Computing and Information Technology, Computer Science, and Computer Engineering.We have gradually improved our approach through suggestion, distribution, supervision, and evaluation activities during the mentioned period.However, as we progressed, new issues and fresh questions arose concerning the way in which the FYPs must be managed within the computing departments.
The FYP, which also has other names such as "capstone project" and "senior project capstone", is the main component of undergraduate computing degrees [1][2][3][4].In addition to different university programs, the importance of this module has been emphasized by major professional bodies in the field.Particularly, Association for Computing Machinery (ACM) and IEEE Computer Society-actively involved in computing curriculum design by organizing joint task forces from academia, professional bodies, and industry for about two decades-have emphasized the significance of the incorporation of this item in almost all their related suggestions.
To illustrate, the recent curricula for Computer Engineering [3] suggests that the program "should include an FYP that requires students to demonstrate the use of a range of knowledge, practices, and techniques in solving a substantial problem".Also, the software engineering curricula [2] mention the capstone project as a significant project which students should complete in their final year study.Similarly, the Information Technology [4] curricula mention a "Senior Capstone Course" that is taken in the final year of study with the aim of solving real-world problems.
Consequently, the essential role of the FYP in undergraduate education in computing suggests the necessity of scholarly investigation and discussion on the methods and processes pertinent to the topics such as assignment, supervision, and evaluation.
The rest of this article is organized as follows.Section 2 explores the related work and provides an overview of the process of FYPs in different universities worldwide.Section 3 investigates the development of the FYP assignment, supervision, and evaluation process during the past decade at UKH. Section 4 discusses the findings and provides an analysis based on the collected data.Finally, Section 5 summarizes the study and addresses the open issues and questions.

Overview
As was presented in Section 1, there is substantial evidence of FYPs' major role in computing.However, the computing literature is not rich regarding the scientific study of, or the discussion about the process of assignment, supervision, and evaluation methods of the FYP module at the undergraduate level.Nevertheless, there have been attempts to address these issues.Below, we review several studies in this regard.
James et al. [5] presented their experience in several universities covering about 40 different projects.Their study addressed the main aspects of the FYPs, provided a taxonomy for the projects, and proposed the quality factors of a "good" project.Also, they addressed the international bodies such as ACM and IEEE Computer Society, and national bodies, for instance, British Computer Society (BSC).The authors also provided an international perspective of the FYPs in which they illustrated evidence of the significant role of FYPs in undergraduate computing studies of both European and US universities.Furthermore, they summarized the environmental factors of education which influence the success of the students' projects.[5].
Olsson et al. [6] discussed FYPs in Computer Science and information systems; they focused on suggestions concerning research-oriented topics for FYPs.The authors suggested several purposes for FYPs, such as preparing students for the work environment, helping them to appreciate the research methodology in order to assist those who are interested in postgraduate studies, and to allow students an interconnected view of the different modules that they have studied in the previous years [6].However, they found it difficult to achieve all these targets at the same time.Moreover, the authors discussed the different roles of the FYP, and particularly proposed that an examiner who is not the supervisor would improve the clarity of the assessment process and its quality.In addition, they itemized a list of criteria which could be used in the project evaluation process.
Lejk et al. [7] studied group assignments in undergraduate computing courses in UK universities; they also addressed FYPs.However, they mentioned that there were issues in the evaluation of final year group projects for which they did not produce proper solutions [7].

Final Year Projects: Different Approaches
Generally, a degree in Computer Science requires an FYP to be carried out by the students to integrate their knowledge acquired in the previous years of study, their communication skills, and new information gained through their work progress in the FYP.Although this practice is known internationally, the method of how to conduct the FYP is different from one country to another or even from one university to another in the same country, mainly dependent on the education system of the university.
Normally, the stakeholders in the FYP include a student and a supervisor who will guide the student during the process of FYP development.Once the project is completed, a student should present a written report describing the work and gives an oral presentation to a committee composed of several members to be assessed.
The UKH follows a western education system, therefore before we explain how FYPs are carried out at UKH, we briefly look into its implementation at the well-established universities in some western countries: France, the United Kingdom, and the United States.

France
In France, computing students can take the FYP module either in the third year of undergraduate studies (License degree) or in the fourth year (Master 1) in any unity or in the fifth year of the Engineering school.Commonly, the FYP can be carried out either at the university or in a company or enterprise (Internship).The duration of the FYP is almost six months, starting from the last day of the second semester.The subject can be proposed by either the supervisor or the student in the case that the student chooses to do it at the university, while, it is suggested by the enterprise in the other case.In the example of the university, the students are advised by only one supervisor, while she/he is advised by two supervisors if the FYP is completed in the enterprise, one from the company and one from the university to assure that the FYP meets the standard of the university.In September, the student should submit a report and give a presentation about the project in front of a committee composed of the supervisor(s) and two internal examiners.Some universities use predefined rubrics for the evaluation of FYPs while others assess FYPs without using such rubrics.However, the assessment is based on several parameters such as the content of the presentation, the body language, and the clarity of the presentation.The final mark will be the sum of the marks for the report and presentation (The provided information is based on the experience of two authors of the article at the French universities).

The United Kingdom
UK universities follow a similar approach to French universities regarding their academic calendar which starts in autumn and ends in spring.The similarity is restricted only to the BSc degree, as the License degree does not exist in the UK educational system.Furthermore, the standard number of examiners is two, including the supervisor, but this number can vary among the universities.There is a particularity in the FYP system in UK universities in which a midway project or poster session is held.When the students are in the middle of their projects, they must prepare a poster of their work and present it to an evaluation committee.In addition, an intermediate report should be submitted, so the final mark of the FYP will be the sum of the intermediate report, final report, and the intermediate and final presentation [8][9][10].

The United States
Unlike the previous two countries, each university in the US has its own system.However, in terms of the FYP duration, two systems can generally be identified: one which takes place from August to May and the other from January to May.Furthermore, a major difference in the US system is that universities offer advanced capstone lectures during this period.In the first part, students are taught concepts such as leadership and team-work, and develop their project management capabilities through seminar presentations and group discussions.In the second part, students propose a project, which is followed by a midway checkpoint, a final project report, and project presentation.The assessment also includes the working performance of the student during the capstone class [11][12][13][14].
Table 1 shows a comparative review of the processes, such as assignment and evaluation, related to the FYP in computing in different countries.The comparison is based on several metrics that we consider essential to these processes.

Assessment
The assessment consists of an oral presentation and a report submitted by the student.At the end of the project, there will be a jury composed of the supervisor and two internal examiners.

Similar to France
Similar but in some universities, the supervisor can be the only person who evaluates or assesses the student.

Location
University/Enterprise University/Enterprise University/Enterprise University

Duration
One Semester One Semester One Semester Two Semesters

Degree credit
In 180-240 European Bachelor according to Bologna Accords, FYP involves 7.5 credits.
In 460 system credit, FYP involves 15 credit.
In 120 system credit, the FYP involves 4 credits In 360 and 480 system, FYP involves 30 credits

FYP report Dissertation
Thesis or project FYP report

FYP subject
Proposed by either the supervisor or student.
For an internship, the subject is proposed by the enterprise.

Similar to France Similar to France
Proposed by either the supervisor or student

Form of the output
Research outcome or product Similar Similar Similar

FYP organization
In the prior academic year, students have already been taught several modules that help them to develop a project, write reports, work in a team, and give a presentation.

Similar to France
The module is divided into two parts: the first part is an introduction to the FYP and the second part is represented by the FYP work accomplished.

Individual work Individual Individual
The first part of the FYP module can involve a team.The second part is individual

Individual
In the following subsections, we provide a brief history of what the undergraduate program in computing has gone through since the UKH was established in 2006.

Final Year Projects in Computing at UKH: The Case Study
This case study focuses on a quantitative method while appreciating a qualitative approach as well.Therefore, we conducted the research on the two bases.
Firstly, we collected a variety of data about the UKH FYPs from the available data sheets which have been archived in the Department of Computer Science and Engineering at UKH and updated it with the data which we acquired from the UKH Registry.We used spreadsheets to organize the data based on the academic years and the programs.We also categorized the project topics and arranged the data accordingly which could be used to investigate the case from other perspectives in the future.For the same reason, we collected the grade of the core modules for each student alongside their performance in the FYP.This part formed the quantitative base of the study.
Secondly, we benefited from sharing our experiences as supervisors, assessors, and coordinators of the FYPs during the period under investigation.In addition, we held several meetings to discuss and summarize our experience, particularly focusing on the reasons behind the changes that we applied to our rules and regulations.These served as the qualitative input to the study.Furthermore, we studied the literature, visited the websites of the other universities worldwide, and benefited from the experience of several colleagues with the academic backgrounds from various countries.

Computing Degrees at UKH
The UKH was established in 2006, and its degree programs started from the academic year 2007-2008.As for computing, the Department of Information Technology (IT) offered a three-year program which led to a Bachelor of Science (BSc) in Information Technology.In 2010, the name of the program and the curriculum were changed to Applied Computing and Information Technology (ACIT).No students were admitted during the academic year 2009-2010, due to a transition in the academic structure.In 2012, the department name changed to the Department of Computer Science and Engineering (CSE).The programs were redesigned into four-year programs, the ACIT program was discontinued, two new programs, namely Computer Science (CS) and Computer Engineering (CE), were offered, and the whole curricula underwent a complete overhaul.This was followed by the introduction of two new programs: a BSc in Telecommunication Systems Engineering (TSE) in 2013 and a BSc in Software Engineering (SE) in 2016.
In the subsection below, we provide a brief history of how the FYP module in computing has been dealt with since the establishment of the UKH.

Background
Table 2 provides a summary of the FYPs' assignment, supervision, and evaluation process at the UKH between the years 2009 and 2017.
In 2009-2010, the staff gathered to discuss the outcome of the FYPs' management process to avoid any significant differences in the report marks before they finalized them.The experience of the first round of FYP evaluation raised two questions among the department staff.The first question was "Does the supervisor assessment alone reflect a fair academic approach in assessing the work conducted by the student?"The second question was "Is it necessary for all members of the oral presentation panel to have expertise in the area of the project topic?"These questions generated a debate about the management method of FYPs in the department, which has continued up to the time of writing this paper.
For several years, the result of the debates over the above questions was in favor of giving the supervisor complete authority in marking the project reports.The proponents' argument was about the supervisors' complete awareness of the students' project and there might be no one else with enough knowledge about the work accomplished to fairly assess the outcome.However, with the newly joined staff and diverse backgrounds in the department, the overall perception about the report marking process started to change.Consequently, in the academic year 2014-2015, some supervisors decided to incorporate one of the oral panelist's opinions in marking the project report to improve the fairness of the assessment.However, this only became a formal practice in the following academic year.As a result, since the academic year 2015-2016, the project report mark was based on the supervisor's mark, weighing 60%, and the second marker, weighing 40%.Also, during these years, a more detailed rubric was developed to make the marking as objective as possible.The rubric categorized the projects into three categories, namely "Product Development-Software", "Product Development -Hardware", and "Research Project".

2009-2010 IT
Staff provided a list of the projects along with a synopsis about each project.Students chose a title by discussing it with the staff.Students were assigned according to a first-in-first-served method.
Weekly meeting with students.
Evaluation at the end of each semester.Project report carried 70% and oral presentation 30%.
The report was only marked by the supervisor.Two rubrics were developed whereby the reports and the presentations were evaluated.
The presentation was in front of a panel of five members of staff.The average of the five marks was taken as the presentation mark.Weekly meeting with students.However, the shortage in department staff made it a heavy burden on the staff.
Similar to the 2009-2010 academic year, except for the number of panelists which was restricted to three.Only a list of the project titles was presented to the students.The students were allowed to suggest their own topics subject to finding supervisors who were interested in the topic.
Weekly meeting with students.
Similar to the 2009-2010 academic year, except for the number of panelists which was restricted to three.The students were asked to select a topic from a provided list or suggest their own topic.
In either case, they were asked to prepare a project proposal which should be signed by both the supervisor and the student.
Weekly meeting with students.Students were asked to provide a weekly report.These reports were part of the marking rubric.
Reports were marked by two examiners: the supervisor and a second marker.The supervisor's mark weighed 60% and the second marker's mark 40%.A detailed rubric was developed for the report and progress marking.The number of panelists was restricted to three.

Findings and Discussion
We have collected and processed the results of FYPs at the UKH during the academic years 2009-2017.These include the distributions of marks (Figure 1), the average, minimum, and maximum (Figure 2), and the median of the results (Figure 3).We discuss these findings in line with the summary of the evolution in the FYP supervision and assessment depicted in Table 2.

The FYP Procedures at the UKH
As the third column of Table 2 shows, the project assignment method was gradually evolved from an inflexible random-based assignment to a more flexible and liberal approach.Several reasons were behind this change.On the one hand, we learned that in some cases the random assignment of the projects using the first-come-first-served approach had disadvantages because projects are assigned regardless of candidates' background and skills.On the other hand, we realized that students might prefer to work with certain supervisors based on their experience in their teaching classes, but they might not find a topic which interests them among the topics suggested by those supervisors.Consequently, the department decided that the students could also provide their own topics, subject to finding a staff member to approve the idea and ready to supervise them.This could increase students' motivation in conducting their FYP.
In addition, to have a better understanding of the project, starting from the academic year 2015-2016, the department decided to add a step to the project assignment procedure wherein the students must prepare a short proposal about their selected/proposed projects.This proposal should describe the approach based on which they conduct their project.This document must be approved by the supervisor, formally submitted to the department, and kept as an FYP record.These activities are handled using an online system.This improvement is found to be effective in resolving disputes about the objectives of the project and students' misunderstandings about the scope and the objective of the projects during the project evaluation.
The fourth column of Table 2 summarizes the supervision procedure.Although from the first time FYPs were offered, the department encouraged students to visit their supervisors on a weekly basis, it did not implement any measure to manage and assess the students' commitment regarding this task.In the academic year 2010-2011, the then Academic Pro-vice-chancellor even instructed the department that students should be given a very short time-frame for the project consultation (no more than 10 min per week) based on the argument that the students should be independent in conducting their projects.However, comparing the staff/student ratio in the mentioned year suggests that perhaps the shortage of staff was the main reason for that decision.
Regardless of this background, the project consultation remained a challenge.In the academic year 2011-2012, the department prepared a weekly report form aiming to show the progress of the students continuously.However, due to the complexity of the form, it was not implemented completely.Also, as it did not contribute to the project assessment and had no obvious impact on their progress, the students prepared it unwillingly.
To address the issue of weekly progress, the department added an item to the evaluation rubric for the project progress report and simplified the project report to include a summary of the progress regarding the project schedule which was given in the project proposal or an updated version of that, the issues found, and the resources consulted.This report was also submitted online.To draw conclusions about the effectiveness of this amendment to the FYP evaluation needs a longer period of time.However, based on our experience, weekly meetings play a crucial role in the success of the students, particularly during the proposal preparation and during the first and the last phase of the projects.However, during the intermediate phases, the meetings could be organized at the students and supervisors' convenience according to the students' need for consultation.
The fifth column of Table 2 shows that the project evaluation rubrics changed at different stages.We used two rubrics for the evaluation: one to assess the reports and the other to evaluate the presentations.The rubrics went through extensive revisions.The rubric versions could be classified into two major versions.The first version was used from 2009 to 2015, and it was replaced by the second version in 2016 which has been in use since then.Table 3 show a summary of the report assessment rubric in the first version.The presentation assessment rubric in this version included three items: Delivery, Organization, and Content.Tables 4 and 5 summarize the reports and presentation assessment rubrics, respectively, which formed the second version.
The first version of the report assessment rubric had the percentage of marks for each item but lacked detail about the criteria based on which marks were awarded.In this version, the presentation did not include a separate section for the product evaluation.The essential improvements to the first version are as follows.We categorized the report assessment rubric into three types according to the major areas of the project: software, hardware, and research.We added a descriptor table for each item to guide the assessors in marking and to reduce the subjectivity in the assessment.We augmented the presentation assessment rubric with a section for the product/outcome evaluation.This was due to the cases that we had observed in which there were considerable discrepancies between the presentations and the real outcome of the projects.Also, in the second version, spreadsheets are used for the assessment in which the final marks are calculated by combining the supervisors and second-markers' evaluations of the reports and taking the average of the marks of the presentation evaluation panel.

The Results of the FYP Assessments at the UKH
The following figures show some statistics about the outcome of the FYP assessments at the UKH according to the previously mentioned procedures from 2009 to 2017.
Figure 1 illustrates the distribution of marks during the study period.In order to provide a clearer visualization, Figure 1 illustrates a histogram based on the letter grade scale, given in Table 6, while in the other two figures the original 100-point scale is used.Figure 2 shows the maximum, minimum, and the average of the project marks during the study period.In this figure, the top of each vertical bar indicates the maximum and the bottom shows the minimum while each small horizontal bar presents the average.Figure 3 shows the median of the project marks during the study period.A closer observation of the grade distribution (Figure 1) shows a skewed distribution in the years 2009-2013 and considerable abnormality in the years 2013-2014.These phenomena are also observable from Figures 2 and 3.This was due to the department's formation period in which it experienced significant turnover in its staff and changes in its aims and programs.The latter abnormality was the result of seeking a higher level of autonomy in marking the reports and forming the evaluation panels by the supervisors, particularly, by the new staff who were clear about the procedures due to the lack of instructive rubrics.
The above issues were tackled by improvement of the evaluation rubric; extension of the categorization of the projects into hardware, software applications, and research providing more relevant details for each category; the random selection of the evaluation panels while considering the relevance of the panel members to the project under evaluation, and more importantly the distribution of marks between the supervisor and a second marker which led to a better distribution of the marks in 2016 and 2017.
Although the comparison of students' performance in the FYPs with their performance in the other modules needs more factual investigation, based on our experience the normal distribution of the marks has been a common feature in most modules.Therefore, achieving a more normal distribution could be interpreted as a less-biased assessment approach.
It is worth mentioning that during the academic year 2016-2017, for the first time, we had some students who did not pass at the first attempt of their project presentation.According to the UKH regulations, the mark of the second attempt is capped to the passing mark only.This raised the students' awareness and encouraged them to spend more efforts on their projects to avoid delay in their graduation and not receive a capped mark.
To summarize, although the role of FYPs in computing is known to academics in the field, the newly established programs could face several challenges.A significant part of these challenges, based on our experience, is to set up the required procedures which show the workflow for managing the module.This should cover all related activities starting with the topic selection up to the project evaluation.It should also be supported by the necessary assessment rubrics, report templates, and student and staff guidelines.Furthermore, the academic staff should have a common understanding of how these procedures should be applied and implemented.The early dissemination of these procedures among the staff and students at the beginning of the academic year could reduce the possible misunderstanding and confusion among the different parties who are involved in running the FYP process.However, based on our experience, the newly established programs might face inconsistency in the approach of the staff members because of differences they possibly might have in their background and their previous educational environments.To reduce the time required to gain a common view towards the implementation of FYPs, we found that having one of the staff members as a coordinator of the FYP module could help in mitigating this risk.The coordinator should communicate the overall approach to the FYPs and the issues which the students might face with colleagues as well as the students.The coordinator should also manage the continuous reviews of the rules and regulations to apply the necessary refinement.It is crucial that the department collectively reviews the outcome of this module every year to benefit from the lessons learned, and to address the problems which might have been raised during the implementation of procedures, and to apply the necessary revision to the related rules and regulations.
We also found that the students find several tasks to be challenging, such as the preparation of the project proposal, finding proper resources, reviewing the literature and report writing.They might also have difficulties in their time management as well as preparation for the presentation.If the topics such as time management and research methods are not covered by the relevant modules, namely Project Management and Research Methods, then a few intensive sessions on these topics could be helpful in reducing the severity of the mentioned issues.

Conclusions
We have presented the results of a case study about the processes concerning FYPs in computing programs at the University of Kurdistan Hewlêr (UKH).Despite the significance of FYPs in undergraduate education in computing, the conduction, supervision, and evaluation processes of this essential component of the programs have not been well researched.
A comparison of the process among various universities in different countries showed the differences and similarities among the approaches.It showed that the UKH approach has ingredients from different schemes and therefore it complies with the approaches which are followed by long-established programs in developed countries.
We have also presented the UKH evaluation process and showed different factors which could contribute to more consistent evaluation results in different periods.These factors include a detailed report assessment and project evaluation rubric; the extension of the categorization of the projects into major topics such as hardware products, software applications, and research projects along with providing more pertinent rubrics for each category; the random selection of the evaluation panel members while considering the relevance of participants' expertise to the project under evaluation, and the distribution of marks between the supervisor and a second marker.
We have addressed the elements of the assessment rubrics which the students find challenging, such as proposal preparation, literature review, report writing, time management, and preparation for the presentation.We suggested that intensive sessions on proposal preparation, research methods, and report writing could help students to manage these challenges.
Finally, we have recommended newly established programs in computing to take the necessary measures to reduce the challenges addressed in this case study by giving particular attention to the FYPs when they start their program.We found this crucial to develop an in-house workflow based on the experiences of well-established programs.However, we believe that it is crucial to adapt this workflow according to each educational institution, the local culture, the background of their prospectus students, and the possibility of enterprise/university cooperation.
In the future, we will continue to monitor the efficiency of our FYP-related procedures and will update and share our findings with the community.Also, we will study the case of FYPs from other perspectives such as finding correlations between the results and the performance of the students in different modules (courses), the classification and categorization of the FYP topics and their relations to students' performance in different areas.
2009-2010 academic year.Weekly meeting with students.Similar to the 2009-2010 academic year, except for the number of panelists which was restricted to three.
Similar to the 2009-2010 academic year.Weekly meeting with students.Similar to the 2009-2010 academic year, except for the number of panelists which was restricted to three.

10 a
Similar to the 2015-2016 academic year.Similar to the 2015-2016 academic year.Similar to the 2015-2016 academic year.The rubrics were slightly modified.26 There were 7 staff members in the department up to October 2009.b No students were admitted during the academic year 2009-2010, due to a transition in the academic structure which caused a gap in the graduation series.

Table 1 .
A comparative review of the processes related to the FYP in computing in different countries.

Table 2 .
FYP assignment, supervision, and evaluation process at the UKH during the academic years 2009-2017.In this table, IT stands for Information Technology, ACIT for Applied Computing and IT, CS for Computer Science, and CE for Computer Engineering.

Table 3 .
Summary of the report assessment rubric, before 2016.
* The updates according to the recommendations from the assessment in the first semester.

Table 5 .
Summary of the presentation assessment rubric, after 2016.