Evaluation of Product Development Success:A Student Perspective

The time pressure on new product development under unpredictable conditions requires the renewal of the project management approach that suggests the prominent role of project managerial competencies in achieving project success. Project management education must be adjusted to understand students’ opinions in the field. The study uses a survey among Hungarian engineering, business, and IT higher education students. The analysis aims to explore opinions about the main barriers to successful product development projects, and the expected ways of regulating the project by ANOVA and principal component analysis. The responses of 126 students confirm the appreciation of management competencies. Gaps in team composition, inadequate communication, common changes in the plans, and the lack of experience in similar development tasks are considered to be the main contributors to product development project failures. Collaboration and competition with external partners were found to be less essential factors. Students believe that regulation of the work is necessary, but the project team should be trusted to establish it. Beyond developing the curricula, the experience of this study can promote the successful execution of collaborative projects between companies and higher education institutions. It can establish expected student competencies to quickly become effective project team members.


Introduction
The traditional methods in project management are increasingly outdated. The market needs new products faster than ever before, and competition between companies is intensified. The accelerated changes lead to unpredictable product requirements and lead times. Reducing the lead time of a new product can be complicated as much by keeping the quality of deliverables and cost constraints as by the iron triangle of project management (Pollack et al. 2018). Companies with industrial products aim to support their customers to increase their performance (Kärkkäinen et al. 2001), but the time urgency causes uncertainty and difficulties. The new conditions affect the related project management approaches and methods. Integrated product development (Vajna 2020) offers the coordination of the related activities from market analysis to sales. Agile tools allow a better focus on customer orientation. Agility in project management becomes general, going beyond software development in line with embedding lean principles in operation. Product development models have long shifted from the traditional predictive to an iterative or incremental project approach, however, the development is ongoing.
Although project management methods are widely available and industry-specific procedures, application success significantly depends on the approach and skills. Successful product development projects are available through carefully selected and applied project management methods accepted by all affected stakeholders. The increasing complexity 2 of 17 of the products is perceptible, and that, together with the characteristics above, raises the need for the collaboration of several professions. Knowledge integration capability (Todorović et al. 2015) and managing the interdisciplinary composition of the project team (Henson et al. 2020) are key factors of sustainable project success. Armenia et al. (2019) offer a framework of sustainable project management based on five key dimensions, including corporate policies and practices, resource management, life cycle orientation, stakeholders' engagement, and organizational learning. The concept of connected leadership (Hayward 2016) can be applied to the project management challenges: • purpose and direction: a common understanding of the project goals is worthwhile, • authenticity: a trusting relationship among the project team members and the stakeholders allows for integrity; • devolved decision-making: authorization of the team members can speed up problem solutions; • collaborative achievements: project outcome is the result of shared efforts, and effective performance requires collaboration; • agility: adaptation to the changing requirements is required in line with the purposes.
The duty of a university can be marked as developing human capital; in particular, its purpose is to prepare the students for their future managerial and employee roles (Bejinaru et al. 2018). An exceptional opportunity is opened if higher education students work on real projects. Pilot projects that involve engineering and management students are excellent instruments for testing new solutions and finding the development gaps.
Improving the collaboration of universities and corporations in product development has several benefits (Soh and Subramanian 2014;Un and Asakawa 2015;Apa et al. 2021). Additional knowledge, laboratory capacity, management skills, and tools can be complemented temporarily or permanently. Flexibility can be achieved by long-term cooperation. Beyond the direct benefits according to the actual deliverables of the projects, the experience can be used to develop the students' skills through a more practice-oriented curriculum in project management. These students will enter the labor market with ready-to-use knowledge.
The research can contribute to preparing the engineering and management students for project tasks. Exploring the students' approach to project management is an essential part of this learning process. The goal of the study is to explore the students' opinions about new product development success factors through a voluntary online survey.
The next sections are organized as follows. Section 2 presents the literature review of the topic, focusing on the key issues related to the survey development. Section 3 shows the research questions and the methodology. Section 4 deals with the discussion of the results, and Section 5 summarizes the conclusions. An appendix includes the main results of the analysis.

Approach to New Product Development
New product development has been in focus for a long time (Karakaya and Kobu 1994;Jones and Stevens 1999;Lewis 2001;Derbyshire and Giovannetti 2017;Iqbal and Suzianti 2021). Based on literature reviews and analysis of product development case studies, the success factors and the process approaches are traceable. Regardless of the technical characteristics of the given era, studies agree that new product development bears a higher risk than other projects. The main characteristics of new product development are high uncertainty and high complexity that can have a negative impact on the product development performance (Ahmad et al. 2013).
There is a significant agreement that a predictive approach to product development is no longer appropriate; adaptive solutions are required. The PMBOK standards (PMI 2017(PMI , 2021 distinguish predictive, iterative, incremental, adaptive, or hybrid development approaches. A predictive (also called traditional) approach with a waterfall planning of the tasks is feasible when the project and product requirements can be defined, collected, and analyzed at the start of the project (PMI 2021). Iterative and incremental (also called Adm. Sci. 2022, 12, 49 3 of 17 adaptive) approaches fit the uncertain conditions better. Iterative work organizing can be preferred if deliverables are not required before finishing the project, while incremental life cycles deliver more often than a single final product. Hybrid approaches aim to change the behavior with the current requirements. Customized models of new product development reflect well the basic approaches above. Integrated Product Development models (Table 1) can give an industry-independent comprehensive framework (Gerwin and Barrowman 2002;Vajna 2020 A synthesis of the achievements of product development models is to be found in Autogenetic Design Theory (Vajna 2020), by integrating the benefits:

•
Marketing, product, and production task integration with management focus according to Olsson (1985) and Andreasen and Hein (1987); • Simultaneous optimization of the product and its production processes, according to Ehrlenspiel (1991); • Collaboration of human, organization, technology, and methodology, according to Meerkamm (1995); • Dynamic product development of Ottosson (1996) for reacting quickly to changing conditions.
The enhancement of product development process approaches in recent decades shows that the scope of project management responsibility moved from the narrower engineering issues to considering the changing requirements and environment. However, the essence of product development remained the output of the project; the understanding of success needs a more comprehensive approach.

Project Success
Project success can be described by the iron triangle model that gives the triple bottom line of success as scope, time, and cost constraints in a simple way, but there has long been a consensus that the contribution to corporate strategy and stakeholders' satisfaction must be considered (Görög 2019;Verzuh 2021). Moreover, intensity, extension, and predictability of the environment force a change in strategy and strategic management (Deutsch et al. 2017), leading to the need for a continuous rethinking of project success. Exploring the project and project management success factors is the focus of interest in general and at the industry level. The success factors can be grouped into three categories by Radujkovića and Sjekavica (2017):

•
The elements of project management competency, including behavioral, technical, and contextual competencies of project manager and project team members; • Organizational culture, structure, competence, atmosphere; • Project management methodologies, software, tools, techniques, risk assessment tools, and communication support tools.
According to new product development, Cooper (2019) identified 20 drivers of success into three categories: • Product: the characteristics of the new product project or the product itself; • Business: Drivers of success for the business, including organizational and strategic factors such as the business's innovation strategy and how it makes its R&D investment decisions; climate and culture; leadership; and how the firm organizes for NPD; • Methods: the systems and methods applied for managing new product development.
Organizational factors influencing new products include visionary leadership, structure, key individuals, teamwork, extensive communications, high involvement, customer focus, the creative and innovatory climate, and the learning organization (Jones and Stevens 1999). A learning organization represents the capability of achieving sustainable competitive advantage due to generative learning processes (Bratianu et al. 2020), where projects play a dominant role. Shenhar et al. (1997) defined four success dimensions by the impact areas as the fulfillment of the triple bottom line of the project: impact on customers, corporate success, and preparing for the future. According to the recent pandemic, Hallstedt et al. (2020) emphasize that adequate responses to digitalization, sustainability, and servitization influence the success of new product development. A new product development project usually showcases state-of-the-art technology and focuses on the future (Table 2). This confirms the need for managing uncertainty (Szabó and Cserháti 2013). However, literature on product development success (González and Palacios 2002;Cooper 2019) reflect the even more agile approach and highlights the strategic focus, communication, team skills, knowledge management, and other soft factors, and a survey among product development experts (Soltész and Berényi 2021) confirmed that beyond that adherence to stability, setting clear project goals is given a high weight. A clearly written set of project goals, as a specification or scope, was evaluated as the most important success factor, followed by the collaboration of the project team.

Project Failures
Failures in achieving the project goals cannot be excluded. Understanding the nature and influencing factors of project failures allows for appropriate strategies for managing the related risks. Non-compliance with the requirements derived from the iron triangle, shortcomings in meeting stakeholder expectations, and strategic contribution may come from the task complexity or environmental uncertainty. Pinto and Mantel (1990) emphasize three aspects of evaluating the project's success or failure: • the implementation process itself, internal efficiency; • the perceived value, quality of the project deliverables; • client satisfaction with the delivered project, external efficiency.
Project success can describe the areas in which the project manager must perform well, but understanding failures provides the opportunity to learn. Project risk management (Fekete and Szontágh 2020) is for mitigating concerns; lessons learned are essential inputs of the related actions. Exploring the reasons for these failures can suggest prevention and other precautionary measures (Gupta et al. 2019). Several blogs and learning materials deal with the reasons for the project failures, including general and industry-specific focus. Common literature items are in line with the categorization of success and failure factors. Emam and Koru (2008)  Sub-optimal team size and composition; • Inconsistent monitoring and control (lack of expert supervision); • Resistance to change (partial cooperation by employees).
Although achieving project goals is a multi-faceted problem, literature agrees that it can be supported by appropriate project management. 'Soft' factors such as project management competencies, communication, and team-level collaboration are appreciated. Networking also came into view (Li and Yu 2022). It cannot mean that all 'hard' factors, such as plans or regulations, would expire; it suggests that more is needed. Since new product development has a passing-through impact on production technology, organizational challenges, and, in parallel, customer habits and satisfaction, a particular emphasis should be paid to success and failure factors already during the education period.

Research Goal
The study deals with the assessment of product development success factors among higher education students. A negative question was formulated, and a set of factors was asked to be rated about the contribution to the failure of a product development project. The goal was to explore the opinions of the students in the field. Regardless of the students' experience in project management, and especially in product development, the responses can help them understand their attitudes toward the projects. The results can be used for curricula development by highlighting the misunderstood factors. Beyond improving education, the information is significant for the companies when hiring new staff.
There were four research questions formulated to cover the investigations: • RQ1: What level of regulation is considered appropriate by the students? • RQ2: What are the main barriers to product development projects' success based on the students' judgment? • RQ3: Can patterns be explored in the students' judgment?

Research Instrument and Analysis Methods
A voluntary online questionnaire was designed for data collection in Microsoft Forms. The questionnaire included twelve factors for assessment, as shown in Table 3. The factors included planning and regulation of the project, competencies of project management and team members, company issues, and external issues. Short names added to Table 3 describe the factors in the results for a simple review of the figures and tables of the paper. The respondents were asked to evaluate the contribution of the statements to project failures. The questionnaire used a 5-point scale with the endpoints 'does not contribute at all' and 'typical reason'. The questionnaire also included four statements to be ranked about the most effective way for achieving success:

•
Keeping the written regulations and plans is the most expedient (Keeping written regulation); • The project team must define the rules for a given project (Rules defined by the project team); • The team must consciously adapt to changing situations (Adapting to the changing situations); • There is no need for overregulation since everyone performs to the best of their ability that leads to the fastest competition in the project (No need for detailed regulations).
The analysis was performed with IBM SPSS 25 software, following the guideline of Pallant (2020) and Babbie (2020). The survey items were described with mean value, standard deviation, skewness, and kurtosis. Beyond the descriptive statistics, the assessment of the students was presented by rank orders calculated based on the mean values and proportion of the high agreement (four or five) answers. The relationship between the sur-vey items was checked by Spearman's correlation method. The impact of grouping factors (profession, gender, study level, and work experience) was tested by the non-parametric Kruskal-Wallis analysis of variance, adjusted to the level of measurement. Based on this, results were considered as significant differences by the grouping factor if the significance level is lower than 0.05.
Since the items of the survey were categorized preliminary according to a researcher's vision, structure validation was a critical issue at this point in the research. Principal component analysis was selected as a dimension reduction method with Varimax rotation to maximize the variance of the components. The applicability of the analysis was tested by the Kaiser-Meyer-Olkin criteria, Bartlett's test of sphericity, and the anti-image correlation matrix method.

Research Sample
The research sample consisted of 126 responses, including 39 business and 47 engineering students at the University of Miskolc and 40 students from the IT field at the University of Miskolc and the Ludovika University of Public Service. The data collection period was the fall semester of the 2020/2021 academic year.
Beyond the profession, the gender, study level (bachelor or master), and work experience (no, internship, yes) were identified as grouping factors for the analysis. Sample characteristics are summarized in Table 4. The reliability analysis showed acceptable results (Cronbach's Alpha = 0.711).

Approach to the Regulation of Product Development Projects
Although the predictability of product development is low, and changes are common, the students considered the rules and regulations as necessary. Based on the ranking of the related questions, only a fifth of them ranked in first place that the project will be completed the fastest, and there is no need to complicate it since everyone performs the tasks to the best of their knowledge. At the same time, the students believed that the rules must have been defined by the project team taking the specificities of the given project into account. Opinions on keeping the written regulations were divided. The mean values of the rank orders are presented in Figure 1, and the distributions of the rankings are detailed in Table 5. completed the fastest, and there is no need to complicate it since everyone performs the tasks to the best of their knowledge. At the same time, the students believed that the rules must have been defined by the project team taking the specificities of the given project into account. Opinions on keeping the written regulations were divided. The mean values of the rank orders are presented in Figure 1, and the distributions of the rankings are detailed in Table 5.

Factors of Product Development Project Failures
Improper communication solutions between team members were the most important contributors to product development project failure based on the mean values of the students' evaluations ( Figure 2). It is followed by selecting the team members, changes in the project plan, and the lack of experience in similar development challenges. According to the grouping of the factors investigated, external issues and the lack of regulation were at the end of the list. The descriptive statistics of the results and the correlations are placed in Appendix A.

Factors of Product Development Project Failures
Improper communication solutions between team members were the most important contributors to product development project failure based on the mean values of the students' evaluations ( Figure 2). It is followed by selecting the team members, changes in the project plan, and the lack of experience in similar development challenges. According to the grouping of the factors investigated, external issues and the lack of regulation were at the end of the list. The descriptive statistics of the results and the correlations are placed in Appendix A.  The low scattering of the mean values and negative skew of the distributions is a characteristic of the sample. Kurtosis of the evaluations suggested the highest agreement on the opinions about the lack of corporate-level cooperation (K = 0.494) and the lack of experience in similar development tasks (K = 0.405). Based on the distributions of the responses, the proportion of respondents with a four or five (typical reason for the failure) rating is highlighted. The factors by this indicator show a different order in the top five positions (Figure 3). Improper selection of the team members was marked as the most common or typical reason for the failure (76.2% of the respondents). Lack of experience in the development task (75.4% of the respondents) was considered more relevant than based on the mean values, while improper communication (74.6% of the respondents) was only in the third place. Too common changes in project plans were considered the typical reason for failure by 69.9% of the respondents (fifth position), while it was the third by the mean values (Table 6). The low scattering of the mean values and negative skew of the distributions is a characteristic of the sample. Kurtosis of the evaluations suggested the highest agreement on the opinions about the lack of corporate-level cooperation (K = 0.494) and the lack of experience in similar development tasks (K = 0.405). Based on the distributions of the responses, the proportion of respondents with a four or five (typical reason for the failure) rating is highlighted. The factors by this indicator show a different order in the top five positions (Figure 3). Improper selection of the team members was marked as the most common or typical reason for the failure (76.2% of the respondents). Lack of experience in the development task (75.4% of the respondents) was considered more relevant than based on the mean values, while improper communication (74.6% of the respondents) was only in the third place. Too common changes in project plans were considered the typical reason for failure by 69.9% of the respondents (fifth position), while it was the third by the mean values (Table 6).

Alternative Factor Structure
The evaluation orders show a scattered picture by grouping the failure factors, except the external issues. Therefore, a principal component analysis was performed with Varimax rotation to check the factor structure assumed in Table 7. Kaiser-Meyer-Olkin test (KMO = 0.709), Bartlett's test of sphericity (Chi-square = 243.251, d f = 66, p = 0.000), and anti-image correlation matrix method (minimum value in the main diagonal is 0.562) confirmed the applicability of the analysis. The results offer an alternative factor structure. The alternative factor structure suggests a different approach to the project success or failure of product development (Table 8). The first factor includes the question of team-level cooperation by proper selection of them and the communication. The second factor consists of external issues and regulation; these can be considered enablers of a project. While the items of team-level cooperation were the most decisive success factors, enablers were considered at the end of the list. The third factor collects the items of project and corporate management competencies in framing and controlling the product development project. The fourth factor consists of professional issues, including project planning success, former experience in product development, and the ability to react to competitors' actions.

Analysis of Variance
Since different professionals contribute to projects, and the emphasis on product development and project management may be different in study programs, it was expected during the compilation of the research sample that the sub-samples would show distinct opinions on product development failures. Figure 4 shows the mean values by profession.

Analysis of Variance
Since different professionals contribute to projects, and the emphasis on product development and project management may be different in study programs, it was expected during the compilation of the research sample that the sub-samples would show distinct opinions on product development failures. Figure 4 shows the mean values by profession. IT students rate the 'too common changes in plans' as being more harmful to the project than other professions, and engineers are the least worried about it. The evaluation of the external factors, including the relationship with external partners and access to market information, show a similar picture. Engineering students' evaluations stand out from the others according to team composition, and especially the cooperation between company management and the project team. However, Figure 4 presents remarkable patterns by professions; the analysis of variance could confirm significant differences in only a few cases. Beyond the profession as a grouping factor, the Kruskal-Wallis H test was IT students rate the 'too common changes in plans' as being more harmful to the project than other professions, and engineers are the least worried about it. The evaluation of the external factors, including the relationship with external partners and access to market information, show a similar picture. Engineering students' evaluations stand out from the others according to team composition, and especially the cooperation between company management and the project team. However, Figure 4 presents remarkable patterns by professions; the analysis of variance could confirm significant differences in only a few cases. Beyond the profession as a grouping factor, the Kruskal-Wallis H test was performed for gender, study level, work experience, and former participation in product development projects. Based on the study level, work experience, and experience in product development projects, the sample is considered homogeneous for each question. The significant results (p < 0.05) are highlighted in Table 9; the analysis results are placed in Appendix A.

Evaluation of the Results
Project management learning materials give a high emphasis to establishing the framework of the project including the work breakdown structure, timing plan, cost, and deliverables. The planning covers personal responsibilities as well. Recent experience questions the possibility of this predictive approach. Software development projects put agile project management in the foreground, and an ever-widening range of sources deals with its applicability in other areas. New product development projects usually must face a less dynamic environment than software engineering, but the technological uncertainty may be higher when delivering before the end of the project is not expected; iterative and incremental approaches fit better to product development, but agile tools and approaches are to be considered. A consequence of the predictability and technological uncertainty is the need for rethinking the way of regulations. RQ1 asked the students about the expected way of regulation. The results confirm that regulated work is important; 46% of the respondents ranked last the option in the questionnaire that project success will be assured by personal skills and competencies while detailed regulation is redundant. Rules defined by the team are the most popular way, followed by a continuous adaption to the changing requirements.
Related to the second research question about the success factors of product development projects, the survey confirmed appreciation of the 'soft' factors. When interpreting the results, it should be taken into account that the respondents are higher education students with limited experience in projects. However, it is encouraging that their opinions and attitudes mirror the challenges of the uncertain environment of the project. All listed factors represent high mean values, but the rankings signal that communication and cooperation are the most relevant success factors. At the same time, there seems to be a sense of trust in the 'hard' factors and external issues of the project.
The third research question investigated the clustering opportunities of the sample. Based on the statistical analysis, only a few significant results were found. The sample seems to be homogenous by the grouping factors even by the work experience of the students. Patterns of the opinions could not be determined.

Conclusions
The increasing complexity of the projects required adaptive project management approaches. New product development is a special area of projects since it must deliver under high technical uncertainty. The increased pressure on introducing a new product to the market faster than before, while costs and quality are controlled asks for new methods and approaches in project management. Changing the culture and refining the new procedures is time-consuming and risky. Investigating the opinions of higher education students has remarkable benefits. First, the experience of the study supports developing the learning materials in the field. The result can be a more comprehensive understanding of product development challenges, leading to more effective integration of the new workforce. Second, corporations can gain relevant information about the preparedness of the students and to find appropriate work for them. In addition, communication and problem-solving can be supported if the values are known. Third, higher education students are more often involved in real-life projects (Berényi and Vadász-Bognár 2019;Bihari and Tóbis 2019). In these cases, effective project management support is not a future aim but a present task.
The results show that the students' opinions reflect the key success factors of product development projects in the literature. 'Soft' factors of projects such as communication and collaboration are appreciated. A former study among product development experts (Soltész and Berényi 2021) found the importance of teamwork, project manager's control, and other factors of success that allow flexibility and responses to the uncertainty of the project. However, a well-defined project goal is among the most important success factors next to these. Both the experts' and students' surveys give a lower emphasis to the 'hard' factors of project management, but it cannot mean that these are not relevant anymore. Experience shows that time and cost planning or assuring the resources of the projects are considered as enablers. In other words, these remained important, but not enough for success. The survey results are encouraging in this sense. The few statistically significant differences by the grouping factors could suggest the homogeneity of the sample and common thinking of the respondents, but this must be questioned.
Despite the thorough planning of the survey, the study has some limitations. The survey included general questions; however, each project is a unique endeavor. The sample composition used a convenient method, and the universities involved were limited. Engineering, business, and IT students were separated, but they cannot cover all responsibilities and positions of product development. Another limitation comes from the fact that the results were based on the responses of higher education students. Although a large proportion of them has work experience and some were involved in product development, a general conclusion from the results is not available. The study can be considered as a pilot investigation.
Further work in the research is expanding data collection and involving new grouping factors. Another task is preparing the practical application of the experience. Projects performed in collaboration between corporations and universities offer experience in new management methods. A survey is also planned that measures the opinions of the students before, during, and after the project work. It will let us make more comprehensive conclusions on the topic and develop a framework for targeted education for future product development experts. Data Availability Statement: Data are available from the corresponding author by request.

Acknowledgments:
The study was conducted as part of the OTKA T139225 project entitled "Management readiness level towards Strategic Technology Management Excellence".

Conflicts of Interest:
The authors declare no conflict of interest.

Appendix A
The appendix presents detailed descriptive statistics, Spearman's correlation coefficients, and Kruskal-Wallis H test results for the survey items.