Results Analysis of Russian Students’ Participation in the Online International Educational Project X-Culture

Abstract

The purpose of the paper is to assess the experience of Russian students in the online project class. The authors studied the impact of the international project “X-Culture” on the level of development of socio-cultural competencies of students and their knowledge of the English language. An additional aspect of the analysis was the question of the connection of students’ perception of the goal of participation in the international project “X-Culture” with their self-motivation and self-assessment of the success of the results achieved. The study used both quantitative and qualitative methods—student testing, database analysis provided by the project “X-culture”, and focus group records. The result of the participation has been inconsistent. As expected, such sociocultural competence as the level of proficiency in English increased among the majority of students, as confirmed by the values of the student’s criterion for the results of language testing, conducted before and after the project. At the same time, average indicators of sociocultural competences such as “interpersonal skills”, “creativity”, “leadership”, and “friendliness” have deteriorated during the project that is confirmed by the trends lines of time series. The focus group revealed differences in terms of participation in the X-Culture project, their connection with self-motivation, and student satisfaction with the results achieved. Students who have achieved high levels of sociocultural competences have set themselves the goals of participation in the project related to the improvement of professional competencies and intercultural communications. At the same time, students were most satisfied with their activities in the project and the results achieved, aimed at obtaining new professional knowledge and skills, and students whose goal was simply to obtain an international certificate of the project participant.

1. Introduction

The era of globalization provides for the integration of university graduates into the system of international economic, social, cultural, and labor relations, which requires modern youth to possess new knowledge and competencies, including sociocultural competence. In the 1970s, the concept of “socio-cultural competence” was developed in foreign literature, especially in demand in the modern world. Today it is interpreted as an integrative personality quality, which is a “system of cognitive, communicative, axiological, and activity components that help to combine theoretical knowledge with practical skills and behavioral strategies defined in relation to social and professional activities of the future specialist and determining the student’s ability to establish contacts with the subjects of sociocultural activities and participate in joint decision-making …” [1].

The competence approach implemented in Russian higher professional education implies the formation of the following universal competencies among bachelors studying in the field of “Advertising and Public Relations”: abilities for social (including intercultural) interaction and realization of their role in a team, for project activities, and to business communication in a foreign language. In this case, the future specialist should be able to apply modern information and communication technologies and technical means [2]. Foreign language knowledge is one of the core competencies that allow one to maintain the diverse and multi-level contacts and forms of communication, but to be a successful international communicator it is not sufficient to make progress in foreign language proficiency. The learner should also develop other global competencies, such as intercultural and interpersonal communicative competence and digital competence, and integrate them into a common set of their skills, which implies an attitude of flexibility, tolerance, and cooperation [3,4]. Today integrated competences in the field of foreign languages and cross-cultural communication are considered to be vital for successful involvement and participation in modern life and society [5,6].

The best conditions for the development of contemporary competencies are created using the educational technology “learning by doing” [7,8]. The task of improving of the skills of intercultural interaction is solved through the participation of students in various international educational projects [9,10]. This is the reason why we decided to introduce one of such projects into the undergraduate students’ curriculum in the fall semester 2018.

The name of this international online educational project is “X-Culture” and it was developed in 2011 at the University of North Carolina (USA) [11]. The essence of the X-Culture project is to provide students with the opportunity to gain practical experience in solving real cases developed by companies specifically for the project. Participation of students from the Peter the Great St. Petersburg State University in this project was provided by the training course “International Business” in the last year of undergraduate study. More than 3800 students from 40 countries took part in the project in the autumn semester of 2018. Divided into small teams, they worked on solving various problems in the field of international marketing and business. Of the project participants, 79.2% were from 9 countries, 20.8% were from 31 countries. 42.8% (1573 people) of the participants were from American universities, while Russian students comprised about 1.9% (69 people).

The project envisages both an assessment of the collective result of each team’s activities by experts, and the monitoring of various aspects of each participant’s personal activities carried out by teammates. The conclusion about the development of sociocultural competence of students who took part in the X-Culture project was based precisely on these assessments.

The purpose of this paper is to investigate the impact of the intercultural online “X-culture” project on the development of socio-cultural competencies of Peter the Great St. Petersburg Polytechnic University students. Another aspect that we would like to investigate is whether there is a connection between the perception of the students of the goal of participation in the international project “X-Culture” and their motivation and self-esteem success achieved.

2. Theoretical Background

The development of new technologies and changes in industrial fields and communication culture are the main triggers of implementing project-based learning (PJBL) in higher education system. PJBL was also stated as a special environment where students obtain not only knowledge but real-life skills. Based on constructivist theory, PJBL moves from traditional teaching methods to student-centered approaches [12]. As the constructivist theory stated, knowledge is not only what is taught, but it must also be formed by the active behavior of students [13,14,15]. According to Sidman-Taveau [16], the learning process occurs when students connect new information with their basic knowledge that builds a long-lasting knowledge. As the responsibility for learning is transferred from teacher to student, the roles of the teacher change—from lecturer to partner, facilitator, guide, and tutor roles. Students are supposed to be active and gain knowledge in the process of active thinking and solving various problems. In this case knowledge becomes not just rote-memorization, tasks, or exercises made for teachers [17], but a selection, preservation, and elaboration of new information that allows learners to satisfy their curiosity. Students’ active participation is essential in PJB; due to this deep-learning approach students are engaged in exploring real-world problems and create an active learning environment through significant and valuable projects. In addition to increasing student engagement in researching important and significant issues, PJBL also helps students develop collaboration and communication skills that are substantial features of students in working life [18,19]. Since projects are often carried out in groups, this joint activity teaches and develops students’ abilities for teamwork and productive collaboration [20].

Many researchers have confirmed the great benefit of project work in improving foreign language learning outcomes [21,22]. Pietila and Virkkula (2011) showed in their research that introduction of project-based learning improves the quality of teaching and learning and contributes to a higher level of cognitive development; involving students in solving complex and innovative solutions to problems, teaching them complex processes and procedures, such as planning and communication, contributes to genuine research and independent learning [23]. Another group of researchers noted that PJBL positively impacted the motivation level of students [24].

Most of the published papers on the effectiveness of PJBL methodology are primarily focused on medicine, engineering, mathematics, and computer science learning and we have not found any papers in which the effect of project-based leaning on overall students’ sociocultural competencies formation was studied.

The works of Merrill Swain and Michael Canale [25], who developed the views of the sociolinguist Dell Hymes, presented in the article “On communicative competence”, written in 1972 [26], played a leading role in determining the substantive characteristics of the phenomenon of socio-cultural competence. Later, the scholar Van Ek, who defined sociocultural competence as “acquaintance with the sociocultural norms of the language, and social competence as the desire and skill to interact with other people, singled out sociocultural and social competence in the structure of communicative competence” [27]. Some researchers considered sociocultural competences in the context of multicultural and intercultural competences [28,29,30,31].

However, the formation of sociocultural competences in the preparation of specialists with higher professional education has so far not been given sufficient attention. We can only mention works in which it was pointed out that the formation of sociocultural competences in education is important because it helps a person to perceive himself as a citizen of the world and trains him to successfully operate with sociocultural knowledge [32].

Our current study is to share the experience of humanitarian students’ participation in PJBL and to fill the existing research gap using mixed methods approach taking into account both the development of students’ sociocultural competencies and students’ perception of the project’s goal and its influence on their motivation and self-assessment.

3. Materials and Methods

A total of 63 undergraduate students voluntarily participated in the study. All of them were from the Advertising and PR Department of the Peter the Great St. Petersburg Polytechnic University. They were enrolled in a two month-long Intercultural online project as part of their required program of study (1 October—24 December, 2018). Nobody had prior experience of participating in such international projects before. To achieve the goal, in this study we used a mixed methodical approach in a consistent explanatory research project (

Table 1. Data collection.

Results	Sort of Data Collection	Type of Data
Sociocultural competencies	X-culture project data, collected through various surveys and questionnaires	quantitative
English level	Testing on Listening, Reading, Writing and Speaking	quantitative
Project goals understanding	Interviewing the focus group	qualitative

).

The first stage of data analysis began with quantitative data that were extracted from the array of educational platform evaluation data of the “X-Culture” project. Extensive project statistics provided an opportunity to study the dynamics of the results of project-based learning for each individual student, and for the entire set of students in the department participating in the project. 63 people were registered for the project, 57 people started to perform, 6 people were not certified, 5 people did not work regularly, and therefore the statistical base of the study was 46 students. A total of about 4000 assessments of students and experts on the team were analyzed. Particular attention was paid to the estimated indicators of universal competencies, designated in the project as “communicability”, “diligence”, “creativity”, “leadership”, and “friendliness”. The first two indicators were evaluated for the period of the project 8 times, the remaining three—9 times.

At the second stage, the level of proficiency in English was assessed using a quantitative method in the form of testing. The test consisted of 4 parts: Listening, Reading, Writing, and Speaking. In Part 1, students watched the video on one of the course topic, and then gave short answers to the questions asked. In the Reading section, students were offered a text (case study) with a volume of approximately 500 words; after reading it was necessary to complete 3 tasks: complete text with correct topic sentences for each paragraph, choose whether the statement is True/False, and answer the questions about the main information from the text. Writing skills were evaluated on the base of answers from Listening and Reading sections. For the speaking test, students conducted small individual presentations and verbally answered few questions.

At the third, qualitative stage of data analysis, researchers studied the variables covered by quantitative data analysis. This was achieved by conducting a focus group with three students - project participants. Researchers on the basis of academic performance, which provided a diverse pool of students, formed the focus group. To participate in the focus group the following participants were selected: a student who received the maximum average score of five indicators—4.8, a student who gained 3.9 points and a student with a minimum average score of 2.7. By analyzing the dialogue from the focus group transcripts, the researchers were able to identify the main points that reflect the students’ opinions.

At the final stage of the analysis, the researchers made meta-conclusions that identified topics in both quantitative and qualitative data that answer the research questions. Within the final phase of analysis, the researcher drew meta-inferences, which identified themes in both the quantitative and qualitative data that are responsive to the research questions.

This paper is based on the following research questions:

(1).

Is there any difference in the level of sociocultural skills before PJBL and after the course?

(2).

Is there a significant development in the level of English proficiency before PJBL and after the course?

(3).

How does students’ perception of the goal of participation in the project affect their motivation and evaluation of the achieved result?

4. Results and Discussion

4.1. Sociocultural Competencies in Online Project

The initial purpose of the study was to identify the increment of competencies, but already the first results of the statistical data analysis forced us to make adjustments to the targets. This was due to the fact that many students during the project did not improve, but rather worsened their performance. An example of the negative dynamics of the average values on the indicator “Diligence (Effort)” is shown in Figure 1.

Data on the results of training groups are shown in

Table 2. Key findings of the group on selected indicators.

	Communication	Diligence (Effort)	Creativity (Ideas)	Leadership	Friendliness
The average final score of the entire group	4.3	3.7	3.7	3.3	4.3
Minimum average final grade	2.7	1.9	1.9	1.6	3.3
Maximum average final grade	5.0	4.8	4.7	4.6	5.0
Scope (max-min)	2.3	2.9	2.8	3.0	1.7
Percentage of students with a final unsatisfactory grade %	4.4	19.6	15.3	32.6	2.2
Percentage of students who have deteriorated grades by the end of the project %	52.2	67.4	65.2	73.9	56.5
Percentage of students with unstable grades (scope more than 1.5) %	47.8	47.8	47.8	60.9	41.3

.

Based on the data of Table 1, a portrait was drawn up of the Russian SPbPU student-participant in the project: sociable and friendly, but not too creative and diligent, working unstable and not loving to play the role of leader in the project. Individual average final scores were as follows: (Figure 2):

This experience was useful not only to students, but also to teachers; first, in terms of developing sociocultural competences, and secondly, from the standpoint of including such pedagogical techniques as a project approach involving teams that communicate in English.

4.2. English Testing Results

As the discipline was taught in English and all communications between project’s participants were totally in English we decided to estimate how the course based on PBL influenced the students’ English level. The testing on English proficiency of all students (n = 63) was conducted two times—before and after the course.

Figure 3 shows the results of the initial testing and final testing of the four groups for comparison of the results obtained.

In general, the overall quality of students’ English knowledge (we measured listening, reading, writing, and speaking skills) was improved. The comparison of results of the two tests (before and after the course) of all participants in the experiment shows that the improvements were significant in all categories excepting the writing test (the significance level was set to 0.05). (

Table 3. Descriptive results of pre-test and post-test on English proficiency.

Category	Test	Results (Average Mean)	SD	T-Value
Listening	Pre-test	14.3	2.14	5.4 ***
	Post-test	17.87	2.18
Reading	Pre-test	14.53	1.99	5.4 ***
	Post-test	18.47	1.89
Writing	Pre-test	17.37	2.5	2.3 *
	Post-test	18.7	1.93
Speaking	Pre-test	16.33	1.78	8.2 ***
	Post-test	20.93	1.48

Note: * p < 0.05; ** p < 0.01; *** p < 0.001.

)

Such results can be explained by the fact that during the participation in X-Culture intercultural project writing skills were rarely used and assessed as the communication was implemented mostly via Skype and different messengers where grammatical mistakes were ignored.

Anyway, we can confirm the efficiency of PBL for English learning purposes.

4.3. Interrelation of Students’ Perception of the Purpose of Participation in the Project “X-Culture” with Motivation and Evaluation of the Achieved Result

The study of the relationship of students’ perception of educational goals with motivation and assessment of achievements is based on the theory of Labaree. David F. Labaree [33,34,35] developed a construction of three competing goals for American education: democratic equality, social efficiency, and social mobility. He believes that students’ understanding of education goals and their attitude towards these goals can play a fundamental role in shaping their educational experience. First, democratic equality implies that the educational system should focus on the training of citizens; second, social efficiency, in which the focus should be on the training of professionals (workers); and third, social mobility, which suggests that education should focus on preparing individuals to participate in socio-economic structures. The first goal, democratic equality, is manifested in three aspects: citizenship education, equal treatment, and equal access. In the process of citizenship education, the education system is seen as a mechanism to strengthen the sense of civic virtue in an attempt to provide a sense of contributing to the common good. The second goal of education, social efficiency, relates to professionalism and educational stratification. Educational stratification, acting as a collective good and focusing on increasing human capital, reflects the situation in the labor market, thereby ensuring people’s readiness for the existing socio-economic structure. The third goal, social mobility, sees education as a private good. We are talking about educational consumers who see as the goal of education not the knowledge they study, but the certificates they receive there. In other words, social mobility provides people not with their skills but with the mandates they need to move forward.

It is assumed that students should have a clear idea of their overall goal of participation in the project, and should strive for a mutually beneficial educational result. Most importantly, these results should be consistent with the results of all stakeholders, including project coordinators and companies.

The analysis of the Peter the Great SPbSTU students’ participation in the project “X-Culture” was supplemented by a focus group study. According to Creswell [36], a focus group is a common and useful tool when conducting phenomenological studies. Focus groups are advantageous in that they facilitate participant interaction and discussion, especially when time is limited. Moreover, focus groups may elicit more information than a one-on-one interview when participants are hesitant to provide information or elaborate.

The purpose of the study of the focus group was to find out how students’ perception of the goal of the international project “X-Culture” was associated with their motivation to participate in the project and influenced on the self-assessment of the success of their results.

The focus group participants were recruited by the researcher in the international business class, which was a requirement for all final-year PR and Advertising undergraduate students, and thus provided the researcher with a diverse pool of students. Upon approval of the questions, the researcher conducted a focus group with three last year bachelor students. Through the analysis of the dialogue from the focus group transcripts, the researcher was able to identify the main provisions that reflect the opinion of students.

The analysis of the transcript records of the focus group revealed that the students perceived three different goals of their participation in the project. The first participant defined the goal in the project as obtaining new professional knowledge and skills. The second student defined the goal of the participation in the “X-Culture” project as an intercultural communication in the international team. And the third participant defined the purpose of participation in the project—to obtain a certificate of participation in the international project for the student’s portfolio.

It is interesting, but the results of the focus group were consistent with the three objectives of education allocated by D. Labaree. The goal of the first student—the expansion of professional competencies—is quite consistent with the idea of social efficiency, where the main actor is a professional and a good worker. Choosing between the ambitions of improving skills or achieving results, the student told about the goal of professional development, which refers to the development of human competence. Since the process of developing professional competencies in this case was more important than the result, the student noted a high degree of interest in the project and constant self-motivation throughout the project. Noting his own new professional achievements, the student noted that the new knowledge will help to feel more confident in the future professional life. It is likely that the focus of the student on the process of work in the project is due to the fact that the student rated his participation in the project as very successful.

As noted earlier, the second focus group participant defined the goal of participation in “X-Culture” in gaining experience of intercultural communication in an international team. This goal corresponds to the “spirit” of the concept of democratic equality, with the establishment of education and training of the citizen of the world, the development of a global democratic society with equal rights, tolerance and equal access to common goods. During the conversation, the student showed exactly this understanding of intercultural communication. In the process of working in the project team, the second participant devoted a lot of time to informal communication, demonstrating the intention to continue to be friends after the project. The motivation for participation in the project was determined by the student as unstable, which was due to the changing psychological climate in the team at different stages of the project. Assessing his own success in the project, the second participant noted that he was not very successful and not happy with the result, as a pair of American students dominated, acted solely in their own interests and did not take into account the opinion of the whole group.

The third student defined the goal of participation in the project as an opportunity to obtain an international certificate, which may be useful both for his personal portfolio and for further employment. We can say that this student acts as an actor–consumer, for whom the private good is most important. Educational consumers in the concept of social mobility are focused not so much on knowledge and skills as on obtaining documents that allow them to move forward. Indeed, employers when hiring young graduates tend to have more confidence in diplomas, certificates and other documents, i.e., everything that officially confirms the specialist qualification. Since the third participant knew in advance that he would receive the certificate of participation, he said that he did not always pay due attention to the tasks in the project and was not very upset by the fact that the team took a low place in the overall ranking. He highly appreciated own participation in the project, successes and expressed full satisfaction with the project activities.

Summarizing the results of the focus group, we can say that the perception of the purpose of participation in the project “X-Culture” by students was different. The goals stated by the participants can be correlated with the theoretical design of D. Labaree about three competing goals in education: democratic equality, social efficiency, and social mobility. Depending on the perception of their goal of participation in the project, students showed different degrees of motivation and differently determined the degree of their satisfaction with the results and felt their own success.

5. Conclusions

The experience of participating in such an international educational project as X-culture was useful not only for students, but also for teachers.

Firstly, weaknesses in the educational process were identified, associated with the development of students’ sociocultural competences. According to the following five indicators—communication skills, diligence, creativity, leadership, and friendliness—by the end of the project, grades were worsened by more than 50% of students.

The discrepancy between the results obtained and the expected results makes us think about the reasons, among which we can distinguish the following:

• The current academic load undoubtedly affected the decline in the quality of students’ work in the project. By the end of the semester, Russian students probably could not properly allocate time between the implementation of course projects in other subjects and tasks in the project teams of the X-culture project.
• For Russian students it was their first experience of participation in such a project and, of course, a new format of participation, the need for active communication in a foreign language, weak software skills training, etc. expressed in modest results and self-assessments.

At the same time, the experience of participating in the international X-culture project was certainly useful. The students received an independent assessment of the level of their sociocultural competences, and teachers included a modern project approach in the arsenal of pedagogical techniques, based on an interactive team solution of real business problems with the implementation of all communications in English.

Secondly, the experience of participating in an international educational project had a positive impact on the level of development of students’ language competence. Constant communication in English with foreign participants of the project for a long time allowed students to feel more confident, significantly expand vocabulary, and gain invaluable experience in this field.

Thirdly, the diversity of the goals of student participation in the “X-Culture” project can serve as an indicator of the ambiguity of the higher education goals perception by Russian students. This may be due to a change in the educational paradigm in Russian society, a decline in the value and significance of obtaining a bachelor’s or master’s degree, the lack of a connection between higher education and successful employment. Awareness of the learning goal directly affects the level of self-motivation of the student to study, the achievement of the final result, and the self-perception of their own success or non-success. In any case, this direction may be the subject of further serious analysis.

The theoretical work of other scholars in this field has been a useful resource for planning and designing, and we expect that our study and the results of students’ work on an international project presented in it can be useful both for teachers who want to improve the effectiveness of training and for researchers working in related fields of knowledge—pedagogy, psychology, and sociology.

Of course, there are some limitations in our study. First of all, it does not take into account a novelty effect—students did not have before this experiment an experience of project-based learning class. At the same time, the sample size was relatively small, because it was the first time we implemented such educational model and the duration of the course was only one semester.

In our further research we are going to reveal the precise reason of negative students’ sociocultural results and conduct an analysis of students’ satisfaction by X-culture project.