Addressing the Artificial Intelligence (AI) Talent Gap: Outcomes of the First U.S. Nationally-Registered AI Apprenticeship Program

Abstract

The purpose of this study was to understand the motivations for and impact of participating in program coursework and related workplace-based learning and provide insights into the emerging field of AI workforce training. This study drew on adult learning theory and the acknowledgement that learning takes place within the context of a complex and uncertain career landscape as described by the chaos theory of careers. A cross-sectional survey research study was conducted to determine participant motivations for attending the AI program as well as the impact of participation in the AI program on individuals in regard to personal benefits and career trajectories. Participants in the AI program were motivated by the opportunity for job growth, whether that was in current or future roles. Personal benefits, such as upskilling, career advancement potential, and growth in confidence were all reported outcomes of participation in the program. Though most respondents in this study indicated holding current roles in computer science areas, a majority were seeking to grow their AI content knowledge and skills in order to assume more responsibilities in this growing area or to move into new roles which emphasize AI specifically.

1. Introduction

With the proliferation of deep learning technologies and growing capacities for data aggregation, artificial intelligence (AI) has come to the forefront across industry sectors as an important means of leveraging data to guide business practices in areas including decision-making, modeling, and streamlining work (NASEM, 2021). From manufacturing to financial services, education, medicine, and social media, AI is transforming the way workers do their jobs at an unprecedented rate of change and in novel ways, leaving workers and employers with skill gaps that affect business’s competitiveness in the global marketplace, employee productivity, and individual-level competitiveness in the labor market (Johnson et al., 2021; Lassébie, 2023; NASEM, 2021; OECD, 2023). These gaps suggest a need to upskill and reskill incumbent and underemployed workers in order to prepare a robust workforce for AI labor market needs (Manca, 2023) and to prepare workers for a job market marked by increasingly rapid evolution of demand for technologically skilled workers (Ciarli et al., 2021) within an uncertain world where career paths are increasingly unpredictable (Pryor & Bright, 2011).

Evidence regarding effective practices for training an AI-competent workforce is relatively sparse. Recent reviews of the literature regarding the AI workforce indicate that there is a paucity of robust research regarding workforce training and education in AI (e.g., Lane et al., 2023; Laupichler et al., 2022; Johnson et al., 2021) and a lack of data regarding returns on investment for reskilling and upskilling programs (Cukier, 2020). This gap is an artifact of rapid changes in skill needs (Johnson et al., 2021), lack of formal and standardized assessments for adult learning programs outside of formal degree programs (OECD, 2019a), lack of organizational capacity for training (Cordes & Weber, 2021; Li, 2022), and underutilization of existing programs because of misalignment between industry and external trainer objectives (Molnar et al., 2022). These difficulties are exacerbated by the disruptive nature of AI technology that makes navigating the practical implications of its widespread use an inherently messy and complex undertaking (Ciarli et al., 2021; Krotov, 2019). Workforce development programs that serve individuals representing a variety of sectors therefore face substantial challenges in aligning programming with the needs of participants and employers.

This article presents findings from a study that examined the perspectives of alumni of the Artificial Intelligence Academy (AIA), a U.S. federally funded AI training program, using the lenses of adult learning theory (Knowles, 1968; Knowles et al., 2005) and career chaos theory (Pryor & Bright, 2011). This exploratory study examined how the AIA attracted participants from a wide range of employment and sectoral backgrounds and prepared them for uncertain career futures. In its focus on participants from a variety of industry sectors, the study aimed to identify crosscutting factors for effective practices for upskilling AI workers that transcend industry boundaries and can be adapted to programs that serve participants from a variety of backgrounds, industries, and career histories. Researchers aimed to answer the following overarching research question: How did the design of the AIA contribute to participants’ preparation for their future career activities? This overarching question was addressed by answering the following three questions:

• What motivated individuals to participate in the AIA?
• What did participants perceive as the benefits of their participation in the AIA?
• What was the impact of program participation on the participants’ career trajectories after completing the AIA?

Researchers aimed to connect participants’ perspectives with key design features of the AIA program to understand how the program met participants’ needs and to identify gaps in program design.

1.1. Conceptual Framework

This study drew on adult learning theory (Knowles, 1968; Knowles et al., 2005) and the acknowledgement that learning takes place within the context of a complex and uncertain career landscape as described by Pryor and Bright’s (2011) chaos theory of careers. The AIA was designed using principles of adult learning theory (Knowles et al., 2005) and practices for effective online professional learning (Herrington et al., 2009).

Professional learning aims to support participants in acquiring new knowledge and skills along with the ability to apply the new knowledge and skills within their professional practice settings (Teräs & Kartoglu, 2017; Walton & Johnson, in press; Webster-Wright, 2009). This understanding of professional learning is grounded in the theory of andragogy (Knowles, 1968; Knowles et al., 2005) that posits that adults acquire new knowledge and skills most effectively when they have ownership of their learning experience (for example, when learners have input into the desired outcomes), when they learn by problem solving in workplace contexts, and when learning activities focus on problem solving (Knowles, 1968; Knowles et al., 2005; McGrath, 2009).

At the same time, it is critical to recognize that adult learners who aim to acquire knowledge and skills related to their career trajectories are learning within complex, unpredictable personal and professional contexts. Bright and Pryor (2011, 2012) addressed several shortcomings of workforce training in their chaos theory of careers. In particular, they noted that programs often failed to account for the multitude of variables and unexpected phenomena that influence participants’ career trajectories, and the unique interpretive structures participants apply to their experiences, resulting in non-linear and unpredictable career trajectories (Bright & Pryor, 2011). In practice, this theory suggests that individuals should approach career growth as an iterative process of exploration, learning, and self-reflection rather than as a linear cause and effect process (Bright et al., 2023). The chaos theory is a particularly apt lens for this study of the AIA since the first cohorts of participants embarked on their AIA journey in the midst of the COVID-19 pandemic. As Bright et al. (2023) noted, the pandemic placed a number of unpredictable pressures on the workplace that resulted in both systemic changes and changes in individuals’ career trajectories—changes that AIA participants were actively experiencing as they engaged with AIA content.

Important ideas from andragogical theory and career chaos theory were synthesized and operationalized by Thompsonowak (2020), who identified effective strategies for workforce training programs that prepare participants for careers within this uncertain environment. The strategies are as follows: carefully and intentionally matching content with both employer and participant needs, incorporating contextually relevant soft skill acquisition across learning activities, and offering meaningful credentials that can enhance participants’ career development.

The importance of incorporating theoretically based principles of adult learning along with the contextual considerations related to career chaos theory are highlighted by evidence indicating that workforce training programs are often underutilized because of lack of content relevancy to participants’ workplaces and misalignment of knowledge and skills between program content and workplace processes (Molnar et al., 2022). On the other hand, evidence indicates that training programs are effective where learning is workplace-based or reflects real life situations (McCarthy, 2016; Mohammadi et al., 2020; Strygacz & Sthub, 2018), is flexible, and grants learners autonomy in learning (Fialho et al., 2019), and where there are opportunities for collaboration and establishing communities of practice (Bartle, 2015; Ebner & Gegenfurtner, 2019; Leon, 2023; Moffitt & Bligh, 2021; Mohammadi et al., 2020; Strygacz & Sthub, 2018).

1.2. Artificial Intelligence Academy

The AI Academy (AIA) was launched in 2020 with the support of a grant from the U.S. Department of Labor as an apprenticeship program of the Employment and Training Administration. The aim of the AIA was to upskill and reskill professionals representing diverse industry sectors and veterans of the U.S. military to meet AI workforce needs. The program, housed at a public university, offers industry credentials to adults seeking to develop and/or enhance their skills in the field of AI. The AIA partners with over 40 employers spanning a broad range of sectors including technology, healthcare, insurance, manufacturing, energy, financial services, entertainment, retail services, transportation, and communications. Partners provide mentored work-based learning experiences for program participants. To date, over 2000 individuals have completed the AIA.

Program design for the AIA drew on effective practices for adult professional learning and was guided by the principles of andragogical theory (Knowles, 1968; Knowles et al., 2005) along with the evidence base for adult professional learning in online settings (Herrington, 2006; Herrington et al., 2009). This resulted in the following evidence-based program design strategies: focusing on active, experiential learning within authentic workplace contexts; employing a learner-centered design that allows learners to apply content within their unique practice settings; leveraging content drawn from field-specific best practices and delivered by experts; and facilitating collaborative learning. In addition, course designers drew on evidence that virtual formats are particularly effective for accommodating professionals’ needs for flexibility (Ajgaonkar et al., 2022; Burris et al., 2018; Cukier, 2020; Moffitt & Bligh, 2021; Gegenfurtner et al., 2020; Mohammadi et al., 2020) and that flipped learning models maximize opportunities for application-based examples, collaboration, and problem-solving during real-time class meetings (Fan et al., 2020; Jacot et al., 2014; Mehta, 2020; Moffitt & Bligh, 2021; Strygacz & Sthub, 2018).

Each of the four AIA courses was designed by university faculty in collaboration with industry-based AI experts. AIA coursework is composed of a series of four fully online non-credit courses, each of which spans ten weeks. Courses are taught by university faculty and staff. Designed using a flipped classroom approach, participants engage asynchronously with interactive pre-recorded seminars (two hours a week) in addition to participating in two live workshop sessions each week (four hours each week) for each of the sequence of four courses—Computer Programming with Python, Data Mining, Introduction to Artificial Intelligence, and Machine Learning (Figure 1). The workplace learning aspect of the AIA program included an industry-based mentor for each participant within their employing organization (or with a partnering organization) with whom participants worked for a minimum of five hours each week to apply course content to organizational problems, provide real-world examples of content application, and increase the effectiveness of both coursework and the on-the-job experiential learning experiences (Bradley et al., 2018; Estrada et al., 2018; Willis, 2021).

Participants were selected through a multi-step process. First, each applicant completed a pre-assessment of their programming skills and knowledge of databases. Those who were proficient were interviewed and 98% were provided admission to the program. Individuals who did not receive a proficient score on the pre-assessment (80%) were offered the opportunity to complete a pre-requisite course for free to build their skills (10 modules with embedded assessments). Upon successful completion, individuals moved forward to the interview and selection stage and 100% were admitted to the program. Participants had the opportunity to earn two industry-recognized credentials—Data Scientist and Artificial Intelligence Associate. AIA coursework was developed collaboratively by computer science faculty, instructional designers, and partnering industry experts working together to determine relevant competencies and associated scope and sequence, as well as real-world industry problems (McCarthy, 2016; Mohammadi et al., 2020; Strygacz & Sthub, 2018) that could be used as context for the program.

Recognizing the difficulties posed by imprecise and varying definitions of AI across industry sectors, AIA program designers operationalized “AI” using the OECD (2019b) definition:

“An AI system is a machine-based system that is capable of influencing the environment by making recommendations, predictions or decisions for a given set of objectives… by utilizing machine and/or human-based inputs/data to (i) perceive real and/or virtual environments; (ii) abstract such perceptions into models manually or automatically; and (iii) use Model Interpretations to formulate options for outcomes.”

(p. 7)

Drawing on this definition, the AIA program acknowledged that a wide variety of skill sets are required to effectively manage AI implementation, including specific technical skills as well as distinctly human skills, such as creativity, collaboration, judgment, and communication, that foster effective and ethical use of AI within a variety of contexts (Dede et al., 2021; Lane et al., 2023).

2. Materials and Methods

A cross-sectional survey research study was employed for this work. As part of AIA activities, participants who had completed the AIA program were asked to complete a post-participation survey consisting of 19 items, six of which were open-ended questions. The open-ended items aimed to determine participant motivations for attending the AIA as well as the participants’ perceptions of personal benefits of program participation and its impact on their career trajectories.

2.1. Participants

A total of 209 alumni of the first two cohorts of the AIA (those who had completed the program by June 2023) were asked to participate; 50 respondents completed the AIA Alumni Survey (24%). Respondents by industry included IT (18%), financial (10%), transportation (10%), manufacturing (3%), retail (3%), healthcare (2%), entertainment (2%), and research or education (2%). This study focused on the following open-ended items in order to gain a deep and nuanced understanding of participants’ experiences with the program and how those experiences related to program design: Why did you decide to participate in the AI Academy? What were the biggest benefits to you personally of participating in the AIA? How have your future career aspirations changed, if at all, as a result of participating in the program?

2.2. Response Rate

A total of 209 individuals (first two program cohorts only) were invited to complete the AIA Alumni Survey; 50 responses were received (24%). This is aligned with what previous research has demonstrated is a typical response rate for an online or electronically-delivered survey. For example, Sheehan (2001) found in their review that the response rate of electronic delivery surveys produced a mean response rate of 24%. Similarly, a study by Nulty (2008) examining survey response rates revealed a typical overall online survey delivery response rate of 33%.

Table 1. Participant Characteristics (n = 50).

Participant Characteristic	Number of Respondents	Percentage of Respondents
Job Role (n = 50)
Data Science/AI	26	52%
Other Sectors	24	48%
Demographic Data (n = 50)
Race/Ethnicity
White	26	52%
Asian	8	16%
Black or African American	6	12%
Hispanic/Latino	1	2%
American Indian/Alaska Native	1	2%
More than one race/ethnicity	3	6%
Choose not to respond	5	10%
Gender
Male	27	54%
Female	17	34%
Non-binary	1	2%
Choose not to respond	5	10%

provides an overview of respondents’ job roles. More than half (54%; n = 26) of respondents described their current positions as focused on data science and/or AI. Over a third (38%; n = 18) were in positions not focused on computer science, including five individuals who were not currently employed.

Participants were asked to self-report demographic information (Table 1). More than half of respondents were White 54% (n = 26), 13% (n = 6) were Black or African American, and 17% (n = 8) Asian or Asian American. One respondent (2%) identified as Hispanic or Latino and one (2%) as American Indian or Alaska Native. Three respondents 6% (n = 3) identified with more than one category of race. Over half of respondents were male (56%; n = 27) male, and 35% (n = 17) were female.

2.3. Data Analysis

Conceptual content analysis (Christie, 2007) was utilized for all open-ended item responses to allow for the identification of themes in participants’ responses. Multiple cycles of manual coding were conducted, following Saldana’s (2013) process of moving from the particular (raw data) to the general (themes). The analysis process began with initial coding by thoroughly reading participant responses and taking notes on commonalities. Next, line-by-line coding was completed with every segment or line being given a code. Three researchers on this project individually conducted initial coding. Next, the entire team met to discuss codes, and these were categorized and assigned to overarching categories identified within the data. Finally, categories were collapsed to reduce overlap or redundancy, and broader themes were then named. Appropriate descriptive statistics (number of cases and percentages) were computed and reported for each theme to demonstrate its importance or weight (Pyrczak, 2008).

3. Results

3.1. Motivations for Participation in the AI Academy

Research Question #1 examined respondents’ motivations for participating in the AIA as reflected by their answers to the question, Why did you decide to participate in the AI Academy? Findings revealed three emergent themes—upskilling for professional purposes, personal growth, and program features. Many of the responding participants (n = 50) cited more than one motivation for participating in the AIA.

Table 2. Participants’ motivations for participating in the AIA (n = 50).

Motivation for Participating in the AIA	Number of Responses	Percent of Responses	Sample of Participant Responses (Participant Identifier)
Upskilling for Professional Purposes	41	82%	“[I participated to] not remain behind the curve when it comes to in-demand AI skills” (#13)
			“facilitate the integration of AI in work-related cloud services…[and] future-proof my career” (#8)
			“I…wanted to explore the potential of data science & machine learning to solve long outstanding problems to give products to provide more value & a competitive edge.” (#11)
			“use it in my world as a software developer” (#39)
			“use this knowledge to help with my work.” (#28)
			“expand my career path” (#19)
			“open up better job opportunities” (#27)
			“[I am] looking to make a career change and re-enter the full-time workforce. I needed a new challenge and felt that AI skills would be the most beneficial in supporting my next career move” (#29)
			“the AI Academy would be the perfect program to help me in my job search process” (#14)
Personal Growth	39	57%	“to broaden my knowledge and provide a foundation for learning the next paradigm shift in technology” (#4)
			“learn about current trends in AI” (#17)
			“update my knowledge” (#23)
			“to get a full understanding of what is happening in the AI world” (#39)
			“had a strong interest in AI and machine learning” (#21)
			“have an avid interest in machine learning and artificial intelligence” (#37)
			“the recent trend of data-driven decision-making” (#49)
Program Features	9	18%	“[learning] in one intensive study” (#22)
			“seemed like one of the coolest, most accessible, most comprehensive to some of the more technical components of AI. And it was!” (#24)
			“real-world experience with an employer” (#30)
			“would lead to an apprenticeship with a partnering company” (#13)
			“[the] scholarship offered, and one-year timeframe made the AI Academy an attractive option” (#38)

summarizes participants’ motivations for participating in the AIA.

3.1.1. Upskilling for Professional Purposes

Upskilling for professional purposes was a primary motivator for participation in the AIA, with 82% of respondents (n = 41) indicating this as a motivation. Upskilling was defined for this study as professional growth for current employees to be used within their current job or to use in a new job role.

Over a third (35%; n = 20) of respondents indicated that upskilling in general, without specific reference to job roles, was a motivation for their participation in the AIA. For example, one applicant shared that prior to starting the program that they believed “the AI Academy would be an excellent upskilling opportunity” (#1), and another shared that the “AI Academy helped me to upskill and reskill myself” (#9).

Other respondents (19%; n = 11) saw the AIA program as a way to equip them with specific skills and knowledge that they could directly apply to problem-solving in their current roles or companies. For example, one participant (#8) discussed that they joined the program to “facilitate the integration of AI in work-related cloud services”. Additionally, participants identified specific areas of need within their companies and job roles and wanted to join the AIA program to further explore how AI or machine learning could provide solutions to outstanding problems. One participant (#11) stated, “I work in product development and wanted to explore the potential of data science & machine learning to solve long outstanding problems to give products to provide more value & a competitive edge”.

Within this group, some participants noted that they were looking for a new job or to expand their job options. One participant said, for example “[I am] looking to make a career change and re-enter the full-time workforce. I needed a new challenge and felt that AI skills would be the most beneficial in supporting my next career move” (#29). Others saw the AIA program credentials as a way to make them more marketable as they searched for new positions. One participant shared that “the AI Academy would be the perfect program to help me in my job search process” (#14).

3.1.2. Personal Growth

The second theme identified as a motivator for joining the AIA program was personal growth. Personal growth was defined as the motivation to develop knowledge and skills based on personal interest, or the desire to pursue participants’ own wants or desires not explicitly tied to career phenomena. Over two thirds (57%; n = 39) of survey respondents identified personal growth as a reason for participating in the AIA program.

More than half of respondents (51%; n = 29) cited as a motivator the desire to generally gain more knowledge and learn new information. As one participant indicated, for example, “[I want] to broaden my knowledge and provide a foundation for learning the next paradigm shift in technology” (#4). Another 19% of respondents (n = 11) saw the AIA program as an opportunity to further explore their interests related to AI and machine learning, citing a general interest in program topics as a motivator for joining the AIA program. Participants said, for example, that they “had a strong interest in AI and machine learning” (#21), and “have an avid interest in machine learning and artificial intelligence” (#37).

3.1.3. Program Features

A small number of participants (18%; n = 9) cited reasons for joining the program based on the format and logistics of the program. Seven respondents (12%) identified specific aspects of the AIA that drew them to the program, including the combination of AI and machine learning, gaining real-world experience, and the condensed nature of the training. For example, one participant stated that the course content drew them to the program because it allowed them to learn both AI and machine learning “in one intensive study” (#22). Another participant noted that the program “seemed like one of the coolest, most accessible, most comprehensive to some of the more technical components of AI. And it was!” (#24). Other participants discussed practical motivations like cost, time, and making connections with employers. One participant, for example, said they joined the program because it “would lead to an apprenticeship with a partnering company” (#13), and another mentioned that it would allow them to gain “real-world experience with an employer” (#30). One participant cited the cost and length of the program as a motivator, noting that the “scholarship offered, and one-year timeframe made the AI Academy an attractive option” (#38).

Only 5% of respondents (n = 3) indicated that employer encouragement was a primary motivator for participating in the program. These participants shared, for example that “my employer offered me the opportunity” (#51) and “someone in my company offered to pay my way through” (#55). The last respondent in this category stated they were “asked, and it seemed like an opportunity to learn” (#47).

3.2. Benefits of Participating in the AI Academy Program

Twelve months after completing the AIA program, alumni were asked to share their perceptions regarding how participation benefited them personally. Findings revealed three categories of benefits—learning about program topics, career advancement, and benefits associated with program design features. Many participants cited more than one benefit of participation. These findings are summarized in

Table 3. Participant perceptions of benefits of AIA (n = 47).

Benefits of Participating in the AIA	Number of Responses	Percentage	Sample of Participant Responses (Participant Identifier)
Learning	41	87%	“allowed me to learn Python, got to learn about AI and data science.” (#48)
			“[I learned about search algorithms…data mining…Bayesian knowledge tracing” (#39)
			“understanding what AI and machine learning mean.” (#2)
			“the knowledge helped me to keep pace with the latest AI tools.” (#29)
			“the capabilities and limitations of AI,” (#7)
			“[p]articipating in the AI Academy has definitely increased my confidence [in program topics]” (#9)
Career Advancement	17	36%	“The learning in Python and Data Science has boost[ed] my learning and career advancement” (#19)
			“It opened up new opportunities at work. I was working as a mechanical engineer before the attending the academy. I now have moved over into the data science area of the business” (#37)
			“It allowed me to break into the data science workforce and allowed me a better opportunity for my career” (#36)
			“It has helped me gained so much knowledge around how to analyze Data and [to] work with it. AI academy helped me in receiving [a] full time career.” (#9)
			“I have used what I have learn[ed] to generate my own deep learning models for personal use. Someday I may produce an application to help the world.” (#40)
Benefits Associated with Program Design Features	14	30%	“breadth of techniques covered with enough detail to select techniques to deep dive for specific problems” (#11)
			“material was just the right size for every week” (#22)
			“appreciated the light introduction to concepts and practice we got” (#51)
			“structured learning environment” (#46)
			“lectures from great instructors” (#50)
			“hands on experience coding” (#6)
			“hands on knowledge of core concepts of AI, ML, Data Mining & HCI” (#26)
			“experience in [AI]/ML in both practice and theory” (#10)
			“interacting with my fellow classmates…many of had a considerably greater depth of knowledge of the foundational basis of the course material than I” (#13)
			“being able to work out assignments as a group” (#32)
			“networking” (#s30, 32, 50)
			“ability to have technical conversations with data scientists and machine learning practitioners” (#47)

.

3.2.1. Learning

Most participants (87%; n = 41) indicated that a benefit of AIA participation was gains in knowledge. Some of these individuals (3%; n = 6) also specifically noted increased confidence resulting from this learning.

A large majority of respondents (80%; n = 38) of the surveyed participants identified as a primary benefit of the program specific areas of growth in their knowledge. Some of these responses reflect gains in disciplinary content knowledge. One participant (#48) shared that the AIA “allowed me to learn Python, got to learn about AI and data science”. Another (#39) shared specific examples of areas of learning, including “search algorithms”, “data mining”, and “Bayesian knowledge tracing”. Others alluded to the depth of their newfound understandings, such as respondent #2, whose primary benefit was “truly understanding what AI and machine learning mean”. Some respondents responded that the AIA prepared them with knowledge appropriate for a rapidly changing world, such as respondent #29, who said that “the knowledge helped me to keep pace with the latest AI tools”.

A small number of participants (6%; n = 3) reflected on how their increased knowledge resulted in gains in confidence to work with AI as a result of participation in the program. For example, one respondent (#9) said that “[p]articipating in the AI Academy has definitely increased my confidence [in program topics]”.

3.2.2. Career Advancement

Opportunities for career advancement were another benefit identified by alumni of the AIA. Thirty-six percent of respondents (n = 17) indicated that they perceived participation in the AIA to be helpful for advancing their careers.

Some participants (19%; n = 9) indicated that they found the AIA program to be beneficial for moving into new jobs or job roles. Some alumni identified specific program content that they believed would be helpful for reaching career goals. For example, one participant stated, “The learning in Python and Data Science has boost[ed] my learning and career advancement” (#19). Some alumni were able to cite specific advancements in their careers due to participating in the AIA. One participant stated, “It opened up new opportunities at work. I was working as a mechanical engineer before the attending the academy. I now have moved over into the data science area of the business” (#37). Another participant considered the possibilities to use their learning in an entrepreneurial manner, for future innovation, saying, “I have used what I have learn[ed] to generate my own deep learning models for personal use. Someday I may produce an application to help the world” (#40).

Other participants (19%; n = 9) saw the ability to apply their learning within their current roles on the job as a benefit of the AIA. These alumni discussed being better prepared to use AI and ML in their current line of work. As one participant said, “I believe my ability to design systems with AI and ML is much better now that I have a more comprehensive understanding” (#38).

3.2.3. Benefits Associated with Program Design Features

Over a quarter of alumni (30%, n = 14) discussed that they believed the greatest benefits of their participation were associated with the AIA program format. These participants noted the course content and delivery structure as benefits, including the opportunities for active learning and forming community of practice and networking.

Some alumni (15%; n = 7) noted that the amount of content, the pacing, and/or instructors were benefits of the program. Respondents said, for example, that they appreciated the “breadth of techniques covered with enough detail to select techniques to deep dive for specific problems” (#11), that they “appreciated the light introduction to concepts and practice we got” (#51), and that the “material was just the right size for every week” (#22). Others commented on the “structured learning environment” (#46) and “lectures from great instructors” (#50) as benefits of participating in the AIA program.

Several participants (11%; n = 5) noted that the active learning format, including hands-on opportunities to apply concepts, was a key benefit of participation. For example, alumni described benefits such as “hands on experience coding” (#6) and “hands on knowledge of core concepts of AI, ML, Data Mining & HCI” (#26. One participant noted the balance of theory and practice in the program, saying they gained “more experience in [AI]/ML in both practice and theory” (#10).

Elements of the program that fostered community building and networking were identified by 17% of survey respondents (n = 8) as an important benefit of participating in the AIA program. Some alumni discussed how the social interactions and direct collaboration with classmates were beneficial to them saying, for example, “interacting with my fellow classmates” was beneficial since “many of [them] had a considerably greater depth of knowledge of the foundational basis of the course material than myself” (#13). Other participants noted as benefits collaboration and working together in groups. Alumni also noted that networking and professional connections were benefits, citing “industry introductions” (#30) and “networking” (#s30, 32, 50) as benefits along with the “ability to have technical conversations with data scientists and machine learning practitioners” (#47).

3.3. Impact on Career Aspirations

The third research question focused on how the career aspirations of respondents (n = 46) were impacted by their participation in the AIA. Most participants (52%, n = 24) reported that the AIA had impacted career aspirations in terms of interest in a new career or new interest in applying learning within their current roles. Some participants responded that both the opportunity to apply their learning within their current job roles and support for future career transitions were ways that the AIA had impacted their career aspirations, and others noted that their aspirations had not changed as a result of participating in the AIA.

Table 4. Impact of AIA participation on career aspirations (n = 46).

Impact on Career Aspirations	Number of Responses	Percentage	Sample of Participant Responses (Participant Identifier)
Interest in a new career	24	52%	“become a data analyst that uses these skill sets to build predictive models or work with larger sets of data & have to clean & process it” (#26)
			“[I aspire to create] ML models to be used in the context of refinery production models” (#46)
			“I found myself interested in the coding aspect of AI but really bored by the data science aspect, so it has allowed me to narrow my career focus to areas I thrive” (#48)
			“now pursuing a master’s degree in computational science with a minor in computational and data science” (#36)
			“learn more about AI while exploring career options” (#6)
			“add to my current skillset and expand it to include machine learning. I have been actively seeking opportunities for a career change” (#23)
			“[I aspire to a] more data-centric role” (#38)
			“from a mechanical engineer to a data scientist” (#37)
Undertaking new activities in current role	11	24%	“creating interactive dashboards for my stakeholders in my company so they can make data driven and informed decision for business growth and profitable sales” (#9)
			“add more AI intelligence to new and current applications so they are able to predict and react to changes outside of nominal conditions” (#18)
			“already an established software developer. [AI Academy will] help my company’s future by building the tools to achieve our challenging goals” (#40)
			“incorporate more of AI/ML in my daily work to improve processes and image quality” (#10)
			“conduct more AI-based analyses” (#24)
			“look out for AI tools that would enhance my current responsibilities” (#29)
No change in career aspirations	11	24%	“[my career aspirations] have not changed. I still aspire to become a data scientist” (#32).
			“I had this career path in mind before the program, but I am very happy to have completed the program” (#20)

provides an overview of participants’ responses.

Eleven alumni (24%) reported that they changed career aspirations based on their experiences in the AIA program. Some participants gave broad indications of how their career aspirations were refined through the coursework in the program. Participants who were specific about the changes in their aspirations said, for example, that after participating in the AIA they aimed to “become a data analyst that uses these skill sets to build predictive models or work with larger sets of data and have to clean and process it” (#26). One participant reflected on how the coursework and experience throughout the program helped them identify a specific area of interest, saying “I found myself interested in the coding aspect of AI but really bored by the data science aspect, so it has allowed me to narrow my career focus to areas I thrive in” (#48). Three alumni (6%) reported that participating in the AIA led them to pursue advanced degrees in concepts related to AIA content. One respondent (#36), for example, is “now pursuing a master’s degree in computational science with a minor in computational and data science” after participating in the AIA.

Eight of the participants (17%) who reported changes in their career aspirations described goals to pursue different career paths. For example, one respondent stated they plan to “learn more about AI while exploring career options” (#6) while another discussed how the program allowed them to “add to my current skillset and expand it to include machine learning. I have been actively seeking opportunities for a career change” (#23). Alumni who had already begun to alter their career trajectories upon completion of the program reported transitions, for example, “from a mechanical engineer to a data scientist” (#37).

When asked about the AIA’s impact on their career aspirations, nearly a quarter (24%; n = 11) of respondents discussed ways they planned to implement AI or ML in their current roles for the benefit of their company and for streamlining work responsibilities. One participant discussed, for example, that they were “creating interactive dashboards for my stakeholders in my company so they can make data driven and informed decision for business growth and profitable sales” (#9). Other responses were less specific. One participant planned, for example, to “incorporate more of AI/ML in my daily work to improve processes and image quality” (#10), and others to “conduct more AI-based analyses in the education field” (#24), or “look out for AI tools that would enhance my current responsibilities” (#29).

Nearly a quarter of alumni (24%; n = 11) reported that their career aspirations had not changed, but that the AIA supported them in their existing career trajectories. For example, participants said “[my career aspirations] have not changed. I still aspire to become a data scientist” (#32) and “I had this career path in mind before the program, but I am very happy to have completed the program” (#20).

4. Discussion

Using Thompsonowak’s (2020) framework for effective workforce training programs as a lens through which to examine findings for participants who are pursuing technical careers in the context of a chaotic and complex career environment lends insight into the overarching research questions for this study, How did the design of the AIA contribute to participants’ preparation for their future career activities? The discussion will examine how the AIA program design contributed to participants’ career preparation by examining what the findings suggest for each of Thompsonowak’s (2020) three effective practices for workforce development programs: carefully and intentionally matching content with employer and participant needs, incorporating contextually relevant soft skill acquisition across learning activities, and offering meaningful credentials that can enhance participants’ career development.

4.1. Matching Content with Employer and Participant Needs

Participants’ responses regarding their motivations for participating in the AIA indicated that they desired instruction that would provide them with new AI and data science skills for application in their jobs, that they were life-long learners who sought to increase their skills and knowledge generally and for career-specific purposes, and that they were motivated by features of the program that would allow them to complete it within a one year timeframe while affording them ample opportunities to practice application of their new knowledge and skills.

Findings suggest that these participant needs were met, with program participants citing as beneficial and impactful program-related phenomena that aligned with each of these initial motivations for participation. Most respondents (85%) reported that the learning they experienced was a primary benefit of their AIA participation, and that the experience allowed them to stay current in their technical knowledge and that they felt more confident in this knowledge. In terms of career-specific learning, over a third of respondents (36%) cited as primary benefits specific skills, such as Python coding, data analysis, and the ability to create deep learning models that could enhance their career growth. This finding is significant since the AIA served participants from a wide cross-sector of employment sectors, meaning that program content could not be tailored specifically to a specific industry sector or job role. This finding suggests that the use of mentoring and on-the-job learning as well as instructors’ use of examples from various settings were effective in meeting the needs of participants from contexts as diverse as utility companies, educational testing firms, and entertainment industry companies.

From a course content and delivery perspective, findings also suggest that participants’ needs were met. In particular, respondents appreciated the hands-on nature of learning, where theory delivered in pre-recorded lectures was translated into practice in the course workshops and in participants’ own work settings. The ability to complete the course within a year was also cited as a benefit, suggesting that the pacing of the course and the amount of content delivered met participants’ needs for balance with their professional duties.

Although this study did not gather data from employers, findings that participants were able to robustly apply their learning within their current job settings strongly suggest that the program content aligned with employer needs. Employer input into program design contributed to an environment in which examples from various industries could be incorporated into instruction, providing participants with a broader view of AI and its applications than they might have gained by an industry-specific training program. In addition, by engaging mentors from employer settings, the program was able to tailor content to participants’ work settings and ensure that course content was translated to meet employer needs.

4.2. Incorporating Contextually Relevant Soft Skill Acquisition Across Learning Activities

Research supports the idea that individuals whose work interfaces with AI need to be proficient not only in technical skills but also in workplace skills, or soft skills, including skills such as problem-solving, creativity, communication, and collaboration (Dede et al., 2021; Lane et al., 2023). The AIA program was designed to incorporate opportunities for communication and teamwork via problem-solving breakout sessions in weekly workshops, and with a focus on applying content to solve problems in the complex and unpredictable environment of the workplace via the on-the-job learning and mentoring components of the program.

Findings from this study suggest that participants were able to engage in soft skill practice while learning important content. When asked about program benefits, many respondents cited the hands-on experiences, indicating that they were able to creatively use their new knowledge for problem-solving within real-world contexts (e.g., “creating interactive dashboards for my stakeholders in my company” and “[I was able to create] ML models to be used in the context of refinery production model”). The incorporation of teamwork in the live synchronous workshop sessions was also perceived as a benefit, as was networking and practicing technical communication skills with peers and other professionals (e.g., “being able to work out assignments as a group” and “ability to have technical conversations with data scientists and machine learning practitioners”).

4.3. Offering Meaningful Credentials to Enhance Participants’ Career Development

The AIA program was designed to offer two distinct credentials for successful completion of coursework, mentoring, and demonstration of course competencies. Participants did not explicitly cite the certificates as primary benefits of the course or when asked about the impact of the program on their career aspirations. Participants’ comments regarding positive impacts on their perceived ability to pursue new jobs or job roles suggests, however, that the program provided adequate support for their career transitions in terms of credentials. The fact that some respondents reported being enrolled in or planning to enroll in graduate studies related to course content suggests that the learning and credentials they gained in the AIA were a useful way to transition into the AI field, but that they perceived that the AIA credential granted did not hold the same weight as a graduate degree.

Participants overwhelmingly affirmed that program completion enhanced their career development. Even those who reported no change in their career aspirations found program participation useful (e.g., “I had this career path in mind before the program, but I am very happy to have completed the program”).

5. Conclusions

The burgeoning growth of the AI field points to the need for evidence-based programs to upskill and reskill workers. This study of the AIA provided some key information that can be used in future program development and that begins to fill the gap in the knowledge of effective practices for AI workforce development programs. The study’s findings have particular implications for program design and for recruitment of participants as well as for future research.

In terms of program design, findings from the AIA study provide additional evidence for the ideas advanced by Thompsonowak (2020) that adult learning programs should be planned with a focus on real-world application, attention to participants’ workplace contexts and soft skills, and credentials. Findings indicating that the ability to translate program learning into real-world problem solving within authentic workplace contexts is a crucial factor for the design or programs to upskill and reskill workers for AI and also points to the value of the mentoring and experiential learning aspects of the program. These findings align with research findings indicating that workforce learning is most effective when grounded in workplace contexts (Fialho et al., 2019; Laupichler et al., 2022), especially for programs that serve participants from multiple sectors where the future of work is impacted by a variety of unpredictable factors (Bright & Pryor, 2011). The multi-sector participant population may actually have provided participants with a more holistic view of AI applications and career opportunities than they would have gained from a program targeted at a specific industry or job role.

The scaffolded approach to problem-solving utilized by the AIA was designed to serve multi-sector participants by implementing progressively more targeted application opportunities. Participants were first presented with problems that mirrored real-life situations in the classroom setting and were challenged to solve these in collaboration with fellow learners. Next, learners benefited from relationships with mentors who shared problems based within their specific workplace or industry sector and engaged in mentored application of AI to create solutions to these problems. These experiences then translated into program participants working with colleagues in the workplace to problem-solve using the knowledge and application skills they gained.

The inverted model of instruction can be especially useful for programs that prioritize utility and application skills since live classroom time can be used for application rather than content delivery. In addition, an inverted and online program delivery model provides adult learners with the flexibility to engage with content knowledge on their own schedules and at their own pace, characteristics that adult learners in this study valued. This is particularly important since employees have reported that the time required for workforce development programs can act as a barrier to professional learning and that effective programs are workplace-based, flexible, and grant participants autonomy in learning (Cordes & Weber, 2021; Fialho et al., 2019), phenomena that are addressed through the inverted, application-based model employed by the AIA.

Research shows that workforce development programs must align with workplace needs in order to be successful and indicates that programs are often underutilized because of lack of content relevancy to participants’ workplaces and misalignment of knowledge and skills between program content and workplace processes (Molnar et al., 2022). The AIA program proactively engaged partners from a wide variety of industry sectors and drew on these partners’ expertise in industry needs, using this expertise to inform curriculum design and employing mentors from a wide variety of workplace settings. This proactive engagement provided a programmatic fit with industry and workplace needs that attracted participants who anticipated the ability to not only apply their learning in their existing settings, but to apply their learning in new roles in their own and new industry sectors. These findings, along with previous research, suggest that programs designed with industry needs in mind are more likely to attract and retain employers and workers looking to upskill and reskill, a phenomenon that is important to individuals, as evidenced by one participant who cited participating in order to “future-proof my career”.

Although this study was limited by its small sample size and short-term perspective on alumni impacts, it suggests several areas for future research. As AI workforce development programs age, longitudinal studies of participants could lend insight into what future upskilling or fine-tuning of knowledge participants might need in order to stay current in the rapidly changing world of AI. Studies of how workforce training programs incorporate principles from the chaos theory of careers, including cycles of exploration, learning, and reflection, could provide insight into how individuals adapt their career trajectories to turbulent situations such as the COVID-19 pandemic and reveal important resilience skills. More research into how participants perceive program-specific credentials, how these credentials could be delivered for maximum usefulness (e.g., digital format appropriate for sharing on social media), and how employers perceive such credentials could provide insight into how credentials are created and earned. In addition, studies regarding how AI workforce training programs could act as train-the-trainer programs, enabling participants to provide important professional development to other employees in their workplaces, could provide additional value to employers who invest in employee training in such programs.

Since a growing research base points to the need for soft skills to successfully work at the human–computer interface (Crumpler & Lewis, 2019; Dede et al., 2021; Lassébie, 2023; OECD, 2023), future studies that investigate how AI workforce development programs foster these skills and what challenges participants face in enacting these skills within the workplace could provide insight into program design and provide a set of innovative effective practices.

Finally, studies that examine employers’ approaches to and investments in workforce development and their potential contributions to program design could be useful in ensuring alignment between industry needs and program content and format. These and other such studies have the potential to inform program design and continuous improvement efforts for workforce development programs.