COMET’s Education and Training for the Worldwide Meteorological Satellite User Community: Meeting Evolving Needs with Innovative Instruction

Abstract

Since 1989, the COMET^® Program’s staff of instructional designers, scientists, graphic artists, and web developers has been creating targeted, effective, and scientifically sound instructional materials for the geosciences in multiple languages and formats. The majority of COMET training materials and services are available via COMET’s online training portal, MetEd. MetEd hosts over 500 self-paced English-language lessons, which are freely available to registered users. The lessons cover a broad array of topics, including satellite meteorology, numerical weather prediction, hydrometeorology, oceanography, aviation weather, climate science, and decision support. Nearly 300 lessons have been translated to other languages. NOAA NESDIS, EUMETSAT, the Meteorological Service of Canada, and the US National Weather Service all provide funding and subject matter expertise for satellite training efforts at COMET. The COMET team is focused on helping our sponsors refine their learning objectives and produce instructional material that is focused on learner engagement, knowledge retention, and measurable performance improvement. The COMET Program has continually transformed its instructional approach to better meet the shifting needs of learners. Our satellite remote sensing educational and training materials provide sound foundational knowledge for existing and new satellite products paired with increasing opportunities to apply that knowledge.

1. Introduction

The COMET^® Program, part of the University Corporation for Atmospheric Research (UCAR) Community Programs, has been a worldwide leader in geoscience distance and in-residence education for 30 years [1]. In this paper, we will provide an overview of COMET’s satellite training activities as well as the meteorological workforce changes worldwide that are influencing how we approach the development and delivery of our training resources. These changes include not only new observational technologies and an expanded range of satellite products and data, but also shifts in the types of services that forecasters are tasked with providing. This paper documents examples of how COMET has been proactive in streamlining its deliverables to better meet the needs of its learner communities. We include a summary of the instructional design principles and practices that are at the core of the training we produce, and we conclude by examining metrics related to COMET’s overall reach and look at the impact of the training as well as its applicability to operational workforce needs in the US and across the globe.

The majority of COMET’s training materials and services are offered as self-paced lessons, and while we do offer in-person training and blended learning solutions, this paper will remain focused on the online environmental satellite lessons and courses available via our training portal, MetEd (https://www.meted.ucar.edu/). On MetEd, COMET hosts over 500 English-language lessons covering topics in satellite meteorology, numerical weather prediction, hydrometeorology, oceanography, aviation weather, climate science, decision support, and other topics, all of which are free of charge for anyone who registers. Nearly 300 lessons have been translated to other languages, including Spanish and French, with some additional materials available in Portuguese, German, Chinese, Indonesian, and Russian, for a total of eight languages, including English.

Approximately 600,000 registered users, including nearly 200,000 international users from over 190 countries (Figure 1), take advantage of this free, on-demand training. MetEd’s registered users include over 142,000 students and 21,000 faculty from over 2000 different universities and colleges around the globe. Anyone can join this large community of learners by signing up at the MetEd registration page (https://www.meted.ucar.edu/registration.php).

1.1. History and Support

From its inception in 1989, COMET has sought to advance geoscience workforce expertise worldwide by offering on-demand training solutions [2]; in the program’s 30 years, this fundamental focus has not changed [1,3]. COMET’s core staff of 24 individuals includes a team of instructional designers, scientists/meteorologists, a graphic artist, multimedia developer, and information technology and administrative professionals working flexibly and efficiently to allow effective use of program funds. COMET’s efforts to develop satellite and remote sensing distance learning began in the mid-1990s, with the introduction of digital satellite imagery and the launches of new generation geostationary (GOES I-M/8-12) and polar-orbiting (NOAA-KLM/NOAA-15, 16, 17) satellite series. As new products and visualization capabilities emerged, COMET’s training deliverables expanded to address a range of fundamental remote sensing topics, helping forecasters and others acquire a working understanding of the underlying science. These foundational training requirements were considered vital, as they ensured forecasters’ preparedness to take advantage of the new observational data products. The overall goal of the training was to increase both the adoption and successful use of products being developed as the new instruments came online.

The US National Oceanic and Atmospheric Administration (NOAA) National Environmental Satellite, Data, and Information Service (NESDIS) and National Weather Service (NWS), the European Organisation for the Exploitation of Meteorological Satellites (EUMETSAT), and the Meteorological Service of Canada support satellite training efforts at COMET by providing direct funding as well as subject matter expertise. COMET develops and delivers training at the request of these program sponsors to meet the training needs identified for their workforces. COMET currently focuses on developing web-based materials that highlight the capabilities and applications of current generation geostationary and polar-orbiting satellites, including the US’ GOES-R+ and JPSS series operated by NOAA, the complementary European MSG and Metop series operated by EUMETSAT, the Japan Meteorological Agency’s Himawari-8, and the COSMIC radio occultation constellation jointly operated by Taiwan and the US.

1.2. Evolving Needs and Challenges

Several factors have guided changes in COMET’s approach to designing and developing online learning materials and in-person instruction. New and evolving satellite capabilities and products, combined with the availability of substantially more data, have led to additional requirements and needs for the operational workforces we train. We have consistently incorporated additional interactions and activities that address the evolution in tools and capabilities as well as how they can be applied. Meteorological workforce responsibilities have shifted increasingly toward impact-based decision support services (IDSS) in response to evolving societal needs, particularly when storms or other phenomena threaten lives and livelihoods. In addition to being able to incorporate new data into their forecasts, operational meteorologists and hydrologists must be able to use these data to support partners and stakeholders in decision-making. These changes in our partners’ needs have required that we remain adaptable. As this shift has occurred, we have adjusted our instructional approach to incorporate the additional skills that forecasters must possess to effectively perform their work, while remaining considerate of the time constraints of our audience.

As meteorologists continue to be tasked with forecasting as well as successfully communicating their forecasts to an expanding variety of users, the increased demands of their work have decreased the available time for training. These changes mean that effective training needs to be focused, efficient (i.e., accessible and completable in a short window of time), and impactful. Increasingly, training needs to incorporate practical elements that include examples and exercises relating directly to decision support services and effective communication.

The evolution in our training deliverables attempts to match the paradigm shift that forecasters are facing. A recent World Meteorological Organization (WMO) survey of the workforces of international meteorological and hydrological services [4] documents identified training priorities (Figure 2). Instrumentation and observation (which includes environmental satellites) and communication and customer interaction scored high as priorities, in line with the new-generation observational data becoming available and the shift in duties to increased focus on IDSS.

1.3. Focus Moving Forward

Advances in environmental satellite technology have created opportunities for new and exciting science, and this translates into increased needs for training, particularly as related to instrumentation and observation. In the US alone, there have been four relatively recent satellite launches (Suomi-NPP, GOES-16, JPSS-1, and GOES-17) that are increasing and improving the environmental satellite information available for a variety of applications. EUMETSAT launched the third polar-orbiter in their Metop series in 2018, and has also placed strong emphasis on both MSG satellite applications and the anticipated capabilities of the next-generation MTG satellites. COMET works collaboratively with these sponsors to define training priorities and needs. In the satellite meteorology community, these needs often stem from new satellites and the increased remote sensing capabilities and data flows that come with them. The resulting training offerings link directly to the WMO Guidelines and Skills for Operational Meteorologists [5], which focus on feature identification as well as on satellite interpretation for atmosphere, land, and ocean systems.

COMET’s more recent lessons work in tandem with existing foundational training materials on MetEd. With the changing technological landscape and increased emphasis on providing shorter, more interactive learning solutions that are both effective and engaging, we are continually attempting to balance the need for continued instruction in foundational science with the changing needs of the workforce. To address these workforce concerns, in recent years, COMET’s offerings have become shorter and more narrowly focused, while also attempting to increase learner engagement. With our newest offerings, we have continued to keep our sights set on one of the primary goals of our training efforts: To provide measurable increases in performance and skill using environmental satellite products.

2. Instructional Methods and Best Practices

2.1. Design Methods and Process

COMET staff work to ensure that our instructional products have been effectively designed to facilitate learning by incorporating established principles of instruction [6]. The breadth of geoscience topics covered, our commitment to making the training accessible to a global audience, and our overall focus on instructional design are all part of what makes the COMET Program unique. Our lessons utilize theoretical underpinnings combined with instructional design best practices [3,7,8,9] and incorporate iterative cycles surrounding content development, design, and programming [10]. COMET seeks to staff each project with an in-house scientist and an instructional designer, who work as a team with sponsors and subject matter experts (SMEs) to identify existing training needs, define the intended audience for instruction, and establish measurable learning objectives that stem from the identified need and expectations for performance. Our in-house team then works more directly and collaboratively with the SMEs, who include both satellite product developers and operational forecasters, to ensure that the training meets high standards for accuracy and relevancy. These SMEs are engaged throughout the lifecycle of a project to provide details about environmental satellite products and their use in operational or environmental monitoring settings. A simplified illustration of the development process is included in Figure 3.

The growth and effectiveness of online learning, and the ways in which online learning complements traditional classroom methods, have been well documented over the years.

“Perceptions of online learning have been shifting in its favor as more learners and educators see it as a supplement to some forms of face-to-face learning. Drawing from best practices in online and face-to-face methods, blended learning is on the rise at universities and colleges. The affordances of blended learning are now well understood, and its flexibility, ease of access, and the integration of sophisticated multimedia and technologies are high among the list of appeals.” The New Media Consortium (NMC) Horizon Report, p. 18 [11].

2.2. Designing for Learner Practice Opportunities and Retention

Typical online lessons produced by COMET include an intuitively navigable and visually appealing layout so the learner can focus on the content they need to. The lessons typically rely on written text, narration, or both, paired with high quality imagery and animations. Video components, interactive data sets, exercises (Figure 4), and mapping tools can be incorporated to further extend the learning experience. Lessons produced in recent years include rigorous pre- and post-assessments (see also Section 3.3) that help the learner and the COMET team understand how effective the training has been.

In recent deliveries, our team has been restructuring our overall content development approach to introduce practical activities at the start of the lesson, which draw the learner’s attention to the relevance of the instruction and provide an initial challenge. This shift in organization stems from Michael Allen’s context, challenge, activity, feedback (CCAF) model [10]. By presenting the type of mental activity that a learner will need to apply when they have completed the training as early as possible within a given lesson, we are setting a context that demonstrates an understanding of the learner’s goals in completing that particular lesson. This method also has the added benefit of increasing engagement, as it provides active participation and the opportunity for failure. Failure in safe learning spaces, paired with effective feedback, clarifies where a learner might need to focus his or her efforts and also reinforces areas where the learner already possesses strengths. This tactic also provides space for adjustment, to correct mistakes without real-world impacts, which is important when building training tailored toward IDSS. Note that the intent of this methodology is not to provide a negative experience, but to make the point that the instruction the learners are receiving is necessary and will help them solve problems that may occur in authentic, operational situations.

Another way COMET engages learners is through the use of shorter, case-based lessons, which seek to provide required knowledge alongside applications of practical skills. An example is the lesson titled Basic Satellite and NWP Integration (https://www.meted.ucar.edu/lesson/1408) published in early 2019. This lesson uses a web map service (WMS) to display real-time data and products for learners to be able to practice the skills and methods they have been taught (Figure 5). The WMS display can help to efficiently regionalize lessons, because learners can focus on data for their particular area of interest.

By relying on Allen’s conceptions related to learner engagement [10], COMET’s instructional designers are able to more easily narrow the instructional content into relevant pieces. This strategy increases the chances that learners will find what it is they actually need to focus on and helps provide instruction that is scoped effectively to meet the increasingly restricted time frames that learners are able to devote to training. Additionally, COMET continues to incorporate effective methods and tools for engaging our intended audience. For example, the use of an interactive slider tool allows learners to efficiently compare two images or graphics. The opportunity to interact with imagery (Figure 6), web map content, or other data products reinforces learning and helps learners retain the essential knowledge and skills they need to perform better in their jobs.

COMET continually strives to provide online learners with opportunities for practical application of the knowledge and skills they should be mastering throughout the lessons. We have built extensive experience with case exercises, and in some topic areas, are developing branching simulations that are scenario based and communication focused. These interactive learning experiences provide learners with multiple decision points in a real-world scenario and the opportunity to make mistakes in a safe space. Learners are able to return to the scenario and try again if they fail to achieve the optimal outcome. By mimicking the complex thinking that would be needed on the job, the simulations give learners the ability to apply their knowledge and incorporate the feedback they receive as they work their way through the scenario.

To help reinforce learning, COMET’s online training offers learners the option to subscribe to booster questions. As of May 2019, there are over 4000 subscribers to this new feature. Learners receive booster questions at spaced intervals starting two days after they have completed a lesson and extending to 80 days after completion. The questions address the central concepts of the learning experience, serve to keep the learner engaged with the content, and ultimately aid retention. The value of “spaced learning” has become better understood and more prominent in recent years. It centers on the notion that providing repeated recall activities at regular intervals that hinge on the essential components of an instructional experience serves as an effective impediment to forgetting [12]. In other words, presenting learners with problems that are connected to the lesson’s learning objectives at spaced intervals after completing the lesson significantly increases the likelihood that they will retain the knowledge and abilities garnered through their learning experience.

2.3. Learning Resources

The techniques described in the previous pages serve as a basis for creating innovative training materials that are then made accessible on the MetEd website. The MetEd online delivery platform provides access to numerous resources spanning an extensive range of over 20 geoscience topics. These instructional resources are available as individual lessons or as distance learning courses, which are collections of lessons pertaining to a broader topic area (Figure 7). Users of both lessons and courses can receive certificates of completion. Here, we focus solely on MetEd resources within the satellite meteorology topic area.

2.3.1. Lessons

Most of COMET’s satellite lessons can be broadly categorized by either satellite constellation, (e.g., GOES-R+ and JPSS), by remote sensing science topic area, (e.g., microwave remote sensing, hyperspectral sounding), or by user application (e.g., forecasting convective weather, monitoring the wildland fire cycle, analysis and short-term forecasting of surface flooding, or land surface analysis). A comprehensive list is included in

Table 1. Sampling of MetEd satellite meteorology lesson categories.

Satellite Constellations	GOES-R, Himawari, Meteosat Second Generation JPSS, Metop, GCOM-W Jason altimetry mission COSMIC constellation
Remote Sensing Topics	GEO vs. LEO orbits and observational coverage Principles of radiation Visible and infrared remote sensing Microwave remote sensing Hyperspectral sounding Scatterometry for wind and wave estimates Land and ocean surface properties Radio occultation Satellite altimetry
User Applications	Satellite imagery interpretation Multispectral satellite products and applications Weather analysis, diagnosis, and prediction Storm analysis Monitoring and prediction of environmental hazards Land and ocean analysis Climate monitoring Atmospheric chemistry

.

2.3.2. Distance Learning Courses

MetEd distance learning courses organize multiple lessons in a particular area of interest, making it easier for learners interested in that topic area to access all of the lessons related to that topic. Courses are self-paced and always available for open enrollment. When a learner passes the quizzes for all the core lessons in a course, he or she can receive a course completion certificate. As of early 2019, MetEd hosts several distance learning courses, which are shown in

Table 2. MetEd courses and course contributions.

MetEd Satellite Meteorology Courses (https://www.meted.ucar.edu/training_detail_courses.php)
GOES-R Satellites Orientation Course (https://www.meted.ucar.edu/training_course.php?id=42)	Introduction to capabilities, products and applications with the new generation GOES-R+ satellites
GOES-R Series Faculty Virtual Course (https://www.meted.ucar.edu/training_course.php?id=60)	Introduction to GOES-R satellite series instruments, capabilities and applications for university faculty
JPSS Satellites: Capabilities and Applications Course (https://www.meted.ucar.edu/training_course.php?id=54)	New generation operational polar-orbiting satellites, instruments, science, products and benefits for monitoring and forecasting various meteorological phenomena and environmental hazards
Microwave Remote Sensing Topics Distance Learning Course (https://www.meted.ucar.edu/training_course.php?id=15)	Foundational topics covering science, products, and applications of satellite microwave remote sensing
Multispectral Satellite Application Topics Course (https://www.meted.ucar.edu/training_course.php?id=35)	Foundational topics covering products and applications from multispectral observations, including methodologies used to derive multispectral image products
Satellite Water Vapor Interpretation - Short Course (https://www.meted.ucar.edu/training_course.php?id=44)	Introduction to interpreting water vapor imagery for analyzing and identifying atmospheric processes
Dynamic Feature Identification: The Satellite Palette (https://www.meted.ucar.edu/training_module.php?id=187)	Lesson series addressing use of satellite imagery to identify various atmospheric dynamic features
Frontal Diagnosis Course (https://www.meted.ucar.edu/training_course.php?id=68)	Identifying and analyzing cold fronts using satellite imagery
Modifying NWP Output Course (https://www.meted.ucar.edu/training_course.php?id=67)	Assessing NWP output using satellite imagery
COMET Contributions to Multi-Partner Courses
Satellite Foundational Course for GOES-R: SatFC-G (https://www.meted.ucar.edu/training_module.php?id=1301)	Series of short lessons and guides addressing GOES-R geostationary satellite instruments, capabilities, essential science and products for NWS forecasters and decision makers
Satellite Foundational Course for JPSS: SatFC-J (https://www.meted.ucar.edu/training_module.php?id=1614)	Series of short lessons and guides addressing JPSS and other polar-orbiting satellite instruments, capabilities, essential science and products for NWS forecasters and decision makers
African Satellite Meteorology: ASMET	Series of lessons focused on the needs of African forecasters. Includes both online lessons as well as in-person training by different African training centers

. COMET, in coordination with other training providers, also contributes individual lessons to larger courses designed to cover multiple topics that promote user readiness for next-generation operational satellite series, such as the GOES (geostationary) and JPSS (polar-orbiting) constellations.

Several of COMET’s most recent training deliveries are topic-focused case exercises that allow learners to work through an event to practice applying satellite products to specific forecast tasks or challenges. Since 2017, COMET has released four stand-alone case examples in the satellite topic area (Figure 8). Additional training case exercises are in development.

The intent in expanding our offerings of scenario-based instructional materials is three-fold. First, these materials provide increased opportunities for higher-level thinking, synthesis, and application of concepts, mimicking the type of mental activity that will be required in the field. Second, they help connect the scientific knowledge that learners need to real world events, increasing the potential for engagement. Lastly, they provide for a flexible presentation of material so that only essential components are incorporated from the events as they happened in the world. This scenario-based focus allows us to develop content targeted toward the skills that learners most need to develop and hone. Sponsors and learners have consistently provided positive feedback regarding this method of instruction and usage metrics are high. In its first 18 months online, the “GOES-16 Case Exercise: 8 May 2017 Colorado Hail Event” lesson received over 1200 unique views. Similar metrics for “GOES-16 and S-NPP/JPSS Case Exercise: Hurricane Harvey Surface Flooding” indicate nearly 650 sessions in the lesson’s first 10 months online.

Additionally, case exercises can be focused by region, so that learners can work on scenarios and events happening within their area of interest. Both case exercises and scenario-based training build on the solid foundation of satellite-platform-specific training already available on MetEd and we look forward to continuing to expand our repertoire of innovative learning solutions in the future.

3. Outcomes

3.1. Users and Reach

The current collection of satellite-focused lessons, courses, and instructional content available on MetEd provides learners worldwide with the foundational knowledge and skills necessary to put satellite products relevant to their work and studies to use. Along with the functional impact associated with the content COMET provides, one of the clearest benefits of MetEd is its worldwide reach.

As of April 2019, MetEd has approximately 600,000 registered users (Figure 9). Of those users, nearly 200,000 are dispersed across over 190 different countries and territories outside the United States, and the total number of users in both categories has been steadily increasing. Due to the international nature of the science, applications, and satellite capabilities addressed in our lessons, they are applicable to interested learners around the globe.

The largest portion of MetEd users are affiliated with educational institutions. Along with providing training support for numerous meteorological agencies in the US and abroad, our instructional materials serve as reinforcement for instruction taking place at the undergraduate and graduate levels. Because MetEd serves learners across multiple institutions worldwide, including university students and faculty, the materials available support professional development throughout the entire career lifecycles of meteorological and geoscience professionals.

International users account for just over 30% of total sessions on MetEd (Figure 10). Many international learners participate in satellite training via MetEd through the WMO VLab, a global network of training centers and meteorological satellite operators focused on improving the utilization of data and products from environmental satellites. Other international efforts include lessons produced as part of the African Satellite Meteorology Education and Training (ASMET) project, a joint effort between EUMETSAT, COMET, and meteorology instructors from several African nations. These lessons focus on specific examples for enhancing weather forecasts using meteorological satellite imagery and products.

Given the volume of session use by international learners on MetEd, it has become imperative that COMET provide translated materials for learners to be able to complete the training in the language of their choice. COMET employs a full-time translator to be able to offer its training materials in languages other than English. Many of COMET’s GOES-R resources, for example, have been translated into both Spanish and Portuguese and are available through a dedicated multilingual resource page (https://www.meted.ucar.edu/satmet/goes_resources/index.html). These resources include the GOES-R Satellites Orientation Course, consisting of three longer-format lessons, which was created in 2014 prior to the launch of the first satellite in the GOES-R series. Spanish and Portuguese translations of this course were made available following the release of the English lessons.

MetEd’s total number of satellite meteorology focused resources has grown considerably over the years (Figure 11). By around 2012, COMET was focusing significant efforts on infusing satellite products into other training areas, including aviation, marine weather, and fire weather. Along with these efforts to integrate satellite products into non-satellite-specific topics, COMET’s sponsors also began requesting more dedicated, satellite-specific trainings. In 2016, COMET released eight lessons as part of the US NWS Satellite Foundational Course for GOES-R (SatFC-G, https://www.meted.ucar.edu/training_module.php?id=1301), along with the production of other sponsor-requested satellite training lessons. In 2018, COMET released five lessons as part of the US NWS Satellite Foundational Course for JPSS (SatFC-J, https://www.meted.ucar.edu/training_module.php?id=1614), in addition to three case exercises and other satellite-focused training. In the first quarter of 2019, COMET released seven satellite-focused lessons: Two of these lessons compose part of a new distance learning course on Frontal Diagnosis while another three serve as core components of a Modifying NWP Output Course.

Lessons produced by COMET for the US NWS as part of the SatFC-G course have seen high usage. The full SatFC-G course, hosted on the Cooperative Institute for Research in the Atmosphere (CIRA) Satellite Hydrology and Meteorology (SHyMet) website (http://rammb.cira.colostate.edu/training/shymet/), contains 37 lessons and was published as a joint effort coordinated by the US NWS Satellite Training and Advisory Team. The team consisted of trainers from the NWS, COMET, CIRA, the Cooperative Institute for Meteorological Satellite Studies, and the NASA Short-term Prediction Research and Transition Center. This group, which also created the SatFC-J course that provides training for JPSS, worked together for multiple years to plan, produce, and make the full course available to forecasters prior to the GOES-16 satellite launch. These lessons are applicable not just to the US NWS but also to meteorological services in Canada, Central and South America, and the Caribbean.

The Satellite Foundational Course lessons are mandatory for US NWS forecasters. In 2017 alone, at least 2500 NOAA staff accessed each of COMET’s eight SatFC-G lessons. Usage statistics for these lessons are shown in Figure 12.

3.2. Impact

John Ogren, Chief Learning Officer for NOAA’s US National Weather Service, summarizes the importance of this training for preparing the operational meteorological workforce in the US and internationally:

“The UCAR/COMET Program has been critically important in helping weather forecasters throughout North and South America and the Atlantic/Pacific Island nations in preparing for utilizing the vast datastreams from GOES 16/17 and NOAA-20. COMET initially educated professionals about the instruments and capabilities of these new satellites far in advance of their launch. COMET was instrumental in developing foundational courses for both the GOES and POES instruments which provided background information on the many new channels, sensors and products, so forecasters were ready to use the new satellite data from day 1. COMET has been diligent in updating their training modules once the actual satellite data was flowing so new forecasters have up-to-date training modules. COMET led many User Readiness workshops for years at professional conferences, workshops and World Meteorological Organization events. The National Weather Service forecasters are now fully making use of this new satellite data to improve weather watches, warnings and forecasts, and informing critical local decision makers all thanks to the outstanding education and training from COMET.” [13]

From the launch of NOAA’s first operational microwave remote sensing instrument in the late 1990s, to the advanced microwave instruments on the current JPSS and Metop polar orbiters, COMET has produced lessons introducing forecasters to foundational science topics and applications, with the goal of enhancing usage of this unique observational capability. Figure 13 shows usage activity for the three most used foundational microwave remote sensing lessons since their initial release between 2005 and 2008. Two of these lessons are now in their second edition, highlighting the ongoing importance and relevance of this training.

Figure 14 highlights three MetEd satellite lessons with consistently frequent use since their publication. The level of difficulty of these lessons ranges across educational levels, from high school, to undergraduate, to more advanced, requiring some prior knowledge of the topic.

COMET regularly reviews usage statistics like these to gauge the popularity, effectiveness, and reach of our training deliveries, which can help inform priorities for lesson updates.

3.3. Assessing Learner Performance

COMET’s metrics allow regular monitoring of learner performance on assessments. These data help us understand the effectiveness of our instruction and can aid us in identifying any issues with the assessments we create. In the past few years, COMET has begun including pre-assessments with every lesson. The pre- and post-course assessments, offered upon launching the course and immediately following completion, are aligned to measurable learning objectives associated with each lesson, and the questions are weighted equally across both assessments (in terms of score, objective, level of cognition, content area, and skill application). Performance on assessments can therefore offer a metric to measure the impact of the lesson on learner comprehension and skills application.

Comparisons of the pre- and post-test scores for three recent satellite training case exercises (Figure 15) show an increase in the mean post-lesson quiz scores over the pre-assessment scores. This increase suggests that upon completion of the lesson, learners have gained enough functional knowledge to better answer a challenging set of questions based on the lesson’s learning objectives. The improvement in scores can help to demonstrate the effectiveness of the instruction. The materials remain available online as a resource for learners so they can return to refresh skills as needed, and if they also enroll in booster questions, there are increased opportunities for practice and retention. The final booster question asks the learner to draw the connection between what they have learned in the lesson and its applications to on-the-job duties, which gives us information about how they are actively implementing their learning. We are hopeful that the increased knowledge and skill imparted through these case exercises and our educational materials remain in use well beyond the lesson.

4. Summary

The COMET Program’s online education and training resources include numerous offerings in the area of satellite meteorology. The resources support domestic and international training related to both geostationary and low Earth orbiting satellite systems to enhance the capabilities and performance of environmental forecasters. These same resources also reach a large university audience to provide full-career professional development for meteorologists and other geoscience professionals.

Instructional designers and staff scientists at COMET make use of well-established instructional principles, maintain an awareness of emerging trends, and clearly delineate forecaster needs to develop training that is operationally relevant and that best suits the intended audience of forecasters and other users of satellite information. The inclusion of interactive exercises as well as the availability of booster questions helps reinforce the content for professionals needing to apply satellite information across a variety of applications and topic areas, including but not limited to, weather analysis and forecasting, hazard monitoring, land and ocean monitoring, climate monitoring, and decision support.

As we look toward the future, COMET’s commitment to sound science and effective and innovative instructional design will continue to guide our offerings. New lessons will emphasize satellite advancements in the context of forecasting applications, IDSS, and overall environmental monitoring. The combination of changing workforce needs and new satellite capabilities coming online, including future Meteosat and EUMETSAT Polar System satellites in Europe, the expanding capabilities of the GOES-R and JPSS satellite series in the Americas, and other new satellite systems from various international satellite operators, will require additional contextual education and training for forecasters and geoscience professionals worldwide.