Automatic Flight Callsign Identification on a Controller Working Position: Real-Time Simulation and Analysis of Operational Recordings
Abstract
:1. Introduction
- Start by the identification or the callsign of the flight being addressed;
- Continue by issuing the command with its qualifiers or information.
- Starts with the command with its qualifiers;
- Ends with the identification or callsign of flight.
- FCs always have to call air traffic control when they are about to enter a new air traffic service, ATS, unit or sector; they make a call prior to the boundary between both airspaces. The FC communicates with the ATCo to make them aware of their presence and confirm that voice communication is feasible for emergency use. In this communication the FC will typically greet the ATCo and provide some information related to the flight. Example:Good morning Ryanair nine zero three five flight level three hundred
- The FC usually starts communications at any time with ATCos to request modifying vertical/horizontal trajectories and/or the speed to fly at the optimum performance of the aircraft.
- Another important reason to initiate a call from the FC is requesting to modify their flight level, route, speed, or any other flight condition because of adverse weather such as encountering cumulonimbus, severe turbulence, icing etc. Example:Air Europa six alfa bravo requesting flight level four zero zero due to severe turbulence
- The voice signal used for speech recognition from controllers’ voice utterances is extracted directly from the jack of the controller. This signal has a low degree of noise.
- Controllers’ language is English or the local language of the ground station [6].
- Usually, controllers of an air navigation service provider will have similar accents when speaking.
- Communications from controller to flight crew can be standardised as [9]: call id + command + qualifier 1 + qualifier 2.
- The voice signal used for speech recognition from flight crew voice utterances is extracted from radio communications. The quality of these communications is highly dependent on:
- The distance of the aircraft to the receiving radio station.
- The signal-to-noise ratio, SNR, can vary from 10 dB to −5 dB [10].
- The quality of the signal transportation from the radio station to the air traffic control facility where the signal is analysed.
- Flight crew language is English or the local language of the ground station [6].
- FCs have very different accents usually, but not always, relating to the flight company country. Countries that are in the routes of international flights have even higher rates of different accents.
- Communications from flight crew to controller can, similarly to the controller’s ones, be decomposed as: call id + command + qualifier 1 + qualifier 2.Alternatively, if it is a readback: command + qualifier 1 + qualifier 2 + call id
2. Materials and Methods
2.1. Requirements to Be Met by the System
2.1.1. Basic ASR Engine Requirements for Callsign Identification
- The voice recognition system, VRS, shall be able to function without connection to sources external to the area control centre, ACC.
- The callsign illumination must be produced as soon as possible once the communication has started.
- The ASR engine shall be able to process the utterance in English and the local language, when local languages are allowed.
2.1.2. VRS Requirements for Callsign Identification
- It is preferable not to have a callsign illumination rather than a wrong callsign illumination.
- The VRS will use the sector flight list from the CWP to improve its performance.
- The radio callsign e.g., Beeline/Cactus.
- The company name e.g., Brussels Airlines/ US Airways.
- ICAO designator using aeronautical alphabet. e.g., Bravo Echo Lima (BEL)/Alpha Whiskey Eco (AWE).
- All of the possible modes to pronounce a number. e.g., one zero zero, ten zero, one hundred.
2.2. Real-Time Simulation
- The FDP has the list of flights that are of interest for the ACC (composed of several sectors). The FDP ensures that the list of flights in each CWP is updated with new incorporations or cancellations once the flight is no longer of interest.
- The CWP has the list of flights that are of interest for the sector. This list is smaller than the previous one, but some flights may not be covered, for example, last minute flights deviated due to weather.
- The sectors have several entry points where the flight crew performs their first call (related to the highlight of callsigns on the CWP from pilot utterances).
- The sectors are quite wide and integrate nine control volumes. This implies that there are very different traffic flows that require different types of control and facilitates the creation of situations where the traffic is focused in one area or disperse along the whole sector (related to both, the highlight of callsigns on the CWP from the pilot and controller’s utterances).
- There are several airports within the control volume, the main one being Madrid- Barajas airport, LEMD in its ICAO code. This airport was used in the north configuration and generated traffic flows to/from both sectors.
2.3. Statistical Approach
- CJL is a sector with good radio coverage whose main traffic flows are to and from Madrid-Barajas Airport, the major Spanish airport. It limits with Madrid TMA and the surface. Control service is provided to all aircraft from FL210 to FL325. Information service is provided from SFC to FL210 outside the TMA/airport areas and airways.
- CJU is a sector with good radio coverage and quality whose main traffic flows are to and from Madrid-Barajas Airport, and over flights to the south of Spain. Control service is provided to all aircraft from FL325 to FL660.
- The SAN sector includes a large proportion of oceanic airspace that has lower radio coverage compared to CJL and CJU. Its main flows are overflights to/from the America, and to/ from United Kingdom. Free route airspace is implemented in this sector. Control service is provided to all aircraft from FL210 to FL660. Information service is provided from SFC to FL210 outside the TMA/airport and control areas.
3. Results
3.1. Technical Results
- Thousands are correctly identified (100%);
- Hundreds are correctly identified (100%);
- Numbers between 11 and 99 (e.g., 13, 18, 34) have very high recognition rates (98%);
- Numbers with triple have different success rates;
- ∘
- 111 (triple one) was transcribed correctly 88% and one time as 341;
- ∘
- 666 (triple six) had 67% success but was transcribed 326;
- ∘
- 777 (triple seven) was the least accurate of the group being transcribed as 37 in 84% of the utterances;
- ∘
- 888 (triple eight) has a success of 75%, but was transcribed as 68 two times.
3.2. Operational Results
3.2.1. Human Performance
3.2.2. Safety
3.2.3. Additional Findings
- During a shift change. The entering controller may sit near the departing controller during a period of time to be able to grasp the situation before actually controlling the flights.
- When new controllers have onsite training. The new controller may be near the experienced controller following the issued commands, or a supervisor may be near the new controller.
4. Discussion
Supplementary Materials
Author Contributions
Funding
Data Availability Statement
Acknowledgments
Conflicts of Interest
References
- ICAO. Procedures for Air Navigation Services (PANS)—Air Traffic Management Doc 4444, 16th ed.; ICAO: Montreal, QC, Canada, 2016. [Google Scholar]
- ICAO. Annex 11—Air Traffic Services, 15th ed.; ICAO: Montreal, QC, Canada, 2018; Para 3.7.3.1. [Google Scholar]
- Helmke, H.; Rataj, J.; Mühlhausen, T.; Ohneiser, O.; Ehr, H.; Kleinert, M.; Oualil, Y.; Schulder, M.; Klakow, D. Assistant-based speech recognition for ATM applications. In Proceedings of the 11th USA/Europe Air Traffic Management Research and Development Seminar (ATM 2015), Lisbon, Portugal, 23–26 June 2015. [Google Scholar]
- Nguyen, V.N.; Holone, H. Possibilities, challenges and the state of the art of automatic speech recognition in air traffic control. Int. J. Comput. Inf. Eng. 2015, 9, 1940–1949. [Google Scholar]
- Pellegrini, T.; Farinas, J.; Delpech, E.; Lancelot, F. The Airbus Air Traffic Control Speech Recognition 2018 Challenge: Towards ATC Automatic Transcription and Call Sign Detection. Interspeech 2019, 2019, 2993–2997. [Google Scholar]
- ICAO. Annex 10. Aeronautical Telecommunications, 6th ed.; ICAO: Montreal, QC, Canada, 2001; Para 5.21.2. [Google Scholar]
- Apesteguia, E. Empleo real de las mujeres en el sector aeronáutico. Fly News N° 4 2017, 80–85. [Google Scholar]
- Delpech, E.; Laignelet, M.; Pimm, C.; Raynal, C.; Trzos, M.; Arnold, A.; Pronto, D. A Real-life, French-accented Corpus of Air Traffic Control Communications. In Proceedings of the Eleventh International Conference on Language Resources and Evaluation (LREC 2018), Miyazaki, Japan, 7–12 May 2018. [Google Scholar]
- Helmke, H.; Slotty, M.; Poiger, M.; Herrer, D.F.; Ohneiser, O.; Vink, N.; Cerna, A.; Hartikainen, P.; Josefsson, B.; Langr, D.; et al. Ontology for transcription of ATC speech commands of SESAR 2020 solution PJ.16-04. In Proceedings of the IEEE/AIAA 37th Digital Avionics Systems Conference (DASC), London, UK, 23–27 September 2018. [Google Scholar]
- Segura, J.C.; Ehrette, T.; Potamianos, A.; Fohr, D.; Illina, I.; Breton, P.A.; Clot, V.; Gemello, R.; Matassoni, M.; Maragos, P. The HIWIRE Database, A Noisy and Non-Native English Speech Corpus for Cockpit Communication; 2007; Available online: https://catalogue.elra.info/en-us/repository/browse/ELRA-S0293 (accessed on 28 February 2023).
- Goyal, A.; Gupta, V.; Kumar, M. Recent named entity recognition and classification techniques: A systematic review. Comput. Sci. Rev. 2018, 29, 21–43. [Google Scholar] [CrossRef]
- Altinel, B.; Can Ganiz, M. Semantic text classification: A survey of past and recent advances. Inf. Process. Manag. 2018, 54, 1129–1153. [Google Scholar] [CrossRef]
- Nigmatulina, I.; Braun, R.A.; Zuluaga-Gomez, J.; Motlicek, P. Improving callsign recognition with air-surveillance data in air-traffic communication. In Proceedings of the Interspeech 2021 Satellite Workshop, Brno, Czech Republic, 30 August–3 September 2021. [Google Scholar]
- Cordero, J.M.; Dorado, M.; de Pablo, J.M. Automated speech recognition in ATC environment. In Proceedings of the 2nd International Conference on Application and Theory of Automation in Command and Control Systems (ATACCS’12), London, UK, 29–31 May 2012; IRIT Press: Toulouse, France; pp. 46–53. [Google Scholar]
- SESAR. SESAR 2020 D3.2.020 PJ.16-04 TRL4 TVALR—Automatic Speech Recognition. V02.00.00; SESAR: Brussels, Belgium, 2019. [Google Scholar]
- Eurocontrol. European Operational Concept Validation Methodology, E-OCVM Volume I—Version 3.0; Eurocontrol: Brussels, Belgium, 2010. [Google Scholar]
- Madson, M. Air Traffic Controllers and Real-time Simulation: A Powerful Combination. In Journal of Air Traffic Control; ATCA: Alexandría, VA, USA, 2004; pp. 24–27. [Google Scholar]
- Publications Office. Single European Sky ATM Research 3 Joint Undertaking, Digital European Sky: Strategic Research and Innovation Agenda. 2020. Available online: https://data.europa.eu/doi/10.2829/117092 (accessed on 28 February 2023).
- ENAIRE. Plan Estratégico de Enaire 2021–2025. Plan de Vuelo 2025; ENAIRE: Madrid, Spain, 2021. [Google Scholar]
- SESAR. SESAR PJ.10-W2-96 ASR Initial Technical Specification. V00.01.00; SESAR: Brussels, Belgium, 2020. [Google Scholar]
- EUROCAE. EUROCAE ED-137 Interoperability Standards for VOIP ATM Components; EUROCAE: Saint-Denis, France, 2012. [Google Scholar]
- Fischer, V.; Ghahabi, O.; Kunzmann, S. Recent improvements to neural network based acoustic modelling in the EML real-time transcription platform. In Proceedings of the ESSV, Ulm, Germany, 7–9 March 2018; pp. 38–45. [Google Scholar]
- Ghahabi, O.; Zhou, W.; Fischer, V. A robust voice activity detection for real-time automatic speech recognition. In Proceedings of the ESSV, Ulm, Germany, 7–9 March 2018; pp. 85–91. [Google Scholar]
- Lin, Y. Spoken Instruction Understanding in Air Traffic Control: Challenge, Technique and Application. Aerospace 2021, 8, 65. [Google Scholar] [CrossRef]
- Cordero, J.M.; Rodríguez, N.; De Pablo, J.M.; Dorado, M. Automated speech recognition in controllers communications applied to workload measurement. In Proceedings of the 3rd SESAR Innovation Days, Stockholm, Sweden, 26–28 November 2013. [Google Scholar]
- Raja, J.B.; Cordero, J.M.; De Pablo, J.M. Reconocimiento y síntesis de voz en sistemas automatizados de control de tráfico aéreo. In Proceedings of the VIII Congreso Ingeniería del Transporte, A Coruña, Spain, 2–4 July 2008. [Google Scholar]
- Insignia. Available online: https://insignia.enaire.es (accessed on 28 February 2023).
- Arnegard, R.; Comstock, J., Jr.; Raimond, J.R. The Multi-Attribute Task Battery for Human Operator Workload and Strategic Behaviour Research; Nasa Technical Memorandum 104174; NASA: Washington, DC, USA, 1992.
- Dehn, D.M. Assessing the Impact of Automation on the Air Traffic Controller: The SHAPE Questionnaires. Air Traffic Control. Q. 2008, 16, 127–146. [Google Scholar] [CrossRef]
- Ohneiser, O.; Sarfjoo, S.; Helmke, H.; Shetty, S.; Motlicek, P.; Kleinert, M.; Her, H.; Murauskas, S. Robust Command Recognition for Lithuanian Air Traffic Controller Tower Utterances. In Proceedings of the Interspeech 2021, Brno, Czech Republic, 30 August–3 September, 2021; pp. 3291–3295. [Google Scholar]
- Helmke, H.; Shetty, S.; Kleinert, M.; Ohneiser, O.; Ehr, H.; Prasad, A.; Motlicek, P.; Cerna, A.; Windisch, C. Measuring Speech Recognition And Understanding Performance in Air Traffic Control Domain Beyond Word Error Rates. In Proceedings of the 11th SESAR Innovation Days, Online Conference, 7–9 December 2021. [Google Scholar]
- Efthymiou, M.; Usher, D.; O’connell, J.F.; Warnock-Smith, D.; Conyngham, G. The factors influencing entry level airline pilot retention: An empirical study of Ryanair. J. Air Transp. Manag. 2021, 91, 101997. [Google Scholar] [CrossRef]
- Shade, W.; Reynolds, D. A comparison of speaker Clustering and Speech Recognition Techniques for Air Situational Awareness. In Proceedings of the Interspeech 2007, Antwerp, Belgium, 27–31 August 2007; pp. 2421–2424. [Google Scholar]
- Ohneiser, O.; Balogh, G.; Rinaldi, W.; Murauskas, S.; Kis-Pál, G.; Usanovic, H.; Helmke, H.; Shetty, S.; Kleinert, M.; Ehr, H.; et al. Understanding tower controller communication for support in Air Traffic Control displays. In Proceedings of the 12th SESAR Innovation Days, Budapest, Hungary, 5–8 December 2022. [Google Scholar]
- Badrinath, S.; Balakrishnan, H. Automatic Speech Recognition for Air Traffic Control Communications. Transp. Res. Rec. J. Transp. Res. Board 2021, 2676, 798–810. [Google Scholar]
- Helmke, H.; Ondrej, K.; Shetty, S.; Arilíusson, H.; Simiganoschi, T.; Kleinert, M.; Ohneiser, O.; Her, H.; Zuluaga-Gomez, J.; Smrz, P. Readback error detection by automatic speech recognition and understanding. In Proceedings of the 12th SESAR Innovation Days, Budapest, Hungary, 5–8 December 2022. [Google Scholar]
Speaker | Callsigns | Detected Call Sings | Detection Rate |
---|---|---|---|
Controller | 859 | 721 | 84% |
Flight Crew | 687 | 457 | 67% |
Speaker | Callsigns | Detected Call Sings | Detection Rate |
---|---|---|---|
Controller | 143 | 127 | 87% |
Flight Crew | 158 | 77 | 49% |
Flight Crew First call/request | 65 | 38 | 58.5% |
Airline | Controller Utterances | Flight Crew | |||||
---|---|---|---|---|---|---|---|
Call Sings | Detected | Rate | Call Sings | Detected | Rate | ||
American Airlines | AAL | 5 | 4 | 80% | 4 | 2 | 50% |
Air Europa | AEA | 8 | 8 | 100% | 7 | 6 | 86% |
Aegean airlines | AEE | 9 | 6 | 67% | 6 | 0 | 0% |
Air Nostrum | ANE | 5 | 5 | 100% | 5 | 5 | 100% |
Condor | CFG | 9 | 8 | 89% | 8 | 3 | 38% |
Iberia | IBE | 8 | 8 | 100% | 11 | 11 | 100% |
Iberia Express | IBS | 7 | 7 | 100% | 4 | 1 | 25% |
Ryanair | RYR | 18 | 17 | 94% | 23 | 11 | 48% |
Tap Portugal | TAP | 7 | 7 | 100% | 11 | 4 | 36% |
Thomson | TOM | 14 | 13 | 93% | 18 | 12 | 67% |
Emirates Airlines | UAE | 11 | 11 | 100% | 14 | 4 | 29% |
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2023 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
García, R.; Albarrán, J.; Fabio, A.; Celorrio, F.; Pinto de Oliveira, C.; Bárcena, C. Automatic Flight Callsign Identification on a Controller Working Position: Real-Time Simulation and Analysis of Operational Recordings. Aerospace 2023, 10, 433. https://doi.org/10.3390/aerospace10050433
García R, Albarrán J, Fabio A, Celorrio F, Pinto de Oliveira C, Bárcena C. Automatic Flight Callsign Identification on a Controller Working Position: Real-Time Simulation and Analysis of Operational Recordings. Aerospace. 2023; 10(5):433. https://doi.org/10.3390/aerospace10050433
Chicago/Turabian StyleGarcía, Raquel, Juan Albarrán, Adrián Fabio, Fernando Celorrio, Carlos Pinto de Oliveira, and Cristina Bárcena. 2023. "Automatic Flight Callsign Identification on a Controller Working Position: Real-Time Simulation and Analysis of Operational Recordings" Aerospace 10, no. 5: 433. https://doi.org/10.3390/aerospace10050433