Toward a Reliability Scale for Assessing Reports of Unidentified Anomalous Phenomena (UAP)

Abstract

Unidentified Anomalous Phenomena (UAP) observations have been reported from ancient times to today, but their true nature remains uncertain. In this paper we propose a rating scale designed to separate “signal” from “noise” in assessing UAP sighting reports. Our intention is that this will help professionals and laypeople alike distinguish cases that warrant further investigation from easily explainable false alarms. We categorize UAP sighting reports according to the quality of their evidence, considering such factors as number of observers, amount and quality of supporting evidence, especially physical evidence, and perhaps most importantly, whether UAP witnesses have made some effort to find an ordinary explanation for what they saw or experienced and whether the evidence has been subject to expert analysis.

1. Introduction

It is possible that other spacefaring civilizations exist in our galaxy, which, by current estimates, contains billions of potentially habitable planets. Technologically advanced civilizations may have found a means to exceed light speed [1,2], although that is not required for interstellar travel if the voyagers are patient enough. Even with our own current technology, robotic probes launched from Earth could reach the nearest stars in just thousands of years, moving at a fraction of the speed of light [3].

Could any such alien spacecraft have reached Earth? Throughout the ages humans have puzzled over mysterious lights in the sky. In the years immediately following World War II, these became popularly known as Unidentified Flying Objects, or UFOs [4], a term that military agencies and others recently have dropped in favor of the more inclusive Unidentified Anomalous Phenomena. The idea that UAP might come from beyond Earth, along with the possibility that they relate to secret military programs, accounts for the continuing strong public interest in the subject. That interest has only increased due to recent hearings in the U.S. Congress (e.g., [5]).

Determining the true nature of UAP and connecting them—if there is indeed a connection—to speculation about extraterrestrial visitors requires that we first recognize most UAP sightings as cases of misunderstanding. Hundreds of thousands of sightings have been reported over the years to organizations such as the National UFO Reporting Center (NUFORC), the Mutual UFO Network (MUFON), the Center for UFO Studies, Enigma Labs, and GEIPAN in France. In 2025, the U.S. Defense Department’s All-Domain Anomaly Resolution Office (AARO) plans to unveil a public website for reporting UAP sightings, which is likely to increase that number. Nearly all the reports to date—or at least those that include enough data to allow for investigation—are explainable as ordinary objects and events [6,7], including airplanes, windborne balloons, drones, rocket launches, satellites, military aircraft, and a wide variety of natural phenomena from oddly shaped clouds to meteors to flocking birds. New scientific discoveries also have shed light on events that were previously unexplained, such as “sprites” and “elves” in Earth’s upper atmosphere. Of the more than 1800 cases in AARO’s database as of August 2025, just two percent remain unexplained. Another 40 percent have been “closed,” i.e., explained, while more than half—57 percent—lack enough data to properly investigate [8].

2. Methodology

In some ways the study of UAP is similar to the problem faced by radio astronomers engaged in the Search for Extraterrestrial Intelligence (SETI). No clear ET radio signal has ever been detected in more than 60 years of listening. Yet SETI researchers continue to work tirelessly to eliminate common sources of natural and artificial noise that might wrongly be interpreted as an intentional signal from an alien civilization.

We propose a rating scale that could similarly help separate “signal” from “noise” in the realm of UAP sightings. This is not the first such attempt; there have been other proposed schemes for classifying and ranking UFO reports, going back to the 1960s. Notable among these are the “Close Encounters” categories proposed by J. Allen Hynek [9], as well as classification schemes suggested by David Saunders, a member of the 1968 Condon Committee that investigated UFO reports [10] and Jacques Vallée [11]. More recent schemes include an “assessment matrix” for evaluating UAP evidence [12]. Organizations such as Enigma Labs and GEIPAN also have their own methods for sorting the wide variety of UAP reports, and UAP investigators in the U.S. Department of Defense may well have developed their own approach to ranking.

While our scale has elements in common with these other proposed scales, its distinction lies in incorporating three separate but equally important factors. It ranks observations based on (1) the type of evidence, considering such criteria as the number of observers, the existence of recorded data, and the presence of physical artifacts. We also assign a higher ranking if UAP witnesses—or some organization to whom they report—have made a diligent effort to (2) research possible explanations for what they saw or experienced. Finally, reports are ranked higher on our scale if they have been subjected to (3) expert analysis. (Unlike some other proposed schemes, we do not rate the credibility of witnesses based on their background or professional credentials. Trained pilots can make errors of judgment, too.)

We also make no attempt to assess whether reported UAPs are of extraterrestrial origin. We can only speculate as to how would-be alien visitors, whose biology we can only guess at, might think or behave. The idea that they might arrive in metal spaceships with blinking lights seems the height of anthropomorphic projection. We cannot even assume they would travel in a biological form—robotic probes piloted by AIs are just as likely, perhaps even more likely. Some UAP researchers have entertained the possibility that certain paranormal experiences, from hearing voices to visions, might have an extraterrestrial explanation. Others have theorized that “cryptoterrestrials” [13] from other dimensions or time periods have already arrived on Earth. As intriguing as these speculations are, we confine ourselves here to the possible presence of physical artifacts that can be detected through ordinary human senses and/or technology.

We expect that our proposed scale will be of most use to scientists and other serious researchers of the UAP phenomenon, if only to help them decide which cases deserve the most attention. But it should also be helpful to the general public trying to make sense of UAP sightings reported in the news. The suggested “Before You Report” checklist described under Category II could be a good starting point for anyone who witnesses an anomalous event and wants to quickly find out what it might be.

Finally, we expect that our reliability ratings, when applied to actual cases, will be subject to change. Many sightings—probably most—will drop to zero on the scale after further investigation, once a natural or known technological explanation can be ascertained.

3. Results

After considering a variety of public UAP reports, we suggest the reliability scale shown in

Table 1. Reliability Scale for UAP reports.

X	Long-duration sighting by multiple individuals/groups at different locations or recorded evidence from multiple sensors; physical evidence; reviewed by experts and so far unexplained
IX	Long-duration sighting by multiple individuals/groups at different locations or recorded evidence from multiple sensors; physical evidence
VIII	Long-duration sighting by multiple individuals/groups at different locations or recorded evidence from multiple sensors; reviewed by experts and so far unexplained
VII	Long-duration sighting by multiple individuals/groups at different locations or recorded evidence from multiple sensors; plausibly explained but explanation disputed
VI	Long-duration sighting by multiple individuals/groups at different locations or recorded evidence from multiple sensors
V	Long-duration sighting by one person or group, with recorded evidence
IV	Long-duration sighting by one person or group, no recorded evidence
III	Short-duration sighting by one person or group, with recorded evidence
II	Short-duration sighting by one person or group, researched but no recorded evidence
I	Short-duration sighting by one person or group, not researched
0	Sighting explained beyond reasonable doubt as a known phenomenon or object

, with UAP observations ranked from 0 (not reliable) to X (most reliable). Note that each higher category assumes that everything in previous categories also applies.

• Category 0. Sighting explained beyond reasonable doubt as a known phenomenon or object

Sighting can be removed from the UAP case list.

• Category I. Short-duration sighting by one person or group, not researched

Many UAP reports, however real-seeming or convincing to the witness, turn out to be unreliable [14]. Our eyes play tricks on us, as do our brains and memories. Ordinary objects seen from an odd angle or under unusual lighting conditions appear strange. This is especially true if the sighting lasts just a few seconds, as when a light in the sky flashes then suddenly disappears. For this reason, a brief (under ~10 s) UAP sighting by a single individual (or group observing together from the same location) receives the lowest rating on our scale. We recommend that such sightings not be reported publicly without further research (see Category II below).

• Category II. Short-duration sighting by one person or group, researched but no recorded evidence

Perhaps the single best way to reduce the number of UAP false alarms would be for witnesses to take the time to do a simple self-guided check before reporting. There are many online resources available to help members of the public research and possibly identify what they saw. Such self-directed research can start with consulting a checklist of websites including, but not limited to those in

Table 2. “Before you report” Checklist.

Could It Have Been…	Where to Check
…an airplane?	FlightRadar24 https://www.flightradar24.com/ ADS-B Exchange https://globe.adsbexchange.com/ Map of US Private Airports https://koordinates.com/layer/22874-us-private-airports/
…a weather balloon?	SondeHub Tracker https://tracker.sondehub.org/
…a rocket launch or atmospheric test?	U.S. Launch Range Locations https://public.ksc.nasa.gov/kscsma/range-locations Vapor Tracers https://www.nasa.gov/soundingrockets/about-vapor-tracers/
…a meteor?	American Meteor Society https://www.amsmeteors.org/fireballs/fireball-report/
…a satellite?	N2YO.com Real-Time Satellite Tracking https://www.n2yo.com/ Starlink Satellite Map https://satellitemap.space/?constellation=starlink
…a military aircraft?	FAA Map of Special Use Airspace https://ais-faa.opendata.arcgis.com/datasets/dd0d1b726e504137ab3c41b21835d05b_0/explore Map of U.S. military bases (https://data-usdot.opendata.arcgis.com/datasets/usdot::military-bases/explore) These maps show where military tests and overflights are likely to take place. Many bases post notices of these events and can answer queries through their websites or public information offices.
…an unusual cloud formation?	Cloud Appreciation Society https://cloudappreciationsociety.org/cloud-library/ Weird Weather https://www.weather.gov/owlie/weird-weather
…stars or planets?	The Sky Live https://theskylive.com/ Stellarium https://stellarium-web.org/
…a flock of birds?	BirdCast bird migration forecasts in real time https://birdcast.info/
…a drone?	Interactive Map of UAV Drones over the USA https://climateviewer.org/history-and-science/government/maps/uav-drones-over-usa/ (a very limited list of the thousands of public and private drones in the air at any given moment. More than a million drones are registered in the U.S. alone.)

Note: all websites were accessed last time on 23 September 2025.

below. UAP witnesses can also consult local news, neighborhood websites, social media sites like Reddit, and other community resources to determine if others reported—and possibly offered an explanation for—the same event.

If a sighting remains unexplained after a reasonable attempt to find its cause, it can be reported to organizations like NUFORC and Enigma Labs, who maintain public databases of sightings and, in some cases, provide a tentative explanation. The U.S. Defense Department’s All-Domain Anomaly Resolution Office (AARO) is also developing a system for public reporting of UAP sightings. Ideally, these organizations could come up with their own standardized checklist or resource center for the public to consult, expanding on the short list outlined in Table 2.

Self-directed attempts to explain UAPs obviously have their limits, however. Members of the public cannot be expected to recognize every type of bird, drone, airplane, or balloon flying overhead, let alone have knowledge of secret military projects. For that reason, artificial intelligence (AI) systems that could interrogate large databases of images of known flying objects under different conditions could go a long way to explaining, even tentatively, many public sightings.

The U.S. AARO and Enigma [15] are among the groups using AI to evaluate UAP data reported by the public, while the Galileo Project [16], the ADEOS (Anomaly Detection and Observation System) system at the University of Würzburg [17], and others are developing AI systems for dedicated UAP observing networks. As these systems become more capable, organizations like AARO, perhaps in cooperation with NASA, the Federal Aviation Administration, and NUFORC, as well as similar international organizations, could create a single, well-publicized clearinghouse for reporting UAP sightings, ideally with a smartphone app that would allow people to report sightings in real time and submit supporting images or data.

Centralizing public UAP reporting in this way would have two big advantages: people would know where to report sightings and how. And with the help of AI, reports that misidentify known objects like aircraft, astronomical objects, satellites, balloons, and perhaps even classified military tests could be quickly and easily identified and dismissed.

• Category III. Short-duration sighting by one person or group, with recorded evidence

Recorded evidence, whether from a smartphone video, doorbell camera, cockpit data recorder, or other device, aids in solving UAP mysteries by allowing witnesses as well as impartial outsiders to replay and study the event. The mere existence of a recording does not constitute proof, however. If the images or data are unclear or ambiguous, they may have little value. A two-second video of an unidentified light speeding past a doorbell camera could be anything from a flying bird to an insect buzzing the lens.

Once again, AI will be extremely useful, even revolutionary, for improving analysis of UAP photos and videos. But the same technology will undoubtedly be used to create fake UAP images, and researchers will need to be on guard for such deception.

• Category IV. Long-duration sighting by one person or group, no recorded evidence

Long-duration (greater than 10 s) sightings have greater credibility because they allow witnesses more time to assess the situation and rule out ordinary objects like balloons and drones, which together accounted for 91 percent of sightings resolved by the AARO in cases reported from 1996 to 2024 [18].

• Category V. Long-duration sighting by one person or group, with recorded evidence

Same principle as in Category IV: The longer the recording of a UAP sighting, the more opportunity there is for witnesses and non-witnesses to interpret the event.

• Category VI. Long-duration sighting by multiple individuals/groups at different locations or recorded evidence from multiple sensors

In cases where many unrelated people, or multiple instruments such as radars or CCTV cameras, report the same event from different locations, we can be more confident that something was seen and that the report is not a mistake, hoax, or due to a single person’s faulty memory. Multiple observations of the same object do not necessarily ensure a correct identification, however. In the case of sophisticated instruments such as advanced military radars and infrared sensors, it is critically important to check for possible calibration and system errors that could lead to misinterpretation.

• Category VII. Long-duration sighting by multiple individuals/groups at different locations or recorded evidence from multiple sensors; plausibly explained but explanation disputed

The more people report a UAP sighting, the more likely it is that local authorities such as police and military bases, as well as local news outlets and social media, will take notice and provide an explanation. Perhaps it was a meteor, an especially bright auroral display, or some event related to a nearby rocket launch. In cases where a conclusive explanation is offered, the sighting would drop to zero on our reliability scale.

Agreeing on what constitutes a conclusive or even plausible explanation is not always easy, however, in a field where camps of “believers” and “skeptics” distrust each other’s arguments, methods, and sometimes even facts. Add to this the widespread and growing societal problem of misinformation, and sorting truth from fiction becomes even more difficult.

Fortunately, there are a number of individuals and communities, including Metabunk, Skeptoid, the American Institute of Aeronautics and Astronautics, and the Society for UAP Studies, making good-faith efforts to evaluate UAP evidence scientifically, with a critical eye. We are fully aware that even in cases where a plausible explanation is offered, arguments may continue. There is unlikely to be a final, authoritative verdict in some cases. But for many people, the mystery will have been solved.

• Category VIII. Long-duration sighting by multiple individuals/groups at different locations or recorded evidence from multiple sensors; reviewed by experts and so far unexplained

Some UAP cases stubbornly resist explanation, even when the evidence is reviewed by experts (sometimes long after the event). In addition, the majority of AARO’s current case holdings remain unresolved simply due to a lack of data that would allow further analysis [19].

Who should qualify as “expert” reviewers of UAP reports? Although that question is likely to be controversial, we propose that, in the tradition of scientific peer review, they be neutral parties (i.e., people outside the community making the claim) with professional backgrounds in science, engineering, psychology, and other relevant fields. Such a group could be constituted within AARO, provided that the information be shared publicly, to the extent that military secrecy allows. An alternative would be to create a standing committee in a respected body such as the American Institute of Aeronautics and Astronautics or the U.S. National Academy of Sciences. Although it was not tasked to examine evidence for particular UAP claims, the 2023 NASA Unidentified Anomalous Phenomena Independent Study [20] could be a model for such an effort. Whoever the expert reviewers are, we should keep in mind that UAP sightings, by their very nature, pose a problem for scientific study, as they cannot be reproduced in a lab setting. Credibility in many cases hinges on the reliability of witness testimony. Thus, positive proof will be difficult to come by even under the best circumstances.

• Category IX. Long-duration sighting by multiple individuals/groups at different locations or recorded evidence from multiple sensors; physical evidence

Observations that include physical evidence have the highest reliability because they lend themselves more easily to scientific study. Physical evidence can be analyzed in the laboratory and, in ideal cases, judged as to whether it might be of extraterrestrial origin. The full inventory of materials found on Earth, including natural isotope variations, is known with some certainty, particularly for common elements such as hydrogen, carbon, oxygen, and nitrogen [21]. Extraterrestrial material might fall outside this known range—for example, in elemental isotope fractionation or abundance. For example, elements such as iridium that are rare on Earth might be abundant in the sample.

Evidence of technological manipulation could be another possible way to determine extraterrestrial origin. On Earth, advanced tests like infrared spectroscopy or chromato graphy can distinguish between natural and synthetic materials based on their molecular structure. Natural materials tend to have complex, diverse structures and often have inclusions and impurities. Synthetic materials tend to have more defined, uniform molecular compositions with repetitive or homogeneous structures, as they are manufactured to exact specifications.

• Category X. Long-duration sighting by multiple individuals/groups at different locations or recorded evidence from multiple sensors; physical evidence; reviewed by experts and so far unexplained

Category X observations include physical evidence for which most experts cannot come up with a reasonable explanation other than it being a non-natural, extraterrestrial object. Despite many claims that extraterrestrial artifacts—including bodies of alien visitors—have been recovered and retained, no convincing evidence has ever been presented to the general public for evaluation by outside experts. This is the highest bar of our reliability scale, which has not yet been reached to our knowledge.

4. Discussion

The objective of this paper is to prioritize truly difficult-to-explain UAP observations so that researchers can focus more efficiently on those few cases that merit detailed investigation. The proposed classification system gives the highest rank to cases where there is physical evidence, because only then can the scientific method be employed best. Without physical evidence, the highest classification is Category VIII. As a result, events ranked higher on the scale are not necessarily more difficult to explain. We offer here two examples of well-known UAP cases, with our proposed ranking, to illustrate difficulties and challenges in using the scale.

4.1. Council Bluffs, Iowa, 1977

This well-known UAP case from 1977 [22] is one of the few instances where physical evidence has been evaluated in the lab. On the evening of 17 December 1977, three people reported seeing a red, luminous object falling from the sky over Council Bluffs, Iowa. When the object made contact with the ground, the witnesses saw a bright flash and flames shooting approximately three meters high. They then drove to the crash site, where they recounted seeing a glowing orange blob with a bluish crystalline substance at its center [23]. The blob ignited a small grass fire, and firemen were called to the scene. Several other witnesses, including police, reported seeing the object in the sky before it fell. Officials at nearby Eppley Airfield and Offutt Air Force Base denied knowledge of any aircraft crash.

The site of the crash was thoroughly investigated at the time, and samples of material were recovered. Decades later, Nolan et al. [24] analyzed this material for elemental composition and isotopic ratios of titanium, iron, and chromium. Their conclusion was that the material was most similar to terrestrial carbon steel.

Various explanations for the sighting have been entertained, from a meteorite or satellite impact to a hoax. Given that there were several unrelated witnesses, the latter is unlikely. Due to the presence of multiple witnesses and the recovery of physical remnants, the sighting would initially be ranked as a Category IX event. But because the material appears to be engineered by humans, it could plausibly have been a piece of space debris fallen to Earth or perhaps classified technology being tested at one of the nearby air force bases, despite official denials. This is a good example of a case where military secrecy may obstruct full investigation of UAP cases that otherwise would be downgraded to Category 0 (fully explained) if the full facts were known.

Military secrecy may also have figured in a case near Haynesville, Louisiana, that occurred on 30 December 1966 [25]. A luminous object witnessed at ground level led to charred tree bark that had been left as physical evidence. In a recent study, Vallée et al. [26] estimated that the trees were exposed to temperatures of about 500 °C and a radiative output of 500 to 1400 MW. If their simulation is correct, only a military experiment or an extraterrestrial source seems likely.

4.2. Nimitz “Tic Tac” Encounters, 2004

In 2004, a number of mysterious sightings and radar observations were reported by members of the USS Nimitz Carrier Group off the coast of southern California. Radar tracking data showed UAP at 24,000 m above sea level before dropping to around sea level in less than a second and hovering. When fighter jets were launched to investigate, several pilots reported encountering a white oval object about 12 m long, which over a period of 5 min appeared to match the flight path of their airplanes before disappearing. A second group of fighter pilots sent to investigate had an advanced infrared camera on board and recorded video of the encounter. Several of these videos are available, as well as interviews with the pilots [27].

Mundane explanations for the incident include errors in interpreting the video evidence, human observational errors (e.g., parallax illusion), weather balloons, and drones or other types of aircraft. Sensor failure is certainly possible, although several different systems would have all had to malfunction, including shipborne radars and the fighter jets’ own electro-optical sensors. Another possible explanation could be “radar spoofing”, in which an aircraft (possibly from an adversary nation) could have intentionally sent out spurious signals to expose weaknesses in US electronic warfare systems [28].

Given the large amount of data and eyewitness testimony available, the Nimitz case represents one of the most puzzling UAP sightings to date. However, with no physical evidence, the highest rating it can be assigned on our scale is a Category VIII (or perhaps Category VII if the spoofing hypothesis is considered plausible). Thus, in some way our scale ranks the type and quality of data that can be analyzed more than it does the extent to which a sighting is truly mysterious.

The case studies presented here highlight a familiar problem facing UAP researchers: military secrecy. Those in charge of classified projects understandably are reluctant to reveal details about new technologies under development or to give the technical specifications of sensors used on advanced military aircraft. Without such information, outside experts have no way to determine the true nature of some sightings. One answer is to declassify certain information related to past UAP reports, but that solution may be slow in coming.

A final point: Our proposed scale is most applicable to modern (including future) UAP observations. Historic accounts such as one associated with the Battle of Phrygia (central Turkey) in 74 BC [29] or the phenomena observed in Nuremberg on 14 April 1561 (e.g., [30]) remain extremely difficult to evaluate. The latter is one of the few historic cases for which some type of evidence (a woodcut illustration) is still available. The highest this case could rank is Category V; however, any follow-up investigation is all but impossible.

5. Conclusions

We propose a reliability scale for UAP reports based on three factors: (1) the type and quality of the evidence; (2) whether the sighting has been researched to rule out common misidentifications; and (3) whether the event remains unexplained after expert analysis. One cornerstone of our suggested approach is that any UAP witness, particularly members of the general public, should first check their observation against available online databases to assess whether it might be explained as being due to natural or known technological causes. This will help the scientific community focus on truly difficult-to-explain reports, which eventually may result in new insights about their origin.

We start from the premise that most UFO sightings are misunderstandings of ordinary objects and events. Unfortunately, most reports to date have lacked enough data to prove this one way or another. This may be an intractable problem given the random and unpredictable nature of UAP sightings. But the situation should improve if a well-publicized reporting mechanism becomes available that includes online resources the public can consult when they see something strange in the sky. We hope that the scale is adopted by some organizations involved in UAP research and look forward to seeing how it works with a large number of real-world cases.