Autonomous Mission Planning Method for Optical Imaging Satellites Based on Real-Time Cloud Cover Information

Abstract

Cloud cover is an important factor limiting the earth observation efficiency of optical imaging satellites. Existing solutions include avoiding cloudy observation time windows by onboard cloud detectors and ground monitors, which are difficult to improve satellite observation efficiency in time. In order to solve the problem, firstly, a Geostationary Earth Orbit (GEO) and Low Earth Orbit (LEO) satellites cooperation scheme by using cloud cover information provided by GEO meteorological satellite to guide the imaging of LEO optical satellites is proposed, and the operation flow and key elements in this scheme are analyzed. Secondly, Fengyun-4 GEO meteorological satellite and its cloud mask (CLM) products are analyzed. Thirdly, an autonomous mission planning algorithm based on real-time cloud cover information is proposed. Computational results have demonstrated the effectiveness of the proposed GEO–LEO satellites cooperation scheme by taking the actual orbit and payload data of Fengyun-4 and Gaofen-1/2 satellites as examples.

1. Introduction

Satellites with optical imaging sensors observe earth surface targets and the lower atmosphere through onboard optical cameras to obtain image data. Common optical imaging satellites include the U.S. KH-12 satellite, the French Pléiades satellite, and China Gaofen-1/2 satellite. The optical remote sensing images have advantages including high spatial resolution, conforming to the visual characteristics of the human eye and are easy to interpret. Consequently, optical images are widely used in military reconnaissance, land and resource survey, environmental monitoring, and other fields. However, optical sensors are passive imaging and have weak penetration ability with clouds, so cloud cover becomes an important factor limiting the earth observation efficiency of optical imaging satellites. According to the International Satellite Cloud Climatology Project [1], the global average cloud cover reaches about 65%, and many optical imaging satellites are affected. For instance, due to cloud cover, about 80% of the observation tasks of the French SPOT satellite fail [2], and over 60% of the optical satellite images in China cannot be effectively captured [3]. When the ground targets are obscured by clouds, it is not only difficult to obtain high-quality satellite images due to the lack of information, but also the corresponding observation time window, onboard storage and power are wasted. Furthermore, satellite–ground data downlink resources are occupied when transmitting the such cloud-covered images. Therefore, how to effectively avoid the impact of cloud cover, acquire high-quality image data and improve the efficiency of satellite observation is a key problem that needs to be solved in the practice of optical imaging satellite applications.

Many previous studies have concentrated on cloud cover problem in optical imaging satellite applications. The solutions in early studies can be summarized into three categories.

Firstly, during the ground mission planning period of imaging, the weather information provided by meteorological departments is considered to generate robust observation schedule based on a priori information on cloud prediction [4]. Liao et al. [5] developed a stochastic integer planning model considering cloud cover based on current meteorological information, and solved by using a Lagrangian relaxation algorithm to obtain the basic scheme, with a rolling adjustment strategy is used to adjust the basic scheme in real-time according to the latest cloud cover conditions. He et al. [6] constructed a spatial geometric model of cloud cover, in the model, the cloud cover time window is obtained by spatial analytical calculations, and then a scheduling model considering cloud cover is established. A heuristic algorithm based on maximizing the observation reward is used for the solution. He et al. [7] established a cloud cover information description model, and designed a cloud cover time window calculation method based on pre-judge and dichotomy, built an agile satellite (i.e., with steerable imaging sensor) mission planning model considering cloud cover information and solved the model with ant colony algorithm [8]. Wang et al. [9,10] studied the optical imaging scheduling problem under cloud cover uncertainty. Based on the a priori information of cloud prediction, stochastic expectation value model and branch pricing algorithm, chance constrained planning model and branch cutting plane/column generation heuristic algorithm, robust model, and exact/random sampling heuristic algorithm were designed. However, the methods mentioned above do not consider the real-time changes of cloud state.

Secondly, during the in-flight imaging period, the cloud cover information is obtained via the onboard cloud detector and the observation schedule is adjusted dynamically. Beaumet et al. [2] studied the autonomous decision making problem of Pléiades satellite which has an onboard cloud detector. In the proposed scheme, cloud cover information in front of the satellite is obtained with the forward-looking cloud detector, and then a reactive/deliberative framework can be achieved through an onboard autonomous decision. He et al. [11] considered an agile satellite with flexible attitude can avoid clouds during imaging using a synchronous cloud detector, and proposed a hierarchical scheduling method for the real-time scheduling problem. The method divided the scheduling process into three steps: pre-assignment, rough scheduling, and fine scheduling, which can compose a hierarchical scheduling algorithm based on ant colony algorithm. Wang, et al. [12] considered high resolution satellites operating on the same orbit, which had wide-swath images and providing the cloud cover information close in time. Then, the model of avoiding clouds under satellite orbital space coordinate system were established to guide the online mission planning of the high spatial resolution satellite. Wagstaff et al. [13] studied a machine learning method for onboard cloud detection and used real-time cloud cover information to guide the autonomous planning of the satellite. However, the limitation of the methods described above are that the optical satellites require incorporating a synchronous payload to obtain cloud cover information, which lacks universality in practical applications.

Thirdly, during the ground data processing period of LEO satellite, cloud detection [14] provides the cloud mask firstly, and then images with much cloud would be discarded. Also, enhancement of optically thin cloud scenes [15], and restoration of thick cloud scenes [16] can be used to recover image information from cloud cover scenes. The main methods of cloud detection include threshold segmentation, texture analysis, change detection, and integrated detection. The threshold segmentation method is based on the difference between cloud and surface brightness values, and cloud area extraction is achieved by threshold segmentation. The selection of the threshold value is very critical, and the fixed threshold value is only applicable to specific image scenes, from which the threshold segmentation method has been gradually developed into dynamic threshold segmentation [17], and multi-band combined threshold segmentation [18], etc. The threshold segmentation method has the advantages of simplicity and ease of implementation, but it is prone to misjudgment for high brightness cloud-like surface targets (e.g., ice, snow, etc.). The texture analysis method identifies clouds based on the similarity within similar features and the discontinuity of boundaries between different features, such as the fractal dimension method [19] and the gray level co-occurrence matrix method [20]. The detection accuracy using only texture features is limited due to the large variation of cloud features. The change detection method uses clouds as a change target in the image and uses two or more images with similar temporal phases of the same area [21], which has the advantage of high detection accuracy and the disadvantage of requiring images in the same regions, short temporal phases, and strict geometric consistency. Integrated detection methods combine radiometric, geometric, and temporal features for cloud detection, such as support vector machines [22], neural networks [23], and object-oriented image analysis [24]. With the development of deep learning and its successful application in the field of computer vision, cloud detection based on deep learning has attracted wide attention. The main methods include full convolutional networks [25], multi-scale convolutional feature fusion [26], super-pixel segmentation combined with convolutional neural networks [27]. Ground data processing methods can improve the utilization of cloud cover images to a certain degree, but they essentially belong to the data processing after mission planning, which is also wasting resources of imaging and data downlink. If the cloud cover rate of images is high, it is difficult to produce effective image products. In addition, the area obscured by clouds is usually resubmitted as a new observation requirement. So the whole imaging task would incur additional costs, such as one day or as long as several weeks, which greatly reduces the timeliness of image data due to the constraints of satellite tracking, telemetry and control, data downlink, and other factors.

Geostationary (GEO) meteorological satellites orbit the Earth above a particular location on the Equator at an altitude of about 36,000 km and provide “full-disk” (about 1/3 of the Earth’s surface area) multi-spectral remote sensing images every 15 min or shorter. In recent decades, the spatial resolution of GEO meteorological satellite images has increased to better than 0.5 km level, and the spectral bands have extended from a few panchromatic and thermal infrared bands to cover many visible, near-infrared, mid-infrared and far-infrared bands, e.g., Chinese Fengyun-4 and Japanese Himawari-8 satellites have as many as 14 spectral bands, for example. The GEO meteorological satellite CLM products have ultra-high temporal resolution, middle spatial resolution, and high cloud identification accuracy.

To address the cloud cover problem, a solution that uses cloud cover information provided by GEO meteorological satellites to guide the mission planning of LEO optical imaging satellites is proposed in this paper. The basic ideas of the solution include two parts. One is that the GEO meteorological satellite acquires multispectral ‘full-disk’ remote sensing images, and generates CLM products with real-time cloud detection algorithms by using a special fast processing chip on board. And the other is that the GEO meteorological satellite can judge the cloud over situation of the target area in real-time based on the CLM products, interact with the LEO optical imaging satellites through the inter-satellite communication link, complete the autonomous cooperative mission planning, and dynamically adjust the observation tasks of the LEO optical imaging satellites. The solution aims to effectively solve the cloud cover problem and improve the imaging efficiency of the optical imaging satellites.

In this paper, a GEO–LEO satellites cooperation scheme is proposed and the operation flow and key elements in this scheme are analyzed. By using the Fengyun-4 CLM products, an autonomous mission planning algorithm based on real-time cloud cover information is developed. Then satellites data of Fengyun-4 and Gaofen-1/2 satellites are used as examples to perform an experimental application of the proposed GEO–LEO satellites cooperation scheme. Concluding remarks are summarized in the final section.

2. Materials and Methods

2.1. Fengyun-4 Meteorological Satellite and Its CLM Products

Fengyun-4A (FY-4A for short) is the latest generation of GEO meteorological satellite in China. It was successfully launched from Xichang Satellite Launch Center on 11 December 2016. FY-4A realizes the combination of GEO observation and infrared hyperspectral atmospheric vertical detection. It takes only 1 min to perform special area scanning, which greatly improves the temporal as well as spatial resolution of earth observation. The sensors loaded on FY-4A satellite mainly include Advanced Geosynchronous Radiation Imager (AGRI), Geosynchronous Interferometric Infrared Sounder (GIIRS), Lightning Mapping Imager (LMI), etc. [28]. With high temporal and spectral resolution, FY-4A satellite can realize all day long multi-dimensional remote sensing monitoring [29]. FY-4A satellite provides a large amount of level 2 product data, mainly including atmospheric vertical detection, real-time cloud detection, cloud types, and other various types of product, which can provide fast and effective data support for agriculture, forestry, industry, transportation, and other fields [30].

The FY-4A satellite ARGI has a wide range of meteorological products, and CLM products are one of the important products. The FY-4A CLM are produced mainly using 6 spectral bands of the ARGI, including 0.65 μm, 3.75 μm, 7.1 μm, 10.8 μm, and 12.0 μm bands. At the same time, a variety of supplementary data sources are combined to distinguish between clouds and clear sky by using the different features exhibited in each spectral bands [28,31]. The CLM product, with a 4 km spatial resolution, contains the value of the full-disk with a grid size of 2748 × 2748 pixels, and the digital number ‘0’ for cloud, ‘1’ for possible cloud, ‘2’ for possible clear sky, ‘3’ for clear sky, ‘126’ for blank, and ‘127’ for invalid fill. A binary image of CLM is shown as Figure 1.

The satellite data used in this study are obtained from the official data website of the National Meteorological Satellite Center. The selected data used is the L2 CLM product data from the FY-4A with 4km resolution. The time range is from ‘00:00:00 1 March 2021’ to ‘00:00:00 2 March 2021’. It is worth noting that we assume that cloud detection is done onboard with mature machine learning based algorithms and special processing chips.

2.2. Methods

2.2.1. GEO–LEO Satellites Cooperation Scheme

The overall scheme of using the cloud cover information provided by the GEO meteorological satellites to guide the mission planning of the LEO optical imaging satellites consists of an operational workflow and a GEO–LEO satellites cooperation scheme framework, as shown in Figure 2 and Figure 3, respectively.

In this strategy, the ground control center develops a 24-h satellite observation schedule based on the imaging requirements and transforms it into payload commands, and then the ground stations upload the commands to the GEO meteorological and LEO optical imaging satellites. During the execution of the observation schedule by the LEO optical imaging satellites, the GEO meteorological satellite performs autonomous mission planning for the LEO satellites based on the acquired cloud cover information dynamically.

In the above framework, satellites should have different levels of intelligent and autonomous capabilities. The concept of satellite intelligence was considered initially by Schetter et al. (2003) [32]. They performed comparisons on different levels of satellite intelligence. Four levels of satellite intelligence have been identified.

(1)

I₁ represents the most “intelligent” agent. Here the “agent” means one intelligent satellite in the satellite system composed by single satellite or multiple satellites. The primary difference between I₁ and the other satellite agents is that it is capable of monitoring all satellite agents in the system and planning for the system as a whole;

(2)

I₂ can interact with other satellite agents in the system. This usually requires the agent to have at least partial knowledge of the full agent-based system;

(3)

I₃ has local planning functionalities onboard. “Local” means the agent is capable of generating and executing only plans related to its own tasks; and

(4)

I₄ represents the most “unintelligent” agent. It can only receive commands and tasks from other satellite agents in the system, or from the ground, and execute them.

Based on the identification of the satellite intelligence levels, we define a GEO meteorological satellite as I₁ level, and LEO optical imaging satellites as I₃ level in the GEO–LEO satellites cooperation scheme. The GEO meteorological satellite has a strong computing and autonomous capability, and the onboard data contains orbital data of LEO satellites, as well as a multi-satellite cooperation mission planning capability. The LEO satellites are defined as a partially autonomous capability satellite, which can report their own status data to the GEO meteorological satellite, and also receive mission commands from the GEO meteorological satellite and execute action sequences autonomously.

The ways of coordination architectures for multiple satellites include top–down coordination, centralized coordination, distributed coordination, and fully distributed coordination. The top–down coordination architecture includes only one single (highly intelligent) I₁ level agent, and the other satellite are (unintelligent) I₄ agents. The centralized coordination architecture requires at least local planning and, possibly, interaction capabilities between satellite, requiring I₃ or I₂ agents. The distributed coordination architecture consists of several parallel hierarchical decision-making structures, each of which is commanded by an I₁ intelligent agent. In the case of a fully distributed coordination architecture, each satellite in the system represents an I₁ level agent, resulting in a totally flat system [33]. In the GEO–LEO satellites cooperation scheme, we define the coordination way between satellites as centralized coordination.

2.2.2. Mathematical Model

In the cooperation scheme of GEO–LEO satellites, autonomous planning is performed on the basis of an initial observation schedule generated by ground control center. The GEO meteorological satellite dynamically adjusts the observation tasks of multiple LEO optical imaging satellites based on real-time cloud cover information to obtain a new observation scheme. Due to the complex temporal constraints and high coupling between observation tasks, local adjustments may have influence on subsequent tasks, resulting in a chain reaction. How to design autonomous planning algorithms with small perturbations and high efficiency is one of the key issues.

Based on the above analysis, we establish the mathematical model of the autonomous planning problem. A summary of notations is presented as

Table 1. Definitions of the problem’s notations.

Notations	Definitions
T	$\mathrm{Observation} \mathrm{tasks} \mathrm{set}, T=\left\{t_1,t_2,\dots ,t_{N_T}\right\}$$,N_T$ represents the number of tasks
S	$\mathrm{EOS} \mathrm{resources} \mathrm{set}, S=\left\{s_1,s_2,\dots ,s_{N_S}\right\}$$,N_S$ represents the number of EOSs
$C^n$	$\mathrm{Orbits} \mathrm{set} \mathrm{of} s_n$$,C^n=\left\{c_1,c_2,\dots ,c_{N_C^n}\right\}$$, N_C^n$ represents the number of orbits
$b_{ki}^n$	$\mathrm{Observation} \mathrm{start} \mathrm{time} \mathrm{of} \mathrm{task} t_i$$\mathrm{at} \mathrm{the} k\mathrm{th}$$\mathrm{orbit} \mathrm{of} \mathrm{satellite} s_n$
$q_{ki}^n$	$\mathrm{Observation} \mathrm{end} \mathrm{time} \mathrm{of} \mathrm{task} t_i$$\mathrm{at} \mathrm{the} k\mathrm{th}$$\mathrm{orbit} \mathrm{of} \mathrm{satellite} s_n$
$o_{ki}^n$	$\mathrm{Observation} \mathrm{time} \mathrm{of} \mathrm{task} t_i$$\mathrm{at} \mathrm{the} k\mathrm{th}$$\mathrm{orbit} \mathrm{of} \mathrm{satellite} s_n$
$d_i$	$\mathrm{Observation} \mathrm{duration} \mathrm{of} \mathrm{task} t_i$
$p_i$	$\mathrm{Priority} \mathrm{of} \mathrm{task} t_i$
$u_{ij}^{nk}$	$\mathrm{Attitude} \mathrm{maneuver} \mathrm{duration} \mathrm{from} \mathrm{task} t_i$$\mathrm{to} t_j$$\mathrm{at} \mathrm{the} k\mathrm{th}$$\mathrm{orbit} \mathrm{of} \mathrm{satellite} s_n$
$e_u$	Energy consumption of satellite attitude maneuver per unit time
$t_{st}$	Satellite attitude stabilization time
$e_{st}$	Energy consumption of satellite attitude stabilization
$m_o$	Memory consumption of observation per unit time
$e_o$	Energy consumption of observation per unit time
$M_n$	Memory capacity per orbit
$E_n$	Energy capacity per orbit

.

We developed two decision variables in the model, which are as follows:

$$x_{ki}^n=\{\begin{array}{c} 1, \mathrm{if} \mathrm{task} t_i \mathrm{is} \mathrm{assigned} \mathrm{at} \mathrm{the} k\mathrm{th} \mathrm{orbit} \mathrm{of} \mathrm{satellite} s_n\\ 0,\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em} \mathrm{otherwise} \hfill \end{array}$$

(1)

$$y_{ij}^{nk}=\{\begin{array}{c}1, \mathrm{if} \mathrm{task} t_j \mathrm{is} \mathrm{the} \mathrm{immediate} \mathrm{successor} \mathrm{of} \mathrm{task} t_i \\ \hspace{1em}\hspace{1em} \mathrm{at} \mathrm{the} k\mathrm{th} \mathrm{orbit} \mathrm{of} \mathrm{satellite} s_n\\ 0,\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em} \mathrm{otherwise} \hfill \end{array}$$

(2)

The objective of optimization is maximizing the schedule profits of all selected tasks.

$$F=max{\displaystyle \sum }_{n=1}^{N_S}{\displaystyle \sum }_{k=1}^{N_O^n}{\displaystyle \sum }_{i=1}^{N_T}{x}_{ki}^n\cdot p_i$$

(3)

where F means maximizing the schedule profits of all selected tasks.

The operations of EOSs should satisfy following constraints:

(1)

Each task can be observed at most once.

$$\forall t_i\in T:{\displaystyle \sum }_{n=1}^{N_S}{\displaystyle \sum }_{k=1}^{N_C^n}{\displaystyle \sum }_{i=1}^{N_T}{x}_{ki}^n\le 1$$

(4)

$$\forall t_i,t_j\in T,i\ne j:{\displaystyle \sum }_{n=1}^{N_S}{\displaystyle \sum }_{k=1}^{N_C^n}{\displaystyle \sum }_{i=1}^{N_T}{x}_{ki}^n={\displaystyle \sum }_{n=1}^{N_S}{\displaystyle \sum }_{k=1}^{N_C^n}{\displaystyle \sum }_{i=1,j=2}^{N_T}{y}_{ij}^{nk}$$

(5)

(2)

Each task must be observed within its available time windows.

$$\forall t_i\in T,x_{ki}^n=1:b_{ki}^n\le o_{ki}^n\le q_{ki}^n$$

(6)

(3)

Setup time between contiguous tasks must be sufficient for transition.

$$\forall t_i,t_j\in T,i\ne j,x_{ki}^n=y_{ij}^{nk}=1:o_{kj}^n\ge o_{ki}^n+d_i+u_{ij}^{nk}+t_{st}$$

(7)

(4)

The memory consumption cannot exceed maximum memory capacity for each orbit.

$$\forall s_n\in S,\forall c_k\in C^n:{\displaystyle \sum }_{i=1}^{N_T}{m}_{ob}\cdot d_i\cdot x_{ki}^n\le M_n$$

(8)

(5)

The energy consumption cannot exceed maximum energy capacity for each orbit.

$$\begin{array}{c}\forall s_n\in S,\forall c_k\in C^n,\forall t_i,t_j\in T,i\ne j:\\ {\displaystyle {\displaystyle \sum }_{i=1}^{N_T}}{e}_{ob}\cdot d_i\cdot x_{ki}^n+{\displaystyle {\displaystyle \sum }_{i=1}^{N_T}}{\displaystyle {\displaystyle \sum }_{j=2}^{N_T}}\left(e_u\cdot u_{ij}^{nk}+e_{st}\right)\cdot y_{ij}^{nk}\le E_n\end{array}$$

(9)

The attitude transition activities in agile imaging satellite are a temporal constraint problem. During imaging acquisition activity, after a ground target is selected, the starting time will be a continuous variable within a visible time window, and different starting time corresponds to different attitude angles, furthermore, different attitude angles mean different imaging quality. During attitude transition activity, for contiguous targets, the satellite needs to complete attitude transition, and the transition duration depends on the ending and starting time of previous and next targets, different transition duration means different energy consumption. Consequently, attitude angle, imaging quality, attitude transition duration, energy consumption are closely related to time. In previous work, we have studied the temporal constraint model and designed calculation methods [34]. And we used the model and methods in this paper.

This mathematical model includes the optimization objective of maximizing the planning rewards of all selected tasks and takes operational constraints into consideration, such as the time window, the attitude transition, and onboard memory and energy constraints, meanwhile, it is characterized by high complexity.

2.2.3. Planning Cycle

Task-driven strategy means how to determine the start-up time of autonomous planning. Currently, three types of strategies are commonly used: cycle-driven, event-driven, and hybrid-driven. Cycle-driven makes planning decisions according to a predefined time interval T. Event-driven makes planning decisions based on events, meaning decisions are started up as soon as an event occurs. Hybrid drive considers both cycle-driven and event-driven, meaning decisions are started up according to a certain time interval, also when a major event occurs.

GEO meteorological satellite usually acquire ‘full-disk’ cloud cover images at fixed intervals (e.g., every 15 min for FY-4A). Based on this ability, we use cycle-driven autonomous planning, where the planning period is the time interval at which the meteorological satellite acquires cloud cover images. It should be noted that this interval is not fixed as 15 min, but it depends on the actual time of the acquisition by the meteorological satellite.

2.2.4. Onboard Autonomous Planning Algorithm

Onboard autonomous planning is a process of response to new tasks dynamically, which requires high timeliness and optimality of a planning algorithm. Heuristic algorithm is intuitive, easy to understand and efficient in solving [35]. It can give feasible solutions with high rewards in a short time and low computing consumption, which meets the requirements of onboard autonomous planning. Therefore, we propose a heuristic algorithm for onboard autonomous planning. The proposed algorithm consists of matching the longitude and latitude of the target regions (observation tasks) with the pixel of the CLM, computing cloud cover rate and screening out tasks requiring adjustment, selection of the tasks and observation windows, and local neighborhood search.

(1)

Matching the longitude and latitude of the targets with the pixel of the CLM. Each ground target has a latitude and longitude. In order to determine the cloud over the target region, the latitude and longitude of the target are matched with the pixels in the CLM products to obtain accurate cloud cover rate in pixel units. The resolution of the CLM products is 4 km, and supplementary data ‘FullMask_Grid_4000.raw’ has a size of 2748 × 2748. The process of matching the longitude and latitude of the targets with the pixel region of the CLM is as following: Firstly, finding the corresponding pixel point in the ‘FullMask_Grid_4000.raw’ data based on the central latitude and longitude of targets. Secondly, matching the pixel point with the CLM pixel area. Thirdly, according to the size of the target, calculating the corresponding pixel area. For example, the actual ground size of the target is 60 km × 60 km, which is 15 × 15 pixel area in the CLM image;

(2)

Computing cloud cover rate and screening out tasks requiring adjustment. The International Satellite Cloud Climatology Project gives the cloud level according to the cloud cover of remote sensing images: cloud cover below 35% is set as low cloud level; cloud cover between 35% and 65% is set as medium cloud level; cloud cover above 65% is set as high cloud level. Referring to this cloud level, we set the cloud cover threshold to 65%, and after each acquisition of CLM images, we judge the cloud cover over all observed targets in this planning cycle. If the cloud cover is larger than 65%, the observation plan of this task in the current planning cycle is cancelled and the task is added to the set of pending tasks;

(3)

Selection of tasks and observation windows. The set of pending tasks consists of multiple new tasks, the tasks and observation windows need to be selected according to certain rules. Here the selection of new tasks is based on the task demand degree, which indicates the urgency of the task planning, and it is calculated using the following equation.

$${\rho }_i=\frac{p_i}{T{W}_i}$$

(10)

where $p_i$ is the priority of task $i$, $T{W}_i$ is the number of available time windows of task $i$.

The available time windows at this point are the time windows after the current planning. Tasks with higher priority and fewer observation windows have a higher task planning urgency. That is, the higher value ${\rho }_j$, the higher the task demand should be.

For the selection of observation windows, it is considered that a task often has multiple observation windows. Here, the selection is based on the temporal order of the observation windows, i.e., the observation window with an earlier start time should be selected first; and

(4)

Local neighborhood search. Given that the neighborhood of the current solution often contains better neighborhood solutions, a local neighborhood search can be performed on the current solution. Local neighborhood search is a kind of heuristic algorithm which has the advantages of timeliness, high efficiency and succinctness. In this paper, insertion neighborhood search is used to arrange the current tasks. Insertion neighborhood means inserting the pending tasks that satisfy operational constraints into the current task sequence, and the basic steps are as follows:

Step 1: Sort the pending tasks in descending order of priority. Select the first one, and calculate of the position of the observation window of the pending task in current task sequence of the satellite;

Step 2: Build a task tabu list to conserve the planned tasks, which means the planned task will not be considered again;

Step 3: Judge whether the operational constraints are satisfied between the pending task and the adjacent tasks before and after, and, if so, insert the task into the current task sequence and write it down in the tabu list. In addition, if the observation window of the task is located before the first task or after the last task, only the constraint satisfaction relationship with the first and last task needs to be judged;

Step 4: Update the planning rewards and consumed storage and power; and

Step 5: Judge whether there are tasks in pending task set, and if so, repeat Steps 1–4. Otherwise, exit the algorithm.

Based on the above design, the flowchart of the onboard autonomous planning algorithm is shown in Figure 4.

2.3. Experimental Study

In order to verify the effectiveness of the GEO–LEO cooperation scheme proposed in this paper, we construct a scenario conforming to the actual application based on the actual orbit and payload parameters of FY-4A and GF-1/2 satellites in

Table 2. Main parameters of FY-4A satellite.

Parameters	FY-4A
Orbit type	GEO
Orbit attitude	36,000 km
Longitude of sub-satellite point	104.7°E
Spatial resolution under full-disk	4 km
Time resolution	15 min

and

Table 3. Main parameters of and GF-1/2 satellite.

Parameters	GF-1	GF-2
Orbit type	Sun-synchronous Orbit	Sun-synchronous Orbit
Orbit attitude	645 km	631 km
Orbit inclination	98.0506°	97.9080°
Swath	60 km	45 km
Maximum rolling angle	±35°	±35°
Spatial resolution (PAN, MSS)	2 m, 16 m	1 m, 4 m

. The CLM/FY-4A data, which has low spatial resolution (4 km) but very high temporal resolution (15 min), provide cloud information covering the same spatial area for the whole swath of GF-1 or GF-2. For example, the swath of GF-1 is 60 km × 60 km, which is corresponding 15 × 15 pixels in the CLM image. Among them, GF-1 series satellites include GF-1-01/02/03/04, a total of 4 satellites, and GF-2 is 1 satellite, thus constructing a scenario of 1 GEO meteorological satellite and 5 LEO optical imaging satellites. The time period of the scenario is (1 March 2021 00:00:00, 2 March 2021 00:00:00).

In terms of observation tasks, the point targets are selected as observation tasks, which randomly distribute on the earth surface with latitude among (50°E, 150°E) and longitude among (50°S, 50°N), the number of point targets is 600. All point targets are distributed within the coverage of FY-4A satellite. Each target is associated with a priority uniformly distributed among [1,10] and an observation duration among [5,15].

The data processing and algorithm implementation are performed via MATLAB 2017b on a PC with Intel core i7 CPU @2.0 GHz, and 16 GB RAM under Windows 10 OS.

3. Results and Discussion

To address the problem of cloud cover, we propose a solution that uses cloud cover information provided by GEO meteorological satellites to guide mission planning for LEO optical imaging satellites. In the traditional mission planning, a 24-h observation schedule is developed by the ground control center and uploaded to the GEO–LEO satellites. Thus, the GEO–LEO satellites cooperation is based on an initial schedule. During the execution of the observation schedule by the LEO optical imaging satellites, the GEO meteorological satellite performs autonomous mission planning for the LEO optical imaging satellites based on the acquired cloud cover information. In the scenario, we set the cloud threshold to 65% and extract the time period of CLM from 0:00 to 9:00 (UTC time). The observation time of the GEO meteorological satellite under this time period is from 08:00 to 17:00 at local time, which meets the basic requirement for optical imaging. The number of CLM data during this time period is 15, which means there are 15 planning cycles. The number of tasks adjusted by onboard autonomous planning, the number of tasks successfully adjusted to the cloud-free window, and the number of tasks fail to adjust to the cloud-free window in the 15 planning cycles are shown in the

Table 4. Adjustments of the initial schedule by onboard autonomous planning.

Type	Number of Tasks	Adjustment Ratio
Initial schedule	325	/
Onboard autonomous planning adjustment	229	70.46%
Adjust to cloud-free window	98	30.15%
Fail to adjust to cloud-free window	131	40.31%

.

As can be seen from the table, there are 325 tasks in initial schedule, the onboard autonomous planning adjustment means that if the GEO–LEO satellites cooperation scheme is not used and the LEO optical imaging satellites perform observation according to the initial schedule, 70.46% of the images will contain over 65% cloud cover. By using the GEO–LEO satellites cooperation and autonomous onboard planning, 30.15% (98/325) tasks are successfully adjusted to cloud-free windows, it means the 98 tasks meet the operational constraints of mathematical model and are adjusted under onboard autonomous planning algorithm, which has improved the effectiveness of data acquisition. Meanwhile, 40.31% (131/325) tasks fail to adjust to the cloud-free window, it means the weather conditions of the 131 tasks are highly cloudy (over 65%), meanwhile the 131 tasks cannot meet the operational constraints of mathematical model. The 131 tasks will still be executed according to the initial schedule under high probability of acquiring cloudy images, consider that the images with over 65% may contain the region of interest. In addition, the task adjustments during each autonomous planning cycle are shown in Figure 5.

In the above experiment, we set the cloud cover threshold as 65%. In order to test the influence of different cloud cover thresholds on onboard autonomous planning. We used 10 cloud thresholds from 5–95% with a value taken at 10% intervals, and the test results are shown in the

Table 5. Influence of cloud cover thresholds on onboard autonomous planning.

Cloud Cover Threshold	Number of Tasks in Initial Schedule	Onboard Autonomous Planning Adjustment	Adjust to Cloud-Free Window	Fail to Adjust to Cloud-Free Window	Adjustment Ratio
5%	325	296	131	165	91.08%
15%	325	277	121	156	85.23%
25%	325	266	112	154	81.85%
35%	325	260	111	149	80.00%
45%	325	251	106	145	77.23%
55%	325	244	101	143	75.08%
65%	325	229	98	131	70.46%
75%	325	218	94	124	67.08%
85%	325	202	87	115	62.15%
95%	325	172	72	100	52.92%

.

As can be seen from Table 5, with increases in the cloud cover thresholds, the number of tasks that are adjusted by onboard autonomous planning, the number of tasks adjusted to the cloud-free window, the number of tasks that fail to adjust to the cloud-free window, and adjustment ratio gradually increase. This is because when the cloud cover threshold is larger, fewer observation tasks in the initial schedule reach cloud cover threshold. While the cloud cover threshold becomes smaller, more and more observation tasks reach that threshold. At a 5% cloud cover threshold, 91.08% of the tasks need to be adjusted. From the above data and analysis, we know that the setting the cloud cover thresholds influence the adjustments of the initial schedule while performing onboard autonomous planning, and it can be determined according to the actual demand.

In a nutshell, the GEO–LEO satellites cooperation scheme can improve the effectiveness of data acquisition and reduce the amount of redundant observation data downlink of LEO optical imaging satellites, which provide a new solution for solving cloud cover problem in the application of optical imaging satellites.

4. Conclusions

In this paper, a GEO–LEO satellites cooperation scheme was proposed that uses the cloud cover information provided by GEO meteorological satellites to guide the autonomous mission planning of LEO optical imaging satellites to address the cloud cover problem. At the same time, the intelligence degree of different satellites in the scheme and the ways of coordination between GEO–LEO satellites were analyzed. Under the scheme, the characteristics of the FY-4A satellite and the application of CLM data were analyzed. Furthermore, a mathematical model of onboard autonomous planning was established, and the period of autonomous planning was determined according to the observation characteristics of GEO meteorological satellites. A heuristic algorithm for onboard autonomous planning, which takes into account the timeliness of real-time cloud cover information and the demand for task adjustment was proposed. An experimental scenario with FY-4A and GF-1/2 as satellite resources and 600 ground targets as observation tasks was constructed, and the effectiveness of the proposed GEO–LEO cooperation scheme was verified. The future works of our study are: (1) This study assumes that CLM products from meteorological satellites can be processed onboard. Consequently, onboard real-time cloud detection algorithms which aims to provide fast CLM products needs to be developed; (2) In the GEO–LEO satellites cooperation scheme, we defined a centralized coordination way between satellites. In the future, the highly autonomous satellite system can be performed in a fully distributed coordination, and autonomous mission planning algorithms; and (3) At high latitude area, the GEO–LEO cooperation scheme has application limitation caused by the parallax of the two satellites observations. Such problems are expected to be solved in further detailed studies.