Visual Information Requirements for Remotely Supervised Autonomous Agricultural Machines

: A study to determine the visual requirements for a remote supervisor of an autonomous sprayer was conducted. Observation of a sprayer operator identiﬁed 9 distinct “look zones” that occupied his visual attention, with 39% of his time spent viewing the look zone ahead of the sprayer. While observation of the sprayer operator was being completed, additional GoPro cameras were used to record video of the sprayer in operation from 10 distinct perspectives (some look zones were visible from the operator’s seat, but other look zones were selected to display other regions of the sprayer that might be of interest to a sprayer operator). In a subsequent laboratory study, 29 experienced sprayer operators were recruited to view and comment on video clips selected from the video footage collected during the initial ride-along. Only the two views from the perspective of the operator’s seat were rated highly as providing important information even though participants were able to identify relevant information from all ten of the video clips. Generally, participants used the video clips to obtain information about the boom status, the location and movement of the sprayer within the ﬁeld, the weather conditions (especially the wind), obstacles to be avoided, crop conditions, and ﬁeld conditions. Sprayer operators with more than 15 years of experience provided more insightful descriptions of the video clips than their less experienced peers. Designers can inﬂuence which features the user will perceive by positioning the camera such that those speciﬁc features are prominent in the camera’s ﬁeld of view. Overall, experienced sprayer operators preferred the concept of presenting visual information on an automation interface using live video rather than presenting that same information using some type of graphical display using icons or symbols.


Introduction
The era of autonomous agricultural machines (AAMs) seems to be just around the corner. Engineers and researchers are working diligently to design AAMs that will enable farmers to increase the productivity of their operations. When contemplating the incorporation of automation into an existing machine, [1] presented a model that can be used by design engineers to identify which functions should be automated. Their model is based on the four-stage model of human information processing and proposes that automation can be applied to four distinct types of functions: information acquisition, information analysis, decision and action selection, and action implementation. The engineer's task is to assign the appropriate level of automation (ranging from manual to fully autonomous) for each of the four distinct types of functions. An AAM is likely to have a high level of automation for each of these independent types of functions. Despite the fact that an AAM is designed to automate information acquisition and analysis (using various sensors), to automate decision-making (through well-designed programming or machine learning approaches), and to automate action implementation (through various actuators linked to on-board sensors), it is imperative that design engineers do not of the information to the operation of the machine. Considering the cost and technical challenges associated with wireless transmission of live video and the risk of overcrowding the interface with multiple video displays, it is essential to determine the visual information that is typically used by the machine operator and if the visual information could be replaced with a graphical display (consisting of icons or symbols) on the interface without impacting the perception of the information. Therefore, the primary objective of the study was to determine and evaluate the visual information that is gleaned from the machine and its surrounding environment that assists the operator with the control of an agricultural machine. A secondary objective was to assess whether visual information present in live video could be replaced with a graphical display consisting of icons or symbols to represent various machine parameters. Some examples of graphical displays in conventional agricultural machines can be found in [13,14]; two examples are shown in Figure 1 below. Recognizing that visual information requirements will be different for distinct agricultural machines, the self-propelled agricultural sprayer was selected for this study.
Appl. Sci. 2020, 10, x FOR PEER REVIEW 3 of 16 challenges associated with wireless transmission of live video and the risk of overcrowding the interface with multiple video displays, it is essential to determine the visual information that is typically used by the machine operator and if the visual information could be replaced with a graphical display (consisting of icons or symbols) on the interface without impacting the perception of the information. Therefore, the primary objective of the study was to determine and evaluate the visual information that is gleaned from the machine and its surrounding environment that assists the operator with the control of an agricultural machine. A secondary objective was to assess whether visual information present in live video could be replaced with a graphical display consisting of icons or symbols to represent various machine parameters. Some examples of graphical displays in conventional agricultural machines can be found in [13,14]; two examples are shown in Figure 1 below. Recognizing that visual information requirements will be different for distinct agricultural machines, the self-propelled agricultural sprayer was selected for this study. Figure 1. An example of a graphical display for a sprayer (adapted from [13]).

Identification of Look Zones During Manual Sprayer Operation
Arrangements were made to observe the operator of a high clearance, self-propelled agricultural sprayer (Rogator 1100 AGCO, Duluth, GA, USA) equipped with autosteer technology during the 2017 growing season. The participant who volunteered his sprayer and farmland for this phase of the study was a farm owner with 12 years of spraying experience who also performed custom spraying. One GoPro camera (GoPro, San Mateo, CA, USA), identified as camera M in Figure 2, was mounted on the windshield inside the sprayer's cab to record the head and eye movement of the operator. The video was recorded on the morning of 6 July 2017 on a flat field in Rosser, MB (137 acres in size), for approximately 42 min of spraying time (this was the time taken to cover the entire field). During the spraying operation, the field boundaries were first sprayed in an anti-clockwise direction (to give room for the sprayer to turn easily without going outside the field), before straight parallel swaths were completed to traverse the remaining (unsprayed) area of the field. Subsequent analysis of the video footage from camera M was performed to identify the various look zones from the operator's perspective as described by [14]. One of the co-authors rode along with the sprayer operator to ask questions (i) about the reasons for viewing the various look zones and (ii) the type of information being acquired when viewing the look zones. Although we asked questions during the actual spraying operation rather than interviewing the operator after data analysis had been completed as suggested by [14], this is an important complementary component of the experimental methodology.
To determine the amount of time the operator spent viewing each look zone, an algorithm for converting video recordings to images was developed using Python 2.7.16 (Python Software Corporation, Fredericksburg, VA, USA). The algorithm was used to convert the in-cab video recordings to images, each corresponding to 10 s of the recorded video footage. These images were then manually classified based on their look zones to measure the relative time the operator spent observing each zone.

Identification of Look Zones during Manual Sprayer Operation
Arrangements were made to observe the operator of a high clearance, self-propelled agricultural sprayer (Rogator 1100 AGCO, Duluth, GA, USA) equipped with autosteer technology during the 2017 growing season. The participant who volunteered his sprayer and farmland for this phase of the study was a farm owner with 12 years of spraying experience who also performed custom spraying. One GoPro camera (GoPro, San Mateo, CA, USA), identified as camera M in Figure 2, was mounted on the windshield inside the sprayer's cab to record the head and eye movement of the operator. The video was recorded on the morning of 6 July 2017 on a flat field in Rosser, MB (137 acres in size), for approximately 42 min of spraying time (this was the time taken to cover the entire field). During the spraying operation, the field boundaries were first sprayed in an anti-clockwise direction (to give room for the sprayer to turn easily without going outside the field), before straight parallel swaths were completed to traverse the remaining (unsprayed) area of the field. Subsequent analysis of the video footage from camera M was performed to identify the various look zones from the operator's perspective as described by [14]. One of the co-authors rode along with the sprayer operator to ask questions (i) about the reasons for viewing the various look zones and (ii) the type of information being acquired when viewing the look zones. Although we asked questions during the actual spraying operation rather than interviewing the operator after data analysis had been completed as suggested by [14], this is an important complementary component of the experimental methodology. recording was completed on the morning of 6 July 2017 on a flat field near Rosser, MB (137 acres in size). Video duration was approximately 42 min (which was the time required to spray the entire field). On a separate occasion (21 July 2017), video was recorded from a drone to provide an overhead view of the entire sprayer. A third party licensed drone operator assisted in capturing the overhead view on a field near Moosehorn, MB (120 acres in size); video was recorded for approximately 11 min and covered only a portion of the spraying operation. Cameras were placed on each dot, while arrows indicate the direction of the camera. Note: camera P = clip P; camera 1 = clip 1; camera 2 = clip 2. Camera 5 was mounted on a drone, while cameras 8 and 9 were third party hand-held cameras that were focused on the back of the sprayer and the field, respectively. Camera M was placed inside the sprayer's cab to record the head and eye movement of the operator. To capture the view of the right and left boom, respectively, as perceived from the typical seated position of an operator in the cab of a self-propelled sprayer.
3, 4, 6, 7, 8, and 10 To capture regions around the sprayer that are currently not visible to operators (from their seated position).

and 9
To capture off-field (third party) views and assess their usefulness to the remote supervisor of the autonomous sprayer(s). Cameras were placed on each dot, while arrows indicate the direction of the camera. Note: camera P = clip P; camera 1 = clip 1; camera 2 = clip 2. Camera 5 was mounted on a drone, while cameras 8 and 9 were third party hand-held cameras that were focused on the back of the sprayer and the field, respectively. Camera M was placed inside the sprayer's cab to record the head and eye movement of the operator.
To determine the amount of time the operator spent viewing each look zone, an algorithm for converting video recordings to images was developed using Python 2.7.16 (Python Software Corporation, Fredericksburg, VA, USA). The algorithm was used to convert the in-cab video recordings to images, each corresponding to 10 s of the recorded video footage. These images were then manually classified based on their look zones to measure the relative time the operator spent observing each zone.

Determination of Relevant Visual Information
Nine GoPro cameras (Hero Session, with horizontal field of view of 120 • and a video resolution of 1920 × 1440) were mounted at different locations on a high clearance, self-propelled agricultural sprayer (Rogator 1100) to capture video footage of various regions of the sprayer and its environment while the sprayer was in operation ( Figure 2). Various literature sources [14,15,17] were consulted to position the GoPro cameras on the sprayer. The rationale for the positioning of each camera is provided in Table 1. Some cameras were positioned to provide familiar views (i.e., look zones that a sprayer operator would typically see from the seated position), while other cameras were positioned to provide non-traditional views (i.e., look zones that are not visible from the seated position). Video recording was completed on the morning of 6 July 2017 on a flat field near Rosser, MB (137 acres in size). Video duration was approximately 42 min (which was the time required to spray the entire field). On a separate occasion (21 July 2017), video was recorded from a drone to provide an overhead view of the entire sprayer. A third party licensed drone operator assisted in capturing the overhead view on a field near Moosehorn, MB (120 acres in size); video was recorded for approximately 11 min and covered only a portion of the spraying operation.
Subsequent to collection of the video footage, clips of approximately 60 s in duration were selected from the full video, which showed a large variation in the spraying operation, the sprayer's operational status, and the field or environmental conditions. Experienced sprayer operators (defined as having at least two years of experience operating a sprayer) were recruited from the prairie provinces (Manitoba, Saskatchewan, and Alberta) through industrial partners (Northstar Robotics), university, and farmers' group contacts to participate in this phase of the research. Participants had the choice to either come in person to the Agricultural Ergonomics Lab at the University of Manitoba to complete the experiment or to complete the study online. All participants were required to give their consent prior to the commencement of the experiment. After answering demographic questions, participants were required to view a series of video clips and complete a questionnaire consisting of six questions. In total, eleven video clips were presented to each participant: one practice clip (clip P) and ten test clips (clips 1-10). The practice clip was included to allow participants to familiarize themselves with the experimental protocol and the questions that followed each video. Video clips showing familiar views of the sprayer and its environment (from the operator's seated position) were presented before non-traditional views. Table 1. Rationale for choosing each camera position.

Camera Rationale
M Ability to observe the operator's head and eye movement (from his seated position) during the spraying operation. 1 To capture the field ahead of the sprayer and other environmental cues.

P and 2
To capture the view of the right and left boom, respectively, as perceived from the typical seated position of an operator in the cab of a self-propelled sprayer.
3, 4, 6, 7, 8, and 10 To capture regions around the sprayer that are currently not visible to operators (from their seated position).

and 9
To capture off-field (third party) views and assess their usefulness to the remote supervisor of the autonomous sprayer(s).
The video clip questionnaire comprised six questions. The first three questions were open-ended and required participants to describe: (i) what they saw in each video clip, (ii) the information gained from each video clip, and (iii) how they could use such information during operation of the sprayer. The fourth question asked if the video clip was something the operator typically viewed when operating a sprayer (yes/no response) and whether such a view is useful to the spraying operation (yes/no response). Question 5 required the participant to rank the importance of the visual information perceived using a 5-point Likert scale. In the final question, participants were asked to assess whether the information would be best presented via live video or whether the information could be encoded and incorporated into a graphical display for a sprayer. The video clip questionnaire was completed after viewing each video clip.

Identification of Look Zones during Manual Sprayer Operation
Analysis of the in-cab video footage (camera M) revealed nine look zones: (i) the view ahead of the operator from his seated position (i.e., front view); (ii) right side; (iii) right boom; (iv) left side; (v) left boom; (vi) upper display/mapping system (located at the top right corner from the operator's seated position); (vii) a second display (located close to the right arm rest); (viii) sprayer's dashboard/steering wheel; and (ix) left mirror/other ( Figure 3). The participant was also seen making adjustments to the sprayer settings (through the displays) and checking his phone. Overall, he spent 39% of his time looking at the front view ( Figure 3). This finding is consistent with the results of [15], who noted that sprayer operators spent the most time looking at the field ahead (i.e., front view), while the least attention was given to the dashboard information. When asked why he was looking at these regions, his reasons were (i) to determine how close the booms were to the field boundary, (ii) to assess if the nozzles were working properly, and (iii) to ensure that there were no obstacles close to the field boundaries and along the sprayer's path that would require him to manually raise the boom. He also stated that the front view helped him to determine when to take over the steering wheel to initiate turning at the headland. Appl. Sci. 2020, 10, x FOR PEER REVIEW 6 of 16 Another method for interpreting eye-tracking data is to represent the data as a function of time to give a sense of the cyclical nature of the task [14]. The data collected in this study are presented as a function of time in Figure 4. The data show that the operator was focused primarily on four look zones (front view, upper display, right side, and right boom) during the first 9 min of video footagethis corresponded to the time when he was spraying the field boundary. The front view enables the operator to assess where he is going and when to initiate turning, while the look zones to the right (i.e., right side and right boom) enable the operator to guide the sprayer along the field's boundary. Therefore, when the operator was spraying along the field boundary, his attention was split between looking ahead and watching the right side of the sprayer. Once the field boundary was completed, the time spent viewing the right boom decreased noticeably. Another method for interpreting eye-tracking data is to represent the data as a function of time to give a sense of the cyclical nature of the task [14]. The data collected in this study are presented as a function of time in Figure 4. The data show that the operator was focused primarily on four look zones (front view, upper display, right side, and right boom) during the first 9 min of video footage-this corresponded to the time when he was spraying the field boundary. The front view enables the operator to assess where he is going and when to initiate turning, while the look zones to the right (i.e., right side and right boom) enable the operator to guide the sprayer along the field's boundary. Therefore, when the operator was spraying along the field boundary, his attention was split between looking ahead and watching the right side of the sprayer. Once the field boundary was completed, the time spent viewing the right boom decreased noticeably.
Appl. Sci. 2020, 10, x FOR PEER REVIEW 6 of 16 Another method for interpreting eye-tracking data is to represent the data as a function of time to give a sense of the cyclical nature of the task [14]. The data collected in this study are presented as a function of time in Figure 4. The data show that the operator was focused primarily on four look zones (front view, upper display, right side, and right boom) during the first 9 min of video footagethis corresponded to the time when he was spraying the field boundary. The front view enables the operator to assess where he is going and when to initiate turning, while the look zones to the right (i.e., right side and right boom) enable the operator to guide the sprayer along the field's boundary. Therefore, when the operator was spraying along the field boundary, his attention was split between looking ahead and watching the right side of the sprayer. Once the field boundary was completed, the time spent viewing the right boom decreased noticeably.

Participant Demographics
Twenty-nine experienced male sprayer operators participated in this phase of the research, 73% of which were from Manitoba, 17% from Saskatchewan, and 10% from Alberta. Among these 29 operators, 26 of them completed the study online, while the remaining operators completed the experiment in person. Approximately 50% of the respondents were within the age bracket of 41-65 and self-reported to have more than 6 years of experience operating a sprayer. Almost 30% of respondents were between the ages of 16 and 25, with two-thirds of these individuals having less than 2 years of experience operating a sprayer ( Figure 5). Two of the participants reported that they had not operated a sprayer within the 12-month period prior to participating in this study. Only 10% of respondents were custom applicators, with the remaining respondents being farm owners (and therefore spraying only their own crops). Self-propelled sprayers were operated by 88% of respondents, with the remainder of the respondents using pull-type sprayers. All but one participant used auto-steer navigation technology when spraying.

Participant Demographics
Twenty-nine experienced male sprayer operators participated in this phase of the research, 73% of which were from Manitoba, 17% from Saskatchewan, and 10% from Alberta. Among these 29 operators, 26 of them completed the study online, while the remaining operators completed the experiment in person. Approximately 50% of the respondents were within the age bracket of 41-65 and self-reported to have more than 6 years of experience operating a sprayer. Almost 30% of respondents were between the ages of 16 and 25, with two-thirds of these individuals having less than 2 years of experience operating a sprayer ( Figure 5). Two of the participants reported that they had not operated a sprayer within the 12-month period prior to participating in this study. Only 10% of respondents were custom applicators, with the remaining respondents being farm owners (and therefore spraying only their own crops). Self-propelled sprayers were operated by 88% of respondents, with the remainder of the respondents using pull-type sprayers. All but one participant used auto-steer navigation technology when spraying.

Preliminary Observations from the Practice Clip
A screen shot from the practice video clip (clip P) is shown in Figure 6. Participants described the practice video clip as the left boom of a sprayer applying chemicals on the field (based on mist present in the video). Specific features that were seen included the spray pattern, properly functioning nozzles, stable but relatively low boom height, no visible obstacles in the field, and the presence of green crop (short in height). Participants also noticed that the field was level, had previously been sprayed (given the wheel tracks), and had several regions where no crop was growing. Most participants identified boom features (left boom height, nozzle, and spray pattern) and crop (or field) conditions as the information that was gained from watching the practice clip. A majority (86%) of the participants indicated that the information provided by the practice clip was something they typically viewed while spraying. Among the 14% of participants who did not indicate that such information was typically viewed during spraying, 50% of them felt that it would useful to see the areas provided by the practice clip. When asked how they would use the information gained from the practice clip, participants indicated that the information would help them to identify if nozzles were plugged, to set their guidance line to match the previous passes or tracks (since the field had previously been sprayed), to determine the type of spraying method that would be economical (i.e., adopting full application or spot spraying), and to make navigation decisions (e.g., deciding whether to go into low areas of the field, considering the risk of getting stuck). Two participants also noted that the information would give an operator the assurance that they were doing a good job. In response to the fifth question, 86% of the participants indicated that the information provided by the practice clip was "very important" or "extremely important". Almost two-thirds of the participants (62%) indicated that it would be difficult to gain the same information if the live video was replaced with an encoded display. Overall, analysis of the data from the practice clip gave us confidence that

Preliminary Observations from the Practice Clip
A screen shot from the practice video clip (clip P) is shown in Figure 6. Participants described the practice video clip as the left boom of a sprayer applying chemicals on the field (based on mist present in the video). Specific features that were seen included the spray pattern, properly functioning nozzles, stable but relatively low boom height, no visible obstacles in the field, and the presence of green crop (short in height). Participants also noticed that the field was level, had previously been sprayed (given the wheel tracks), and had several regions where no crop was growing. Most participants identified boom features (left boom height, nozzle, and spray pattern) and crop (or field) conditions as the information that was gained from watching the practice clip. A majority (86%) of the participants indicated that the information provided by the practice clip was something they typically viewed while spraying. Among the 14% of participants who did not indicate that such information was typically viewed during spraying, 50% of them felt that it would useful to see the areas provided by the practice clip. When asked how they would use the information gained from the practice clip, participants indicated that the information would help them to identify if nozzles were plugged, to set their guidance line to match the previous passes or tracks (since the field had previously been sprayed), to determine the type of spraying method that would be economical (i.e., adopting full application or spot spraying), and to make navigation decisions (e.g., deciding whether to go into low areas of the field, considering the risk of getting stuck). Two participants also noted that the information would give an operator the assurance that they were doing a good job. In response to the fifth question, 86% of the participants indicated that the information provided by the practice clip was "very important" or "extremely important". Almost two-thirds of the participants (62%) indicated that it would be difficult to gain the same information if the live video was replaced with an encoded display. Overall, analysis of the data from the practice clip gave us confidence that participants accurately understood the purpose of the research and the questions in the questionnaire.
Appl. Sci. 2020, 10, x FOR PEER REVIEW 8 of 16 participants accurately understood the purpose of the research and the questions in the questionnaire. Figure 6. A screen shot of the practice video clip (clip P).

Information Gained from Test Clips
Screen shots from the ten clips used in this study are shown in Figure 7. Participant responses regarding the information gained from each video clip and how that information could be used (i.e., two of the open-ended questions from the questionnaire) are tabulated in the appendix. A summary of the responses to questions 4-6 is provided in Table 2, with full details provided in the appendix. Only two of the video clips (1 and 2) were typically viewed by a majority of the participants when operating their sprayers ( Table 2). Not surprisingly, these were the only two video clips where a majority of participants identified the level of importance of the information provided to be "very important" or "extremely important". Video clips 3 through 7 were not perceived to provide visual information that is relevant to the task of operating an agricultural sprayer.

Information Gained from Test Clips
Screen shots from the ten clips used in this study are shown in Figure 7. Participant responses regarding the information gained from each video clip and how that information could be used (i.e., two of the open-ended questions from the questionnaire) are tabulated in the Appendix A. A summary of the responses to questions 4-6 is provided in Table 2, with full details provided in the Appendix A. Only two of the video clips (1 and 2) were typically viewed by a majority of the participants when operating their sprayers ( Table 2). Not surprisingly, these were the only two video clips where a majority of participants identified the level of importance of the information provided to be "very important" or "extremely important". Video clips 3 through 7 were not perceived to provide visual information that is relevant to the task of operating an agricultural sprayer.
Appl. Sci. 2020, 10, x FOR PEER REVIEW 8 of 16 participants accurately understood the purpose of the research and the questions in the questionnaire. Figure 6. A screen shot of the practice video clip (clip P).

Information Gained from Test Clips
Screen shots from the ten clips used in this study are shown in Figure 7. Participant responses regarding the information gained from each video clip and how that information could be used (i.e., two of the open-ended questions from the questionnaire) are tabulated in the appendix. A summary of the responses to questions 4-6 is provided in Table 2, with full details provided in the appendix. Only two of the video clips (1 and 2) were typically viewed by a majority of the participants when operating their sprayers ( Table 2). Not surprisingly, these were the only two video clips where a majority of participants identified the level of importance of the information provided to be "very important" or "extremely important". Video clips 3 through 7 were not perceived to provide visual information that is relevant to the task of operating an agricultural sprayer.   The utility of the information provided in the video clips is more difficult to summarize. Generally, participants used the video clips to obtain information about the boom status, the location and movement of the sprayer within the field, the weather conditions (especially the wind), obstacles to be avoided, crop conditions, and field conditions. However, not all video clips were equally effective at providing information for these relevant parameters. Participants were able to gather information regarding the field conditions and travel speed of the sprayer from almost all of the clips. This outcome is not surprising given that the field was present in all the video clips and all clips featured some aspect of the sprayer against the stationary background of the field. It should be noted, however, that information about many of the other parameters was available in only a subset of the clips. Information about boom height, leaks, crop condition, wheel alignment with previous tracks, machine integrity, weather (wind), and obstacles could be determined from five to seven video clips. Information about the sprayer's location, spray pattern, nozzle status (not plugged), spray drift, and how close the boom end was from the field boundary could be determined from only three or four video clips. Features such as sprayer clearance, section control, boom stability, turning at headland, size of the field, object behind the sprayer, field boundary, side of the sprayer, and knowing that the sprayer was in the right field could be determined from only one or two clips. Participants' ability to identify any useful visual information from the sprayer and its surrounding environment was not dependent on how frequently such information was presented. As an example, 28% of participants considered "turning at headland" to be valuable, even though it was only presented in one of the clips (as perceived by participants), while fewer participants identified the "approximate travel speed" as visually important, even though it was perceived in all the video clips (see Appendix A). Table 2. Summary of responses from experienced sprayer operators (n = 29) who completed the video clip questionnaire.
Furthermore, environmental cues were considered important by participants when such cues were presented in relation to the spraying operation. For example, a higher number of participants perceived the field boundary to be valuable information when the end of the boom was visible with respect to the field boundary. However, fewer participants perceived the "field boundary" to be useful in clips that didn't relate it to the spraying operation (boom), despite being visible. In other words, the field boundary is only important with respect to the position of the sprayer-the user does not care about the location of the field boundary unless it is presented in relation to the position of the boom. A summary of the information gained by operators based their look zones is presented in Figure 8.

Effect of Operators' Experience on Information Gained
The years of operating experience of the participants did not influence their responses towards the level of importance of each video clip or with respect to replacing the clips with a graphical display; however, it did influence the responses to the open-ended questions in the questionnaire. Participants with intermediate or extensive experience operating a sprayer were able to articulate more useful or relevant information from the video clips than their peers with less operating experience (Table 3). Machine speed and integrity (0.8%) Figure 8. Information gained based on from the operator's look zone during spraying operation.

Effect of Operators' Experience on Information Gained
The years of operating experience of the participants did not influence their responses towards the level of importance of each video clip or with respect to replacing the clips with a graphical display; however, it did influence the responses to the open-ended questions in the questionnaire. Participants with intermediate or extensive experience operating a sprayer were able to articulate more useful or relevant information from the video clips than their peers with less operating experience (Table 3).

Influence of Camera Position
The experimental data were analyzed to determine the effect of the camera position on the visual information gained. When specific features of either the sprayer or its environment were prominently displayed within the frame of the camera, participants were more likely to comment on the information gained from that specific feature than information that could be gained from other less prominent features. For example, clip 2 focused mainly on the "right boom" and many participants identified features that were related to the boom as being valuable. Similarly, clip 9 emphasized the field and most participants focused on the relevant information that was gained from the field. This finding suggests that designers can influence which features the user will perceive by positioning the camera such that those specific features are prominent in the camera's field of view.

Alternate Camera Placement
Other views that were suggested included a camera at the end of the boom facing forward (24%), a close-up view of the nozzle and its tip (24%), a view from under the sprayer facing backward to see the spray pattern behind the sprayer and wheels (10%), dashboards or displays (10%), and a camera that would focus on the wheel to show how well an operator was either following old tire tracks or steering within the rows of a corn crop (7%).

Live Video or Graphical Display of Information
A majority of participants indicated a preference for retaining important visual information in the format of live video on a hypothetical automation interface for remotely supervising an autonomous agricultural machine. For clips 1 and 2 (perceived to display the most important information), participants noted that it would be difficult to perceive the same information if they were replaced with graphical displays. However, they indicated that replacing clips 3, 5, 6, and 7 with a graphical display would not have any substantial impact on their ability to operate the sprayer. This divergent result may be due to the fact that most participants did not perceive any valuable information from the latter set of clips. Observed results from several of the other clips (i.e., clips 4, 8, 9, and 10) is more difficult to interpret. In-depth analysis revealed that the participant's response to this question was related to the information perceived from the clip. For example, in clip 8, some participants identified relevant information such as spray drift, spray pattern, and backing up; these aspects of sprayer operation might be difficult to portray on a graphical display, perhaps leading these participants to prefer live video. For the same video clip, other participants reported no substantial impact when replacing live video with a graphical display. These participants identified features such as wheel alignment or following a previous wheel track-features that could perhaps be displayed more easily using a graphical display. Overall, experienced sprayer operators reported higher preference for presenting 1 Front view from sprayer cabin (operator's sitting position).  No information gained-34 Windy-31 Sprayer is in the right field-10 Path taken by sprayer-10 Sprayer is still moving-7 Spray drift-3 How close is the boom end from the field boundary-3 Boom height-3 Field condition-3 Size of the field-3 -Not useful -Possibly stop spraying -Confirm that proper crop is being sprayed and that machine is in the correct field. View of spray/rinse tank and field.
No information-59 Leaks on rinse tank-17 Weather-7 Driving straight-7 Sprayer is still moving-3 Obstacles-3 Travel speed-3 Obstacle-3 Travel speed-3 Field condition-3 Machine integrity-3 Tank latch is secure-3 -Assist when passing low hanging wire or shed.  n = 29, unless otherwise stated. Blank means no response was made. LoI: level of importance. L1 = not at all important; L2 = of little importance; L3 = of average importance; L4 = very important; L5 = extremely important. a = believe it would be more difficult to gain the information required to operate the sprayer if the live video was replaced with a graphical display; b = I believe it would be easier to gain the information required to operate the sprayer if the live video was replaced with a graphical display; c = I do not believe there would be any substantial impact on my ability to operate the sprayer if the live video was replaced with a graphical display.