While there has been much work and progress in developing automated, commercial phenotyping systems in controlled environments (e.g., LemnaTech Conveyor Scanalyzer, Phenospex TraitFinder), there remains a need to continue developing systems for phenotyping at larger field scales. Field-based systems that employ proximal sensing approaches enable data collection at high spatial resolutions necessary for measuring a variety of morphological and physiological traits in realistic growing conditions over an entire growing season. These FB-HTP systems commonly utilize image, spectral, and climate sensors to collect data at the plant, row, or plot level in crop systems, operating with varying levels of autonomy. These field systems are typically built by interdisciplinary academic research teams for highly specialized phenotyping needs. Although the capabilities of technology used for high-throughput phenotyping have improved and costs decreased, there have been few, if any, successful attempts at developing turnkey field-based phenotyping systems. To address this issue, this work presents and implements a framework for characterizing system utility and complexity and evaluates FB-HTP systems through a meta-analysis to identify bottlenecks in adoption, and identifying areas for future work in system improvement.
It has been suggested that resources for developing these systems, maintaining standards for data collection and management, integrating data from multiple sources, environmental challenges, and the need for specialized solutions may all be limiting factors of FB-HTP development. The phenomics community has acknowledged difficulties due the complexity and scale of data needed by the academic sector and breeding programs [1
]. Additionally, it has been observed that FB-HTP platforms are built by large, interdisciplinary teams for specialized needs and require significant resources [3
], prompting recommendations for phenotyping systems to be low cost [3
], usable [2
], and open source [1
], but unfortunately these recommendations have yet to be adopted across a wide range of field-based phenotyping applications. There remains a lack of development and adoption of turnkey FB-HTP systems, particularly ground-based systems, suggesting that a systematic investigation into development challenges is warranted.
To motivate the need of understanding bottlenecks in developing FB-HTP systems, a citation search using the Web of Science database was conducted to understand recent trends of field-based phenotyping systems in the literature. The search queried all records with the words “field AND phenotyping” present in the title, abstract, or keywords which resulted in 2823 records. A second search was similarly conducted and filtered using the terms “field AND phenotyping AND high throughput” to focus on the development of HTP systems, which resulted in 299 records. It can be easily seen from Figure 1
that the number of publication records focusing on field-based phenotyping systems has steadily increased over the past decade, indicating a trend of increased efforts in development, evaluation, and deployment of this technology. Although work in FB-HTP development is temporally increasing, this field is likely still in its infancy, and a framework for conducting systems-level analysis is merited to identify potential areas of technology advancement and adoption as this trend continues.
The purpose of developing an evaluation framework is to assess the quality, performance, and sophistication of the system [5
]. The development of a framework also enables the evaluation of the intrinsic properties, usability, and utility of FB-HTP systems in their environment. Related work has been done to develop methods of evaluating usability of agent-based information systems [6
], including methods which decompose a central domain into smaller dimensions and use qualitative scales to determine ratings based on a set of predetermined criteria [7
]. Comparatively, a systems approach can also assess subsystem properties to evaluate emergent properties of the system in its entirety [9
] and evaluate interactions between subsystems [10
]. These types of analysis tools are useful for FB-HTP systems due to their system complexity, multiple and diverse users, and the necessity to perform system decomposition for cross-platform analysis and comparison.
One particularly useful framework for software evaluation developed by Boloix and Pierre [5
] enabled assessment of the system’s quality and sophistication by investigating and consolidating multiple points of view. Additionally, their framework simplified complex mechanisms to make large amounts of available information useful and provide a basis for comparison. Their framework [5
] was adapted for FB-HTP platforms by taking a systems approach and integrating it with an approach for project complexity evaluation [11
]. This newly developed framework provides a tool for evaluating FB-HTP systems that considers the following points of view: the research project goals, the phenotyping platform requirements, the crop system requirements, and the data analysis pipeline. The evaluation outcome is valuable because it identifies gaps in current research efforts and points to future research directions that are necessary to overcome the current challenges related to developing turnkey hardware and software for FB-HTP platforms.
This paper is organized as follows: Drawing on insights from the literature and a qualitative study, Section 2
presents the systems approach and framework development for evaluating FB-HTP systems. Section 3
describes the implementation of the framework on a set of 10 field-based phenotyping systems and includes results from the framework evaluation. Finally, Section 4
presents a discussion of the results, and Section 5
presents the conclusions and future directions of this work.