Vision-Transformer Model Validation Image Dataset

Abstract

The removal of plastic contamination from cotton lint is a critical issue for the U.S. cotton industry. One primary source of this contamination is the plastic wrap used on cotton modules by John Deere round module harvesters. Despite rigorous efforts by cotton ginning personnel to eliminate plastic during module unwrapping, fragments still enter the gin’s processing system. To address this, we developed a machine-vision detection and removal system using low-cost color cameras to identify and expel plastic from the gin-stand feeder apron, preventing contamination. However, the system, comprising 30–50 ARM computers running Linux, poses significant challenges in terms of calibration and tuning, requiring extensive technical knowledge. This research aims to transform the system into a plug-and-play appliance by incorporating an auto-calibration algorithm that dynamically tracks cotton colors and excludes plastic images to maintain calibration integrity. We present the image dataset that was used to validate the design, consisting of several key AI Vision-Transformer image classifiers that form the heart of the auto-calibration algorithm, which is expected to reduce setup and operational overhead significantly. The auto-calibration feature will minimize the need for skilled personnel, facilitating the broader adoption of the plastic removal system in the cotton ginning industry.

1. Introduction

The prevalence of plastic contamination in cotton lint has emerged as a critical concern for the U.S. cotton industry, particularly in light of the introduction of novel harvester designs utilizing plastic-wrapped cotton modules. This development has resulted in an increased incidence of plastic within cotton bales, as reported by textile mills. U.S. cotton classing offices have identified the primary source of contamination as plastic wraps from John Deere harvesters’ modules. Despite rigorous industry efforts to eliminate plastic during module unwrapping, residual contamination persists throughout the cotton gin processing system [1]. Figure 1 shows an expensive unwrapping system that is utilized in a high-end U.S. commercial cotton gin. Unfortunately, due to the cost, most gins do not have the budget for these high-end systems and subsequently have more issues with plastic contamination.

The economic implications of plastic contamination on the U.S. cotton market value are significant. Historically, U.S. cotton commanded a premium of 0.02 USD/kg on international markets due to its superior cleanliness. However, following the introduction of plastic-wrapped modules, economic projections suggest that U.S. cotton now trades at a 0.01 USD/kg discount, resulting in a total loss of 0.034 USD/kg compared to pre-plastic-wrapping market values. Extrapolating these figures to typical annual cotton yields, the estimated financial impact on U.S. producers exceeds USD 750 million per annum, raising substantial concerns among cotton growers and in the gin industry [2].

While it is acknowledged that plastic contamination may not be solely responsible for this economic downturn, economic analyses indicate that it is a major contributing factor. Further research is warranted to quantify the precise impact of plastic contamination on cotton’s quality and market value.

The primary aim of this technical manuscript is to introduce the software architecture of the author’s machine-vision plastic contamination detection system, shown in Figure 2.

The author’s main aim is for the VISN system to detect and record plastic contamination in order to be used by cotton gin management to help assess plastic contamination reduction strategies. The primary purpose of the AI image dataset is to help develop an AI model so that the VISN system can operate in a fully autonomous mode, needing at most minimal training. To achieve this, several Vision-Transformer AI models were developed. The image dataset presented here was used to train and validate these models [3,4,5,6,7,8,9].

2. Materials and Methods

In creating the Vision-Transformer AI models, ViTs, the images were each manually classed. The training data were obtained in the 2022 season at a commercial gin. The validation data were obtained in the 2023 season from 3 completely different commercial cotton gins. Each image was manually classed into one of four classes {cotton, empty tray, plastic, hand intrusion}. In the original validation dataset, there were over 6000 manually classed images. A minor fraction of the images were removed from the dataset for each image, if it was unclear from the visual inspection which class the image should belong to, and were labeled into the non-determinant, “ND” class. Hence, these data provide a high-quality set of fully independent training and validation image datasets. The images were collected at three commercial cotton gins during the normal course of their cotton ginning operations. The images were collected on multiple gin stands, with one gin in the mid-west, one in south, and one in W. Texas, all of which were in the U.S.A. The image datasets can be downloaded from [10].

The image sensor utilized to collect the images was a Sony IMX219 imager module using Camera Serial Interface, CSI (Sony Corp. Tokyo Japan; sourced from Amazon.com as a Raspberry-Pi camera v2 module). The exposure was set to 750 us. Fixed white balance was utilized under structured lighting, with a light color temperature of 6500 K. The object distance from the lens was 358 mm, with a depth of field of at least +/− 50 mm. The VISN system was tuned to ignore normal cotton colors, utilizing the author’s Negative Classifier [9]. Anything that was not in the prestored natural cotton color lookup table was captured as a plastic image that became part of this image dataset. The image dataset was supplemented with natural cotton images from an additional VISN system that simply captured images at a rate of 1 per second. As this led to an extraordinarily large dataset, the natural cotton image dataset was sampled so that the samples spanned across the entire cotton ginning season. This would avoid overloading the image dataset and thereby avoid a size unbalance bias during the AI model training. We tested the dataset with two different General-Purpose (GP), Vision-Transformer (ViT), and Image-to-Text (caption) models (Blip, Git), that were combined with a semantic classifier for image classification, and again with a custom ViT model. In both cases, the validation results were in excess of 98% prediction accuracy. The results from both of these tests and models are in the process of being published in subsequent publications that are now in a draft state and soon to be submitted. The combined GP model was used with minimal training to enable large scale automated image annotation of the image dataset [10] that was then used to train several stand-alone high-speed dedicated ViT models; at significant reduction in manual image annotation labor. Of note is that the GP models are slow, so throughput was one image per 10 s; so only useful if used in an off-line batch processing; such as to provide automated image annotations. The upcoming paper will focus on this semi-automated image annotation approach and the performance of the high-speed task-specific ViT models that this approach enables.

3. Summary

The image dataset provides two image datasets: a training set and an independent validation dataset. As the image dataset is too large to include as a supplemental file, it was stored in CERN’s Zenodo permanent scientific data repository [10].