Next Article in Journal
Simulating Canopy Temperature Using a Random Forest Model to Calculate the Crop Water Stress Index of Chinese Brassica
Previous Article in Journal
Resource Recovery from Synthetic Nitrified Urine in the Hydroponic Cultivation of Lettuce (Lactuca sativa Var. capitata L.)
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:

Check-All-That-Apply Questions including the Ideal Product as a Tool for Selecting Varieties in Breeding Programs. A Case Study with Mandarins

Sensory and Consumer Science Research Group, Postharvest Technology Center, Valencian Institute for Agricultural Research (IVIA), Carretera Moncada-Náquera, km. 4.5, Moncada, 46113 Valencia, Spain
Instituto de Agroquímica y Tecnología de Alimentos (IATA-CSIC), Agustín Escardino 7, Paterna, 46980 Valencia, Spain
Centre of Citriculture and Plant Production, Valencian Institute for Agricultural Research (IVIA), Carretera Moncada-Náquera, km. 4.5, Moncada, 46113 Valencia, Spain
Author to whom correspondence should be addressed.
Agronomy 2021, 11(11), 2243;
Received: 12 October 2021 / Revised: 26 October 2021 / Accepted: 3 November 2021 / Published: 5 November 2021
(This article belongs to the Section Crop Breeding and Genetics)


Obtaining superior quality varieties are one of the main objectives of fruit breeding programs worldwide. In this study, we investigate employing check-all-that-apply (CATA) questions, which include the ideal product, as a method to select new varieties according to consumer organoleptic quality requirements. To this end, mandarin cultivars were used as a case study. Four new cultivars from the IVIA breeding program (‘Pri-88′, ‘Pri-89′, ‘Pri-90′ and ‘Tri-707′), and two commercial cultivars related to them (‘Clemenules’ and ‘Oronules’), were evaluated by a consumer panel using the CATA questions method, which include their ideal product. Our results reveal that this method is a useful tool for selecting varieties based on consumer descriptions of their sensory properties in relation to those of their ideal cultivar. This allows the different consumer preference patterns and differences in sample perceptions to be taken into account. A penalty analysis was performed, including the ideal product, to identify the “must-have” attributes; i.e., those that consumers included in their ideal mandarin description and, whenever present in samples, they significantly increased acceptance. For the mandarins herein evaluated, these attributes are “refreshing taste”, “very aromatic”, “sweet”, “juicy”, “very intense taste”, “sour” and “not very fibrous”. The fruit characteristics that contributed to improve the quality of the new varieties vs. the original varieties were also identified. ‘Pri-89′ and ‘Tri-707′, obtained from ‘Clemenules’, came closer to consumers’ ideal variety, because besides the aforementioned “must-have” attributes, these mandarins have small segments. ‘Pri-90′ implied improvement in relation to ‘Oronules’ and is an appropriate variety for those consumers sensitive to bitterness and who like mandarins with a certain level of sourness.

Graphical Abstract

1. Introduction

In the last few decades, numerous breeding programs have been established all around the world in fruit-producing countries. In many cases, these programs have focused on improving crop agronomic characteristics, such as adapting grapevine fruit to high temperatures [1] or improving disease resistance in citrus fruit [2]. In parallel to these agronomic objectives, obtaining new varieties that satisfy consumer convenience and sensory quality requirements has always been in breeders’ minds. Clear examples of convenience-based breeding are obtaining seedless varieties of watermelon [3] and citrus fruit [4], or easy-peeling citrus fruit varieties [5] and tomato [6]. Although the evaluation of these traits is relatively easy, determining to what extent new varieties satisfy consumers’ sensory quality demand may be more complicated. Moreover, exploring characteristics of new spontaneously generated varieties can be especially interesting because some can be more desirable than those of original varieties [7,8]. For decades, the selection of new varieties for fruit quality has been based mainly on evaluating physico-chemical properties [9,10,11]. However, when only physico-chemical information is available, consumer responses are difficult to predict. For this reason, the inclusion of sensory studies as a step in the selection process of new varieties has gradually become essential [12,13].
Initially, the approach to describe the sensory attributes that characterize new varieties was by means of trained panels [14,15]. However, this methodology has certain drawbacks, like the time and resources needed to train assessors and the fact that some of the information produced by them may be irrelevant for consumers [16]. Thus, new sensory attributes description techniques have been developed to be used with a consumer-based approach [16]. Of them all, check-all-that-apply (CATA) questions have gained ground in recent years as a reliable and quicker alternative to trained assessor panels [13,17,18]. CATA questions consist of a list of words or phrases from which consumers should select all the attributes that they consider appropriate to describe samples [16,17]. In a recent study, Maheeka et al. [19] reported that consumer and expert CATA panels are not interchangeable and panel type should be chosen in accordance with the study objective. CATA questions with consumer panels have proven to be a useful tool for the characterization of new fruit varieties, such as strawberries [13] and citrus fruit [20], and obtained information is more relevant from a commercial point of view. In such studies, the combination of CATA questions and consumer preference evaluations allows the main drivers of liking to be identified [20].
Moreover, several studies have claimed the usefulness of using CATA questions including the Ideal Product (IP) description for optimizing formulations of elaborated products and their sensory profile, in order to come close to consumers’ IP [20,21]. In this approach, after completing the CATA questions for each evaluated sample, consumers are asked to use the same list of attributes to define their IP [21,22,23,24]. Initially, the aim of this practice is to identify how product characteristics differ from consumers’ IP to reformulate the product [18]. As far as we know, including an IP description in this quite novel methodology of CATA questions has never been used during the process of evaluating new varieties. Based on our expertise in breeding [25,26] and new variety evaluations [8,20,27], we believe that it would be very useful to include this procedure in breeding programs.
On the one hand, it can help to understand consumer preferences based on the characteristics that new varieties share with consumers’ IP. On the other hand, it can allow breeders to select varieties as starting material for future obtainments based on the specific characteristics that make them interesting.
In this context, the objective of this study was to assay CATA questions, including the ideal product description, as a tool for evaluating new varieties. To do so, citrus fruit varieties were selected as a case study because citrus is the most grown fruit tree crop worldwide [28]. We specifically applied this methodology to select the new early-season mandarin varieties preselected in the breeding program hosted by the Valencian Institute of Agriculture Research (IVIA, Spain). In addition to the new varieties, commercial cultivars related to them that ripen at the same time of the season were also included in the study.

2. Materials and Methods

2.1. Samples

In the present study, six early-season mandarin varieties were evaluated: ‘Tri-707′, ‘Pri-88′,’Pri-89′, ‘Pri-90′, ‘Clemenules’ and ‘Oronules’. As shown in Table 1, three of them (‘Tri-707′, ‘Pri-88′ and ’Pri-89′) are new mandarin cultivars developed in the IVIA breeding program and the IVIA-Industry program, whose main objective is to obtain unseeded and superior quality early-season varieties. The new variety ‘Pri-90′ was spontaneously generated from cv. Oronules in a grower orchard. As it has drawn breeders’ attention, it is being studied in the program. ‘Clemenules’ and ‘Oronules’ were included in this study as the main varieties from which the new ones derived. These two commercial varieties, which spontaneously originated more than 40 years ago, are also early varieties.
By mid-October, commercial varieties were obtained upon their arrival at a commercial packing house in Valencia (the FONTESTAD S.A. Company, Valencia, Spain), while the new cultivars were harvested from the IVIA’s experimental orchards. All the fruit were transported to the IVIA’s Postharvest Department for the physico-chemical analysis and consumers’ sensory evaluations.

2.2. Physico-Chemical Evaluation

The following physico-chemical parameters were evaluated: colour, firmness, titratable acidity (TA), total soluble solids (TSS), juice yield (JY) and maturity index (MI).
Peel colour was measured by a Minolta colorimeter (model CR- 300; Minolta Co. Ltd., Osaka, Japan) using 20 fruits per variety and two measurements were taken in the equatorial zone of each fruit. The mean lightness (L), red-green (a) and yellow-blue (b) Hunter parameter values were calculated for each fruit and expressed as a colour index (CI = 1000 a/Lb) [29].
Firmness measurements were taken by an Instron Universal Testing Machine (model 3343, Instron Ltd., Buckinghamshire, UK) on 20 fruits per variety. The results were expressed as the percentage of millimetres of fruit deformation that resulted from 10 N pressure, applied by a 3.5 cm plunger on the longitudinal axis at constant speed.
For the TSS and TA, four samples of five fruit each per treatment were squeezed into an electric juice extractor with a rotating head and were determined following the procedure described by Morales et al. [30]. Maturity Index (MI) was calculated as TSS/TA. Firstly, the juice yield was measured and expressed as a percentage, calculated by dividing the volume of juice by the total fruit weight.

2.3. Sensory Study

In total, 134 consumers participated in the study. Consumers were recruited based on their mandarin consumption frequency (at least once a week during the season) and their interest in participating. Participants were aged between 19 and 58 years, and the male/female ratio (%) was 47/53.
The evaluations were carried out in a standardized test room (ISO 8589; ISO 2007). The fruits were hand-peeled and each sample was composed of four segments of the same mandarin (presented together). Samples were coded with three-digit random numbers and were presented monadically following a Williams’ Latin square design, and consumers were provided with water to cleanse palate between samples. Consumers were instructed to separate the segments before tasting, and were asked to firstly score their overall liking using a 9-point hedonic scale anchored at 1-“dislike very much” and 9-“like very much”, and then to indicate their purchase intention on a 5-point scale ranging from 1-“I definitely would not buy” to 5-“I definitely would buy”. Next, they were asked to answer the CATA questions with 20 terms related to the sensory characteristics of mandarins. After testing each of the six samples, consumers were asked to complete the CATA questions to describe their ideal mandarin.
The descriptors included on the CATA list were initially selected according to previous research [19,31]. Then a group of six semi-trained panellists evaluated the samples to adapt the initial list to their specific characteristics. On the final list, attributes were ordered in such a way that they were likely to be perceived. The selected terms were: stain hands when eating, intense odour when separating segments, small segments, big segments, bitter, very intense taste, not very aromatic, very aromatic, not very sour, sour, very sour, refreshing taste, tasteless/dull, not very sweet, sweet, very sweet, fibrous, not very fibrous, juicy and juiceless (Figure S1).
Finally, when consumers had finished their mandarin assessments, they answered demographic questions about the gender, age and frequency of mandarin/orange consumption during the season. The response options for fruit consumption were as follows: “Almost every day”, “At least twice weekly” or “Less than twice weekly”.
The protocol and procedures used in this study were revised by the scientific directorate of Valencian Institute for Agricultural Research, which stated a waiver consent. All articles from the Declaration of Helsinki and the 2016/679 EU Regulation on the protection of natural persons regarding the processing of personal data and on the free movement of such data were met. The experimental procedure was explained and a written consent indicating voluntary participation was obtained from each participant prior the beginning of the study.

2.4. Statistical Analysis

One-way ANOVA was used to evaluate the difference in the physico-chemical parameters among cultivars. Significant differences between means were determined by calculating the Least Significant Difference test (p ≤ 0.05).
The Kruskall–Wallis test followed by Dunn’s multiple comparisons test were applied to evaluate difference in acceptance scores among cultivars (p ≤ 0.05).
A hierarchical cluster analysis (HCA) was performed with the liking data (6 columns × 134 rows) to identify groups of consumers with similar preference patterns. Euclidean distances (dissimilarity), Ward’s techniques (agglomeration method) and automatic truncation were selected for this analysis. The number of groups was selected by automatic truncation minimizing entropy. Smaller values of overall entropy indicate less disorder, which indicates a better clustering.
The frequencies of citation of each attribute in the CATA question were determined for each sample. The non-parametric Cochran’s Q test was performed on the raw binary CATA data to determine significance among samples for each sensory attribute (p ≤ 0.05). In order to assess the relation between CATA responses and acceptance scores, a Multiple Factor Analysis (MFA) was performed on the acceptability data of the two clusters, and the CATA attributes frequency of mentioning were also included as supplementary variables.
Penalty analysis of the CATA data including the ideal, was used to determine the attributes with impact on acceptability. For each attribute, for those consumers who selected it in the ideal, the difference in average liking of those mandarins having the attribute and those mandarins not having the attribute was calculated. Those attributes showing a significantly (α = 0.05) higher acceptability when the attribute was present were considered “must have” attributes. For each attribute, for those consumers who did not select it in the ideal, the difference in average liking of those mandarins having the attribute and those mandarins not having the attribute was calculated. Those attributes showing a significantly (α = 0.05) lower acceptability when the attribute was present were considered “must NOT have” attributes.
All the calculations were carried out with the XLSTAT software (version 2019, Addinsoft Inc. New York, NY, USA).

3. Results and Discussion

3.1. Physico-Chemical Parameters

Of the citrus fruit’s physico-chemical parameters, external colour, MI and juice yield are especially important for EU countries because they are included in the regulation for exporting and marketing citrus fruit within the European Union [32]. Thus the minimum required MI and JY values are 6.5% and 33% for satsuma mandarins, 7% and 40% for clementines, and 7.5% and 33% for other mandarins or hybrids, respectively. However, the regulation does not specify any minimum colour index values because it only refers to the variety’s typical colour: ‘colour must be typical of the variety on at least one third of the fruit surface’.
In this study, mandarins ‘Oronules’ and ‘Pri-90′ were highlighted for presenting a homogenous orange-coloured skin with CI values of +16 and + 14 at harvest time, respectively, while the part of the skin surface of the other varieties was still green (CI < +4). In this study, external colour was not considered a quality-limiting factor for harvesting because previous studies performed in our department (personal communication) have demonstrated that the four evaluated new varieties adequately respond to the degreening treatment habitually applied to early varieties to enhance colour change. Moreover, a recent study, which evaluated the degreening treatment effect on a range of mandarin and orange varieties, demonstrated that this treatment does not modify the sensory properties that consumers perceive [30].
The determination of the physico-chemical parameters linked with internal quality revealed that all six varieties obtained the minimum required MI (Table 2). The highest MI, with values of 12.9 and 13.6, were respectively determined in ‘Pri-90′ and ‘Tri-707′. The other cultivars had similar MI values to one another, which ranged between 9.6 and 11.5. As previously explained, MI is the ratio between TSS and acid concentration, and it is worth mentioning the new variety ‘Pri-89′, which, despite being that with the highest TSS (13.7 °Brix), presented the lowest MI due to its high acidity level (1.43 g citric ac./100 mL).
The minimum required JY values were met for all the evaluated varieties, except ‘Pri-88′, which had 31.87% JY. The other varieties obtained JY values over 40%.
Regarding size and texture, the ‘Pri-88′ mandarins were the firmest (2.22% deformation) and the biggest (220 g), while the ‘Tri-707′ variety produced the smallest fruit with a mean weight value of 59 g.

3.2. Preference Patterns and Purchase Intention

The mean liking scores and the percentage of consumers who stated that they would buy each mandarin variety are shown in Table 3. The Global column includes the results by taking into account the total dataset. Similar liking scores (around 6) were determined for three new varieties (‘Pri-89′, ‘Pri-90′ and ‘Tri-707′) and for ‘Oronules’. The commercial cultivar ‘Clemenules’ and ‘Pri-88′ obtained liking scores below 5.
For most samples, liking scores showed wide dispersion (according to the histograms and whisker-box plots; data not shown), which indicates distinct patterns in consumer responses to the different varieties. Previous studies that have evaluated new varieties of mandarin [20] and other fruit [21], reveal that consumer preferences usually respond to different patterns, because not all people like similarly. A hierarchical cluster analysis (HCA) was performed to further study the data on this aspect, and two different consumer groups, were identified based on their preferences. Cluster 1 (Cl1) was made up of 61 consumers (46%) and Cluster 2 (Cl2) included 73 consumers (54%). Significant differences were detected in the acceptance scores for varieties in both clusters (Table 3). For the Cluster 1 consumers, the preferred mandarins were ‘Tri-707′ and ‘Pri-89′ (mean liking value of 7.5 and 6.8, respectively) and the least preferred one was ‘Pri-88′ with a liking value of 5. The other three varieties (‘Pri-90′, ‘Oronules’ and ‘Clemenules’) obtained intermediate liking values between 6 and 6.4.
A different preference pattern was observed for the Cluster 2 consumers, who generally scored the varieties with lower values than Cluster 1, except ‘Pri-90′, which was their most preferred mandarin. They disliked samples ‘Pri-88′ and ‘Clemenules’ (both with mean liking values below 4). Varieties ‘Tri-707′ and ‘Pri-89′, i.e., those preferred by the Cluster 1 consumers, received intermediate values by the Cluster 2 consumers.
In agreement with preference, a higher percentage of the Cluster 1 consumers stated that they would purchase ‘Tri-707′ and ‘Pri-89′ (82% and 62%, respectively), and a low percentage of consumers (26%) indicated that they would buy ‘Pri-88′. In Cluster 2, 55% of the consumers would buy sample ‘Pri-90′, and very few (11% and 8%) would buy ‘Pri-88′ and ‘Clemenules’.
Analysis of demographic data from Cluster 1 and 2 consumers revealed a not significant influence of gender, age or consumption frequency in their pattern of liking.

3.3. Consumers’ Sensory Descriptions of Varieties and Their Ideal Mandarin Variety

The sensory description of the mandarin varieties from the CATA questions, which include the ideal variety, was studied independently for both consumer clusters. After determining the frequency of mention for each attribute (Tables S1 and S2), a correspondence analysis (CA) was performed (Figure 1A,B). In both cases, the first and second dimension explained around 80% of the variability.
The CA plot for the Cluster 1 consumers (Figure 1A) showed that variety ‘Tri-707′ came very close to the ideal mandarin on the left side of the space. This closeness in allocation meant that this group of consumers found most of the properties of their ideal mandarin in ‘Tri-707′, such as “juicy”, “sweet”, “sour”, “small segments”, “very aromatic”, “intense odour when peeling” and “refreshing taste”. This similarity between the description of the ‘Tri-707′ mandarins and the ideal one explained why this variety was that which the Cluster 1 consumers liked the most (Table 3).
If we pay the attention to the first dimension, which explained most variability, we observed that variety ‘Pri-89′ came quite close to the IP, along with ‘Tri-707′, for sharing characteristics of “very intense taste”, “sour”, “juicy”, “very sweet” and “very aromatic”. However, consumers described ‘Pri-89′ as more sour than their ideal mandarin and ‘Tri-707′.
In the upper right side of the plot, ‘Pri-88′ was the variety that moved the furthest away from the IP and was associated mainly with the terms “big segments”, “bitter”, “fibrous” and “juiceless”. Consumer perceptions of the attributes did not match their ideal mandarin, which would explain why this variety obtained the lowest liking scores (Table 3). This characterization also agreed with the physico-chemical data as ‘Pri-88′ was the biggest variety with the lowest juice yield percentage.
Finally, varieties ‘Oronules’, ‘Pri-90′ and ‘Clemenules’, which obtained similar liking scores (6.1–6.4), lay between the Pri-88 and consumers’ ideal mandarin on the first dimension. These three varieties shared characteristics with the ideal one, but also with ‘Pri-88′. The main difference in them was that consumers perceived more sourness in ‘Oronules’ compared to ‘Pri-90′ and ‘Clemenules’, and the latter stained hands when eating.
Figure 1B shows the results of the cluster analysis performed with the data corresponding to the participants from Cluster 2. In this case, the plot revealed that none of the varieties came very close to the ideal one, which agrees with the lower liking scores detected in Cluster 2 compared to Cluster 1 (Table 3).
‘Pri-88′ and ‘Clemenules’ were located furthest away from the IP and were associated with the terms “not very aromatic”, “not very sweet”, “tasteless/dull”, “not very sour” and “fibrous”. In these two varieties, consumers did not find the characteristics that they wished to perceive in their ideal mandarin, which was why they disliked the samples and scored them with values lower than 5 (Table 3).
Cultivars ‘Oronules’, ‘Pri-90′, ‘Pri-89′ and ‘Tri-707′ were located more closely to the IP and, among them, two different groups were detected. The first group, which came closer to the IP on the second dimension, comprised ‘Oronules’ and ‘Pri-90′. To a greater or lesser extent, these cultivars shared most of the characteristics of Cluster 2 consumers’ ideal mandarin, mainly “juicy”, “sweet”, “sour”, “refreshing taste” and “not very fibrous”. However, these varieties presented characteristics like “big segments”, “not very sour”, and “stain hands when eating”, which make them differ from the IP. ‘Pri-89′ and ‘Tri-707′ formed the second group, which obtained negative values on the second dimension. These varieties shared some characteristics with the ideal mandarin, such as “sweet”, “sour”, “juicy”, “very intense taste” and “very aromatic”. The separation between these cultivars and the ideal mandarin was mainly due mainly to the fact that consumers used the terms “very sour” and “small segments” to describe them.
A global view of the two plots revealed three results worth mentioning.
The first one is the relevance of the information obtained by consumer segmentation. Application of this technique to sensory data is quite recent, and it has been mainly based on ‘emotional’ response to different products [33,34,35,36]. In the mentioned studies, the evaluated products (beer, tea-break, snacks, etc.) showed relevant differences in organoleptic properties among the evaluated samples. In the present study, a priori, accused differences among samples were not expected, as the cultivars under evaluation are related to each other. However, our results showed that consumer segmentation is a powerful tool to obtain a deeper understanding of consumer behaviour, even when there are small differences among samples.
The second result is that all the new varieties except ‘Pri-88′ shared more characteristics with the ideal mandarin than the two commercial varieties from which they originated. The ideal product concept refers to the standard of perfection, i.e., the consumers’ description of the characteristics that a mandarin should have to be perfect. Therefore, our results indicate that the objective of the breeding program of obtaining superior quality varieties is being achieved. The exception was ‘Pri-88′, which was initially preselected for its big size fruit, among other reasons. However, our results revealed that consumers do not like this characteristic.
The third one to highlight is that despite showing different preference patterns for the evaluated varieties, Clusters 1 and 2 similarly described their ideal mandarin. Thus, in both cases (Figure 1A,B), the attributes used to describe the ideal mandarin were: “intense odour when separating segments”, “very aromatic”, “refreshing taste”, “very sweet”, “sweet” “very intense taste”, “sour”, “juicy” and “not very fibrous”. To better understand the preference differences between the two groups, the relation between the description of sensory characteristics and the liking values for each cluster was studied.

3.4. Relations between Acceptability and the Sensory Description of the Mandarin Varieties

First by combining CATA questions and liking data, a penalty analysis was used to determine the relevance of each attribute that consumers included, or not, in their ideal mandarin (Figure 2).
Figure 2A,C shows the “must-have” attributes that corresponded to those included in consumers’ ideal mandarin and significantly (p< 0.05) increased consumer acceptability when they found the attribute in a mandarin variety (compared to those cases in which that attribute was not selected). For both clusters, “refreshing taste” and “very aromatic” were the most relevant attributes. Around 70% of all the consumers included these attributes in their ideal mandarin. When they tasted the mandarins and found these attributes, their average acceptability was around 2 points higher than for those mandarins in which these attributes were not selected. Being juicy, not very fibrous and very tasty were also characteristics that increased consumer acceptability. The only difference observed between both clusters was that having small segments was relevant only for Cluster 1 and “sourness” had a much stronger impact on Cluster 2 with an increased consumer acceptability of 1.9 compared to Cluster 1 (0.8).
Figure 2B,D shows the “must-NOT-have” attributes that corresponded to those that were not included in the consumers’ ideal mandarin, and which significantly (p< 0.05) decreased average acceptability when consumers found the attribute in a mandarin (compared to those cases in which the attribute was not selected). For both clusters, attributes “tasteless/dull”, “juiceless” and “not very sweet” had a strong impact. When one of these attributes was used to describe a mandarin, average acceptability decreased by 1.5 to 2 points. Having slight aroma and being fibrous also decreased acceptability. The differences between clusters lay in big segments being a negative attribute only for the Cluster 1 consumers, while being “bitter” and “not very sour” significantly impacted acceptability, but only for the Cluster 2 consumers.
In short, some differences between both clusters were found for those attributes that impacted mandarin acceptability. Segment size was relevant for the Cluster 1 consumers, who penalized big segments and liked small segments in mandarins. Sourness and bitterness were relevant for the Cluster 2 consumers, who penalized samples when they were slightly sour and bitter. However, the more relevant attributes that impacted acceptability were the same for both clusters.
To ascertain if a different use of descriptors between clusters could explain preference differences, a MFA was used to compare the mandarin sensory descriptions of the two consumer clusters in relation to acceptability (Figure 3). The samples in the plot were distributed according to the acceptability of both clusters. The position of attributes (included as supplementary variables) revealed the similarities and differences in the way that groups of consumers used descriptors to describe the six studied varieties. The position of some attributes came close for Clusters 1 and 2, which indicates that both groups of consumers similarly used them. This was the case for the appearance and texture attributes, such as “not very fibrous”, “stain hands when eating”, “big segments”, “small segments”, “juiceless” and “juicy”.
However, the attributes related to taste and aroma, especially those with a positive impact on acceptability like “refreshing taste”, “very intense taste”, “sweet”, and “very aromatic”, were differently positioned for Clusters 1 and 2. In each case, they went close to the corresponding acceptability variable. This meant that the samples described with these attributes in Cluster 1 were the preferred ones (‘Tri-707′ and ‘Pri-89′). The same can be stated for Cluster 2, for which these attributes were used mainly to describe ‘Pri-90′ and ‘Oronules’. This result falls in line with that reported by Ares et al. [21] in a study about commercial apple cultivars. Those authors found that consumers’ use of CATA terms was markedly affected by their preferences, and mainly for terms referring to odour and flavour.
It is known that a genetic background has a very marked effect on volatile fruit composition [8,37]. Thus for the mandarin cultivars herein studied, as ‘Tri-707′ and ‘Pri-89′ both descend from ‘Clemenules’, they may have a similar volatile profile, but one that differs from that of ‘Oronules’ to some extent and its spontaneous mutation ‘Pri-90′. Differences in volatile profile would result in specific odour and flavour characteristics that would explain the different preference patterns obtained for Clusters 1 and 2 in relation to these varieties. Odour and flavour descriptions provided by a trained panel would help to clarify to what extent varieties have characteristics that differ from one another [38]. A gas chromatography analysis would be also a useful tool to this end [39].
In agreement with what the penalty analysis found, attributes “sour”, “bitter” and “not very sour” were also placed differently for Clusters 1 and 2, but were related to acceptability only in Cluster 2. Cluster 1 did not much differentiate samples according to bitterness or sourness, which could be due to consumers being less sensitive to this taste stimulus or because they paid less attention to them. However, the Cluster 2 consumers used the term ‘sour’ to describe the two samples that they liked the most (‘Pri-90′ and ‘Oronules’), and two attributes that were not desirable (“bitter” and “not very sour”) to describe the two mandarins that obtained the lowest liking scores (‘Pri-88′ and ‘Clemenules’). It is known that the perception of sensitiveness to bitterness vastly varies among people [40]. The Cluster 2 consumers’ greater sensitiveness to bitterness would explain why this group gave a much lower score to ‘Pri-88′ and ‘Clemenules’ than the Cluster 1 consumers, who also used this attribute to describe these two varieties, but to a much lesser extent.

4. Conclusions

Our results showed that CATA questions, which include the “ideal” description, is a useful tool for selecting varieties in breeding programs. It allows new varieties to be selected based on consumers’ descriptions of their sensory characteristics in relation to their ideal cultivar, and take into account consumers’ different preference patterns and differences in sample perceptions. It also allows those attributes that have contributed to improve the quality of new varieties versus original varieties to be identified.
Moreover, this method allows identifying “must-have” and “must-not-have” attributes, which is very valuable information for breeders. Undoubtedly, in the context of breeding programs, information revealed by CATA-questions including the Ideal is very useful not only to select varieties with some guarantee of success but also for breeders to set future objectives.
The herein presented case study of mandarins identified two consumer groups according to their preference pattern, i.e., the liking scores of samples. However, for all the consumers, “refreshing taste”, “very aromatic”, “sweet”, “juicy”, “very intense taste”, “sour” and “not very fibrous” were the attributes that mandarins must have to obtain good liking scores. These desirable attributes were detected more in ‘Pri-89′, ‘Tri-707′ and ‘Pri-90′ than in their progenitors ‘Clemenules’ and ‘Oronules’. Therefore, these attributes can explain the increased consumer liking associated with quality improvement. More specifically, ‘Pri-89′ and ‘Tri-707′, obtained from ‘Clemenules’, came closer to the consumers’ ideal cultivars because, besides the aforementioned characteristics, these mandarins have small segments. ‘Pri-90′ implied quality improvement in relation to ‘Oronules’ and is an appropriate variety for those consumers sensitive to bitterness and who like mandarins with a certain level of sourness.

Supplementary Materials

The following are available online at, Figure S1. List of attributes included in the CATA-questions. Table S1. Frequency of mention of consumers from Cluster 1 (61 consumers, 46% of the total number of consumers). Table S2. Frequency of mention of consumers from Cluster 1 (73 consumers, 64% of the total number of consumers).

Author Contributions

Investigation, writing—original draft preparation, A.G.-S.; formal analysis and writing—review and editing, A.T.; investigation, P.T.; resources, P.A.; conceptualization, supervision, writing—review and editing, funding acquisition, C.B. All authors have read and agreed to the published version of the manuscript.


This study has been co-funded by the European Regional Development Fund (ERDF) of the Generalitat Valenciana 2014–2020 (IVIA project number 51910).

Institutional Review Board Statement

The protocol and procedures used in this study were revised by the scientific directorate of Valencian Institute for Agricultural Research, which stated a waiver consent. All articles from the Declaration of Helsinki and the 2016/679 EU Regulation on the protection of natural persons regarding the processing of personal data and on the free movement of such data were met.

Informed Consent Statement

Informed consent was obtained from all subjects involved in the study.


Author Adrián Giménez-Sanchis wishes to thank the Conselleria de Innovación, Universidades, Ciencia y Sociedad Digital (Generalitat Valencians) for the financial support of his contract (FDEGENT/2020/004).

Conflicts of Interest

The authors declare no conflict of interest.


  1. Bigard, A.; Berhe, D.T.; Maoddi, E.; Sire, Y.; Boursiquot, J.M.; Ojeda, H.; Péros, J.P.; Doligez, A.; Romieu, C.; Torregrosa, L. Vitis vinifera L. fruit diversity to breed varieties anticipating climate changes. Front. Plant Sci. 2018, 9, 1–16. [Google Scholar] [CrossRef][Green Version]
  2. Wang, L.; Chen, S.; Peng, A.; Xie, Z.; He, Y.; Zou, X. CRISPR/Cas9-mediated editing of CsWRKY22 reduces susceptibility to Xanthomonas citri subsp. citri in Wanjincheng orange (Citrus sinensis (L.) Osbeck). Plant Biotechnol. Rep. 2019, 13, 501–510. [Google Scholar] [CrossRef]
  3. Liu, J.; Li, S.; Liu, B.; Wang, S.; Zhou, Z.; Yi, C.; Luo, Y.; Liu, H. Breeding of new seedless watermelon of Xiangke 1. J. Hunan Univ. Nat. Sci. 2009, 35, 73–75. [Google Scholar]
  4. Ali, S.; Khan, A.S.; Raza, S.A.; Rehman, R.N.U. Innovative breeding methods to develop seedless citrus cultivars. Int. J. Biosci. 2013, 3, 191–201. [Google Scholar] [CrossRef]
  5. Yu, X.; Zhang, X.; Jiang, D.; Zhu, S.; Cao, L.; Liu, X.; Shen, W.; Zhao, W.; Zhao, X. Genetic diversity of the ease of peeling in mandarins. Sci. Hort. 2021, 278, 109852. [Google Scholar] [CrossRef]
  6. Dick, J.A. Commercial Tomato Breeding in a Small Independent Seed Company. Acta Hort. 2009, 823, 25–31. [Google Scholar] [CrossRef]
  7. Koornneef, M.; Alonso-Blanco, C.; Vreugdenhil, D. Naturally occurring genetic variation in Arabidopsis thaliana. Annu. Rev. Plant Biol. 2004, 55, 141–172. [Google Scholar] [CrossRef] [PubMed][Green Version]
  8. Besada, C.; Sanchez, G.; Gil, R.; Granell, A.; Salvador, A. Volatile metabolite profiling reveals the changes in the volatile compounds of new spontaneously generated loquat cultivars. Food Res. Int. 2017, 100, 234–243. [Google Scholar] [CrossRef] [PubMed]
  9. Suzuki, K.; Okadome, H.; Nakamura, S.; Ohtsubo, K. Quality evaluation of various “new characteristic rice” varieties based on physicochemical measurements. Nippon Shokuhin Kagaku Kogaku Kaishi 2006, 53, 287–295. [Google Scholar] [CrossRef][Green Version]
  10. Cantin, C.M.; Torrents, J.; Gogorcena, Y.; Moreno, M.A. Fruit quality attributes of new peach and nectarine varieties under selection in the ebro valley conditions (Spain). Acta Hort. 2009, 814, 493–500. [Google Scholar] [CrossRef][Green Version]
  11. Nunoo, J.; Ofori, E.; Quartey, E.; Gasu, E.; Ewusie, E.; Odai, B.; Torgby-Tetteh, W.; Agbemavor, W. Characterization of Some Physico-chemical Properties of F5 Breeding Lines of Tomatoes. Brasilian J. Appl. Sci. Technol. 2014, 4, 3967–3975. [Google Scholar] [CrossRef]
  12. Predieri, S.; Ragazzini, P.; Rondelli, R. Sensory evaluation and peach fruit quality. Acta Hort. 2006, 713, 429–434. [Google Scholar] [CrossRef]
  13. Lado, J.; Vicente, E.; Manzzioni, A.; Aresb, G. Application of a check-all-that-apply question for the evaluation of strawberry cultivars from a breeding program. J. Sci. Food Agric. 2010, 90, 2268–2275. [Google Scholar] [CrossRef]
  14. Hampson, C.R.; Quamme, H.A.; Hall, J.W.; MacDonald, R.A.; King, M.C.; Cliff, M.A. Sensory evaluation as a selection tool in apple breeding. Euphytica 2000, 111, 79–90. [Google Scholar] [CrossRef]
  15. Vaysse, P.; Reynier, P.; Roche, L.; Lavialle, O. Sensory evaluation of new pear cultivars. Acta Hort. 2005, 671, 341–347. [Google Scholar] [CrossRef]
  16. Varela, P.; Ares, G. Sensory profiling, the blurred line between sensory and consumer science. A review of novel methods for product characterization. Food Res. Int. 2012, 48, 893–908. [Google Scholar] [CrossRef]
  17. Ares, G.; Jaeger, S.R. Check-all-that-apply (CATA) questions with consumers in practice: Experimental considerations and impact on outcome. In Rapid Sensory Profiling Techniques and Related Methods: Applications in New Product Development and Consumer Research, 1st ed.; Delarue, J., Lawlor, B., Eds.; Woodhead Publishing: Sawston, UK, 2015; pp. 227–245. [Google Scholar] [CrossRef]
  18. Ares, G.; de Andrade, J.C.; Antúnez, L.; Alcaire, F.; Swaney-Stueve, M.; Gordon, S.; Jaeger, S.R. Hedonic product optimisation: CATA questions as alternatives to JAR scales. Food Qual. Prefer. 2017, 55, 67–78. [Google Scholar] [CrossRef]
  19. Maheeka, W.N.; Godfrey, A.J.R.; Ellis, A.; Hort, J. Comparing temporal sensory product profile data obtained from expert and consumer panels and evaluating the value of a multiple sip TCATA approach. Food Qual. Prefer. 2021, 89, 104141. [Google Scholar] [CrossRef]
  20. Tarancón, P.; Tárrega, A.; Aleza, P.; Besada, C. Consumer description by check-all-that-apply questions (CATA) of the sensory profiles of commercial and new mandarins. Identification of preference patterns and drivers of liking. Foods 2020, 9, 468. [Google Scholar] [CrossRef][Green Version]
  21. Ares, G.; Dauber, C.; Fernández, E.; Giménez, A.; Varela, P. Penalty analysis based on CATA questions to identify drivers of liking and directions for product reformulation. Food Qual. Prefer. 2014, 32, 65–76. [Google Scholar] [CrossRef]
  22. Oliveira, E.W.; Esmerino, E.A.; Carr, B.T.; Pinto, L.P.F.; Silva, H.L.A.; Pimentel, T.C.; Bolini, H.M.A.; Cruz, A.G.; Freitas, M.Q. Reformulating Minas Frescal cheese using consumers’ perceptions: Insights from intensity scales and check-all-that-apply questionnaires. J. Dairy Sci. 2017, 100, 6111–6124. [Google Scholar] [CrossRef]
  23. Ares, G.; Varela, P.; Rado, G.; Giménez, A. Identifying ideal products using three different consumer profiling methodologies. Comparison with external preference mapping. Food Qual. Prefer. 2011, 22, 581–591. [Google Scholar] [CrossRef]
  24. Bruzzone, F.; Vidal, L.; Antúnez, L.; Giménez, A.; Deliza, R.; Ares, G. Comparison of intensity scales and CATA questions in new product development: Sensory characterisation and directions for product reformulation of milk desserts. Food Qual. Prefer. 2015, 44, 183–193. [Google Scholar] [CrossRef]
  25. Aleza, P.; Juárez, J.; Cuenca, J.; Ollitrault, P.; Navarro, L. Recovery of citrus triploid hybrids by embryo rescue and flow cytometry from 2x × 2x sexual hybridisation and its application to extensive breeding programs. Plant Cell Rep. 2010, 29, 1023–1034. [Google Scholar] [CrossRef] [PubMed]
  26. Aleza, P.; Juárez, J.; Hernández, M.; Ollitrault, P.; Navarro, L. Implementation of extensive citrus triploid breeding programs based on 4x × 2x sexual hybridisations. Tree Genet. Genomes 2012, 8, 1293–1306. [Google Scholar] [CrossRef]
  27. Tarancón, P.; Giménez-Sanchis, A.; Aleza, P.; Besada, C. Selection of New Late-Season Mandarin Cultivars Based on Sensory Changes and Consumer Acceptance after Fruit Cold Storage. Agronomy 2021, 11, 116. [Google Scholar] [CrossRef]
  28. Food and Agriculture Organization of the United Nations. Available online: (accessed on 15 January 2021).
  29. Jimenez-Cuesta, M.; Cuquerella, J.; Martinez, J.M. Determination of a color index for citrus fruit degreening. Proc. Int. Soc. Citric. 1981, 2, 750–753. [Google Scholar]
  30. Morales, J.; Tárrega, A.; Salvador, A.; Navarro, P.; Besada, C. Impact of ethylene degreening treatment on sensory properties and consumer response to citrus fruits. Food Res. Int. 2020, 127, 108641. [Google Scholar] [CrossRef]
  31. Goldenberg, L.; Yaniv, Y.; Porat, R.; Carmi, N. Mandarin fruit quality: A review. J. Sci. Food Agric. 2018, 98, 18–26. [Google Scholar] [CrossRef]
  32. UNECE STANDARD FFV-14 Citrus Fruit. 2017. Available online: (accessed on 15 February 2021).
  33. Nijman, M.; James, S.; Dehrmann, F.; Smart, K.; Ford, R.; Hort, J. The effect of consumption context on consumer hedonics, emotional response and beer choice. Food Qual. Prefer. 2019, 74, 59–71. [Google Scholar] [CrossRef]
  34. Jaeger, S.R.; Xia, Y.; Le Blond, M.; Beresford, M.K.; Hedderley, D.I.; Cardello, A.V. Supplementing hedonic and sensory consumer research on beer with cognitive and emotional measures, and additional insights via consumer segmentation. Food Qual. Prefer. 2019, 73, 117–134. [Google Scholar] [CrossRef]
  35. Pierguidi, L.; Spinelli, S.; Dinnella, C.; Prescott, J.; Monteleone, E. Liking patterns moderate the relationship between sensory, emotional and context appropriateness profiles: Evidences from a Global Profile study on alcoholic cocktails. Food Qual. Prefer. 2020, 83, 103904. [Google Scholar] [CrossRef]
  36. Low, J.Y.; Lin, V.H.; Yeon, L.J.; Hort, J. Considering the application of a mixed reality context and consumer segmentation when evaluating emotional response to tea break snacks. Food Qual. Prefer. 2021, 88, 104113. [Google Scholar] [CrossRef]
  37. Sanchez, G.; Besada, C.; Badenes, M.L.; Monforte, A.J.; Granell, A. A non-targeted approach unravels the volatile network in peach fruit. PLoS ONE 2012, 7, e38992. [Google Scholar] [CrossRef][Green Version]
  38. Baviera-Puig, A.; García-Melón, M.; Ortolá, M.D.; López-Cortés, I. Proposal of a New Orange Selection Process Using Sensory Panels and AHP. Int. J. Environ. Res. Public Health 2021, 18, 3333. [Google Scholar] [CrossRef] [PubMed]
  39. Besada, C.; Salvador, A.; Sdiri, S.; Gil, R.; Granell, A. A combination of physiological and chemometrics analyses reveals the main associations between quality and ripening traits and volatiles in two loquat cultivars. Metabolomics 2013, 9, 324–336. [Google Scholar] [CrossRef]
  40. Risso, D.; Morini, G.; Pagani, L.; Quagliariello, A.; Giuliani, C.; De Fanti, S.; Sazzini, M.; Luiselli, D.; Tofanelli, S. Genetic signature of differential sensitivity to stevioside in the Italian population. Genes Nutr. 2014, 9, 401. [Google Scholar] [CrossRef]
Figure 1. Correspondence analysis for Cluster 1 (A) and Cluster 2 (B).
Figure 1. Correspondence analysis for Cluster 1 (A) and Cluster 2 (B).
Agronomy 11 02243 g001
Figure 2. Penalty analysis. Impact on the average liking scores of the “must-have” and “must-NOT-have” attributes for Cluster 1 (A,C) and Cluster 2 (B,D) consumers. Only the attributes with a significant impact on liking (p < 0.05) were included. For each attribute, the percentage of consumers who selected it for their ideal mandarin was indicated (see brackets).
Figure 2. Penalty analysis. Impact on the average liking scores of the “must-have” and “must-NOT-have” attributes for Cluster 1 (A,C) and Cluster 2 (B,D) consumers. Only the attributes with a significant impact on liking (p < 0.05) were included. For each attribute, the percentage of consumers who selected it for their ideal mandarin was indicated (see brackets).
Agronomy 11 02243 g002
Figure 3. MFA of the CATA question considering liking scores for each of the two identified clusters as active datasets and attributes selected to describe samples and ideal as Supplementary Data. (A) Vocabulary and liking (Cluster 1: red font; Cluster 2: green font). (B) Samples representation.
Figure 3. MFA of the CATA question considering liking scores for each of the two identified clusters as active datasets and attributes selected to describe samples and ideal as Supplementary Data. (A) Vocabulary and liking (Cluster 1: red font; Cluster 2: green font). (B) Samples representation.
Agronomy 11 02243 g003
Table 1. Origin of varieties.
Table 1. Origin of varieties.
Tri-707Triploid hybrid obtained by open pollination of cv. Clemenules 4x
Pri-88Triploid hybrid obtained by controlled pollination of cv. Clemenules 4x (♀ parental) and Satsuma mandarin(♂ parental)
Pri-89Triploid hybrid obtained by controlled pollination of cv. Clemenules 4x (♀ parental) and Satsuma hybrid (♂ parental)
Pri-90Spontaneous mutation of clementine cv. Oronules
ClemenulesSpontaneous mutation of clementine cv. Fina
OronulesSpontaneous mutation of clementine cv. Fina
Table 2. Physico-chemical parameters of the six mandarin cultivars.
Table 2. Physico-chemical parameters of the six mandarin cultivars.
CultivarsFirmness (%def)TSS 1 (°Brix)TA 2 (g citric ac./100 mL)MI 3 (TSS/Ac)JY 4 (%juice)Weight (g)
Pri-882.2 a10.6 b1.05 c10.1 a31.9 a219.7 f
Pri-893.8 b13.7 e1.43 d9.6 a47.0 c107.8 c
Pri-904.6 c11.2 c0.87 ab12.9 c40.1 b123.0 d
Tri-7075.4 d12.7 d0.93 b13.6 c46.2 c58.8 a
Oronules3.8 b10.7 bc0.93 b11.5 b43.5 b140.0 e
Clemenules5.8 d9.2 a0.83 a11.1 b47.2 c88.4 b
1 TSS: total soluble solids, 2 TA: titratable acidity, 3 MI: maturity index, 4 JY: juice yield. Different letters in the same column indicate significant difference among cultivars, according to LSD test (p ≤ 0.05).
Table 3. Mean liking scores and purchase intention of all consumers (Global) and from cluster 1 (Cl1) and cluster 2 (Cl2) consumers.
Table 3. Mean liking scores and purchase intention of all consumers (Global) and from cluster 1 (Cl1) and cluster 2 (Cl2) consumers.
LikingPurchase Intention (%)
CultivarGlobalCl 1Cl 2GlobalCl 1Cl 2
Pri-884.1 a5.0 a3.4 a17.926.211.0
Pri-896.1 c6.8 c5.5 b53.762.347.9
Pri-906.1 c6.1 b6.3 c50.747.554.8
Tri-7076.4 c7.5 d5.5 b56.782.037.0
Oronules6.1 c6.4 b5.7 cb50.039.345.2
Clemenules4.8 b6.1 b3.8 a21.655.78.2
Liking was evaluated on a 9-point hedonic scale; scores not sharing letters within each column were significantly different (p ≤ 0.05), according to Kruskal–Wallis test. Purchase intention is expressed as the percentage of consumers that responded that they were willing to buy (probably would buy + definitely would buy).
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Share and Cite

MDPI and ACS Style

Giménez-Sanchis, A.; Tárrega, A.; Tarancón, P.; Aleza, P.; Besada, C. Check-All-That-Apply Questions including the Ideal Product as a Tool for Selecting Varieties in Breeding Programs. A Case Study with Mandarins. Agronomy 2021, 11, 2243.

AMA Style

Giménez-Sanchis A, Tárrega A, Tarancón P, Aleza P, Besada C. Check-All-That-Apply Questions including the Ideal Product as a Tool for Selecting Varieties in Breeding Programs. A Case Study with Mandarins. Agronomy. 2021; 11(11):2243.

Chicago/Turabian Style

Giménez-Sanchis, Adrián, Amparo Tárrega, Paula Tarancón, Pablo Aleza, and Cristina Besada. 2021. "Check-All-That-Apply Questions including the Ideal Product as a Tool for Selecting Varieties in Breeding Programs. A Case Study with Mandarins" Agronomy 11, no. 11: 2243.

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop