Next Article in Journal
Transformational and Transactional Factors for the Successful Implementation of Enterprise Architecture in Public Sector
Previous Article in Journal
Decision-Making for Risk Management in Sustainable Renewable Energy Facilities: A Case Study in the Dominican Republic
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Visual Prompts or Volunteer Models: An Experiment in Recycling

1
Sustainable Behaviour Research Group, Department of Environmental Science and Engineering, Fudan University, Shanghai 200433, China
2
Values and Sustainability Research Group, University of Brighton, Watts Building, Lewes Road, Brighton BN2 4GJ, UK
*
Author to whom correspondence should be addressed.
Sustainability 2016, 8(5), 458; https://doi.org/10.3390/su8050458
Submission received: 15 March 2016 / Revised: 26 April 2016 / Accepted: 4 May 2016 / Published: 9 May 2016

Abstract

:
Successful long-term programs for urban residential food waste sorting are very rare, despite the established urgent need for them in cities for waste reduction, pollution reduction and circular resource economy reasons. This study meets recent calls to bridge policy makers and academics, and calls for more thorough analysis of operational work in terms of behavioral determinants, to move the fields on. It takes a key operational element of a recently reported successful food waste sorting program—manning of the new bins by volunteers—and considers the behavioral determinants involved in order to design a more scalable and cheaper alternative—the use of brightly colored covers with flower designs on three sides of the bin. The two interventions were tested in a medium-scale, real-life experimental set-up that showed that they had statistically similar results: high effective capture rates of 32%–34%, with low contamination rates. The success, low cost and simple implementation of the latter suggests it should be considered for large-scale use. Candidate behavioral determinants are prompts, emotion and knowledge for the yellow bin intervention, and for the volunteer intervention they are additionally social influence, modeling, role clarification, and moderators of messenger type and interpersonal or tailored messaging.

1. Introduction

Waste is a problem worldwide, and is especially acute in urban areas where land is less available for large scale waste processing in landfills or incinerators. Ironically, some components such as organic waste could be used to combat other problems: for example if used to replace some fossil fuels with biogas, or to produce soil conditioners that may reduce the need for artificial fertilizers and can also increase carbon content held in soils. Organic waste is the component of waste most urgently needing an alternative route: it produces methane in landfills and requires added fuels in incinerators as it contains so much water. Commercial food waste, and “green” waste from gardens can be easily be collected, but food and kitchen waste from households has been challenging, especially in dense urban areas. Cities around the world are trying many approaches, but the success stories are still slow to come.
Successful citywide recycling of residential waste must start with successful segregation at source—the residents need to segregate or sort their waste [1]. This practice is now well established in many countries for more valuable recyclable material streams such as plastic bottles, paper, metal cans, clean paper and cardboard, although segregation in high-density urban areas with high-rise apartments is generally significantly less successful. Food waste has proved much more challenging; some success is reported, mostly for special small-scale programs (see for example [2,3]) mostly in non-academic literature, and usually for less dense urban areas. Effective approaches for cities and multiple-occupancy housing are still in early development and rare.
Source segregation requires the involvement of residents, and scores of studies have explored different ways to encourage this. Many studies focus on the effect of different types of interventions on recycling behavior, such as door-stepping campaigns [4,5], financial and in-kind incentives [6], feedback [7] or awareness raising education [8]. Recycling programs can be variously driven by government bodies, businesses, communities, non-governmental organizations (NGOs) or combinations of such stakeholders. The underlying motive can be for reduction of public costs (e.g., public health needs met with sanitary landfills), for reduction of resource use (e.g., in requiring fuel to run incinerators with low calorific waste), for circular economies (e.g., biogas and fertilizers from organic waste), or for pro-environmental reasons (reducing production of greenhouse gases). However, despite these various approaches to recycling and source segregation in general, no frameworks for guidance have emerged from academia that are made use of in practice, for planning purposes. For example, academic areas focusing on behavior change have produced tens of theories and constructs, but they are reported to not be predictive or useful for planning details of large-scale, real life programs, e.g., for recycling [9,10,11].
In this context of a clearly defined pragmatic need and an unclearly defined academic approach, this study has been designed to answer two explicit calls for progress that are in the literature. The first call is for future intervention studies, regardless of their fundamental approach, should more carefully identify behavioral determinants (i.e., psychology based) either singly or in combinations (such as they might in practice) (see, for example [11,12,13]). The reason is that reporting practices currently focus often on operational parameters such as “door-stepping” without careful consideration of the range of behavioral determinants involved in a significant way [5] thus holding back greater bridging between theory and practice, and thus improvements in large scale implementation. The second call is for intermediary-scale research in tandem with policy makers and business to do the bridging work needed to translate the insights from behavior science in small studies, into scaled interventions that are effective [14,15]. We take these two calls into account by first considering the most significant operational element of a (rare) known, large scale, highly successful intervention type recently reported [16]: that of volunteers taking shifts beside recycling bins for four hours a day for 2–3 months. As interviews had revealed that prompting, positive emotion and interpersonal interactions were the main determinants of the volunteer shifts which were perceived to be effective in changing behavior, we designed an intervention which was a more practical operational alternative and less complex with respect to determinant analysis: we covered the recycling bins on three sides with bright yellow sunflower covers. We then tested the new intervention against the original one, and a control group, using the tangible weighing of sorted (recycled) materials as a measure of behavior. This was complimented by an exploratory study via interviews to indicate if the same behavioral determinants were perceived to be triggered. The primary aim was thus to design and test a new type of intervention which was operationally less expensive but still effective—a goal in waste management—by considering behavioral determinants. The secondary aim was to produce data which explored variations in behavioral change determinants that could potentially feed into psychology based research, as specifically called for by Steg and Vlek [12] and Abrahamse et al. [9]. In this way contributions to both practical large-scale programs and academic studies might be made.
Our context is the pilot trials for food waste source segregation (“recycling”) in Shanghai Municipality which started in 2010. Their emphasis was information provision, and a study of the 5000 communities (circa five million households) found the result that the “recycled” waste was indistinguishable from the “residual waste” via compositional analysis. This was in contrast to outstanding success—a 95% purity rate—in similar communities using a different approach led by a non-governmental organization (NGO) and described by them as “more personal” [17]. An in-depth study of one of those successful communities (N = 1600 households) indicated that the use of volunteers to stand beside newly installed communal food waste recycling bins during peak hours for three months to encourage residents and answer questions was a key element [16], contributing to role clarification, seriousness of message, interpersonal influences, and habit formation. The result was an outstanding 70% food waste capture rate with a negligible contamination level (<1%). Importantly, the effect was still strong 74 weeks later, with a 45% capture rate. Although these studies showed that the NGO-led approach is effective, it is not always possible to recruit so many volunteers for so many hours, and for such a long period, and even when they are not paid there are costs involved in training and organizing them.
For this pragmatic reason, the current study was designed to explore a non-human, cheaper, and easily scalable alternative intervention that did not require specialists. This was done by considering the candidate behavioral determinants involved, developing a new intervention and then testing it experimentally.

2. Materials and Methods

2.1. Design of the Study

In the successful NGO-led Shanghai program there were several key operational elements for success identified, obtained by in-depth semi-structured interviews with key informants from the main actors [16]. Elements included clarification of roles and responsibilities; the usefulness of a “broker” (here, the NGO) to co-develop new boundaries for stakeholder responsibilities; the serious intention of the local government to implement the policy; relationships with the local government; and the use of volunteers to demonstrate and interact on a personal level with residents. Several actors were involved, but the volunteers were cited as key in most elements. Thus, the volunteer element was chosen for special study.
In the work reported here, the element of the volunteers was isolated, analyzed and reconfigured for experimental testing. The use of volunteers to stand next to the recycling bins at peak hours to remind and encourage residents to recycle, to physically demonstrate how to do it, and to answer questions, was to be reproduced. Candidate behavioral determinants as proposed in the study by Xu et al. (2016) [16] are mentioned here, and summarized in Table 1, and then further nominations are discussed, and used in designing the intervention. The interpersonal interaction of the volunteers with the residents was seen as key, but more as a moderator to strengthen effects of behavioral determinants. For example, the residents had already been provided with basic knowledge about the program at the launch, with leaflets given out, posters and signage. However, being individually told (verbally) by volunteers seemed to make an extra impact on embedding the knowledge received. Furthermore, the residents generally responded to “serious” requests from the local government for action, but many information-based policies were not seen to be serious, and the presence of the volunteers made that distinction to them. Similarly, the residents had information sources stating it was their responsibility to get their food waste, separated from residual waste, to the correct bins, as opposed to having cleaners or machines to separate it. However, having the volunteers reinforce this made a deeper effect of role clarification. There were several mentions of a respect for the volunteers and an affiliated feeling of motivation via emotion: a strong emotion although only described as slightly positive [16].
Referring to the wider literature, social influences include a broad domain of determinants of behavior change. It has been reported that despite a proliferation of social influence approaches for behavior change, there is a lack of systematic research regarding the extent of their contribution and the conditions under which they are most effective [18]. More specifically, most intervention studies do not link the specific social influence mechanisms to the behavior changes, especially non-lab based ones [9], and there is a tendency for real-life interventions to contain several behavior change determinants, without further work to separate out the impacts [12]. These issues are reflected in the complexity in the NGO-led program, as the volunteers could trigger different types of social influences due to being local community members, having local social networks, representing the local government and its message, eliciting human-human respect for their long hours, or being seen as supervisors. Thus, the design of the alternative intervention in the experimental study would be greatly simplified by omitting the use of volunteers completely, if possible, and thus removing associated social influences.
Another potentially important candidate determinant in the NGO-led intervention originates from the volunteers’ modeling—physically demonstrating the act—of emptying bags into the correct bin. Several researchers have argued that subtle modeling enhances the use of signs and prompts in increasing pro-environmental behavior [19,20]. Huffman et al. (1995) [21] indicated visual prompts are less consistently effective in directing behavior than modelers. In social learning the process of learning through observation of others is well established [22] (as opposed to e.g., verbal instruction), and a wide range of behaviors have been shown to be increased if artificial “bystanders” are placed so that they can model the behavior to others, for example to reduce litter in public places [23], and sign a petition [24]. Although modeling was not flagged as a key aspect of the impact of the volunteers in the original study we realized its importance from this literature.
The fact that the original volunteers wore brightly colored tabards also indicated the possibility of them being effective prompts—not only reminders to those arriving at the bins, but even to passers-by who could see them from a distance. Sussman and Gifford (2013) [25] compared the effectiveness between modelers and prompts on waste diversion in a student cafeteria. The “modelers” in that case had some similarities to our volunteers in that they performed the act of the correct segregation of the waste in front of others. It was found in a 2-by-2 experiment that the effects of the modeling were much greater than signs alone, and the combination of both did slightly better [25].
Bearing these determinants in mind, and in particular the usefulness of removing complex social influences and retaining prompting, we designed an “alternative” intervention which involved covering the recycling bin on three of its sides with a noticeable, bright, colorful and pleasant design (Figure 1). This was thought to be a possible proxy source for the emotional impact of the volunteers e.g., through their brightness or style, and so is the subject of this study. If found to be true, it would meet the requirements for a well-defined, cheap, and easily transferrable action applicable in most places worldwide: bright bins.
A bright color rather than just a different color was necessary in our design to ensure residents would be prompted when passing by, not just on arrival at the bins when they had already sorted or not sorted their waste. Bennett [26] pointed out that a pleasant or funny visual prompt may be successful because it creates a positive emotional state in the reader. This knowledge led us to design our experiment with a cheerful and pleasant bin cover. Since residents were told the food waste could be composted, we choose a cover with large yellow sunflowers for this association. From experience we also knew that the cover should appear clean and bright to mitigate any possible negative emotions associated with a bin, which is inherently considered to be dirty: this ruled out several alternatives e.g., with multi-colored flowers or complex landscapes, which can appear less clean looking from a distance.
We note here that although this study is concerned not with the recycling of “dry” recyclables such as plastic bottles and cans, but with food waste sorting (so that it can later be used for a useful purpose such as compost or bio-gas or pig food—i.e., “recycled”), they are both in the context of residential waste sorting, and there are significant overlaps of behavioral determinants, typical program designs, challenges and solutions [11].

2.2. Site Details

The researchers chose to carry out the study in their resident city of Shanghai, for convenience and because of their access to contextual and recent historical information about related food waste sorting and other programs: food waste pilot programs had been taking place since 2011, sometimes involving the use of local volunteers [5], but almost all with poor results [1], and alternative approaches were being explored [16]. The NGO which we generally worked alongside had been allocated a large new set—thirty—of randomly chosen communities in compounds, in one of the sixteen districts, so we decided to select one of those for this project. Designated as Community #30, it was selected because it comprised a large number (23) of similar apartment blocks to choose from to randomly allocate to interventions. The only difference was that some had fewer stories (floors) than others. Each had its own waste station in a convenient area outside the main entrances but fully covered, which meant that rain would not affect the disposal patterns and thus collection of valid data. Of these, 13 buildings were chosen for baseline measurements, as this was the maximum number that the research team could manage for data collection at any given time: the tallest were chosen to maximize the number of households involved—circa 100 each.
Other community characteristics were important in choosing the site. For accurate and valid objective measurements of the residual waste and segregated food waste, the catchment areas of the bins would need to be contained (i.e., the waste not being removed or added to before measurement) and inaccessible to residents from elsewhere. The waste stations should be out of sight of each other, thus limiting any cross-building influences, yet the bins should be visible from a distance by the residents of the related building to allow prompting at a distance. The residents should be accessible for later interviews so that richer, qualitative data on the perceived influences on behaviors could be explored. It was typical of centrally planned residential communities in Shanghai in that it was informally gated and walled, and had communal garden areas between the buildings, and parking areas, manned by site-specific cleaners.
The local level of government, which was the Community Committee for this residential compound, reported that the residents in each building had similar demographics. However, as with our general experience in Shanghai, it was not appropriate to be provided with details or to use questionnaires to obtain them. A baseline measurement was thus taken one week after the program launch to indicate any variations in behavior due to innate population differences. Buildings were then randomly allocated to the two interventions and a control group from those with similar baselines. The interventions were carried out in November 2015.
The first intervention was the use of a volunteer to stand near the bins to model the waste sorting, to inform residents of the food types applicable, and to be positive and encouraging. It would reproduce the volunteers element of the NGO-led intervention reported in Xu et al. (2016) [16] as much as possible, but only one, not two volunteers was able to be resourced, and for only one hour, twice a day, rather than two. The second intervention was to cover three sides of the food waste bins with brightly colored, (yellow) sunflower pictures (Figure 1): no volunteers would be present. The control group had neither volunteers nor bright bin covers: the conditions remained the same before and after baseline measurements were taken for all buildings.
Two buildings would be randomly allocated to each experiment condition, and measurements of the food waste sorting behavior would be taken twice. In total, four independent measures of the result in each condition would thus be available.

2.3. Program Launch

Prior to the program launch the residents were not separating out food waste. Common practice was that residents fill small bags of waste as it arose in the kitchen, e.g., four-liter size, and to frequently take them downstairs to the communal bins, which were typically one or two, 220-liter rectangular, dull orange plastic bins, with open lids. These bins were emptied several times a day by the on-site “cleaner”, but left un-cleared through the evening and early morning until 7:00 a.m. each morning. It is common practice for residents to save up their valuable recyclables and sell them to visiting informal waste collectors for cash and so they are not a significant part of the waste stream in this study.
The food waste separation program was launched to the entire community with the use of several tables in the central communal garden areas in October 2015. Young volunteers from a local NGO gave out three-liter kitchen caddies to passing residents, as well as information on the type of waste to sort, and demonstration of the new (brown) 120-liter bin to represent the new ones to be found at each buildings’ waste stations. Residents who did not obtain their caddy at this event had them delivered or dropped off by their block leaders. From that day onwards, 1–2 new 120-liter, deep brown colored, lidded wheelie bins were available adjacent to the normal, dull orange colored bins used for residual waste. The wheelie bins were labeled, and informative posters were provided near each bin, indicating acceptable items. The cleaner was given full information on the upcoming trials, and asked not to interfere with them.

2.4. Methods for Evaluation of Behavior

There are various indicators and data collection methods commonly used in the study of waste and recycling management, and in behavior change. Our work is focused on developing results that are directly useful and meaningful to government and business practitioners, and so the common practice of using surveys for self-reported attitudes was not used, as they are known to be both exaggerated and irregularly related to actual practice [27,28]. Participation rates are often used for households living in houses, where each house has a recycling container, which can simply be counted [3]. But in this work the residents from each building used a communal bin—sometimes several times a day—and thus accurate participation rates were not feasible. Recycling Rates can measure the ratio of kg of recycled material (contaminated or not) to the kg of total waste, but another indicator which is more useful is the Capture Rate (CR) [29], (also known as the Diversion Rate, or Source Separation Rate [30]) representing how much of the total food waste present in ALL bins (recycling or residual) is successfully diverted from the mixed waste. To calculate that, it is necessary for researchers to collect all of the waste in both bin types and determine the composition of food waste (FW) and non-food waste (nonFW) in each: the “Food Waste Capture Rate”, CR FW, can then be calculated:
CR FW = 100 × FW (in recycling bins) / (total FW in all waste)
where FW denotes food waste (with any contamination removed).
Most indicators can be misleading if there is significant contamination which is not quantified. For example, if all the waste, completely unsorted, were to be placed in recycling bins, the Capture Rate (CR) would appear to be excellent at 100% by definition, whereas in fact the recycling would be useless. It is therefore necessary to use some measure of the contamination levels in detailed studies. In our case with this is defined as:
Cont.L = the percentage of nonFW in the recycling bin = [100 × (nonFW/(FW + nonFW))] recycling bins
The general definition, applicable to any specified material type, is also known as the mis-sorting ratio or purity of recyclables [4,30].
Different city programs have different thresholds of contamination, so it is not possible to develop a methodological standard from that perspective e.g., to set a reference. A further complication is that the Cont.L in any situation will vary depending the typical percentage of nonFW present in the waste generally—which varies e.g., by season and city. For this reason we decided to obtain both CR and Cont.L, but to then introduce a new indicator, which sensibly combines and communicates the usefulness of both CR and Cont.L. We denote it the Effective Capture Rate, effCR:
effCR = βCR
where β = (proportion of nonFW-Cont.L)/(proportion of nonFW) and where (proportion of nonFW) = [nonFW/(nonFW + FW)] all bins.
This indicator, effCR, provides a much better indication of the behavior of the residents, because the higher the contamination levels, the closer to zero it gets. But when the contamination level is low, the values of effCR and CR will approach each other. The value β can be negative when the bin is heavily contaminated.
Typically in waste management studies a sample of otherwise large amounts of daily or weekly collected waste is taken, and separated into the classifications of interest. In this case there were just two categories used: food waste (FW) and non-food waste (nonFW). To be representative it is generally considered necessary to sample 91 kg of the total waste when classifying for n ≤ 10 categories [31]. In this study no sampling was carried out: All of the waste was analyzed for each building, collected over 24 h. Each building produced more than 91 kg in almost every case. The proportion of nonFW was calculated across the entire set for each day of measurement.

2.5. Setting Up the Control and Two Trial Approaches

As mentioned above, N = 13 buildings had baseline measurements taken (shown in Table 2). Normality tests showed these had a normal distribution and ANOVA tests showed that n = 12 had similar baselines. Two each were randomly allocated to the conditions of Control (C), Yellow Bins (Y), or Volunteers (V).
The food waste bins at buildings Y1 and Y2 were wrapped on three sides with colorful covers designed for the purpose and publicly available online for purchase for that purpose. In this case a brightly colored design with large yellow sunflowers was used (see Figure 1). The back side of the bins was not covered as it is not usually visible. These bins were prominently visible. The bin covers were regularly checked and replaced if needed, but they were waterproof and oil-proof, so they did not lose their effect easily.
In the buildings labeled V1 and V2 we arranged for experienced volunteers to take turns to stand singly in the vicinity of the bins, between 7:30 and 8:30 a.m. and between 6:30 and 7:30 p.m., for three consecutive weeks before the measurements were taken. Volunteers were instructed to do the following, while at all times remaining pleasant and friendly: (a) encourage residents to start and continue doing the waste sorting; (b) politely demonstrate (model) and explain (verbally) to the residents how to do the waste sorting if they did not know; (c) help residents to physically do the sorting if they did it incorrectly; and (d) never blame residents if they were not doing well.
Two buildings labeled C1 and C2 were controls. They had neither decorated, colorful bins, nor volunteers: only the same new bin types, labels and posters as used in the other buildings, with no changes before and after the baseline measurements.

2.6. Exploratory Qualitative Data Collection

In addition to the quantitative evaluation data to measure the waste sorting behavior, an appropriate form of qualitative data collection was designed to provide exploratory information on the possible behavioral determinants involved. Although in some countries a comprehensive questionnaire might be suitable to systematically obtain responses from residents on a multitude of potential determinants, previous experience of the researchers in Shanghai suggested that residents would not engage properly with that approach but instead give superficial answers, biased for political and social correctness. It was agreed with indigenous researchers that residents were most likely to give an unbiased answer if open questions were asked face to face, and in a manner which clearly gave them freedom to express themselves in their own way, such as referring to ”other residents” instead of themselves personally, in some questions. General open questions were thus used for this purpose, in a semi-structured interview. However, experience also showed that residents did not typically pre-consider determinants to their behavior to the extent common in many other countries, and thus a list of candidate answers was provided after each open question, as prompts for more specific answers. The questions are listed in the results section.

3. Results

The weights of the food waste and non-food waste found in each of recycling and residual waste are given in Table 3 below, for the baseline measurements taken on the fifth day after the launch and for the two measurements on the 31st and 32nd days. The interventions started on the eightth day.
The results in Table 3 show that the control buildings had high contamination levels (48%) in three of the four post-intervention measurements. Since the general proportion of non-food waste was only 50% and 51% respectively, this suggested that no sorting was taking place, as reflected in the near-zero values of the Effective Capture Rate. To clarify whether the contamination could be sorted waste that had been left in the bags and then misplaced into the incorrect bin, a note was made of the heterogeneity of those bags. The findings were that virtually all contained unsorted waste. Thus, the deposited waste in the control building appeared to generally not be sorted.
The experimental groups had consistently low contamination levels Cont.L ≤ 19%, and Effective Capture Rate values which were consistently high at 23%–42%. Table 4 shows that the results from the two experimental groups were statistically different from the control group, but not from each other: the Tukey Test is used to make pairwise comparisons between means after the analysis of variance has been completed.
The exploratory interviews gave exploratory information about candidate determinants for the behavior (shown in Table 5). Of the eight residents interviewed from the volunteer buildings only two mentioned the volunteers, both stating that they considered the main job of the volunteer to be instructing residents in how to do the sorting. For instance, the volunteer shows them which bin is for which waste, what kind of waste belongs to the food waste category, and modeled the emptying of the food waste into the bin from the bag used to transport it. The other interviewees did not notice the volunteers. This low proportion of 2/8 = 25% is not surprising in that the volunteers were only present for two hours each day, and thus would have not been seen by residents who deposited their waste at other times. Although eight residents are not representative of the group (n = 200) if that low proportion is even roughly indicative of the number currently influenced by the volunteers to give the rather good result of an Effective Capture Rate of 34%, then the potential for even greater achievement with more exposure to volunteers seems possible.
One of the volunteers was also interviewed. He said that some residents commented on his efforts (using the Chinese expression, “xinku”, which is considered an emotional reaction), some indicated regret at not sorting better, and some later began to sort. He commented that in many instances he was able to, in an appropriately positive manner, physically assist residents to sort their waste if they had difficulty doing it correctly. He further commented that there were residents who were impatient and not interested in the waste sorting.
Four of the nine interviewees from the yellow bins buildings noticed the bright yellow bin for food waste. They said it was very obvious that this bin was for the food waste. One resident said she got to know about the waste sorting program because of the yellow food waste bin (knowledge).
The nine interviewees from the control buildings reported that their bins for different categories were confusing. They felt that the local community committee was not very responsible for the program, and indicated that they did not believe the program would be successful or make any difference.
Most of interviewees in all buildings mentioned that they initially started to do the sorting because of the activities on the launch day, or because of the distributed kitchen caddy. Most residents did not indicate a deep environmental awareness—something that the researchers have experienced generally in studies in Shanghai and which is the subject of a separate study, in progress. Some of them only know the waste sorting is “good for the environment”, i.e., using a particular government slogan. Many did not even mention the environment, but only that, “The government requires us to do the waste sorting, so it should be a good thing.”
The extra cost of the yellow bins intervention was small, at 28 US dollars (Table 6). The extra cost of the volunteer intervention was 231 US dollars. In view of the good results achieved, it seems reasonable to recommend that the yellow bins intervention represents excellent value for money and could be used more widely.

4. Discussion

This study set out to design an effective and scalable food waste sorting program element to replace a more costly one previously found to be key to success in a recently reported program [16]. This was done by considering the potential behavioral determinants that could be associated with that key element—the use of volunteers to stand beside bins to interact with residents—and thence to design an alternative operational intervention that could be tested alongside it in an experimental set up. In this way, we hoped, as a secondary aim, to also provide exploratory findings about impacts of related determinants in a real world context for use by behavior change researchers and thus to contribute in the long term towards better bridging between theory and practice.
The findings show that the alternative intervention of using covers on the recycling bins, which are bright yellow displays of sunflowers produced good results, with low contamination rates and good capture rates of the targeted food waste. The results are statistically similar to those obtained for the volunteer intervention. As the cost of the yellow bin covers and their application is very low and their implementation simple to achieve, this indicates that, operationally in terms of waste management, it would be very worthwhile to make use of this intervention.
As shown in Table 1, the yellow bin intervention is thought to differ by not involving determinants of social influences or modeling, nor interpersonal or tailored moderators. Its main determinants were thought to be prompting and emotion, with a possible contribution to knowledge. The interviews support this, with four of the nine specifying the prompting effect, and none of them being unclear about bin categories unlike the control group. One person volunteered that her “knowledge” about the program came from noticing the colorful bin. The possibility of emotional triggering was only indicated by nuances in comments: future studies might need to design a different approach for its measure.
The findings do not obviously suggest a strong prompting effect of the volunteers, as only two of the eight interviewed specified it. Although this initial result seems low compared to the number prompted by the yellow bins, the latter were visible all day whereas the volunteers were only present one hour per day. Concerning levels of knowledge, no one indicated uncertainty of waste categories as a hindrance, unlike the control group, which might suggest the volunteers might have influenced knowledge levels. Of the two interviewees who mentioned the volunteer, neither implied an emotional engagement: a larger number of interviews or more in-depth ones might be needed to determine that in future work. A related topic more developed in human-computer-interaction (HCI)literature is the use of technology to “open space” to raise awareness and break habits [32]; there may be links between the effects of our yellow bins and technology which issues reminder, and this could be an area for further consideration.
The consistently high number of references of interviewees to the kitchen caddy deserves further careful attention. As none of the residents were observed to be actually using them, (consistent with studies in five other communities), it would be worthwhile to find ways of determining if their impact was due to being “silent reminders” in the kitchen (even if unused), or because of being viewed as “gifts”, or as artifacts which embedded the seriousness of the intention of the authorities to implement this particular policy.
In this study the control groups had very similar conditions to the 5000 communities in the pilot government implementation of the policy: that strategy emphasis was on information provision, kitchen caddies and new, brown communal recycling bins. The results were also similar: no discernible food waste sorting was found to be taking place [17]. The results of our control group are thus entirely consistent with the results of that wider study.
Beyond the main aim of this work—to design an effective scalable program—it is worthwhile to reflect on several related variables and complexities to consider as discussed in the literature, and we discuss them here.
Both interventions involved prompting, but the volunteers also modeled to residents. Modeling is known to be more consistent in effectiveness compared signs [21], and visual prompts [33]. A combination of modelers and signs has been shown to work well in three small-scale studies [34,35,36]. Our findings of similar Effective Capture Rates with both interventions might seem inconsistent with those other studies, but then we have to remember that our volunteers did not overlap hugely with residents, as they were only present two hours a day. The observation of modeled behavior is also well established in social learning as an effective way to learn [21] and in fact the observation of two modelers interacting has been found to be even more effective than one [35,36]. Thus, the apparently small change from two to one volunteer may have been an important one. We suggest that future studies take these points into account carefully.
The use of pleasant visual prompts has been suggested as successful because it promotes a positive emotional state [26], and the bin covers used may have done that to some extent. In the field of waste management a surprising result was found by McDonald and Oates (2003) [37]: that the appearance of the bins (especially the “terrible color”) was given as one of the main reasons for not participating in recycling scheme. They thus suggested that bins should be conceptually redesigned. Volkswagen (2009) [38] recently initiated the “Fun Theory” program, to advocate the concept that “fun” was a simple way to change people’s behavior towards sustainable living. Bins painted to resemble cows—“moo bins”—took the place of normal large garbage bins to promote a sense of fun, with the intention to thus make a difference in waste recycling. “Funology” has been also developed in more academic areas [39] in terms of playful design, but there is no clear link with associated systems of knowledge in psychology or behavior change.
Another psychological and operational factor that is mentioned occasionally but not systematically reported, is personal interaction. When the volunteers were present they acted as interactive sources of information, tailored to the needs of each individual. Some studies have used “tailored information” as a category of its own [9], suggesting it makes “information” more salient and thus more deeply embedded. However, our volunteers can also be considered to differently influence determinants of behavior change because they involve personal, one-to-one and face-to-face interaction with residents. This factor is often overlooked as a key aspect in intervention studies: an example is in reports of “tailored” approaches of information or feedback in energy reduction studies where it is emphasized as a variation of a type of information rather than personal interaction. Abrahamse and Steg (2013) [18] indicated that face-to-face interaction can make social influence more powerful compared to an information and feedback approach. Delmas et al. (2013) [40] also suggested that information provided personally is more effective than information delivered through media. A study of student accommodation found that commitments obtained through personal contact resulted in higher recycling rates than commitments through indirect contact or the provision of information [41,42]. We suggest that future studies try to separate out the two.
Ideally measurements should be taken for longer periods in order to understand the extent to which the different approaches help embed new and lasting behavior. Although this would have been useful in principle, we note here that the calls in many papers for more durability (or longevity) studies are often unrealistic, as in non-laboratory, real-life studies there are always many parameters which can cause variations and decreases in recycling rates which are not often mentioned, such as labels getting dirty, bins looking unclean, and turnover of residents which often run at 30% per year. It is unlikely that any studies of durability would be valid without taking such factors into account.
In summary, we have tried to link elements of successful interventions with candidate determinants, especially in the vocabulary of (psychology based) behavior change researchers (instead of, e.g., waste management researchers). However, we would note here that this is an onerous task on the context of so many determinants used variously in the literature, not only lacking a generalized uniting or linking theory but actually multiplied by variations in definition when affiliated to constructs of specific theories. We note that a framework such as that developed and well used in public health by Michie (2005) [43] would be useful: it reduced 125 constructs from 37 theories into 12–14 domains of determinants which were more relatable to practice. By considering the determinants involved in a recently reported rare successful program, a scalable and cheaper alternative was developed.

5. Conclusions

This study built on that of Xu et al. (2016) [16] by adapting a proposed key element of a known successful intervention for experimental testing, against an alternative intervention which was simultaneously designed to be useful pragmatically in the field and in using a less complex mix of determinants. The design approach made use of the concepts of domains of determinants used by behavior change researchers.
The two interventions were the use of bright yellow bin covers, and the use of volunteer advisers standing beside normal bins. The results showed that both produced good results, with effective capture rates of 32% and 44% respectively.
These results are statistically different to those from the control group (effective capture rate of 7%), but not from each other. The new alternative intervention using bright bin covers had a very low additional cost to the control method, did not require specialist staff, and could easily be implemented by typical waste management operators. It is thus recommended that it be considered for general, large-scale use.
The volunteer intervention is more expensive and requires specialist trainers but we note that in this study the overlap time with residents did not produce a saturation effect, and thus the resulting waste sorting behavior could potentially be higher. Use of volunteers for more hours per day might increase results significantly. Due to a limitation in this work of only a small number of residents being interviewed, this issue would be better explored with a specially designed study focusing on post-intervention interviews e.g., for psychological factor exploration.
Candidate determinants of the waste sorting behavior were identified for further specific studies. For the yellow bin intervention they are prompts, emotion and knowledge, and for the volunteer intervention they are additionally social influence, modeling, role clarification, and moderators of messenger type and interpersonal or tailored messaging.
This work shows that a multiple-step approach to bridge behavior change theory-based systems of knowledge with pragmatic, operationally-based waste management knowledge can be effective. However, we suggest it would be improved with a bridging framework such as that used by Michie et al. (2005) [43] to ensure that no important domains of determinants are missed.

Acknowledgments

Marie K. Harder acknowledges her support via the China National Thousand Talents Program, which also provided funds for Open Access costs.

Author Contributions

Zi Yin Lin and Marie K. Harder conceived and designed the experiments; all authors performed the experiments; Zi Yin Lin, Xiao Wang and Marie K. Harder analyzed the data, and Zi Yin Lin and Marie K. Harder wrote the paper.

Conflicts of Interest

The authors declare no conflict of interest.

References

  1. Huang, W.F.; Wang, J.; Dai, X.Y.; Li, M.R.; Harder, M.K. More than financial investment is needed: Food waste recycling pilots in Shanghai, China. J. Clean. Prod. 2014, 67, 107–116. [Google Scholar] [CrossRef]
  2. Bench, M.L.; Woodard, R.; Harder, M.K.; Stantzos, N. Waste minimisation: Home digestion trials of biodegradable waste. Resour. Conserv. Recycl. 2005, 45, 84–94. [Google Scholar] [CrossRef]
  3. Woodard, R.; Harder, M.K.; Bench, M.; Philip, M. Evaluating the performance of a fortnightly collection of household waste separated into compostables, recyclates and refuse in the south of England. Resour. Conserv. Recycl. 2001, 31, 265–284. [Google Scholar] [CrossRef]
  4. Bernstad, A.; la Cour Jansen, J.; Aspegren, A. Door-stepping as a strategy for improved food waste recycling behaviour—Evaluation of a full-scale experiment. Resour. Conserv. Recov. 2013, 73, 94–103. [Google Scholar] [CrossRef]
  5. Dai, Y.C.; Gordon, M.P.R.; Ye, J.Y.; Xu, D.Y.; Lin, Z.Y.; Robinson, N.K.L.; Woodard, R.; Harder, M.K. Why doorstepping can increase household waste recycling. Resour. Conserv. Recycl. 2015, 102, 9–19. [Google Scholar] [CrossRef]
  6. Harder, M.K.; Woodard, R. Systematic studies of shop and leisure voucher incentives for household recycling. Resour. Conserv. Recycl. 2007, 51, 732–753. [Google Scholar] [CrossRef]
  7. Timlett, R.E.; Williams, I.D. Public participation and recycling performance in England: A comparison of tools for behaviour change. Resour. Conserv. Recycl. 2008, 52, 622–634. [Google Scholar] [CrossRef]
  8. Read, M.; Gregory, M.K.; Phillips, P.S. An evaluation of four key methods for monitoring household waste prevention campaigns in the UK. Resour. Conserv. Recycl. 2009, 54, 9–20. [Google Scholar] [CrossRef]
  9. Abrahamse, W.; Steg, L.; Vlek, C.; Rothengatter, T. A review of intervention studies aimed at household energy conservation. J. Environ. Psychol. 2005, 25, 273–291. [Google Scholar] [CrossRef]
  10. Darnton, A. GSR Behaviour Change Knowledge Review: Reference Report: An Overview of Behaviour Change Models And Their Uses; Centre for Sustainable Development, University of Westminster: London, UK, 2008. [Google Scholar]
  11. Jackson, T. Motivating Sustainable Consumption: A Review of Evidence on Consumer Behavior and Behavioral Change; A report to the Sustainable Development Research Network, Survey; Centre for Environmental Strategies: Guildford, UK, 2005. [Google Scholar]
  12. Steg, L.; Vlek, C. Encouraging pro-environmental behaviour: An integrative review and research agenda. J. Environ. Psychol. 2009, 29, 309–317. [Google Scholar] [CrossRef]
  13. Davies, J.; Foxall, G.R.; Pallister, J. Beyond the Intention–Behaviour Mythology: An Integrated Model of Recycling. Market. Theory 2002, 2, 29–113. [Google Scholar] [CrossRef]
  14. Allcott, H.; Mullainathan, S. Behaviour and energy policy. Science 2010, 327, 1204–1205. [Google Scholar] [CrossRef] [PubMed]
  15. Secondi, L.; Principato, L.; Laureti, T. Household food waste behaviour in EU-27 countries: A multilevel analysis. Food Policy 2015, 56, 25–40. [Google Scholar] [CrossRef]
  16. Xu, D.Y.; Lin, Z.Y.; Gordon, M.P.R.; Robinson, N.K.L.; Harder, M.K. Perceived key elements of a successful residential food waste sorting program in urban apartments: Stakeholder views. J. Clean. Prod. 2016. [Google Scholar] [CrossRef]
  17. Dai, Y.C.; Lin, Z.Y.; Li, C.J.; Xu, D.Y.; Huang, W.F.; Harder, M.K. Information strategy failure: Personal interaction success, in urban residential food waste segregation. J. Clean. Prod. 2016. [Google Scholar] [CrossRef]
  18. Abrahamse, W.; Steg, L. Social influence approaches to encourage resource conservation: A meta-analysis. Glob. Environ. Chang. 2013, 23, 1773–1785. [Google Scholar] [CrossRef]
  19. Geller, E.S. Actively caring for the environment—an integration of behaviourism and humanism. Environ. Behav. 1995, 27, 184–195. [Google Scholar] [CrossRef]
  20. Lutzenhiser, L.L. Social and behavioural aspects of energy use. Annu. Rev. Energy Environ. 1993, 18, 247–289. [Google Scholar] [CrossRef]
  21. Huffman, K.T.; Grossnickle, W.F.; Cope, J.G.; Huffman, K.P. Litter reduction—A review and integration of the literature. Environ. Behav. 1995, 27, 153–183. [Google Scholar] [CrossRef]
  22. Bandura, A. Social Learning Theory; Prentice-Hall: Oxford, UK, 1997. [Google Scholar]
  23. Cialdini, R.B.; Reno, R.R.; Kallgren, C.A. A focus theory of normative conduct—Recycling the concept of norms to reduce littering in public places. J. Personal. Soc. Psychol. 1990, 58, 1015–1026. [Google Scholar] [CrossRef]
  24. Bégin, G. Sex makes a difference. 1. Evidence from a modelling study conducted in a natural setting. Psychol. Rep. 1978, 43, 103–109. [Google Scholar] [CrossRef]
  25. Sussman, R.; Gifford, R. Be the change you want to see: Modeling food composting in public places. Environ. Behav. 2013, 45, 323–343. [Google Scholar] [CrossRef]
  26. Bennett, R. Customer recall of promotional displays at supermarket checkouts: Arousal, memory and waiting in queues. Int. Rev. Retail Distrib. Consum. Res. 1998, 8, 383–399. [Google Scholar] [CrossRef]
  27. Williams, I.D.; Kelly, J. Green waste collection and the public’s recycling behaviour in the Borough of Wyre, England. Resour. Conserv. Recycl. 2003, 38, 139–159. [Google Scholar] [CrossRef]
  28. Perrin, D.; Barton, J. Issues associated with transforming household attitudes and opinions into materials recovery: A review of two kerbside recycling schemes. Resour. Conserv. Recycl. 2001, 33, 61–74. [Google Scholar] [CrossRef]
  29. WRAP. Improving the Performance of Waste Diversion Schemes—A Good Practice Guide To Monitoring and Evaluation (WRAP Project EVA092–000); WRAP: Banbury, UK, 2010. [Google Scholar]
  30. Dahlen, L. To Evaluate Source Sorting Programs in Household Waste Collection Systems. Master’s Thesis, Lulea University of Technology, Lulea, Sweden, November 2005. [Google Scholar]
  31. Sharma, M.; McBean, E. A methodology for solid waste characterization based on diminishing marginal returns. Waste Manag. 2007, 27, 337–344. [Google Scholar] [CrossRef] [PubMed]
  32. Comber, R.; Thieme, A. Designing beyond habit: Opening space for improved recycling and food waste behaviors through processes of persuasion, social influence and aversive affect. Pers. Ubiquitous Comput. 2013, 17, 1197–1210. [Google Scholar] [CrossRef]
  33. VanHouten, R.; Nau, P.A.; Merrigan, M. Reducing elevator energy use—A comparison of posted feedback and reduced elevator convenience. J. Appl. Behav. Anal. 1981, 14, 377–387. [Google Scholar]
  34. Ferrari, J.R.; Chan, L.M. Interventions to reduce high-volume portable headsets—Turn down the sound. J. Appl. Behav. Anal. 1991, 24, 695–704. [Google Scholar] [CrossRef] [PubMed]
  35. Aronson, E.; O’Leary, M. The relative effectiveness of models and prompts on energy conservation: A field experiment in a shower room. J. Environ. Syst. 1982, 12, 219–224. [Google Scholar] [CrossRef]
  36. Sussman, R.; Greeno, M.; Gifford, R.; Scannell, L. The Effectiveness of Models and Prompts on Waste Diversion: A Field Experiment on Composting by Cafeteria Patrons. J. Appl. Soc. Psychol. 2013, 43, 24–34. [Google Scholar] [CrossRef]
  37. McDonland, S.; Oates, C. Reasons for non-participation in a kerbside recycling scheme. Resour. Conserv. Recycl. 2003, 39, 369–385. [Google Scholar] [CrossRef]
  38. Volkswagen 2009. The World’s Deepest Bin. Available online: http://www.thefuntheory.com/worlds-deepest-bin (accessed on 25 June 2015).
  39. Blythe, M.E.; Monk, A.F.; Overbeeke, K.; Wright, P. Funology; Kluwer Academic Publishers: Dordrecht, The Netherlands, 2004. [Google Scholar]
  40. Delmas, M.A.; Fischlein, M.; Asensio, O.I. Information strategies and energy conservation behaviour: A meta-analysis of experimental studies from 1975 to 2012. Energy Policy 2013, 61, 729–739. [Google Scholar] [CrossRef]
  41. Bryce, W.J.; Day, R.; Olney, T.J. Commitment approach to motivating community recycling: New Zealand kerbside trial. J. Consum. Aff. 1997, 31, 27–52. [Google Scholar] [CrossRef]
  42. Reams, M.A.; Ray, B.H. The effects of 3 prompting methods on recycling participation rates—A field-study. J. Environ. Syst. 1993, 22, 371–379. [Google Scholar] [CrossRef]
  43. Michie, S.; Johnston, M.; Abraham, C.; Lawton, R.; Parker, D.; Walker, A.; “Psychological Theory”, Group. Making psychological theory useful for implementing evidence based practice: A consensus approach. Qual. Saf. Health Care 2005, 14, 26–33. [Google Scholar] [CrossRef] [PubMed]
Figure 1. The “yellow bin” intervention: the recycling (food waste sorting) bin had a cover on three sides showing large, bright yellow sunflowers.
Figure 1. The “yellow bin” intervention: the recycling (food waste sorting) bin had a cover on three sides showing large, bright yellow sunflowers.
Sustainability 08 00458 g001
Table 1. Likely determinants/moderators of behavior relevant for each proposed experimental group.
Table 1. Likely determinants/moderators of behavior relevant for each proposed experimental group.
Volunteer GroupYellow Bin GroupControl Group
Interpersonal tailored *√√××
Social Influence√√××
Emotion√√×
Role clarification√√×
Messenger/authority *√√××
Modeling #××
Knowledge #√√
Prompting #×
* These are more likely to be moderators, than determinants, of behavior; # Determinants not specifically proposed in the original study of the volunteer intervention.
Table 2. Baseline measurements of the buildings allocated to the three experimental conditions of control, yellow bin or volunteer interventions.
Table 2. Baseline measurements of the buildings allocated to the three experimental conditions of control, yellow bin or volunteer interventions.
Control GroupYellow Bin GroupVolunteer Group
Building No.C1C2Y1Y2V1V2
Baseline effCR0.230.210.160.130.170.22
SD0.010.020.04
p-value a0.12 a
a Based on ANOVA among three groups (α = 0.05).
Table 3. Baseline and two daily Measurements (kg) of food waste FW and non-food waste nonFW in recycling (Rec) and residual (Res) waste bins for three experimental conditions of Control, Volunteers and Yellow bins.
Table 3. Baseline and two daily Measurements (kg) of food waste FW and non-food waste nonFW in recycling (Rec) and residual (Res) waste bins for three experimental conditions of Control, Volunteers and Yellow bins.
Control GroupYellow Bin GroupVolunteer Group
Building Label:C1C2Y1Y2V1V2
Baseline (Day 5)
FW in Rec bins (kg) *#19.0219.5811.1814.6810.0012.54
FW in Res bins (kg) *60.2069.3247.1067.8447.9417.92
nonFW in Rec bins(kg) *0.200.560.921.920.123.02
nonFW in Res bins(kg) *42.3849.8824.9470.1640.7833.28
CR0.240.220.190.180.170.41
Cont.L #0.010.200.080.120.010.19
effCR0.230.210.160.130.170.22
Proportion of nonFW = 0.42
Measurement 1 (Day 31)
FW in Rec bins (kg) *#30.5818.3434.1819.2224.1812.14
FW in Res bins (kg) *44.4629.1631.5438.0424.0817.28
nonFW in Rec bins(kg) *5.1221.443.643.224.240.40
nonFW in Res bins(kg) *62.7237.2668.1683.1234.6832.56
CR0.410.390.520.340.500.41
Cont.L #0.140.540.100.140.150.03
effCR0.29−0.030.420.240.350.39
Proportion of nonFW = 0.50
Measurement 2 (Day 32)
FW in Rec bins (kg) *#13.3420.7419.1613.9221.7810.86
FW in Res bins (kg) *30.4040.4440.1424.9222.8417.78
nonFW in Rec bins(kg) *14.9819.102.481.382.662.26
nonFW in Res bins(kg) *58.3427.5646.8041.4622.8448.10
CR0.390.340.430.350.490.38
Cont.L #0.530.480.110.090.110.17
effCR−0.010.020.340.290.380.25
Proportion of nonFW = 0.51
* For reference, all have propagated errors for measurement of ≤ 0.1 kg; # Almost all bags of contamination contained unsorted, versus sorted waste.
Table 4. A summary of the Effective Capture Rates (effCR) across the three experimental conditions.
Table 4. A summary of the Effective Capture Rates (effCR) across the three experimental conditions.
Control GroupYellow Bin GroupVolunteer Group
Building No.C1C2Y1Y2V1V2
Measurement 10.29−0.030.420.240.350.39
Measurement 2−0.010.020.340.290.380.25
Mean0.070.320.34
SD0.150.080.06
p-value with respect to Control Group *0.0180.011
p-value with respect to Volunteer Group *0.950
* (α = 0.05), Tukey test.
Table 5. The number of responses to the multiple choice part of the questionnaire (each being asked following an open ended question first) for each of the three experimental conditions.
Table 5. The number of responses to the multiple choice part of the questionnaire (each being asked following an open ended question first) for each of the three experimental conditions.
Control Group n = 11Yellow Bin Group n = 9Volunteer Group n = 8
Do you know about the program?YES998
NO200
Have you started waste sorting?YES477
NO521
If yes, what are your reasons to start doing waste sortingthe launch told me it was now possible454
good to environment330
a new kitchen caddie434
response to government’s call211
the neighbours are doing it001
other family members doing it010
If yes, what kind of things can remind residents to do the waste sorting?clear signage000
brightly-coloured bins040
kitchen caddie201
volunteer’s shifting002
neighbours001
What kind of reasons may hinder residents from doing the waste sorting?it is troublesome223
don’t trust government110
limited propaganda100
unclear bin categories400
they can’t get used to it100
neighbours don’t do the sorting200
Table 6. Extra resources required for each of the two interventions (for 200 households in two buildings).
Table 6. Extra resources required for each of the two interventions (for 200 households in two buildings).
Yellow Bins
Choosing style of covers2 hours * = 100 RMB
Installing covers1 hour ** = 20 RMB
Cost of covers × 260 RMB
TOTAL cost180 RMB (28 USD)
Volunteers
Recruit and training of volunteers6 hours ** = 300 RMB
Expense of volunteers, 24 days2 × 24 = 48 hours * = 960 RMB
Hot drinks, snack10 × 24 = 240 RMB
TOTAL cost1500 RMB (231 USD)
*Specialist person at 50 RMB/h; **non-specialist person at 20 RMB/h.

Share and Cite

MDPI and ACS Style

Lin, Z.Y.; Wang, X.; Li, C.J.; Gordon, M.P.R.; Harder, M.K. Visual Prompts or Volunteer Models: An Experiment in Recycling. Sustainability 2016, 8, 458. https://doi.org/10.3390/su8050458

AMA Style

Lin ZY, Wang X, Li CJ, Gordon MPR, Harder MK. Visual Prompts or Volunteer Models: An Experiment in Recycling. Sustainability. 2016; 8(5):458. https://doi.org/10.3390/su8050458

Chicago/Turabian Style

Lin, Zi Yin, Xiao Wang, Chang Jun Li, Micheil P. R. Gordon, and Marie K. Harder. 2016. "Visual Prompts or Volunteer Models: An Experiment in Recycling" Sustainability 8, no. 5: 458. https://doi.org/10.3390/su8050458

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop