Theory-of-Change Development for the Evaluation of Forest Stewardship Council Certification of Sustained Timber Yields from Natural Forests in Indonesia

Abstract

To illustrate the importance of theories-of-change (ToCs) for evaluation of conservation interventions, we consider the global ToC from the Forest Stewardship Council (FSC) and then develop a more explicit ToC focused on the sustained timber yield (STY) aspiration for natural forest management in Indonesia. We use these ToCs to consider certification implementation processes vis-à-vis indicators for STY extracted from FSC’s Indonesian Stewardship Standard that mentions STY explicitly in 45 and implicitly in 21 of 237 indicators. Analysis of 38 audit reports about 23 enterprises (2001–2017) revealed that only 77 of 504 major non-conformities assigned by auditors addressed STY. This apparent lack of attention to STY is surprising given the exhaustion of timber stocks in many production forests and the closure of many forest enterprises over the past two decades, but our ToC reveals numerous unsatisfied and unsatisfiable assumptions in certification that preclude detection of unsustainable harvests. Furthermore, compliance with governmental regulations on harvest intensities does not allow full timber recovery. To sustain yields, logging intensities need to be reduced and/or silvicultural treatments applied to increase yields, both of which reduce short-term profits. Declining yields might be accepted if the capacity of logged forests to grow timber is not impaired, but forest abandonment due to timber stock depletion is worrisome if it fosters illegal forest conversion.

1. Introduction

Impact evaluations are used to assess and then improve policy-based programs and projects in a range of public interventions on health and education, and increasingly in recent years, conservation [1,2,3,4,5,6]. Given limits on time, money, and other resources, it behooves intervention planners to specify the causal pathways towards their intended impacts, but causal frameworks can also be used to assess whether and why on-going or terminated interventions worked, and if so under what conditions, for whom, in what ways, and at what costs to whom. One such tool is the capture of hypothesized pathways to desired impacts in a graphical theory-of-change (ToC) with assumptions specified [7,8]. Well-informed ToCs, which remain uncommon in conservation, can rely on information provided by analyses that explicitly recognize nature and evolution of contextual factors, relevant actors, their histories, and power dynamics and actions, to specify transitions and assumptions that underlie the planned or realized implementation trajectories of interventions and intended outcomes [9,10,11]. Here we build and then use a ToC to clarify how the prominent and continuing conservation intervention of forest certification addresses one of the core components of sustainable management of renewable natural resources, non-diminishing timber yields using Indonesia as a case-study.

Among the ongoing efforts to improve the fates of tropical forests and forest-dependent people, voluntary third-party certification of responsible forest management by the Forest Stewardship Council (FSC) rose to prominence over the past 25 years. This intervention aims to contribute to the broad sustainability goals for the forest sector through adoption of management practices that maintain environmental, social, and economic values [12]. Evaluations of certification have recently proliferated, but most are based on secondary data (e.g., audit and other reports: [13,14,15,16,17,18,19,20,21]) or remote-sensing-data [22,23,24] and are not informed by explicit ToCs so the causal pathways underlying any reported changes/differences are not clear. Two recent exceptions were studies of FSC impacts on deforestation based on remote-sensing data analysis that employed ToCs to clarify assumptions and in attempts to assign causality for observed changes to the FSC intervention [25,26]. Even less effort has been invested in on-the-ground observations that inform robustly designed counterfactual impact evaluations [9,10]. This deficit of information is unfortunate given the potential for implementation/process evaluations to partially account for why complex initiatives, like FSC certification, achieve or fail to achieve their goals [9,10,27], and, in doing so, improve their practices.

Even in the absence of field data, we make the case in this article that ToC construction and use provides insights into the in-theory (i.e., the FSC’s principles, criteria, and indicators) and in-practice (i.e. results of audit visits into aspiring or certified units) capacity of FSC certification to deliver on one of its many objectives, the maintenance of timber yields from managed forests. We justify this narrow outcome focus by pointing out that despite FSC’s broad intentions, the long-term ecological and economic viability of natural forests managed principally for timber depends on the continued availability of that renewable resource. In other words, maintenance of the range of socio-economic values of managed forests in FSC’s stated goal will only be possible if there are still quality timber stands to manage. We compare our ToC with FSC’s ToC [12] (visualization in Figure S1] specifically represented in FSC’s Stewardship Standard for Indonesia [28]. Additionally, in an effort to reveal how the desired impact of sustained timber yield (STY) might be achieved via the FSC certification process, we derive evidence from Corrective Action Requests (CARs) assigned to forest management enterprises (FMEs) during certification audits to assess how in practice, FSC assesses the maintenance of timber yields. CARs assigned when third-party auditors detect discrepancies between management practices and certification standards are often used for indirect (i.e., not based on first-hand observations by researchers) assessments of FSC impacts [13,14,15,16,17,18,19,20]. Despite concerns about these analyses [18], we focus here on CARs related to STY.

1.1. Theories-of-Change (ToCs) in Impact Evaluations

Causality is at the core of the scientific method but is difficult to assess when dealing with complex systems. Causal inference frameworks can nevertheless inform researchers about the natures of problems of interest and reveal underlying relationships in a system [29]. These frameworks can also help disentangle the likely mechanisms that under-pin causal relationships [30]. ToCs can be developed to decompose the drivers of expected (i.e., at the design stage) and observed changes (i.e., at the evaluation stage). In doing so, they can also reveal our knowledge/ignorance about the extent to which the specified drivers indeed play roles, and overall, uncover the underlying assumptions behind the proposed mechanisms of change and their plausibility. The ToC building process also often illuminates research needed to clarify the roles of mediators who are affected by a policy/program and then affect the outcomes, as well as of moderators who represent external/contextual factors that may influence the outcomes of a policy/program [5]. ToCs can also reveal the pathways through which unexpected factors may have driven change and enable their interpretation [7,8].

Although causal models can be considered reductionist/mechanistic, they provide a powerful way to understand and map drivers of change and their varying effects through time, as well as relationships with other elements of the system. To that end, clear definitions of boundaries and scopes of those inferences need to be made explicit in terms of contexts, times, and conditions. For instance, [31] used an agent-based approach to clarify some of the factors that influence the decision-making of tree fellers in tropical forests when confronted with large trees suspected of being hollow, and, as such, not sources of marketable timber. The rudimentary agent-based model built with input from a multi-stakeholder workshop and interviews with workers in the field highlights the roles of actors other than the fellers themselves (e.g., forest supervisors, government inspectors) and some of the meditating factors (e.g., safety risks, fiscal policies) as well as the ecological and other effects of felling such trees. With the insights gained through this sort of participatory modeling process, experiments can be designed to test incentives and other mechanisms to elicit behavioral changes that can result in improved environmental and socio-economic outcomes.

1.2. Justification for Focus on FSC and Background about This Intervention

FSC certification of forest management is a well-established conservation intervention with global scope. This non-governmental initiative to incentivize adoption of responsible management practices is based on a set of principles, criteria, and indicators agreed upon through consultations by members that represent industry, economic, environmental, and social interest groups (https://ic.fsc.org/). Although FSC also certifies plantations and supply chains all over the world, we focus exclusively on natural forest management only in Indonesia, which has a 20-year history with this conservation intervention [32] and is where both authors have in-country experience with the timber sector and certification. FSC certification in Indonesia has also been promoted locally over the years by a range of international initiatives (e.g., Tropical Forest Trust, Global Forest & Trade Network, The Borneo Initiative). As of June 2018, the country hosted 23 certified natural forest management enterprises (FMEs) [33].

1.3. Sustained Timber Yields (STYs) and the FSC

Although the concept of sustainable forest management evolved over the past decades from an initial focus on timber supplies to include considerations of many more goods, services, and other values [34], we here focus on sustained timber yields (STYs). We believe that this focus is justified because if timber yields decline with every harvest, companies will not remain in the forest management business for long, which will have cascading negative consequences well beyond the economic prospects for the specific firm (e.g., lost jobs, foregone governmental revenue from taxes and royalties, and multiple deleterious environmental impacts).

The science that underpins claims of STYs has a long history but continues to develop with accumulation of data and new insights into their assessment. These analyses are primarily based on data on tree recruitment, growth, and mortality collected from monitored permanent sample unlogged plots. Because the available monitoring data are generally from periods shorter than even a single harvest cycle (i.e., the time between harvests), simulation models are employed that use stand dynamics to predict future yields [35,36,37]. Variation in data quality, differences among models, and the wide variety of interpretations of the outputs mean that STY calculations are not straightforward or simple. Furthermore, there are no widespread agreements about some basic conventions for calculation of STY [38]. For example, for projections of future yields there is neither consensus nor legal requirements about whether timber from smaller trees or newly commercial species can be included in yield calculations. Even more fundamentally, there is no agreement about whether to qualify as “sustainable” projected yields should equal the first harvest from primary forest, or after cashing in the “primary forest premium” (i.e., non-diminishing yields after the first harvest). For this study we use the most liberal conventions insofar as to qualify as STY the number of harvested species can increase, the minimum diameter of harvested trees can decrease, and yields only need to be sustained after the first harvest.

Due to high tree diversity and the lack of markets for the timber from most species, logging in Indonesia is selective. Trees are typically cut with chainsaws and then extracted from the forest to roadside log landings with bulldozers (i.e., crawler tractors). After a selective harvest from these multi-aged stands, the volume of the next cut is mostly supplied by trees already present at the time of the initial harvest (i.e., future crop trees) [39]. Protection of these future crop trees from logging damage is therefore critical to STY as is prevention of soil damage that reduces future stand productivity.

Although STY is not referred to in those explicit terms in FSC’s standards, the goal of non-diminishing yields appears frequently in the organization’s principles, criteria, and indicators with mentions concentrated in Principles 5, 6, and 7 (https://ic.fsc.org/). Overall, non-diminishing yields and economic viability are among FSC’s core principles (e.g., Principle 5. Benefits from the forest. Forest management operations shall encourage the efficient use of the forest’s multiple products and services to ensure economic viability and a wide range of environmental and social benefits). FSC’s ToC document (visualization in Figure S1) [12] describes four pathways of change (engagement, standards, assurance, and market) and several strategies as necessary conditions to achieve its intended impacts [12]. Those most pertinent to STY include Economic Impact 2 (“Harvesting activities are based on the principle of sustained yields: there is a balance of growth and yields of the forest species composition”), Environmental Impact 7 (“Minimized degradation of natural forests, no conversion of forests to other land use in certified areas”), and General Impacts 10 (“FM [forest management] operations develop strategies to diversify the portfolio of forest products, and manage a broad portfolio to increase environmental and economic resilience”).

It should be noted that since its founding in 1993, FSC generally eschewed loose use of the phrase “sustainable forest management (SFM).” Nevertheless, the tension between sustainable and responsible forest management is evidenced in the accompanying statements by FSC about their ToC in which both terms are used [12]. The preamble to this document states, for example, that: “The FSC supply chain is a tool for companies to demonstrate their commitment to the principles of sustainable FM [forest management]…Some forest management units with FSC certification can be seen as proof, for the relevant government, that it is indeed feasible to balance economic, ecological and social interests, and to achieve sustainable FM.” In contrast, in the ToC visualization itself FSC states that certification promotes “responsible” forest management (Figure S1) [12].

2. Materials and Methods

We first explore FSC’s global ToC [12] (visualization reproduced in Figure S1) to establish how concerns regarding STY are framed and to identify the underlying elements of STY-related causal pathways. Because this ToC does not specify assumptions and to further our understanding of how FSC certification is suited to address STY goals in Indonesia, we extracted from the FSC’s Stewardship Standard for that country (FSC-STD-IDN-01-01-2013 Indonesia Natural PC&I) [27] the indicators that are directly (i.e., on-the ground management practices) or indirectly (e.g., record-keeping and reporting procedures) pertinent to STY (Tables S1 and S2). The authors individually classified indicators as direct or indirect with disagreements resolved by arbitration by two Indonesian certification practitioners. Henceforth we focus only on direct indicators as we think they more readily reveal how auditors assess STY. We grouped STY-related indicators into those that are most critical (non-conformance implies a fundamental failure to achieve STY), critical (necessary but not sufficient for STY), and tangential (potentially important but not instrumental to achievement of STY).

With inputs from a wide range of stakeholders over several years, we then developed a high-level ToC on which we map a plausible causal model theory that would deliver STY in managed natural forests. This model is intended to show the necessary conditions for achievement of STY, identify the underlying causal pathways, and describe the information required to test these assumptions (indicators). To assess the extent to which FSC certification process can adequately address STY requirements, we contrast the list of direct indicators identified from FSC’s Indonesian Stewardship Standard with those derived from our STY theory-based ToC.

Finally, to assess how FSC certification tracks STY concerns in practice, we built a database of all major STY-related CARs assigned to natural forest FMEs in Indonesia. We focus only on major CARs because these non-conformities must be addressed before certification is either granted or renewed. CAR information was derived from publicly available main assessment and reassessment reports about 23 FSC-certified FMEs (2001 through December 2017) [40]. We emphasized these types of audits because they represent the most detailed assessments of management practices against the standards, as compared to annual surveillances that tend to focus on a few agreed-upon issues. Only non-conformities are included in audit reports, so we assume that failure to mention a specific indicator implies conformance with it. To assess whether the process of FSC certification is in theory equipped to determine whether or not timber yields are sustained, we compared the content of indicators derived from our ToC and that of STY indicators extracted from FSC’s Indonesian Stewardship Standard. To determine how the application of the national standard yielded STY information, we contrasted major CAR analysis results with our ToC framework.

3. Results

3.1. Stand Management Regulation Pertaining to STY in Indonesia and How to Assess STY

Regulations governing the selective harvest of timber from natural forests in Indonesia are set by its Ministry of Environment and Forestry [41]. Harvests are approved via the preparation of forest management plans (valid for 10 years) and detailed annual plans for permits with durations of up to 70 years [41]. In the early 1970s the minimum cutting diameter in production forest was set at 50 cm DBH (diameter at 1.3 m or above buttresses) and the minimum duration of a harvest cycle (the years between selective harvests from the uneven-aged stands) was 35 years. These rules were based on the assumptions that in forests that occupy a diversity of ecological conditions across the 17,000 island archipelago country, all of its hundreds of commercial tree species grow 1 cm in DBH per year, and commercial timber accumulates at 1 m³/ha/year [42,43]. Ostensibly to improve these rules, FMEs are required to establish and annually monitor tree growth, death, and recruitment in permanent sample plots, enter the data in computer files, and send them to the Ministry of Environment and Forestry for analysis.

In spite of evidence that started to accumulate in the 1990s that timber yields declined with each harvest if these minimum guidelines were followed [42,43,44,45,46,47,48,49,50], in 2009 the government changed the regulations to allow increased harvest intensity. For production forests, the minimum cutting diameter was reduced from 50 cm to 40 cm and the minimum cutting cycle duration from 35 to 30 years. For FMEs that agreed to do enrichment planting along cleared lines through logged forest, the minimum cutting cycle was further reduced to 25 years [32]. This lack of correspondence between governmental regulations and STY contributed to the decline in the number of FMEs in Indonesia from 580 in 1992 to 265 in 2016 [51].

3.2. Indonesian FSC’s Standard and STY-Related Indicators: Ranking and Citation Frequencies in Audit Reports

FSC’s Standards for Indonesia include 45 indicators of conformance that are related to the STY goal directly (i.e., refer to on-the-ground management practices) or indirectly (i.e., refer to management procedures; 21 indicators). Direct indicators are found in Principles 1, 5, 6, 7, and 8 with indirect indicators in Principles 1, 5, 6, and 7 (Tables S1 and S2). We focused only on direct indicators of STY of which we consider 17 to be most critical (Principles 5, 6, and 8), 22 as critical (Principles 1, 5, 6, 7, and 8), and 6 as tangential (Principles 1, 5, and 7; Table S1 and Table S2).

Of the 504 major CARs assigned to 23 FSC-certified FMEs, only 77 (15.5%) were related to STY. Most cited STY-related direct indicators belong to Principle 7 (critical indicators 7.3.2–7.3.4) with at least 6–8 occurrences (7.7–10.3% of all major CARs, respectively). These direct indicators reflect concerns about worker and subcontractor training and supervision as they might affect their capacity for proper and safe practices. Eight other less cited non-conformances (3‒4 times; 3.8–5.1% of major CARs, respectively) referred to Principles 1, 5, 6, and 8 and related to respect for national regulations, economic viability of management operations, assessment of environmental impacts, and monitoring activities, respectively (critical indicators P1: 1.5.1, 1.5.3; P5: 5.3.1, 5.3.4; P6: 6.5.6, 6.5.10; and P8: 8.2.3, 8.4.1). These less-cited indicators are classified as either most critical or critical to STY, with one tangential indicator exception (

Table 1. Major CARs for indicators directly related to STY in 38 FSC main and reassessment audit reports in Indonesia (2001–2017) from national FSC Stewardship Standard. Green shaded and **: indicators are most critical to STY, no shading and *: indicators critical for STY, and beige shaded and no asterisk indicators are tangential to STY achievement yet important (see Table S1 for full ranking and indicator information). Empty cells indicate that no non-conformances were reported.

Principle	Indicator	# Times Major CARs Reported	% of Reported Major CARS
1. Forest management shall respect all national and local laws and administrative requirements.	1.5.1 *	4	5.1
	1.5.3	3	3.8
	1.5.4 *	1	1.3
	1.6.1
5. Forest management should strive toward economic viability, while taking into account the full environmental, social, and operational costs of production, and ensuring the investments necessary to maintain the ecological productivity of the forest.	5.1.3 *	1	1.3
	5.1.4 *	2	2.6
	5.2.2 L
	5.3.1 **	4	5.1
	5.3.2 **	2	2.6
	5.3.3 **	1	1.3
	5.3.4 **	3	3.8
	5.6.1 **	1	1.3
	5.6.2 **	1	1.3
	5.6.3 **	1	1.3
	5.6.5 **	1	1.3
6. Assessment of environmental impacts shall be completed—appropriate to the scale, intensity of forest management and the uniqueness of the affected resources—and adequately integrated into management systems. Assessments shall include landscape level considerations as well as the impacts of on-site processing facilities. Environmental impacts shall be assessed prior to commencement of site-disturbing operations.	6.3.1 **	2	2.6
	6.3.2 **	1	1.3
	6.5.6 *	3	3.8
	6.5.10 **	3	3.8
7. A management plan—appropriate to the scale and intensity of the operations -shall be written, implemented, and kept up to date. The long term objectives of management, and the means of achieving them, shall be clearly stated.	7.1.6 *
	7.1.7 *	1	1.3
	7.1.8 *
	7.1.10 *	1	1.3
	7.1.11 *
	7.1.12 L *
	7.1.13 *	2	2.6
	7.1.14 *
	7.2.1 *	1	1.3
	7.2.2 *	1	1.3
	7.2.3 *	1	1.3
	7.2.4
	7.2.5	1	1.3
	7.3.2 *	6	7.7
	7.3.3 *	8	10.3
	7.3.4 *	6	7.7
	7.3.5	2	2.6
8. Monitoring shall be conducted—appropriate to the scale and intensity of forest management —to assess the condition of the forest, yields of forest products, chain of custody, management activities and their social and environmental impacts.	8.1.2 **	2	2.6
	8.1.3*	1	1.3
	8.1.4 **
	8.2.1 **	1	1.3
	8.2.2 **	2	2.6
	8.2.3 *	4	5.1
	8.3.2 L **		3.8
	8.4.1 **	3	3.9
	8.4.3 *

, Table S1).

Of the 504 major CARs assigned by FSC auditors, none were reported for 11 STY-related indicators (Table 1; see Table S1 for details about each indicator). For example, no CARs were assigned for the most critical indicators of availability of monitoring information about harvested volumes (8.1.4) and sources of such data (8.3.2 L). As for critical indicators of STY, no CARs were assigned for commitment to long-term forest management practices, promotion of harvesting of lesser-known species, or assignment of personnel for timely management plan updates (1.6.1, 5.2.2 L and 7.2.4, respectively). Similarly, no CARs were assigned for six tangential indicators that cover a range of STY-related issues (7.1.6, 7.1.8, 7.1.11, 7.1.12 L, 7.1.14, and 8.4.3). The proportion of CARs related to STY did increase over time; FMEs certified within the last three years of the study accounted for 60% of the major STY-related CARs (46), of which 37% were most critical and critical to STY.

3.3. Our ToC for Assurance of STY: Assumptions, Indicators, and Data Needs

Judgements about whether or not timber yields are being sustained need to be based on the outputs from stand projection models populated with data on rates of tree recruitment, growth, and mortality rates. These data need to be derived from permanent sample plots established and monitored in areas that are geographically close, on the same soil type, and at more or less the same altitude as the managed stands. Equally important, the history of logging and other interventions in the monitored plots should resemble those in the stands for which volumes are being projected. To assure data quality, guidelines for plot location, establishment, monitoring, and data handling need to be followed [52]. One big challenge in species-rich tropical forests is that for each species (or species group) harvested, the plots need to include sufficient numbers of trees of near harvestable size that are monitored over sufficiently long periods for growth and mortality data to be reliable. Also needed for accurate estimates of future harvestable volumes is the proportion of logged areas that constitute the permanent logging infrastructure (main skid trails, roads, and log landings), areas rendered un-productive by severe soil damage (e.g., poorly used skid trails), and deforested areas. Evidence that utilizable wood waste was minimized should be based on field observations on the proportions of the boles of felled trees that were extracted from the forest. Field observations are also needed to determine the extent to which reduced-impact logging practices were effectively utilized.

Our STY-focused high-level (i.e., not very detailed) ToC identifies regulatory frameworks and governmental capacities along with available FME financial and technical resources and commitment to long-term forest management goals as key ingredients to STY aspirations (Figure 1). The different stages in the ToC represent conditions needed to achieve STY through the forest management cycle. For selectively logged forests to recover their timber volumes within a harvest cycle, logging activities need to minimize damage to future crop trees and other sorts of environmental harm that reduces regeneration, growth, and yield (e.g., soil damage). Because they are so seldom applied in Indonesia, we disregard the potential yield benefits of silvicultural treatments such as future crop tree liberation and enrichment planting. Only as an illustration of the usefulness of detailed ToCs, we develop a ToC with the requisite assumptions for the impacts of worker training in reduced-impact logging (Figure S2 and Table S3).

To reveal some of the requirements for assurance of STY in our general ToC (Figure 1), the indicators and assumptions are articulated for each stage of the model from inputs and short-term outcomes to long-term outcomes and the ultimate impact of STY (

Table 2. Assumptions, indicators and metrics derived from our high-level ToC for STY assurance in managed natural forests in Indonesia (Figure 1). RIL = reduced-impact logging.

ToC Stage	Variable	Indicators	Assumptions
Short-Term Outcome 1	Logging Damage Minimized	-# trees/ha killed or damaged (by severity class), soil damage (m2/ha) due to skid trails, log landings, and roads.	- Data available and verified on worker training in RIL. - System in place that motivates workers to apply RIL. - Trained workers retained by FME. - Post-logging evaluation of tree damage verified. - Soil damage measured in the field and/or with remote sensing but verified in the field. - No loss of area in production due to severe degradation.
Short-Term Outcome 2	Wood Waste Minimized	- Wasted utilizable commercial timber volume less than threshold (e.g., <10 m3/ha).	- Chainsaw operators follow FME’s wood utilization standards. - Post-logging inventory of wood waste available and field verified.
Short-Term Outcome 3	Future Crop Trees Sufficient	- Threshold # trees of good form of commercial species after the harvest (e.g., >20 trees 20–40 cm DBH per hectare).	- Post-logging stand inventory data available, field verified, and used in stand projections that indicate sufficient stocking for STY.
Short-Term Outcome 4	Seed Trees Sufficient	- Reproductively mature trees of commercial species (# per hectare).	- Post-logging stand inventory data available, field verified, and used in stand projections that indicate sufficient sources of regeneration to secure future yields.
Short-Term Outcome 5	Fire, Illegal Logging, and Deforestation	- Area burned, illegally logged, or cleared (hectares and % total FME).	- Post-logging field inventory and remote sensing data show no loss of area under timber stand management due to severe degradation or deforestation.
Long-Term Outcome 1	Commercial Timber Volume Recovers	- Volume of harvestable timber recovers before next harvest m3/ha). For example, for STY after a harvest of 60 m3/ha with a cutting cycle of 30 years the volume increment on FCTs 20–40 cm DBH must be >2 m3/ha/year. - Field data related to specific practices (silvicultural interventions implemented and effects, for example liana removal, liberation thinning)	- Field-verified permanent sample plot data from logged stands and suitable stand projection models indicate sufficient growth of FCTs for full volume recovery by the next harvest. - Field data available and verified on specified silvicultural practices. - Silvicultural treatments required by forest management plans are implemented as needed and are successful.
Long-Term Outcome 2	Adequate Commercial Recruitment	-# of seedlings, saplings, poles, and FCTs of commercial species per hectare after logging activities (m3/ha/year)	- Permanent sample plot data are available, field verified, and used in stand projections that indicate that regeneration is adequate sustain future harvests.
Long-Term Outcome 3	Forest Retained	- Volume lost to deforestation or forest degradation by illegal logging or fire occurs (m3/ha/year)	- Remote sensing data available and field verified.
Impact	Sustained Timber Yields (STY)	- Volume of commercial timber available for harvest at least equal to previous harvest (m3/ha)	- Reliable data from permanent sample plots in logged forest available and field verified. - Results of a stand model projection based on the permanent sample plot data checked. - Reported logging intensity (m3/ha hauled from log-yards) accurate and field verified.

). The great majority of requirements pertain to data availability, quality, and use by trained and motivated personnel. There are also several requirements related to field verification of management practices. Given the abundant evidence that compliance with the Indonesian government’s minimum forest management rules for its production forests (i.e., a 30-year minimum harvest cycle and a minimum harvest diameter of 40 cm DBH) results in declining yields [42,43,44,45,46,47,48,49,50], we assume that achievement of STY requires lower harvest intensities than the allowed ones and/or application of silvicultural treatments that effectively increase yields [42].

3.4. The Contrast between Our ToC, Aspirations from FSC’s Indonesian Stewardship Standard, and STY-Related Audit Data

There is substantial but not complete convergence between our STY-based ToC and the direct indicators of STY in FSC’s Indonesian Stewardship Standard. For instance, both indicate concern about the use of improved logging techniques by trained personnel to avoid unnecessary damage and about the implementation of silvicultural and other practices to enhance commercial tree regeneration and growth (indicators 6.3.2 and 7.3.5, respectively, in the FSC Stewardship Standard, and Long-Term Outcome 2 and Short-Term Outcome 1 in our ToC). Similarly, the two systems both emphasize that decision-making processes should be based on updated and quality information and the need for these decisions to “ensure that annual production is in accordance with the capability of forest productivity” (most critical indicator 5.6.5) and our indicator for Long-Term Outcome 1. Finally, there is commonality on the requirement that growth-and-yield data are used to adapt management prescriptions so as to reflect timber accumulation and other characteristics of logged stands (our Impact indicator and indicators from FSC’s Indonesian Stewardship Standard Principles 7 and 8).

There are noteworthy differences in indicators included in our ToC and FSC’s Indonesian Stewardship Standard. In particular, we emphasize field verification of STY-related logging and other field operations, the cross-validation of the use of obtained information, and consider enabling conditions to incentivize STY-promoting management decisions. Field verification is time-consuming and therefore expensive, but is critical for assessment of logging damage, wood waste, and compliance with harvesting guidelines, as well as to determine whether any required treatments were indeed applied and were effective (e.g., skid trail drainage, road bed stabilization, or future crop tree liberation from lianas). Deforestation assessments can also be carried out on the ground, but conformance assessment bodies and their auditors might instead utilize the increasingly available remote sensing technology coupled with geographical information systems for this and several other indicators [53]. Our ToC also stresses that the motivational context for worker performance should be understood to evaluate the existence and effectiveness of an enabling environment for STY-favoring decisions. We prefer outcome-based indicators collected in the field because, for example, even after training in reduced-impact logging, workers may not implement the practices they studied or the trained workers may no longer work for the FME.

Other aspects on which our system and FSC’s Indonesian Stewardship Standard differ refer to our attention to issues being adopted and not only planned (i.e., whether management plans were indeed updated and how—most critical indicator 8.4.1; whether the rate of annual harvest reflects what the data say and do not exceed actual replenishment rates—most critical indicator 5.6.3; the accuracy of inventories—critical indicator 8.2.2). In other words, information on specific issues may be available in documents and files that auditors review during their visits, but whether the information they contain is of sufficient quality and is used properly by both auditors and forest managers remains to be established.

4. Discussion

Comparisons of the 10 STY-related indicators in our ToC and the 38 in FSC’s Indonesian Stewardship Standard show both commonalities and contrasts. We focus here on the differences because this analysis helps us explore the extent to which FSC’s Standard addresses STY concerns. Although both indicator groups include expectations about management planning, training, and logging practices as well as the issues of data quality, availability, and use, our ToC places more emphasis on field verification of the outcomes of training, data use, and the employment of recommended management practices. Our ToC also helped us clarify the verifiable conditions that contribute to STY from tree to stand levels (e.g., post-logging stocks of future crop trees and soil protection from compaction). We could even use available research results from Indonesia to propose threshold values for some of those field verifiable indicators in full recognition that adjustments will be needed based on local stocking, growth, and yield data. In contrast, the protocols used by FSC auditors lack this level of specificity.

We assessed what information about STY is collected by FSC auditors based on the STY-related major CARs they assigned by FSC auditors and the STY indicators specified in our ToC. Although all the available evidence indicates that timber yields are declining in Indonesia [42,43,44,45,46,47,48,49,50], for the purposes of this comparison we assumed that indicators missing in these non-conformity audit reports were satisfied by the FMEs (e.g., that regeneration is monitored). If this assumption is accepted, then it is encouraging for STY that 15 of the 17 indicators that are most critical to STY and 16 of the 22 classified as critical were the focus of CARs at least once in the 38 certification and re-certification audits of the 23 FMEs. This finding is promising because it means that both the issues were detected and addressed given that certification was either granted or renewed. Among these non-conformities, prevalent were those related to worker and subcontractor performance and training. Although these issues are important to STY, other indicators that are more directly linked to STY were less often the focus of non-conformance reports. For example, excessive damage was seldom reported (indicators 5.3.1 and 6.5.10).

Our analysis revealed a number of practical constraints on auditors and the auditing process, some of which are not unique to STY concerns. First of all, auditors are faced with various challenges and choices when they interpret indicators and assign CARs, as discussed elsewhere [18,20]. Given the little time auditors can spend observing logging operations and their outcomes or checking permanent sample plots, these challenges loom particularly large. We cannot assess whether auditors have the necessary skills and training to judge conformance with complicated technical guidelines related to STY (e.g., whether wood waste or skid trail densities are excessive), but recognize that even experienced foresters grapple with these issues. Secondly, the costs of both logging operations and auditing are increasing in Indonesia as managed forest become more remote. Even after arriving at the main camp of a FME, which now often requires a full day of travel, auditors might need to travel an additional several hours to reach active logging areas; even more time is needed to visit permanent sample plot established in areas to which access roads are no longer passable.

In our exploration of FSC assessments of STY we use data from third-party audits in full recognition that they suffer from some fundamental problems. First of all, audits are paid for by the FMEs being audited; if certifying bodies or auditors are too stringent in their interpretation of the standards, they will soon find themselves without work [9,18,54]. The six conformity assessment bodies that work in Indonesia as well as many of the auditors employed by those firms are known by FME administrators; when auditing firms are chosen, both costs and the expected rigor of the assessment are considered. For example, an FME that has social problems might select an assessment body that regularly employs auditors known to pay more attention to environmental matters, even if audit teams must include experts on social as well as environmental and economic issues.

Given that certification audit reports only present non-conformities, their use for making conclusions about FSC’s impacts requires many dubious assumptions, all with strong implications for the assessment of STY. For instance, the observation that few major CARs were assigned to STY-related indicators (Table S1) might suggest that FSC’s auditors possessed all the information needed to make these complex decisions (e.g., regarding timber volumes harvested and subsequent rates of timber volume increment). Whether this is indeed the case cannot be determined from our ToC-based analysis insofar as the assumptions about information availability, quality, and handling would all require independent verification.

More fundamentally, for timber yields to be sustained in selectively logged forests in Indonesia and elsewhere in the tropics [55,56,57], FMEs would have to harvest less timber and less frequently than allowed by law or apply silvicultural treatments to increase volume. In other words, if FSC-certified concessions are to meet the goal of STY, FSC would need to insist on management controls more stringent than those of the government. If reductions in revenues due to these limitations are compensated by the market benefits of certification, perhaps a transition from timber mining to timber stand management could be motivated [39]. Unfortunately, the hoped-for market benefits of FSC-certification are small and seldom realized, at least by Indonesian FMEs, and especially relative to the costs of certification ($ 4.76 /ha) [32]. Clearly, much more could be learned from direct field observations as part of rigorously designed impact evaluations that compare the outcomes in FSC-certified and appropriate counterfactual FMEs [9], but this mode of assessment is not without its challenges. For example, implementation evaluations should be performed in collaboration with relevant stakeholders, require substantial time and money, and require political support to gain access to both FSC-certified and non-certified FMEs.

5. Conclusions

While the resources, skills, and collaborators for a fully-fledged, independent, and on-the-ground impact evaluation of the FSC remain to be assembled, we could learn much about one aspect of forest certification by assembling a ToC focused on the goal of sustained timber yields. When this ToC was compared with FSC’s indicators and the major CARs assigned by its auditors, numerous logical and process-related gaps were revealed that cause us to be wary of the effectiveness of FSC certification vis-à-vis STY. More fundamentally, the ToC analysis that guided considerations of the economic and governance-related conditions in the forestry sector in Indonesia raises concerns about STY’s suitability as a realistic objective.

If timber yields can really only be maintained with constraints on harvests and the application of silvicultural practices that severely reduce short-term financial profits to loggers [39,58], perhaps concession closures should not be mourned. Unfortunately, at least from an environmental perspective, it needs to be recognized that forests under active management, even those from which yields are not sustained, resist deforestation more effectively than those that are abandoned [22].

Exacerbating the financial constraints on natural forest managers in Indonesia is the fact that most of the country’s lowland and accessible forests on the mega-islands of Borneo and Sumatra have been cleared and converted into oil palm and tree plantations [59]. What forest remains in accessible areas is badly degraded and when loggers return for the second and third harvests, they find commercial timber stocks that are a small fraction of what they were initially [42]. Most of the better-stocked forests are in remote areas and on steep land where harvest and transport costs are high and profits are consequently low [60]. Timber yields can be sustained only if, in addition to reduced harvest intensities and the application of silvicultural treatments, alternative timber yarding techniques are employed (e.g., long-line cable yarding instead of cutting switchbacks up steep slopes with bulldozers) [39,42,43,48,49,58]. Given these seemingly insurmountable short-term constraints, FMEs are only likely to make STY-friendly choices and responsible natural forest management will only become a financially viable land use if substantial policy changes are enacted. It would also help if the large hydrological, biodiversity, carbon, and long-term socioeconomic benefits of improved forest management were recognized but, unfortunately, due to a lack of funds and insufficient political will, they remain “externalities”.

We hesitate to suggest it, but rather than expect timber yields to be sustained, perhaps they should be allowed to decline as long as the forest remains mostly intact and otherwise retains its capacity to recover its timber stocks. This relaxation of expectations is in keeping with the concept of “weak” sustainability [61,62], which is the basis of claims regarding sustainable development, sustainable infrastructure, and even sustainable mining [63]. Given the declining profits from natural forest management in Indonesia, the scarcity of ecosystem service payments, the paucity of premiums for certified wood, widespread governance failures, high opportunity costs of forest retention, and the environmentally disastrous impacts of forest conversion, perhaps expecting timber yields to recover between harvests does not serve the cause of conservation.