A Two-Phase Hybrid Trading of Green Certiﬁcate under Renewables Portfolio Standards in Community of Active Energy Agents

: The future distribution network is a community involving numerous active energy agents (AEAs) and a local operator. Each AEA is obligated to meet the renewables portfolio standards (RPS) with enough green certiﬁcates (GCs), which can be obtained from renewable energy consumption or from GC trading. This paper concentrates on the GC trading in AEA community and proposes a two-phase hybrid mechanism, which combines the peer-to-peer (P2P) phase and the centralized phase. In Phase 1, GCs are traded among AEAs in P2P manner. All AEAs are classiﬁed into two types: naïve and sophisticated, each of which has the speciﬁc preference in GC trading. Additionally, each AEA ﬁnds trading partners by adopting multi-option-based matching. In Phase 2, the remaining GCs are traded between AEAs and the local operator. Numeric studies are performed on a 30-AEA community in three different market scenarios: globally balanced, undersupplied, and oversupplied. Simulation results indicate the optimality of bi-option, verify the effectiveness of the hybrid trading, and reveal the economic advantages over the sole centralized counterpart. The impact of AEA type is also discussed on both updating quotation and concluding deals.


Introduction
Vigorously developing renewable energy has become an effective solution to environmental pollution and energy crisis. Feed-In Tariffs (FIT) and Renewables Portfolio Standards (RPS) are two mainstream policies for renewable energy accommodation [1]. FIT refers to the premium price paid by the government for encouraging renewable energy production. FIT used to be widely implemented but suffers from low efficiency on renewable energy accommodation and high pressure on governmental funds for subsidies [2]. On the contrary, RPS mandatorily stipulates a specific quota of renewable energy accommodation for each undertaker, and the quota should be completed by physically consuming renewable energy or by economically purchasing green certificates (GCs) [3]. With the support of GC trading, RPS has been successfully put into practice in Australia [1], the United States [4], the United Kingdom [5], Denmark [5], Germany [5], Japan [6], etc.
The applications of RPS demonstrate its effectiveness in renewable energy accommodation. It is proved that RPS can optimize the power supply structure [7], increase the renewable energy capacity [8], reduce the social costs [9], attract investment on green power development [10,11], as well as lower the wholesale price of electricity [12].
RPS is strongly dependent on GC trading, and there already exist many studies on GC trading under RPS. Considering the sustainable development of renewable energy, a system dynamic model is proposed with data-based simulation [13]. The multi-agent approach is used to study the interaction behaviors of users in GC trading and the positive effects of increasing the frequency of transactions are presented [14]. An equilibrium model of GC market is established based on oligopoly competition equilibrium theory [15]. A multi-oligarch dynamic power sales game model for three types of heterogeneous power supply enterprises is established and the importance of eliminating information asymmetry is revealed to keep the stability of the GC market [16].
The future smart grid is a community involving numerous active energy agents (AEAs) and a local operator [17]. AEA is a unified behavioral model of various dual-role entities in smart grid. In the physical layer, an AEA is an electrical entity, which can be a producer, a consumer, or a prosumer of electricity; in the financial layer, an AEA is an autonomous intelligent agent, which actively acquires and analyzes market information, and actively participates in market transaction, so as to maximize individual profit. Obviously, AEAs are required to trade GCs to meet the RPS regulation and the peer-to-peer (P2P) mode is suitable for trading GCs among AEAs regarding the community structure.
The P2P trading of electric energy in smart grid is thoroughly investigated [18] and the trend of personalization and decentralization is revealed [19]. A multi-layer system is proposed to identify and classify the key elements of P2P trading, and a platform of P2P energy trading is developed for game theory simulation. It is proved that P2P transaction can improve the balance of local energy production and consumption [20]. P2P transaction is also adopted to address the energy coordination among multi-microgrid systems [21]. A parallel multidimensional willingness bidding strategy is proposed for the high concurrency of P2P energy trading among microgrids [22].
Considering the structure of AEA community, it is reasonable to introduce the P2P mode to GC trading in order to improve the effeciency and flexibility. However, to the best of the authors' knowledge, most existing publications focus on the energy market, and the research work on the P2P trading of GCs under RPS regulation is quite limited. A blockchain-based trading scheme is proposed but is regardless of RPS policy [23]. A non-cooperative game model is designed, but only applicable in small-scale problems [24]. A blockchain-based platform for GC trading is developed, but the behavioral differences of users are not taken into account [25].
On this basis, we propose a hybrid trading of GCs including two phases: peer-to-peer (P2P) trading and centralized settlement. In Phase 1, majority GCs are traded among AEAs in a P2P manner; in Phase 2, the remaining deficit and surplus GCs of AEAs can be purchased from or sold to the local operator, respectively.
The contributions of this paper can be outlined as follows: (1) The GC trading of AEA community under RPS is considered, and a two-phase hybrid mechanism combining P2P trading and centralized settlement is proposed. (2) The diversity of individual behaviors in GC trading is regarded, two types of AEAs are defined: naïve and sophisticated, and the impacts on updating quotation and concluding deals are quantitatively investigated. (3) The overall efficiency of GC trading is taken into account, a multi-option-based matching is designed to facilitate the P2P order-matching, and the parameters are numerically optimized.
The remainder of this article is organized as follows. Sections 2 and 3 respectively present the system description and mathematical formulation of the two-phase hybrid trading of GC in an AEA community. A case study is performed in Section 4 to numerically analyze the effectiveness and efficiency in three different market scenarios. Finally, we draw the concluding remarks in Section 5.

Structure of Market
The RPS policy stipulates a quota of renewable energy accommodation on each AEA in terms of GCs. AEAs that successfully complete the RPS quota will be rewarded by the local operator for the surplus GCs; AEAs that fail to complete the RPS quota will be charged by the local operator for the deficit GCs. Denote the reward and charge rates as P and P, respectively; in other words, the local operator is a GC broker that trades with AEAs at the two different prices: P and P.
Accordingly, we propose a two-phase hybrid mechanism of GC trading. As shown in Figure 1, the trade mechanism consists of two sequential phases: P2P trading and centralized settlement. In Phase 1, majority GCs are trade among AEAs in P2P manner. AEAs with deficit and surplus GCs trade as buyers and sellers, respectively. Each AEA initializes trade order including quotation and amount of GCs, broadcasts on trade platform, freely chooses trading partners, and repeatedly negotiates quotations to conclude deals. In Phase 2, remaining GCs are settled through centralized trading with the local operator: AEAs buy GCs from the local operator to satisfy the RPS regulation, or sell the surplus GCs to the local operator for additional reward.

Process of Trading
The process of the two-phase hybrid trading of GCs is developed as follows: {Phase 1: Peer-to-Peer Trading} Step 1: (Initializing order). Each AEA initializes its trade order (including quotation and amount of GCs), and broadcasts on the trade platform.
Step 2: (Finding partner). Each AEA searches for partners, requests trading by using the multi-option-based matching.
Step 3: (Updating order). Each AEA updates both quotation and amount of GCs in case a deal is concluded; otherwise only updates quotation.
Step 4: Each AEA repeats Step 2 and Step 3 till the deficit/surplus GCs are cleared or the maximum trading round is reached.
{Phase 2: Centralized Settlement} Step 5: The local operator settles all the undealt GCs with AEAs: charges the deficit at P and rewards the surplus at P.

Types of AEA
Due to the individual diversity, we consider two types of AEA: naïve and sophisticated, whose behaviors are different in finding partner (Step 2) and updating order (Step 3).

•
Naïve AEA: Naïve AEAs are work-sensitive and aim to conclude deals with minimum work. In Step 2, a naïve AEA attempts to find partners which provide abundant amount, and in Step 3, a naïve AEA adjusts quotation by compromising the market price and the latest quotation of its own. • Sophisticated AEA: Sophisticated AEAs are profit-sensitive and aim to conclude deals for maximum profit. In Step 2, a sophisticated AEA attempts to find partners which provide favorable quotations, and in Step 3, a sophisticated AEA adjusts quotation by compromising not only the market price and its latest quotation, but also the time pressure.

Initializing Order
Due to the existence of centralized settlement phase, the quotations of AEAs in P2P phase can only vary within P and P, which denote the reward rate of surplus GCs and the charge rate of deficit GCs, respectively. The AEA with deficit GCs is a GC buyer. As for buyer A i , its trade order can be expressed as Q r buy,i , P r buy,i , where Q and P denote the amount and the quotation of the deficit GCs, respectively; the superscript r indicates the index of r-th round. As a sequel, the initial order of A i is Q 1 buy,i , P 1 buy,i , and the initial quotation is set as follows: where τ is a random number for simulating individual diversity among different buyers, and 0 ≤ τ ≤ 1. Similarly, as for seller A j , its trade order is denoted by Q r sell,j , P r sell,j , where Q and P denote the amount and the quotation of the surplus GCs, respectively; the superscript r indicates the index of r-th round. The initial order of A j is Q 1 sell,j , P 1 sell,j , and the initial quotation is where τ is a random number for simulating individual diversity among different sellers, and 0 ≤ τ ≤ 1.
All trade orders are broadcast on the trade platform. Both buyers and sellers can comprehensively evaluate each other by scoring to find partners.

Scoring Partner
Assume that AEAs consider two factors when searching for partners: GC quotation and GC abundance.

•
Factor 1: GC quotation The quotations of two parties determine whether a deal can be concluded. As for seller A j , its score in GC quotation is Similarly, the score of buyer A i in GC quotation is It reveals that the higher GC quotation, the lower score for seller and the higher score for buyer.

•
Factor 2: GC abundance Since naïve AEAs are work-sensitive, they tend to find trading partners with abundant GCs so as to clear trading orders in fewer rounds. Define the score in GC abundance of any seller A j with respect to buyer A i Equation (5) indicates that buyer A i assigns a full score if a seller has abundant GC to satisfy all its demand, otherwise the lower abundance, the lower score. The score in GC abundance of any buyer A i with respect to seller A j has a similar expression.
Each AEA scores all possible partners by weighed aggregation of the scores in GC quotation and GC abundance where W r i,j denotes the score rated by A i for all possible partners A jS in r-th round, α i and β i are the weights of GC quotation and GC abundance, respectively. If A i belongs to a sophisticated type, α i ≥ 0.5; otherwise, α i < 0.5.

Multi-Option-Based Matching
In order to improve the efficiency of order-matching, we designed the multi-optionbased matching. The procedure is shown in Figure 2. A buyer is permitted to have multiple options of a trading request, i.e., a buyer can request at most N sellers for trading. On one hand, each buyer evaluates the scores of sellers, sorts them in descending order, selects the top-N ones, and sequentially requests for trading. On the other hand, each seller evaluates the scores of buyers requesting for trading, and accepts the buyer with the highest score. The buyer will move to its 2nd option if the trading request is denied by its 1st option. The multi-option-based matching provides each buyer N with preceding chances.
A trading pair is formed if a buyer and a seller simultaneously choose each other, and then they can conclude a deal. Since the P2P market stipulates that each AEA can trade at most once in a round, those who have successfully formed pairs cannot match any other AEA in this round. As for a trading pair including buyer A i and seller A j , we denote the deal price of GC, the deal amount of GC, and the market price by P (i,j) , Q (i,j) , and P, respectively. Then we have: where D is the deal set and the superscript r indicates the variable is associated to r-t round.

Updating Order
Once a trading round ends, both of the two parties update their orders to prepare for the next round.
A naïve buyer A i updates the quotation by statically compromising two factors: latest quotation of its own and market price: where δ is a constant weight, and 0 ≤ δ ≤ 1.
A sophisticated buyer A i updates the quotation by dynamically compromising three factors: charge on deficit GC, latest quotation of its own, and market price: where ε is the time-varying weight indicating the time pressure on this buyer.
Similarly, the quotation updating of a naïve seller A j is shown in Equation (14): where δ is a constant weight, and 0 ≤ δ ≤ 1.
A sophisticated seller A j updates the quotation by dynamically compromising three factors: reward on surplus GC, latest quotation of its own, and market price: where ε is the time-varying weight indicating the time pressure on this seller: We set P r = P r sell,j in case no deal is concluded in r-th round. Moreover, both of the two parties simultaneously update the GC amount: where x r (i,j) is a Boolean value. x r (i,j) = 1 if a deal is concluded between buyer A i and seller A j in r-th round, and otherwise x r (i,j) = 0.

Simulation Setups
Numeric studies are performed on a community including 30 AEAs to verify the effectiveness of the proposed two-phase hybrid mechanism of GC trading. Generally, the consumption of 1MWh of renewable energy corresponds to one unit of GC, and GCs are fractional. The two-phase hybrid GC trading is a monthly market. The charge rate of deficit GCs P = 130 CNY/Unit, and the reward rate of surplus GCs P = 90 CNY/Unit. The simulation environment is, Intel (R) Core (TM) i7-7700 @ 3.6 GHz, 8 GB DDR3 RAM, MATLAB-2018a, CPLEX12.8.
As shown in Table 1, we consider three typical scenarios: globally-balanced community, undersupplied community, and oversupplied community. Figure 3 further illustrates the monthly initial GC amount of each AEA in the three scenarios, and the surplus and deficit GCs are presented by positive and negative numbers, respectively. The parameters of 30 AEAs in the community are shown in Table A1.

Determination of Key Parameters
This subsection determines two key parameters of the model: the option number N and the maximum trading round R. We consider the following three settings of "multioption-based matching":    Figure 4a illustrates the frequency of the round at which all order-matchings are completed. As for the average case, it is most likely to take 13 rounds for the uni-option to complete all order-matchings; while the results of bi-option and tri-option are both 9. As for the worst case, uni-option may spend at most 17 rounds to complete all order-matchings; while the results of bi-option and tri-option are both 14. Furthermore, Figure 4b illustrates the number of successfully matched orders in each option of the three multi-option settings. It is revealed that both bi-option and tri-option are superior to uni-option on the efficiency of order-matching; however, tri-option can hardly improve the efficiency as compared to bi-option. To summarize, bi-option (N = 2) is the optimal setting for a globally-balanced community. The similar results of the undersupplied and oversupplied community are shown in Figures 5 and 6, respectively. It is also revealed that bi-option (N = 2) is the optimal setting for both communities.   Table 2 indicates the average round at which all order-matchings complete under the three settings of multi-option in the three scenarios. The 2nd column of Table 2 shows the experimental results in the globally balanced community. Notice that the average round at which all order-matchings complete of uni-option, bi-option, and tri-option is 12.53, 8.94, and 8.97, respectively. Obviously, bi-option is far superior to uni-option in terms of time efficiency (8.94 vs. 12.53) and bi-option is slightly better than tri-option (8.94 vs. 8.97). The 3rd and 4th columns of Table 2 show the results in the oversupplied and undersupplied community, respectively. It is also revealed that bi-option is the optimal setting for the multi-option-based matching in these two scenarios. Based on the aforementioned experimental results, we can determine the key parameters of the model: N = 2 and R = 16 for this 30-AEA community. We adopted these settings in the following sections.

Verification
In this subsection, we perform simulations to verify the effectiveness of the two-phase hybrid trading in details. The simulation scenario is the undersupplied community as defined in Table 1.

Overall Trading Results
The two-phase hybrid mechanism provides each AEA with two alternatives of GC trading: P2P and centralized. Generally, AEAs give preference to the former, and adopt the latter if there are unsettled GCs. Figure 7 illustrates the results of buying or selling GCs for each AEA in the undersupplied community. Notice that the positive and negative bars indicate the bought and sold GCs, respectively. Since the community is undersupplied, i.e., the total amount of deficit GCs is much higher than that of surplus GCs, only a few buyers can buy enough GCs in P2P phase. As illustrated in Figure 7, some buyers (A 5 , A 11 , A 15 , etc.) still have deficit GCs by the end of P2P phase, and have to buy from a local operator in the centralized phase. Table 3 demonstrates the trading results of 19 deals among 30 AEAs in the undersupplied community. The 1st column lists the indexes of concluded deals, the 2nd and 3rd columns show the two parties of each deal, and the 4th and 5th columns present the GC amount and price of each deal. Notice that the same seller (buyer) may conclude deals with different buyers (sellers) due to multiple trading rounds, e.g., seller A 18 trades with buyers A 21 , A 3 , A 24 , and A 30 in Deal #8, #12, #15, and #19, respectively; buyer A 1 trades with sellers A 4 , A 23 , and A 8 in Deal #4, #9, and #10, respectively.  Figure 8 depicts the whole process of P2P trading. Since AEAs can freely choose partners, many orders can be matched in parallel and multiple deals can be concluded simultaneously. As shown in Figure 8a, it only takes 5 rounds for all sellers to clear their orders. By the end of P2P phase, all the surplus GCs have been sold, and 11 buyers still have deficit GCs. As shown in Figure 8b, the numbers of concluded deals in the first five rounds are 5, 4, 4, 4, and 2, respectively. It can be observed that the numbers contributed by the 1st option are 3, 4, 3, 2, and 2, and the numbers contributed by the 2nd option are 2, 0, 1, 2, and 0. It is verified that the setting of bi-option is effective to improve the efficiency of order-matching.

Economic Advantages
The two-phase hybrid market is compared with the sole centralized one to reveal its economic advantages. Table 4 shows the total revenues/expenses of AEAs by participating in the two different markets in the undersupplied community. It is demonstrated that by participating in the two-phase hybrid market instead of a sole centralized market, the total expenses of buyers decreases by 7.0% (from 14,718.33 CNY to 13,688.55 CNY) and the total revenue of sellers increases by 25.7% (from 5493.24 CNY to 6904.91 CNY). To summarize, both buyers and sellers can benefit from participating in the hybrid market. Furthermore, we compare the individual expense/revenue of each AEA by participating in the two markets. As shown in Figure 9, the positive and negative bars indicate the individual revenues and expenses, respectively. It is observed that each AEA can benefit from participating in the hybrid market due to the existence of P2P phase. Since the P2P trading stipulates all quotations vary between P and P, sellers can expand their revenues and buyers can save their expenses by participating in the hybrid market. As a result, AEAs are more active to take the initiative and participate in the hybrid market to trade GCs within the community. Due to the economic favorability to all AEAs, the proposed two-phase hybrid trading is also environmentally friendly, which can promote the implementation of RPS. In this subsubsection, we study the impact of AEA type on the quotation updating process. The simulation scenario is the globally balanced community defined in Table 1. Four AEAs are selected as typical representatives and investigated: A 16 (naïve buyer), A 8 (sophisticated buyer), A 28 (naïve seller), and A 7 (sophisticated seller).
The same simulation is repeatedly conducted for 200 trials to mitigate the influence of randomness. Since it takes, at most, 11 rounds to close the P2P phase among the 200 trials, we only illustrate and investigate the quotation updating of the first 11 rounds. Figure 10a,b depict the pointclouds of quotation updating of a naïve buyer (A 16 ) and a sophisticated buyer (A 8 ), respectively. It is observed that the quotation of the sophisticated buyer converges faster and to a higher level than that of the naïve buyer. The sophisticated buyer tends to have a quotation higher and close to p near the end of P2P phase, since it attempts to clear the deficit GCs in P2P phase and avoid buying GCs from a local operator.
On the contrary, the quotation of the naïve buyer is not significantly affected by the trading round and exhibits more randomness. Similarly, the pointclouds of quotation updating of a naïve seller (A 28 ) and a sophisticated seller (A 7 ) are demonstrated in Figure 11a,b, respectively. It is shown that the quotation of the sophisticated seller converges faster and to lower level than that of the naïve seller. The sophisticated seller tends to have a quotation lower and close to p near the end of P2P phase, since it attempts to clear the surplus GCs in P2P phase and avoid selling GCs to the local operator. On the contrary, the quotation of the naïve seller does not significantly vary with the trading round and has little evidence of convergence.   As for naïve buyers and naïve sellers, they do not have urgent needs to buy or sell GCs in P2P phase. Their quotations slightly decline with the increasing round since they quote by compromising the market price and their latest quotations.
As for sophisticated buyers, their quotations decrease in the first three rounds since the median of their initial quotations is higher than the market price, and then their quotations gradually increases due to the increasing time pressure of the closing P2P market.
As for sophisticated sellers, the median of their initial quotations is also higher than the market price. It is observed that the quotation monotonically decreases with the increase of trading round and descends rapidly near the end of P2P phase so as to improve their competitiveness in selling GCs.

Impact on Concluding Deals
Hereinafter we investigate the impact of AEA type on concluding deals. The simulation scenario is an oversupplied community defined in Table 1. The parameters are modified for the ease of comparison: there are 14 buyers and 16 sellers in the community, each with 5 units of initial GCs for trade, and all initial quotations are 122.59 CNY/unit. Two indexes are defined as follows: • average deal price: as for an AEA, the average price of all concluded deals in P2P phase. • P2P deal ratio: as for an AEA, the average ratio of the amount traded in P2P phase and the total amount.
The same simulation is repeatedly conducted in 200 trials to mitigate the influence of randomness. Figure 13a,b show the results of concluded deals of buyers and sellers, respectively. The P2P deal ratios are presented by using solid lines, and average deal prices of naïve and sophisticated AEAs are presented by using dark and light bars, respectively.
In Figure 13a, it is observed that the P2P deal ratios of sophisticated buyers are slightly higher than those of naïve buyers. All sophisticated buyers can complete 100% of deficit GCs in P2P phase, and naïve buyers can also complete 80% by the end of P2P phase. Consequently, the two types of buyers have similar average deal prices. On the contrary, Figure 13b illustrates that the P2P deal ratios of sophisticated sellers are significantly higher than those of naïve sellers. All sophisticated sellers can sell 100% of surplus GCs in P2P phase, while most naïve sellers still have over 50% of surplus GCs by the end of P2P phase. As a result, the average deal prices of sophisticated sellers are much higher than those of naïve sellers. In this oversupplied community, buyers and sellers are at superior and inferior side, respectively. The two types of buyers have similar performance in concluding deals despite the different behaviors in updating quotations. However, the performances of sophisticated sellers are much better than those of naïve sellers. Thus it is crucial for AEAs at the inferior side to be "sophisticated" rather than "naïve".

Conclusions and Future Work
This paper addresses the GC trading problem under the RPS regulation policy and proposes a two-phase hybrid trading combining the P2P and centralized settlement. The market structure describes the trading mechanism among AEAs and local operators. Two types of AEAs are defined: naïve and sophisticated, which are work-sensitive and profitsensitive, respectively. The whole process of P2P trading is developed and formulated. A multi-option-based matching is proposed to improve the efficiency of order-matching for AEAs.
Numeric studies are performed on a 30-AEA community and three different market scenarios are considered: globally balanced, undersupplied, and oversupplied. It is experimentally proved that bi-option is the optimal setting of multi-option-based matching. Simulation results verify that the two-phase hybrid trading is effective in terms of overall trading and the P2P order-matching. It is also demonstrated that both buyer and seller can benefit from participating in the hybrid market. We also investigated the impact of AEA type. The quotations of sophisticated AEAs vary more significantly than those of naïve counterparts during P2P phase. Additionally, sophisticated AEAs gain higher economic benefits than the naïve counterparts, especially for those at the inferior side.
Our future work may include two aspects: Firstly, to expand the community scale and study the trading results among hundreds of AEAs; secondly, to employ the blockchain technology and implement a prototype system.

Acknowledgments:
The authors thank Ning Wang for his explorative work that provided the basis of this paper.

Conflicts of Interest:
The authors declare no conflict of interest.