Toward a Chatbot for Financial Sustainability

Abstract

This study examines technology effectiveness for industry demand in which artificial intelligence (AI) is applied in the financial sector. It summarizes prior studies on chatbot and customer service and investigates theories on acceptance attitudes for innovative technologies. By setting variables, the study examines bank revenue methodologically and assesses the impact of customer service and chatbot on bank revenues through customer age classification. The results indicate that new product-oriented funds or housing subscription savings are more suitable for purchase through customer service than through chatbot. However, services for existing products through chatbot positively affect banks’ net income. When classified by age, purchases by the majority age group in the channel positively affect bank profits. Finally, there is a tendency to process small banking transactions through the chatbot system, which saves transaction and management costs, positively affecting profits. Through empirical analysis, we first examine the effect of an AI-based chatbot system implemented to strengthen financial soundness and suggest policy alternatives. Second, we use banking data to increase the study’s real-life applicability and prove that problems in customer service can be solved through a chatbot system. Finally, we investigate how resistance to technology can be reduced and efficiently accommodated.

1. Introduction

Professor Yoshua Bengio, the winner of the 2019 Turing Award, gave a lecture on core technologies in deep learning, such as meta-learning and reinforcement learning, at the Samsung AI Forum 2020 in November 2020. He refuted what Professor Carl Benedict Frey had argued, citing success stories in the application of information technology (IT) in the financial sector. Professor Frey had argued that less than half of financial jobs were set to disappear with the increasing use of artificial intelligence (AI). However, Professor Bengio predicted, Professor Frey’s arguments would lose their convincing power [1], as it had happened for Professor Zoonky Lee who had published articles in a Korean newspaper to combat prejudice against artificial intelligence (Special Series of JoongAng Daily, “Lee Zoonky, Ask about the Future”) [2]. The common points between Lee and Frey are as follows. Considering the history of technology adoption, technological innovation should be considered as a digital transformation that changes roles rather than kills jobs. Hence, as AI grows, digital transformation occurs and people seek new roles. A chatbot, which provides advice on financial products to customers, applies AI to the financial industry. Both Lee and Frey conclude that a chatbot does not eliminate jobs; rather, humans use the chatbot system to venture into new areas. The lack of insight, imagination, and responsiveness to new variables of the chatbot algorithms require humans to resolve them. Thus, AI creates a new ecosystem within the industry, and the role of humans changes for a new era in which machines and humans coexist in a complementary way [3].

Natural language processing technology and speech recognition technology are currently providing personal assistant services that communicate directly through personal mobile devices [4]. Chatbot, an interactive AI, has been widely deployed in finance, retail, public, and manufacturing industries. Apple’s Siri, Amazon’s Alexa, Google’s OKgoogle, and Samsung Electronics’ Bixby are good examples of voice conversion personal assistant services. In addition, Naver and Nugu of SK Telecom provide high-quality voice recognition services through the Korean search platform and communication market. As such, major IT companies, portal sites, and telecommunication companies worldwide are developing commercial voice recognition services and investing significant financial resources to provide AI services with higher accuracy [5].

This study investigates the role of AI in the financial industry from several aspects. First, the social demand for and expectations from artificial intelligence in the financial sector are high. The amount of investment in this field is larger compared to other industries such as the distribution, manufacturing, and public sectors [6]. Second, despite this interest, there are many misconceptions around using AI, and whether AI has been properly applied to the financial industry has been questioned [7]. Finally, the systematic criticism of AI technology applied to financial products is lacking in extant research. Recent media comments about AI describe the positive and negative effects of AI in a stimulating tone [8]. However, it is difficult to find an in-depth comparative study. Many recent studies examine the combination of AI and the financial sector because anyone who engages in economic activities is a financial consumer. Furthermore, even software used exclusively by traditional asset managers can be downloaded easily and used by ordinary people [9]. Hence, AI in the financial industry is simply a tool that individuals can access and use; it is not the exclusive property of experts. This study compares and analyzes the impact of customer service through the existing automatic response service (ARS) with the chatbot system currently being used by banks. In addition, we empirically analyze data to determine how each of these two channels (customer service and chatbot) affects bank profits and then derive practical implications based on the results [10].

The article structure is as follows. Section 2 reviews prior extant research, divided into four areas. First, through the latest research on financial chatbot systems, we investigate AI technologies and their effects in financial environments. Second, we summarize the research on problems faced by customer service counseling staff and on coping strategies and techniques to solve them. Third, we study theories of effective ways to introduce technology. Finally, we examine prior research on indicators representing bank contribution from a methodological perspective. In Section 3, we set the hypothesis for this study and perform statistical analysis based on bank data for new products and existing services by channel. We conduct quantitative analysis using statistical techniques to establish and verify the hypotheses while considering prior studies and descriptive statistics. Section 4 evaluates the theoretical underpinnings verified by data and summarizes the study. Finally, we conclude this study by revealing implications, limitations, and future research plans.

2. Background

2.1. Financial Chatbot Service

The term “chatbot” is a combination of “chatting” and “robot,” which is commonly used for text messages or messengers. A chatbot is a communication software that can store appropriate answers to questions on a server, create models that continuously develop correct answers through conversations with customers, control exceptions, and provide accurate answers [11]. Chatbots create a self-learning model through computer programs and mathematical calculations and provide customers with answers and other relevant information as close as possible to user questions in real time. For companies, a chatbot is an interface that provides information required by customers and marketing through communication with financial consumers [12]. The first chatbot service in the financial sector was Bank of America’s Erica introduced in May 2017. Erica’s early look was similar to Apple’s Siri. Erica provided simple text and voice-based responses, including transaction details, limit amount, and account balance. Additionally, it provided advanced services such as credit rating upgrade application, fund product introduction, bank loan application, interest rate guidance, utility bill payment, and fund management consulting services [13,14]. The chatbot learned customers’ personal profile information, past financial product purchase history, location information, and personal routine data by applying machine learning and deep learning technologies to provide accurate and customized services. Customers could enjoy convenience by securing personalized financial services quickly and easily through a chatbot [15]. In Korea, most commercial banks and other types of financial institutions have introduced chatbot services for customers (

Table 1. Financial chatbot services in Korea.

Type	Financial Institution	Chatbot Name	Service Platform	Starting from
Banking Corps.	Shinhan	Aurora	Shinhan Sol	2018. 02
	Kookmin	Smartly (TalkTalk)	Liiv TalkTalk	2017. 07
	NH	Consultation Talk	NH banking	2018. 11
	Hana	HAI	Hana Members	2017. 09
	Woori	Wibee-bot	WibeeTalk	2018. 09
Credit Card Company	Shinhan	FANi	Shinhan Paypal	2017. 06
	Samsung	Sam	Chatbot Sam	2019. 03
	Hyundai	Henry & Fiona	Buddy	2017. 08
	Lotte	LOCA	The Loca Lab	2018. 04
Others (Securities, Insurance, and Third Bank Sector)	Daishin (Sec.)	Benjamin	Kakao Talk	2017. 09
	Samsung (Ins.)	Tabot	TABOT	2017. 06
	Welcome (3rd S.)	Welcomebot	Kakao Talk	2017. 09
	OK (3rd S.)	Oktok	Kakao Talk	2017. 08
	JT (3rd S.)	JT Mobile Chatbot	Kakao Talk	2018. 05

).

Chatbots can be classified into a retrieval model and a generative model according to the implementation method in web or mobile applications. First, a chatbot based on the retrieval model is a rule-based method that provides prepared answers according to conditions of a specific topic or question. Most early chatbot versions in financial institutions were developed based on rules [16]. However, with the commercialization of chatbot, sophisticated machine learning has become possible as industry data continue to accumulate. Second, the generative model chatbot is a deep learning method that improves the accuracy of new responses through self-evolution as customer and communication data accumulate [17]. With the latest developments in deep learning technology, the system understands the customer’s question and the intent of the sentence and presents the appropriate answer to the customer [18,19]. Therefore, it is possible to recommend personalized products for customers. Studies for commercialization are being actively conducted to capture current emotions of users through individual routines and basic profiles. Despite its many advantages, cost is an issue with the generative model because it requires the accumulation of vast amounts of data for continuous self-evolution [20].

Chatbots are important in terms of technology and user interface (UI). The chatbot is a technology service that implements communication between users and AI-based on text and voice and is a representative non-face-to-face service. Most chatbot services are implemented through conversational interactions based on customer questions and chatbot responses [21]. Through the interaction process with machines, customers perceive chatbots as objects of communication rather than simple machines [22]. Therefore, the chatbot service should be designed to reflect user needs and planned as an efficient and proven system with clear interactions. Chatbot services are mostly text-based messengers in online or mobile applications [23]. Therefore, a UI that enables customers to input and check information on a small screen effectively is essential. As shown in

Table 2. Interface design elements of the financial chatbot.

Process	Design Element	Interface Example
Access Screen	Functional Design	Chatbot location
	Value Design	Chatbot icon and name by function
Start Screen	Visual Design	Background color and overall layout
	Functional Design	Help on key features
Answer Screen	Functional Design	Speech bubble space utilization and option selection function
	Value Design	Character and profile image
Information Screen	Visual Design	Graphic information

, design elements, such as product composition, button position, and background color, may vary depending on the screen. Therefore, the chatbot’s design needs to project user experience on the screen elaborately. In addition, the screen of the chatbot is a publicity vehicle that presents the image of the company [24].

2.2. Telemarketing and Technical Elements of Alternative Systems

Customer services centers provide online consultation with and for customers. They operate under various names such as customer support centers, call centers, contact centers, and customer relationship management (CRM) centers, depending on the company [25]. Initial customer service began as an organization that performed simple response services by receiving calls from customers [26]. In recent years, it has transformed into an organization that creates added value by enhancing the corporate image, providing information on products, conducting marketing and promoting activities, providing customer service, and communicating with customers. Customer service is an organization that provides non-face-to-face interactions with customers, but these interactions require emotional labor beyond face-to-face channels [27]. Customer service’s emotional labor is an essential element of a company, as it can retain existing customers, attract new customers, and maintain a company’s competitive advantage. However, this causes considerable stress on workers due to the incongruity of internal emotions and external expressions. These difficulties have led many companies to build systems that replace customer service [28,29].

Many technical elements are required to build an alternative customer service system. The customer service helper system must respond appropriately by inferring the meaning of the customer’s question in real time [30]. For this reason, a semantic reasoning technique that can infer the meaning of a customer’s query and provide an appropriate answer is essential. Semantic reasoning techniques can be classified into two broad categories according to their development process. First, knowledge-based question-and-answer (Q&A) structures are used by humans in the system using an ontological method. This method finds the result by inferring the large-scale knowledge database built in a logical form. Second, information retrieval-based Q&A orders a list of answers through probabilistic calculations by searching for answers to questions based on indexes in a large document set [31,32].

For the alternative customer service system to provide intelligent services, a Q&A method through ontology-based reasoning should be used, rather than a simple rule-based search. Recently, owing to the development of deep learning technology, ontology-based Q&A technology has been used in industries and chatbots in the financial sector (

Table 3. Components of intelligent virtual assistant technology.

Division	Component
Interactive interface	Speech recognition, multimodal, context recognition
Semantic reasoning	Intelligence level	Assistant chatbot, intelligent assistant, cognitive assistant
	Conversation process	Goal-oriented conversation processing, question and answer skills
	Knowledge	Semantic Web, ontology-based technical data
Other services	Modeling, big data analysis, web service

) [33]. Recently, the application of AI and advanced statistical analysis has enabled users to control local information, weather guidance, Internet searches, route searches, and product searches. These systems can provide advanced services based on user experiences [34].

Another essential element for alternative customer service systems is voice recognition technology. In 1952, the AT&T Bell Laboratory in the United States developed the first technology to convert recordings into text. Since then, various laboratories have attempted to develop speech recognition, but the accuracy has not exceeded 80%. The low accuracy of voice recognition is due to different accents, volume, degree of dialect, and background noise [35].

Figure 1 illustrates a recently developed two-step voice recognition and conversion model that leverages deep learning techniques to recover ambiguous speech and further clarifies speech semantic transmission by considering speech characteristics and the surrounding environment. Voice recognition techniques are evolving into deep learning-based systems that can recognize speech, including long sentences or dialogues [36].

2.3. Intention to Accept New Technology and Its Spread

Due to internal conflicts and external situations of the system, it is difficult to accommodate and apply innovative technologies to existing systems to create a completely different paradigm [37]. In the financial sector, especially in organizations that perform customer services using mainly call centers, considerable conflicts, along with trial and error, will occur when applying chatbot services initially [10].

This study examines five theories on technology acceptance and diffusion. First, the theory of reasoned action (TRA) is the basis for acceptance and proliferation of new technologies, which argues that consumer attitudes influence behavior and that behavior can be predicted if attitudes are accurately measured. In particular, this theory presupposes that people are highly rational and systematically use the information they have. TRA has three components—attitude, subjective norm, and intention [38].

Second, the newly defined technology acceptance model (TAM) is based on TRA and focuses on user evaluation of the technology as a model to emphasize individual characteristics or beliefs in the process of accepting technology [39]. TAM argues that the greater the perceived ease and perceived helpfulness of users, the greater the behavior and intention of using technology. Used by several researchers, TAM is recognized as an excellent model that demonstrates simple and high explanatory power in explaining users’ IT acceptance and utilization behaviors [40].

Third, diffusion of innovation theory comprehensively describes the process by which a new paradigm of innovation is accepted and adopted by a particular organization or individual [41]. The theory considers the psychological rejection of accepting new techniques. Innovation resistance is the tendency of individuals to maintain their status quo. Created perceived risk is an important concept in accepting technologies. Perceived risk is the user’s subjective perception of uncertainty about the future and possible negative consequences [42].

Fourth, the theory of planned behavior (TPB) is different from all of the above because it includes intentional action and strategic intention, and planned behavior control. This theory argues that control of intended and planned actions should be added to the performance of actions. TPB emphasizes that the main determinants of behavior are not the individual’s attitude toward it but the intention to perform it and that it is under human control. From this perspective, we add a new concept, a critical variable called perceived behavior control, which sufficiently compensates for the weaknesses in rational behavior theory (Figure 2) [43].

Fifth, the unified theory of acceptance and use of technology (UTAUT) is a highly descriptive model, because it has been selected as a significant proven factor through numerous trials and verification procedures [44]. In particular, studies analyzing acceptance of fintech payment services, by applying additional variables called reliability to UTAUT, show that individual effort, social impact, and reliability have a positive impact on the acceptance of fintech services. Furthermore, studies using UTAUT in consumer use of internet banking have shown that variables, such as information security risks, uncertainty risks, and transaction efficiency, have a negative impact on the dispersion of internet banking. Prior research results demonstrate that UTAUT is suitable for measuring the intent to use chatbot services introduced by many financial institutions. Studies have found that variables, such as consumer performance expectations, social impact, and promotion conditions, have a significant effect on bank performance [45].

Through the various technology acceptance models described above, we deduce a positive effect of lowering internal resistance and encouraging pro-sustainability behavior, even though there is the disadvantage of being slightly expensive strategically as several variables are added. In addition, we expect to be able to develop models for advancing theory improvement and environmental policy formulation [46].

2.4. Profitability Indicators

Research using detailed profitability indicator data from companies is limited. However, many studies in finance and accounting have used stock market data through the Open API (Application Programming Interface) as a dependent variable. Research has been conducted on the quality of services that are difficult to determine quantitatively [47]. In addition, there are many studies on how political factors cause instability in the financial industry. An empirical analysis of the Bank of Korea’s profitability determinants and policy measures that conducted a regression analysis using independent variables such as equity ratio, per capita expenses, assets per capita, total receivable growth rate, and corporate bond yields [48]. A study of profitability determinants for commercial banks in Japan empirically analyzed how the classification of ownership structures affects profitability. The study used gross asset net profit margins, return on equity (ROE), and net interest margin (NIM) as indicators of profitability [49]. Furthermore, a Korean study conducted a multi-regression analysis using major financial indicators and macroeconomic data of general banks from 2000 to 2009 to identify the profitability determinants of banks. The study found that the non-profit loan ratio (NPL) had a statistically significant effect on the profitability of commercial and local banks, and that poor loan management in banks had a significant impact on asset size [50].

Research in the financial sector, which specializes in financial profitability, examined bank profitability determinants in Europe, North America, and Australia, using gross asset net return, return on capital (ROC), and value-added return on total assets as indicators of profitability [51]. Another study identified the impact of each independent variable on the subsidiary variable using gross capital operating profit, gross capital net income, gross capital net return, and net sales net income, of which gross asset net income was the most effective indicator [52]. Other studies compared the profitability and efficiency of commercial and local banks to examine the impact on the bank’s management performance and suggested ways to stabilize the profitability of local banks. This was an empirical analysis of the factors affecting profitability with time-series cross-section regression, using portfolio mix as a methodology, and using changes in stock prices and gross capital return as an indicator [53]. In addition, long-term time-series data from 22 general banks were used to ascertain the determinants of the bank’s management performance using the net return on assets and the ratio of non-profitable loans as an indicator of the general bank’s profitability. These results demonstrate that macroeconomic variables affect bank asset portfolio and productivity variables. Another study used approximately 10 years of accounting data from Spain, Portugal, Germany, and France to analyze the relationship between net return on assets and net return on equity and profitability on commercial banks [54,55].

3. Methods

3.1. Samples and Data Collection

This study analyzed product data from a large Korean banking company to determine the impact of customer product and service purchases on bank profits (return rate increase) based on two channels using ARS. We analyzed the statistical significance of how much each channel contributed to bank profits based on customer information using financial products and services through customer service calls or chatbot systems. We anticipate that our analysis will help banks derive measures to secure financial stability. Furthermore, we expect to empirically derive the extent to which AI-based chatbot can replace the existing customer service business for all financial affiliates, including banks. From Bank A, we collected 34,089 personal data of four major products and services sold through the chatbot channel for 36 months (on a daily basis) from January 2018 to December 2020, when the chatbot was first introduced at this bank. In addition, we collected 317,438 unstructured voice data acquired through customer service based on similar products at the same time. We standardized the unstructured data through a text conversion system and used a two-step voice recognition and conversion model. Except that each of the four products was handled through customer service or chatbot, all conditions were completely the same; therefore, it is safe to assume that the statistical effect is controllable in advance. Bank A is a nationwide commercial bank, with its target customers individuals residing in Korea; it handled all products during the time of the study. Therefore, the conditions for recognition of region, seasonality, environment, and age are the same. In addition, statistical sampling bias is assumed to be controlled, because the data handled were not part of the extracted data but the parameter data for the entire product. However, unlike chatbots, in the case of responses through counseling staff, there may be a promotional event depending on the period. Therefore, the purchase of a product different from the original purpose may occur due to a specific promotion. However, this cannot be measured quantitatively, and it can be assumed that the effect of counselor promotion is negligible in a situation in which the response to customer purchases is the primary purpose of inbound calls. We deleted sensitive information from Bank A’s customer data through blur-masking. In addition, we made the response to the information protection request by performing mixed-combination conversion of the primary key and set it as data that can be analyzed through data cleansing. Financial product information as final analysis data is shown in

Table 4. Classification of sample data.

Financial Goods	Customer Service	Chatbot	Total	%
Fund subscription	28,435	2531	30,996	8.8
Housing-subscription savings	49,937	4365	54,302	15.4
Loan interest payment	54,833	6350	61,183	17.4
Utility bill	187,233	18,843	206,076	58.5
Total	320,438	32,089	352,527	100.0

.

Bank A’s main products are new sales of funds and home subscription savings products, loan interest payment and repayment services, and local taxes and utility expenses payment services, with a total of 351,527 cases. Since we used the analysis data based on the number of cases, we counted all duplicate product purchases. The data collected included contract channel (contract manager, chatbot unique allocation code), contract date and contract product, contractor’s identification information, and contract number for the individual number of all products. Based on the data, we performed basic statistical information, data preprocessing, hypothesis setting, and testing. We used SAS University Edition, an open-source software, for statistical analysis and the Oracle virtual machine to prepare the software operating environment.

3.2. Operational Definition and Preprocessing

Information on the four financial products selected for this study and product information for each channel through offline counters and online ARS is summarized as follows. First, in the case of funds, the total assets of listed funds (ETFs) handled by six major Korean banking companies amounted to USD 50 billion at the end of 2019. Adding unlisted funds, the amount is over USD 100 billion, which is an increase of 26.1% year-on-year—this is classified into 335 domestic products and 115 overseas products. By investor entity, individual investors account for 38.6% and institutional and foreign investors, 61.4%.

The second product group is housing subscription savings. As of August 2019, the number of Korean subscribers exceeded 25 million, accounting for 50% of the population, and the total amount exceeded USD 80 billion, with savings per person averaging at USD 3000.

The third product group is service products related to loan interest payment and repayment. At the end of 2019, the total amount of personal loans exceeded USD 1.3 trillion, including the amount on credit cards. The average loan per capita is USD 60,000, and interest expenses were, on average, over USD 300 per month.

The final product group, the amount paid in utility bills including local taxes and administrative fees is not large; however, recently commercial banks are promoting a policy to increase the number or amount through promotions. These policies have positive benefits for high interest rates and currency exchange; hence, consumers are also actively using this system.

Table 5. Data preprocessing according to variable classification.

Variable	Preprocessing	Remarks
Customer Number	Assign a unique number after masking	Excluding the first 2 digits
Age	Age of subscribers
Age Group *	Age category of subscribers	0 = under and equal 45, 1 = over 45
Purchase Date *	Date of first contact
Approval Date	Subscription savings, loan payment, and utility bills are processed in real time (same with Purchase Date)	Fund needs to adjust date according to conditions
Amount1	Subscription amount of Funds
Amount2	Amount of housing-subscription savings
Amount3	Amount of loan interest payment
Amount4	Amount of utility bills payment	Includes national tax, local tax, and other utility expenses
Purchase Channel	- Customer service: employee #- Chatbot: HQ unique code (CB0-#)
Channel Classification *	Customer service and chatbot channel classification	0 = customer service, 1 = chatbot
Net profit1 *	Revenue from Funds—Expenses	-Exp1: Counselor salary -Exp2: Chatbot cost (develop and maintenance)/average IT infra depreciation period (daily-base)
Net profit2 *	Revenue from housing-subscription savings—Expenses
Net profit3 *	Revenue from loan interest—Expenses
Net profit4 *	Revenue from utility bills—Expenses

* marked variable is newly created data for preprocessing.

shows the data preprocessing status.

As shown in Table 5, the * marked variable is newly created data for preprocessing. However, in the case of the fund’s approval date, it may differ from the sale date depending on the product’s contract terms and the buyer’s credit terms. For this study, age groups were classified as “Junior” for individuals less than 45 years old, and “Senior” for individuals of 45 years and older. Purchasing channels were classified by the employee number—58 employees of the ARS team at the bank’s head office—and the unique codes of employees of five other inbound marketing service companies. In the case of the chatbot, the purchasing channels were classified with Bank A’s own chatbot allocation code starting with “CB00”. All amount-related variables were calculated based on the total amount received by the bank for each product in the period. To estimate the effect of product-specific returns on bank contribution, we created a new variable of net increase or net income excluding costs from profits by using the gross return on assets (ROA), which was used as a dependent variable in previous studies. We set the customer service cost formula by dividing the number of customers by the sum of labor cost and organizational operation cost. We used the average annual depreciation cost ratio of general system infrastructure of 11.3% and general management cost for server operation to calculate the formula of development and operation cost for the chatbot. We divided this amount by the number of chatbot users and calculated the average cost per chatbot use. As a result, the final cost was set at USD 1.03 per case for customer service and USD 0.39 for the chatbot.

3.3. Descriptive Analysis

As shown in

Table 6. Descriptive statistics.

Channel	Goods	Age Groups				Total (%)
		Junior (%)		Senior (%)
Customer Service	Fund	15,665	(46.9)	17,770	(53.1)	33,435	(10.5)
	H.S.S.	23,469	(42.7)	31,468	(57.3)	54,937	(17.3)
	L.I.	22,845	(44.1)	28,988	(55.9)	51,833	(16.3)
	Bills	81,729	(46.1)	95,504	(53.9)	177,233	(55.8)
	Total	143,708	(45.3)	173,730	(54.7)	317,438
Chatbot	Fund	2,023	(79.9)	508	(20.1)	2,531	(7.4)
	H.S.S.	2,798	(64.1)	1,567	(35.9)	4,365	(12.8)
	L.I.	3,787	(59.6)	2,563	(40.4)	6,350	(18.6)
	Bills	13,105	(62.7)	7,738	(37.1)	20,843	(61.1)
	Total	21,713	(63.7)	12,376	(36.3)	34,089
Total	Fund	17,688	(49.2)	18,278	(50.8)	35,966	(10.2)
	H.S.S.	26,267	(44.3)	33,035	(55.7)	59,302	(16.9)
	L.I.	26,632	(45.8)	31,551	(54.2)	58,183	(16.6)
	Bills	94,834	(47.9)	103,242	(52.1)	198,076	(56.3)
	Total	165,421	(47.1)	186,106	(52.9)	351,527

, in the specific classification of each channel-product group, among all consumers who purchased all financial products using ARS, the number of customers who purchased products through customer service was about 9.3 times more than those who purchased the same products through the chatbot. Therefore, 90.3% of the parameter data purchased products through customer service, whereas purchases through chatbot only remained at 9.7%. In terms of age groups, the purchase of products and services through customer service is higher in the Senior group (54.7%) than in the Junior group (45.3%). This trend is the same for all four products sold through customer service. In particular, in the case of housing subscription savings, the gap widens by 14.6%, which is approximately 5% more compared to the average of 9.5%. In terms of the product purchase rate, 55.8% of customers use customer service to pay utility bills.

Regarding consumers using the chatbot, the distribution of purchases is completely different from that of customer service. First, in terms of frequency of use, the Junior group (63.7%) clearly used the chatbot more than the Senior group (36.3%). However, in terms of the product purchase rate, 61.1% of users, which is higher than customer service, used the chatbot for utility bill payment services. In addition, the frequency of purchases of funds and housing subscription savings, which are subscriptions for new products, is completely different from payment of loan interest or utility bills, which are services for existing products. The most striking statistic related to the difference between the chatbot and customer service channels is that customer service occupies a higher proportion of handling new products at 10.5% and 17.3%, compared to 7.4% and 12.8% of the chatbot.

3.4. Hypotheses

Considering the statistics in the case of new product sales, the ratio of total purchases per channel was lower in chatbot than in customer service. Conversely, in terms of loan interest payment and utility bill management, the chatbot has a higher relative ratio than customer service. Based on these data, we posit the following hypothesis to fit the assumption of the null hypothesis that there is no basis for expecting that new product purchases through customer service will have a greater positive effect on bank profits than purchases through the chatbot:

Hypotheses 1 (H1).

Comparing customer service and chatbot users, there is no difference in their impact on bank contribution according to product classification.

Considering age groups, the data demonstrated that the relative proportion of seniors is considerably large for products handled through customer service than for those handled through the chatbot. Conversely, in the case of product handling through chatbot, the proportion of Junior users was higher than that of Seniors. Therefore, we expect that specific age groups will have a greater positive effect on bank profits in the division by channel, and we propose the following hypothesis to fit into the null hypothesis assumption similar to H1:

Hypotheses 2 (H2).

Comparing customer service and chatbot users, there is no difference in their impact on bank contribution according to customer classification.

Finally, we examined the concrete effects of the two hypotheses. We created a cube model with a combination of four cases in the form of 2 × 2 by mixing product groups and customer age groups. We analyzed the effect of each combination on the increase or decrease in the bank’s net income. We categorized the sale of funds and housing-subscription savings products as “new product sales,” and categorized loan interest and payment of utility bills as “existing service provisions.” Utilizing these categories and the two age groups, we developed the four areas as follows: (1) New product sales–Junior group, (2) Existing service provision–Junior group, (3) New product sales–Senior group, and (4) Existing service provision–Senior group.

Table 7. The relative ratio of rows and columns by cube combination.

Combination		Channel			Col. Ratio	Row Ratio
		Customer Service	Chatbot	Total
Junior	New Products Sales	39,134	4821	43,955	8.1	26.6
	Provision of Existing Services	104,574	16,892	121,466	6.2	73.4
	Total	143,708	21,713	165,421	6.6
Senior	New Products Sales	49,238	2075	51,313	23.7	27.6
	Provision of Existing Services	124,492	10,301	134,793	12.1	72.4
	Total	173,730	12,376	186,106	14.0
Total		317,438	34,089	351,527	9.3

presents the data of the cube combination.

We present the following hypotheses for each of the four combinations to investigate their bank contribution:

Hypotheses 3a (H3a).

In the case of the Junior group who purchased new products, there was no difference in the degree of contribution to the bank according to the classification by channel;

Hypotheses 3b (H3b).

In the case of the Junior group that received the existing services, there was no difference in the degree of contribution to the bank according to the classification by channel;

Hypotheses 3c (H3c).

In the case of the Senior group who purchased new products, there was no difference in the degree of contribution to the bank according to the classification by channel;

Hypotheses 3d (H3d).

In the case of the Senior group that received the existing services, there was no difference in the degree of contribution to the bank according to the classification by channel.

4. Results

4.1. Statistical Hypothesis Testing

To test Hypothesis 1, which assumes that there is no difference in the impact on bank contribution of customer service and chatbot users according to product classification, we performed an analysis of covariance (ANCOVA), as shown in

Table 8. Covariance analysis test for Hypothesis 1 (H1).

	DF	SS	MS	F-Value	p-Value
Model	2	145.3548	72.6774	18.8893	<0.0001
Error	357,435	1,375,248.4582	3.8475
Total	357,437	1,375,393.8130
Parameter	DF	Estimate	S.E.	T for H0	p-value
Intercept	1	−1.4275	0.05251	−2.719	0.0014
T	1	0.0215	0.1457	0.148	<0.0001
NT	1	0.0378	0.2437	0.155	<0.0001

.

The results show that both purchases of new products and existing services have a significant effect on the increase or decrease in bank profits according to customer service and chatbot channels. This means that new product-oriented funds and housing subscription savings are more suitable for customer service than the chatbot. Conversely, services for existing products, such as loan interest or payment of utility bills, are more suitable for processing through chatbot, which has a positive effect on bank net income.

Hypothesis 2 assumes that there is no difference in the impact on bank contribution of customer service and chatbot users according to customer classification. We performed ANCOVA, as shown in

Table 9. Covariance analysis test for Hypothesis 2 (H2).

	DF	SS	MS	F-Value	p-Value
Model	2	645.3548	322.6774	70.1013	<0.0001
Error	357,435	1,645,278.4582	4.6030
Total	357,437	1,645,923.8130
Parameter	DF	Estimate	S.E.	T for H0	p-value
Intercept	1	3.4572	0.4251	8.133	0.073
T	1	0.0035	0.0024	1.458	<0.0001
NT	1	0.0081	0.0075	1.080	<0.0001

, to test two or more elements, as in Hypothesis 1.

Hypothesis 2 secured model suitability according to the F-test result (F = 70.1013). From the results (Table 9), we conclude that both Junior and Senior customers have a significant effect on the increase or decrease of bank revenues according to the two customer channels—customer services and chatbot. In the case of product purchase through customer service, the proportion of Seniors was higher, while the proportion of Juniors was larger for the chatbot. In conclusion, the age group that occupies a relatively large proportion has a positive effect on bank profits.

The total number of samples in Hypothesis 3a is 43,955, which are Junior group customers purchasing new products. The dependent variable is the net increase in bank revenue. We tested the statistical significance of the difference according to the classification by channel.

In the case of Hypothesis 3a (

Table 10. Two-sample t-test for Hypothesis 3a (H3a).

	Variance	DF	t-Value	p-Value
Pooled	Equal	43,953	1.4352	0.312
Satterthwaite	Unequal	43,864.245	1.4345	0.416
Equality of Variance	Num DF	Den DF	F-value	p-value
Folded F	39,134	4821	8.12	0.357

), the assumption of equal variance is satisfied by the F test (F = 8.12). Therefore, we refer to the pooled t-test, and the test result accepts the hypothesis (t = 1.4352). Hence, when comparing customers who purchase products through customer service and customers who purchase products through chatbot, that there is no difference in the bank net profit (New products–Junior group).

Hypothesis 3b is classified by product–customer, and the total number of samples is 121,466 users: Junior customers receiving existing services. The dependent variable is the net increase in revenue for the bank. We tested the statistical significance of differences in channel classification.

In the case of Hypothesis 3b (

Table 11. Two-sample t-test for Hypothesis 3b (H3b).

	Variance	DF	t-Value	p-Value
Pooled	Equal	121,464	18.2142	0.012
Satterthwaite	Unequal	121,435.328	14.2146	0.011
Equality of Variance	Num DF	Den DF	F-value	p-value
Folded F	104,574	16,892	6.19	0.452

), the assumption of equal variance is satisfied by the F test (F = 6.19). Therefore, we referred to the pooled t-test, and the test result rejected the hypothesis (t = 18.2142). That is, when comparing customers who purchase through customer service and those who purchase through chatbot, bank net profits from the customer groups (Existing service–Junior group) are statistically different. In the case of the Junior group receiving only existing service, the bank profit was higher from the chatbot group than from the customer service group. The junior group’s handling of small amounts of multiple utility bills through the chatbot has a positive effect on bank finances due to the regular transaction costs of customer service. Therefore, inducing the use of chatbots with low operating costs is a positive contribution to the bank, due to the nature of existing services involving a small amount of money but a larger number of transactions.

In Hypothesis 3c, the total number of samples classified by product and customer is 51,313 because they are customers who purchase new products and belong to the Senior group. The dependent variable is the net increase in bank revenue. We tested the statistical significance of differences in channel classification.

For Hypothesis 3c (

Table 12. Two-sample t-test for Hypothesis 3c (H3c).

	Variance	DF	t-Value	p-Value
Pooled	Equal	51,311	21.0113	<0.0001
Satterthwaite	Unequal	51,304.525	34.1223	<0.0001
Equality of Variance	Num DF	Den DF	F-value	p-value
Folded F	49,238	2075	23.73	<0.0001

), the assumption of equal variance was not satisfied by the F test (F = 23.73). Therefore, we refer to the t-test of the Satterthwaite method, and the test result rejects the hypothesis (t = 34.1223). When comparing customers who purchase through customer service and those who purchase through chatbot, bank net profits of these group customers (New product purchase–Senior group) are not statistically equal. This result is due to the large number of Senior group customers who purchase new products such as funds and savings through customer service. Additionally, the amount of fund products is large. This increases the average bank receipts. In addition, since the housing subscription savings product has regularity, it is expected to have a positive role in terms of bank contribution.

In Hypothesis 3d, when categorized by product and customer, the total number of samples is 134,793; they are users receiving existing services and customers belonging to the Senior group by age. The dependent variable is the net increase in revenue for the bank. We tested the statistical significance of differences in channel classification.

In the case of Hypothesis 3d (

Table 13. Two-sample t-test for Hypothesis 3d (H3d).

	Variance	DF	t-Value	p-Value
Pooled	Equal	134,791	−13.1452	0.026
Satterthwaite	Unequal	134,731.583	−12.1025	0.025
Equality of Variance	Num DF	Den DF	F-value	p-value
Folded F	124,492	10,301	12.09	<0.034

), the assumption of equal variance is not satisfied by the F test (F = 12.09). Therefore, we refer to the Satterthwaite method t-test, and the test result rejects the hypothesis (t = −12.1025). Contrary to Hypothesis 3b, in the case of the Senior group, handling existing services with high transaction frequency and small monetary amounts through customer services has high transaction costs and a negative effect on bank revenue.

4.2. Cube Model Interpretation

To plot the results of Hypothesis 3, the combination of two conditions by product and by age was made into a 2 × 2 cube model. The X-axis is divided into the age group of customers, and the Y-axis is divided into product characteristics. In addition, we divided the channels into customer service and chatbot. We plotted the four combinations and analyzed the effect of each combination on bank revenue. The analysis results for each combination are shown in

Table 14. Interpretation of the hypotheses from the cube model.

New Products (Y1)	X1–Y1 (H3a) Not significant	X2–Y1 (H3c) Positive in net profit from Customer Service
Existing Service (Y2)	X1–Y2 (H3b) Negative in net profit from Chatbot	X2–Y2 (H3d) Negative in net profit from Customer Service
	Junior Group (X1)	Senior Group (X2)

.

As for X1–Y1, the hypothesis of the study was adopted, so there is no difference in the effect on the net profit of banking operations between the two channels. In the case of X1–Y2, the analysis result was significant, because multiple small transactions were able to save labor and management costs through automated processing. Additionally, X2–Y1 positively affected contribution based on the behavior of the Senior group purchasing new products with large amounts of money. Finally, in the case of X2–Y2, multiple micro-transactions using a chatbot rather than using customer service positively affect bank finances.

5. Conclusions

This study conducted an empirical analysis to pursue the expansion of the use of AI-enabled chatbot in banking financial products and bank policy changes, based on the ARS data of leading banks. For empirical analysis, we summarized the practical implications through the results of hypotheses setting and testing. First, we empirically analyzed the effect of the AI-based chatbot system and suggested policy alternatives to strengthen the financial soundness of large banks. We evaluated the performance of the chatbot system, newly introduced to the existing ARS system in January 2018. In addition, we presented alternatives on how this system contributed financially to banks and what aspects should be supplemented to optimize customized profits in the future. The findings indicate that reinforcing customer service expertise according to product and age classification increases bank profits. In some chatbot cases, the increase is greater. Second, companies, especially in the financial sector, are furiously building AI platforms. However, applying new technologies to the field, including acceptance and adaptations, requires considerable time and public relations, and may result in internal friction. This can affect short-term profits and may lead to economic opportunity losses. If companies fail to make the right investment at the right time, they may forfeit future opportunities. Therefore, this study categorized whether banks are investing with an eye to profits and analyzed the effectiveness of these investments. This study can be applicable to financial institutions other than banks in the future.

We examined previous studies in four dimensions and in that backdrop, summarize the academic contributions of this study. First, considering the financial chatbot system, we examined AI technologies and effects introduced in various financial environments through prior research. Second, in relation to the ARS system, we summarized the practical problems of customer service counseling staff and the countermeasures and techniques to solve them. Third, we studied the properties of resistance to the introduction of technologies and theories related to alternatives that help reduce the resistance and increase acceptance. Fourth, we investigated prior research on actual indicators representing bank contribution from a methodological perspective. Thus, this study provides a real-world situation through data and meaningful statistical inference.

Despite the various academic significances and practical contributions described above, there are problems and limitations of this study. First, data handled at offline counters that account for most product management were excluded. Banks sell bank-specific savings and loan products, and they offer specialized products such as insurance and bonds. The percentage of products sold through ARS is less than 5% of the bank’s total sales. Of these, sales through chatbot are insignificant, less than 10%; hence, it may be unreasonable to closely associate them with bank profits. However, building a new infrastructure for a chatbot is an important factor, considering the unknown impact for the new era. Therefore, continuous research on the introduction of the AI financial system is necessary. Second, the four products and services presented in this study are all parameters of the data accumulated for two years after the chatbot was introduced. These data were developed through trial and error at the time of initial settlement, and the stability of the sample is poor. In addition, there are many macro-environment variables that should be considered along with the impacts presented in this study. This is expected to be a problem that can be resolved naturally as data are continuously accumulated and the system is stabilized in the future. Nonetheless, it remains practically and academically necessary to continuously correct these problems for research. Third, we also need to design an experiment by separating the cases of failure from the success cases in the chatbot service and additionally analyze the service failure factors [56]. In other words, we need meticulous research to control situations that are unfamiliar to customers through further investigation of chatbot service failures. Fourth, we overlooked dealing with digital governance issues. The main challenge in digital governance is not technical but the people involved in the decision-making process [57]. In other words, it is important to create a governance structure so that people participate in decision making and at the same time do not fall into the trap of knowledge issues. Therefore, we need to provide multiple processes at different levels for a sustainable transition to digital governance. Finally, we omitted the study of distorted trust between social cognition and the cognitive ability of chatbots [58]. In other words, we need to list the significant negative impacts of a number of faulty interfaces that could be considered in the conceptual model of a chatbot and provide reasonable evidence of its impact on users. We expect that through the process of closing this set of limitations, we will be able to more accurately relocate the contributions of our research to the digitization of society through chatbots.