Next Article in Journal
Supporting Informed Public Reactions to Shipping Incidents with Oil Spill Potential: An Innovative Electronic Platform
Previous Article in Journal
Optimal Grid Flexibility Assessment for Integration of Variable Renewable-Based Electricity Generation
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Enhancing Data Management Strategies with a Hybrid Layering Framework in Assessing Data Validation and High Availability Sustainability

by
Paniti Netinant
1,
Nattapat Saengsuwan
1,
Meennapa Rukhiran
2,* and
Sorapak Pukdesree
3,*
1
College of Digital Innovation Technology, Rangsit University, Pathum Thani 12000, Thailand
2
Faculty of Social Technology, Rajamangala University of Technology Tawan-ok, Chanthaburi 20110, Thailand
3
Faculty of Information Technology and Innovation, Bangkok University, Pathum Thani 12120, Thailand
*
Authors to whom correspondence should be addressed.
Sustainability 2023, 15(20), 15034; https://doi.org/10.3390/su152015034
Submission received: 1 October 2023 / Revised: 11 October 2023 / Accepted: 17 October 2023 / Published: 18 October 2023

Abstract

:
Individuals and digital organizations deal with a substantial amount of collected data required for performing various data management strategies, such as replacing, upgrading, and migrating existing data from one system to another, while supporting the data’s complexity, authenticity, quality, and precision. Failures in data migration can result in data and service interruptions, financial losses, and reputational harm. This research aims to identify the specific challenges of a data management strategy, develop a comprehensive framework of data migration practices, and assess the efficacy of data validation and high availability for optimizing complex data and reducing the need to minimize errors during data migration. Combining trickle and zero-downtime migration techniques with a layering approach, a hybrid-layering framework was designed to encompass the entire spectrum of data migration techniques, beginning with system requirements and data transformation, rigorous functions, and evaluation metrics for sustainable data validation. The evaluation metric criteria are defined to evaluate data migration based on data consistency, integrity, quality, accuracy, and recall. The experiment demonstrated a real-world scenario involving a logistics company with 222 tables and 4.65 GB of data. The research compared various data migration strategies. The outcomes of the hybrid-layering framework’s examination of the final system’s functionality are satisfactory, emphasizing the critical importance of data migration sustainability to ensure data validity and high availability. This study is useful for individuals and organizations seeking to sustainably improve their data management strategies to minimize disruptions while preserving data integrity.

1. Introduction

Data are essential elements for the operation of many different computer systems, ranging from smartphones and Internet of Things (IoT) devices to powerful server infrastructures, as well as significant operations for modern business environments, including business analysis, operational decisions [1], driving innovation [2,3], and enabling competitive differentiation. Data, in their volume and variety, affect every aspect of operations, making them not only a strategic asset [4] but also something that must be managed systematically and efficiently. Effective data management is necessary for success-making opportunities and challenges in the ever-evolving landscape of present business and organizational growth. Data management encompasses a vast data volume, diversity, and velocity perspective to support organizations’ data management strategy, data quality standards, governance, security and operations, platform and architecture, and sustainability [5]. In addition, the complex nature of data management concerning the central role of various platforms and architectures focusing on database management systems (DBMS) has resulted in numerous obstacles when an organization spreads out data. There are numerous types and brands of DBMS software available. The two types of DBMS software products and licensing models [6] are commercial and open-source software. Some companies may utilize commercial database management systems like Microsoft SQL Server [7], IBM DB2, or Oracle DBMS. Meanwhile, free, open-source alternatives are provided, including PostgreSQL, MongoDB, MySQL, and MariaDB [8].
While digital organizations have expanded with an enormous amount of information, the present issues for data management are the endless increase in data, the complexity of datasets, and data quality problems. A data management strategy is one of the goals for achieving an organization’s roadmap, including communications, data management functions, business case, and funding [5]. Data migration is one of the data management functions and the most challenging task [9] for data-driven system implementation [10], consolidation, changes, and upgrades [11]. The principle of data migration enables data transfer from one or more storage spaces to the same or different computer system platforms or locations to improve data scalability, accessibility, and portability, as well as reducing the cost of information technology operations in businesses and organizations [12,13,14,15,16]. In recent years, numerous data migrations have been focused on storage [17], application [18], business process [19], DBMS [20,21], and platforms [13,20,21,22]. Many studies have identified significant concerns in data migration strategies, such as data validation [23], integrity checks, efficient migration techniques [24,25], heterogeneous data migration [26,27], and data migration frameworks for sustainable decision making [28]. In addition, existing data migration methodologies and frameworks are typically generic regarding the data pipeline and computation platform integration [29]. Migration frameworks frequently do not include methods for large-scale experimental testing and validating the accuracy and completeness of migrated data. Recently, most businesses and organizations alternatively migrate existing data from proprietary data management systems to open-source ones [30]. Due to the complexity and difficulty of data migration, the business or organization may cease services or fail [31]. Without applying an appropriate data migration technique, a failed data migration can result in unreliable data, service interruptions, financial losses, and brand damage [32].
Many recent studies have utilized data migration techniques, such as big bang data migration [33,34,35], trickle data migration [33,36,37], and zero-downtime data migration [21,38,39]. While trickle data migration allows for manageable increments [36] of utilizing the database to improve real-time stream processing without an automated migration process, the big bang migration method [34] involves migrating all data at once, which requires automated migration, testing, and validation. The zero-downtime strategy is a method that integrates the principles of continuous delivery with the objectives of high availability and data consistency [39]. However, the most challenging obstacles encountered in this approach are scheduled and unplanned data downtime. Many studies highlight only one technique out of big bang, trickle, or zero-downtime data migration, whereas others point out their limitations. This disparity underscores the need for a combination technique of data migration to support a large-scale and a complex data relationship to ensure continuous data availability throughout the entire migration process. Moreover, no comprehensive, sustainable frameworks or methodologies integrate data validation and high availability strategies with data migration, particularly in practical scenarios involving the development of cost-effective high availability solutions that balance data accessibility with resource optimization, specifically for organizations with limited financial resources.
The implementation of layering technology in system and software development yields numerous advantages. The importance of testability and error-proofing is highlighted in the concept of separate concerns in data design and development [3,40]. The ability to filter out unnecessary information and easily adapt functions are also emphasized [41,42,43].
Therefore, this study aims to address these research gaps by combining the data migration approaches of trickle and zero-downtime with a layering technique named the hybrid-layering approach. The entire data migration process is divided into layers consisting of procedures from an analysis phase to a system reconciliation phase. This study aims to identify the specific challenges of a data management strategy, to develop a comprehensive framework of data migration practices, and to evaluate the effectiveness of data validation and high availability for optimizing complex data and reducing the need to minimize errors during data migration. In order to accomplish the research objectives, the following research questions have been formulated:
-
What are the challenges of developing a comprehensive framework that integrates data validation and high availability techniques within a data migration practice, ensuring data integrity and continuous accessibility?
-
How can a comprehensive framework be evaluated for the sustainability of data validation techniques that can detect and address data quality issues, ensuring data accuracy throughout the data migration?
-
How can the effectiveness of sustainable criteria metrics and methodologies be defined and established for evaluating the efficiency and effectiveness of different data migration techniques, thereby providing decision-makers with clear guidelines for evaluating migration options?
This study makes research-significant contributions to data management strategies in data migration by providing a comprehensive hybrid-layering framework and by presenting practical case studies concentrating on the sustainability concerns of offering validation measurement together with high availability, thus supporting technological adaptation and reducing data and service disruptions.
  • The framework combines hybrid data migration approaches, layered approaches, data validation, and high availability strategies. The collaborative approach to stakeholder collaboration and communication within the framework of data mapping and transitions is essential for successful data migration and promotes effective communication between team members and stakeholders. Implementing the recommended data migration techniques ensures business continuity during migration, reduces data and service corruption risks, and improves productivity, efficiency, and availability.
  • Further, the study defines recommendations for evaluating and validating the accuracy and completeness of migrated data. This guidance is crucial for organizations to ensure data reliability and correctness during migration.
  • This research offers practical insights into the effectiveness of the data migration framework by providing a real-world case study of converting proprietary databases to open-source ones. This empirical demonstration lends credibility to the framework for hybrid layering.
  • The study assists organizations in adapting to new database system technologies when streamlining technological change and business expansion, emphasizing the significance of understanding and implementing data migration techniques to safeguard data migration strategies and sustain technological agility.
The subsequent sections of this article are structured in the following manner: Section 2 provides a theoretical background pertaining to data migration techniques and a hybrid framework. Section 3 of this study presents a comprehensive analysis of the research model and methodology that underpin the design, development, and evaluation of the proposed framework for data migration. Additionally, the study provides an overview of the outcomes obtained from a real-world case study that employed the hybrid-layering framework, and these outcomes are subsequently summarized. The discussion of the results is presented in Section 4, whereas Section 5 summarizes the migration framework and the findings derived from this study.

2. Theoretical Background

2.1. Comparison of Existing Data Migration Techniques and Hybrid Framework

Successful data migration is essential for organizations seeking to adapt to changing technological landscapes, to optimize system performance, and to guarantee data accessibility and scalability. With various data migration techniques available, such as big bang data migration, trickle data migration, and zero-downtime data migration, each approach is essential to carefully evaluate the merits and suitability of each method for the needs of a particular project. Table 1 presents a comprehensive methodology encompassing these three migration techniques, providing a holistic and well-informed approach for guiding organizations through the data migration process, as well as our proposed combination approach.
The methodology aspects of migration approaches consist of research and analysis, technique selection, planning and preparation, data profiling and preprocessing, testing and validation, migration execution, post-migration evaluation, documentation, and knowledge transfer. By examining the complexities of each technique and integrating them into a cohesive methodology, organizations can make informed decisions, minimize risks, and achieve successful and seamless data migrations, thereby enhancing data management practices and bolstering their competitive advantage in a dynamic digital landscape [44,45].
The combination of trickle and zero-downtime data migration techniques is presented in Table 1 as a comparison of data migration techniques in the preprocessing phase. This strategy aims to capitalize on the advantages of both methodologies, providing a balanced solution for complex data migration projects. Organizations can sustainably achieve seamless data integration and maintain system functionality throughout the data migration process through trickle migration and zero-downtime techniques. The methodology emphasizes meticulous planning, testing, and documentation to ensure data integrity, accuracy, and optimal system performance, empowering organizations to manage their data migration projects effectively and efficiently with minimal disruption and improved user experience.
Table 2 compares and contrasts the various post-processing data migration strategies. Following the migration procedure, the post-processing phase is essential for ensuring data accuracy, consistency, and operational efficiency. This table compares big bang migration, trickle migration, zero-downtime migration, and the proposed hybrid layering data migration framework. These techniques are analyzed and compared in depth, considering crucial aspects such as testing and validation, migration execution, post-migration evaluation, documentation, and knowledge transfer. This study aims to conduct a comprehensive analysis of these methodologies so that organizations can make well-informed choices when selecting the most appropriate approach for their data migration requirements. This comparative analysis is useful for IT professionals and decision-makers engaged in data migration projects.

2.2. Novel Criteria Metrics of Effectiveness Evaluation in Data Migration Sustainability

The criteria depicted in Table 3 are necessary for evaluating the sustainability and efficacy of data migration activities within data management.
Maintaining the alignment and efficiency of primary components, rapidly changing technologies, and knowledge of computer systems, including hardware, software, peopleware, and data, requires data migration. The computer system includes leveraging hardware advancements, adopting the most recent software capabilities, enhancing people’s skills, fostering collaboration, consolidating data, and ensuring data integrity while transforming data. With data migration skill enhancement, peopleware requires collaboration between multiple teams, including data administrators, developers, IT operations, and business stakeholders. A challenging aspect of data migration is the heterogeneity of suitable methods for data compatibility, accuracy, reliability, transparency, transition preparation, and persistent operations [19,53]. Multiple considerations must be optimized for data migration concerns. Therefore, a criteria matrix for an effectiveness evaluation is proposed to support the final data migration process prior to an organization’s implementation of a new open-source data management system. Each criterion plays a vital role in evaluating the efficacy, dependability, and long-term viability of data migration projects, ensuring that data are transferred without interruption while retaining their quality and integrity throughout the migration technique.

3. Research Model and Methodology

This section describes the research materials and methodology used to explain the data migration methods, as depicted in Figure 1. This study focused on the data transformation technique required to develop a hybrid layering data migration framework emphasizing data validity and high availability sustainability, as well as studying data migration techniques, as presented in Table 1 and Table 2. In addition, reviewing the sustainability of data management for effective and efficient data migration, designing sustainable assessment criteria for data migration, as in Table 3, evaluating and justifying the practical case using the data migration, and summarizing the research methodology results were presented. The proposed methodology utilized a layering approach and a hybrid migration of trickle and zero-downtime data migration strategies. An industrial practical success case demonstrates evaluation accuracy by applying the hybrid layering framework based on designed assessment criteria, yielding valid and high availability results for an efficient data migration method. The authors propose a novel hybrid layering framework and sustainable migration criterion metrics to address the research challenges associated with advancing the understanding of the evaluation in data migrations for sustainability of validity and high availability.
From Figure 1, several steps of this study’s methodology transfer the existing commercial data management system to the target open-source data management system. The following steps are included in the methodology for data migration transformation processes: analyzing the current source data management system, including data models, data relationships, data constraints, database structures, and functionalities. The source data included database names, table names, field names, primary keys, candidate keys for each table, constraints, and data storage locations. Significant characteristic differences of source and target data management systems, including the database model, data definitions, data constraints, data manipulating languages, database system commands, license, and cost, affected and were designed to analyze strategic concerns.

3.1. Methodological Development of Hybrid-Layering Migration Framework

As illustrated in Figure 2, system information is gathered in the requirement planning phase of a system transformation, and a data migration strategy is developed. This phase entails gaining an understanding of the current system, which is based on the source data management system, and developing a strategy for transforming data from the source data management system to the target system environment by communicating with and involving stakeholders, such as database and system administrators, users, and the chief information officer (CIO). The team examines the data models, schema, tables, attributes, data constraints, data records, and application architecture to identify potential mapping incompatibilities between the two systems. This step is essential for ensuring an effortless migration, laying the groundwork for subsequent processes in later phases, identifying the requirements of the new database system, and determining the source server database’s compatibility with the target server. The procedure entails the creation of a migration plan outlining the migration timeline, the sequence of the migration of datasets, and the roles and responsibilities of team members and stakeholders involved in the data migration. A back-up and recovery plan should also be included in the migration plan to ensure the safety of data management system services with zero downtime during the migration process.
This execution stage is subdivided into several phases with distinct activities and responsibilities. This stage consists of data migration verification and functional/performance accuracy testing phases that involve the practical migration of data. In the data migration verification phase, the focus shifts to the actual data migration, including compliance and security considerations, back-up and recovery preparations, incremental migration datasets, and data validation and verification processes with stakeholder approval. The team executes the data extraction, transformation, and loading procedures, transferring the data from one system to another table by table. Following the completion of the migration, the dataset will be transferred. Data consistency and integrity checks ensure that all crucial data have been successfully migrated. The data migration technique concludes with the data verification stored in data storage devices, where the migrated data is validated and verified for precision and completeness. In addition to the termination data on the source data management system, the migrated datasets on the target data management system are accessible for servicing application connections. In the functional/performance accuracy testing phase, the team conducts rigorous verification tests to validate the accuracy and completeness of the transferred data in conjunction with the stakeholders’ verification tests.
The framework methodology emphasizes careful planning, validation, and testing, contributing to the data migration’s overall precision and success. By focusing on data extraction, transformation, and loading processes, the framework ensures the accuracy and completeness of migrated data while minimizing the risk of errors and inconsistencies, as concluded in Table 4.
This study presents a comprehensive and systematic framework for efficient data migration from source and target data management systems to facilitate successful and accurate data migrations. Five practical processes define the framework: system transformation, data migration verification, functional/performance accuracy tests, system tuning, and system delivery. Each procedure is meticulously crafted to address specific obstacles and ensure a seamless data migration experience for organizations undergoing technological transformations. The framework allowed the company to transfer critical data tables gradually without interruptions or downtime.
Table 4. Efficient practice processes to increase the probability of a successful data migration.
Table 4. Efficient practice processes to increase the probability of a successful data migration.
Framework PhaseEfficient Practice ActivityDescription
System TransformationThorough PlanningDevelop a comprehensive migration plan, including analysis, objectives, timelines, and milestones.
Data Assessment and MappingAssess the source data, map it to the target data, and ensure compatibility and integrity.
Expertise and CollaborationEngage experienced professionals and collaborate closely with the migration team for comprehensive understanding.
Risk Management and Contingency PlanningIdentify potential risks and develop contingency plans to mitigate them during the migration process.
System Transformation
Data Migration Verification
Functional/Performance
Accuracy Tests
Compliance and Security ConsiderationsEnsure compliance with relevant data regulations and implement appropriate security measures in the new data.
System Transformation
Functional/Performance
Accuracy Tests
Delivering System
Communication and Stakeholder EngagementMaintain open communication with stakeholders and involve them throughout the migration process.
Data Migration Verification
Functional/Performance
Accuracy Tests
Testing and ValidationTo ensure accuracy, completeness, and integrity, perform extensive testing and validation of the migrated data.
System Transformation Data Migration Verification
Functional/Performance
Accuracy Tests
Back-up and RecoveryImplement robust back-up and recovery mechanisms to safeguard the data during the migration process.
Data Migration Verification
Functional/Performance
Accuracy Tests
Incremental MigrationConsider migrating data into smaller batches or modules to facilitate monitoring, validation, and troubleshooting.
Data Validation and VerificationValidate the migrated data’s accuracy, completeness, and integrity through various verification techniques.
Tuning SystemMonitoring and Performance OptimizationContinuously monitor the migration process and optimize the performance of the target data.
Delivering SystemDocumentation and Knowledge TransferMaintain thorough documentation to aid future reference and knowledge transfer to other team members.
Post-Migration Support and MaintenanceProvide support and maintenance after migration to address any post-migration issues or performance optimization.
Training and Knowledge Transfer for End-UsersConduct training sessions and knowledge transfer activities to ensure end-users are familiar with the new data.
Regular Performance Monitoring and Maintenance ReviewsRegularly monitor the performance of the target data and conduct maintenance reviews to optimize functionality.

3.2. Evaluation Criteria Metrics and Formula of Data Validation Sustainability

The hybrid layering framework is essential for assessing the accuracy of data migration to ensure the successful, efficient, and dependable transformation of data from the source data management system to the target ones. In this study, several novel key formulas were developed and utilized to evaluate the hybrid layering migration framework accuracy quantitatively. Data consistency rate (DCR), data integrity score (DIS), data accuracy index (DAI), error rate (ER), data completeness percentage (DCP), data matching ratio (DMR), data consistency index (DCI), and data accuracy percentage (DAP) are all proposed factors. In addition, precision and recall factors contribute significantly to evaluating the framework’s accuracy [54]. According to Table 5, the authors noted that comprehensive insights into the precision and dependability of results can be obtained.
Precision and recall are widely employed performance metrics that provide valuable insights into the quality of the migration activity, introducing a crucial aspect of evaluating the accuracy and dependability of the system’s results [55,56]. A framework for data migration can ensure data consistency, identify potential issues, and make informed decisions by measuring precision and recall results to enhance the migration process and preserve data integrity. Positive matches are data records that have been successfully migrated and match the source and target data. Negative matches, on the other hand, are records that were improperly migrated and did not match between the two data. Establishing clear criteria for positive and negative matches is essential for accurate assessment. Positive matches and negative matches should be included in the validation dataset. The evaluation result can identify true positive matches (correctly migrated matching records), false positive matches (incorrectly migrated matching records), true negative matches (correctly migrated non-matching records), and false negative matches (incorrectly migrated non-matching records) by comparing the data records between the source and target data. As a result, precision is calculated as the ratio of true positive matches to the total number of positive matches predicted by the migration process, thereby reflecting the accuracy of positive predictions made during migration.
The hybrid layering data migration framework (Figure 2) was developed iteratively, with regular feedback and input from stakeholders, to ensure that the framework is effectively tailored to the specific needs and requirements of the data migration project. However, this study’s conditional requirement was to migrate the entire database system. By mapping and schema design between source and target database systems, the target schema was designed using entity relationship diagrams (ERD) for the samples of entity relationships, tables, attributes, data type, keys, and constraints, as shown in Figure 3 and Figure 4. In addition, the preparation process included data normalization, data quality checks, and data transformation to ensure compatibility and integrity in the target system. In Figure 5, the final programming implementation of the data migration sustainability for data validity and high availability was to develop and validate the successful transfer of data from the source data to the target data management system, which includes data extraction, transformation, loading, and reconciliation.
On the other hand, the recall ratio is the proportion of true positive matches to the total number of actual positive matches in the validation dataset and measures the migration process’s ability to identify positive matches accurately. Precision and recall results are indispensable to gaining insight into the framework’s performance for the migration process. A higher precision indicates fewer false positives, indicating that the migrated data are less likely to match non-matching records and vice versa incorrectly; a high recall indicates that the migration process accurately identifies a more significant proportion of the actual positive matches.

3.3. A Real-World Case Study to Evaluate the Hybrid Layering Framework

A real-world case study of data migration begins with a company whose enterprise resource planning (ERP) systems have used a commercial database server to manage sales, products, ordering customers, purchasing orders, suppliers, and logistics data for several years. Due to changing business requirements, the company has decided to migrate the entire database to the target database management system. The goal of the migration project is to transfer the data without loss or corruption of all vital data tables, including those for supplier, product, product details, customer, purchasing transaction, sale transaction, logistic transaction, and product status. Data structure, data integrity, minimization of downtime, and data volume are just a few of the significant issues that arise from complex data migration processes. The source and target database systems have distinct data structures, including different data types, syntax, and query syntax. Over the years, the system’s provider has amassed a substantial amount of data. The entire dataset consisted of 223 tables and contained 4.65 GB of data. The processes must ensure that all data can be transferred to minimize downtime and disruptions, as prolonged system downtime could negatively affect customer and supplier satisfaction. In addition, it is essential to maintain data integrity and precision during the migration process to prevent inconsistencies and errors. Figure 3 and Figure 4 depict the entity–relationship diagram (ERD) for the sample data scenario of the company. The example ERD consists of data for the purchasing order (PO) information management system, which includes the supplier, product, product details, customer, sale transaction, product status, and supplier–product business. Each table has its unique primary, foreign, and candidate keys to ensure data integrity and uniqueness.
In data migration, it is essential to consider every aspect of migration results involving a change within the same DBMS model, a move between different DBMS models, or a move between different DBMS models while maintaining the same entity–relationship (ER) model. Each of these situations requires a unique set of considerations and outcomes. This is similar to DBMS model modifications when the current data management system is stable and meets the organization’s operational needs well. An intelligent option for migration within the same DBMS model is to either upgrade the version or optimize the data. This method typically reveals opportunities to enhance the current data structure and overall system performance. Typically, adhering to the same DBMS model reduces the number of problems associated with how tasks are carried out.
When modifying models of data management systems, it is rational to switch the DBMS model when the target model offers advanced functionality, scalability, economics, or features that work well with the organization’s changing operational needs. Data mapping and transformation issues are frequently raised between various DBMS models. Different schema structures, query languages, and data storage methods must be planned and implemented with precision. The most important step is evaluating the new DBMS’s support infrastructure, licensing terms, and overall compatibility, as switching to a different DBMS model requires a substantial investment of time and resources. This study demonstrates that changing DBMS models while maintaining the same ER model is preferable to maintaining a uniform data structure. However, the process is not simple, as mapping the ER model to different DBMS implementations can be difficult. Despite the consistency of the ER model, data types, constraints, and query languages vary across DBMS models. This study illustrates the importance of meticulous planning in complex data transformation projects. Developing solid data transfer plans and conducting a comprehensive analysis of how well the new DBMS models integrate with the existing applications and tools should always be the primary focus in these circumstances. Preparations must be made to adapt or improve applications so that the transition has no negative effect on operations and is seamless.

3.4. A Practice Implementation for the Sustainability of the Hybrid Layering Framework

As shown in Figure 5, the implementing diagram flow represents the particular concerns of sustainable data migration processes. This study involves numerous users, including developers, administrators, end-users, and stakeholders, who must be effectively communicated with by migration teams. The Python package should be installed to support pyodbc, psycopg2, csv, and time to implement an efficient and enduring data migration using Python. The pyodbc library connects to the source database management system, and the csv library is used to export the source dataset in CSV format. This practical case study connects to the target database management system using the psycopg2 library. The implementation modules begin with the CSV export of the source dataset. To execute the migrated processes, numerous functions are defined, including the definition of target table function, the definition of the migrate data function, the definition of the migrate sequence function, the definition of the check consistency function, the definition of the check integrity function, the definition of the check constraint function, and the definition of the perform migration function. The dataset will be rolled back if a single data migration function returns an error. The migrate_sequences process migrates sequences (used for serial primary keys in the target dataset) from the source database management system to the target database management system. This procedure ensures that the equivalent sequences in the target dataset are assigned the correct value based on the current identity values in the source tables and dataset. The check_consistency process is designed to check the consistency of data between the source and target datasets during data migration by comparing the data retrieved from both the source and target datasets, row by row, and determining whether the data matches in each row. To check the integrity of the data migration from source to target database management system, the integrity process must verify that the schema elements (such as tables, columns, constraints, and indexes) are consistent between the two databases, followed by a check to ensure that the data types and constraints were replicated correctly. Each table’s constraints between the source and target database management systems will be verified. If there is any inconsistency, an error message will be generated to indicate the discrepancy. If there are no errors, the result will indicate successful migration. Following the completion of data migration, the perform migration procedure verifies all processes. The script will indicate which checks failed if any do not pass. If all validations pass, the script will output a success message. The differences between Figure 1, Figure 2 and Figure 5 are, respectively, the overall research methodology to develop the data migration, the strategies and processes of the layering framework for efficient data migration, and the information system architectures of practical implementation.
The functional/performance testing accuracy phase involves validating and verifying the migrated data to ensure that transferred data are accurate, application-compatible, and complete. The testing procedure evaluates the database’s functionality and performance. The team conducts a series of tests to confirm that the database functions as expected in the target environment, with the approval of the primary stakeholders. Functional tests evaluate the behavior of the new system’s applications and queries to ensure that all functionality is intact. Simultaneously, performance tests are conducted to evaluate the database’s response time and overall efficiency, thereby identifying potential bottlenecks that could affect performance. This phase is divided into two processes, each with its own activities and responsibilities. The data validation process involves comparing the migrated datasets on the target database system to the original source server database to ensure no data loss or corruption issues. The application validation process ensures that migrated datasets function correctly within applications.
The tuning system phase focuses on optimizing the database’s performance and efficiency. The data migration team identifies areas requiring fine-tuning to improve system responsiveness and resource utilization based on the results of performance tests. Index optimization, query optimization, and configuration adjustments are performed in the target database system environment to optimize the database’s performance. This phase prepares the system for anticipated workloads and user demands. To ensure the success of the data migration from commercial to open-source database management systems, the data migration must adhere to the framework outlined above. This framework includes recommended planning, execution, and validation procedures for migration processes. In addition, data migration success requires a skilled and experienced team to manage the migration process. This team must include stakeholders such as database administrators, developers, and project managers.
The delivery system process is the last phase of the data migration framework. The fully migrated and optimized data are deployed to the production environment during this phase. The team coordinates the transition from the previous source server system to the new target system, ensuring minimal downtime and an effortless change. At this stage, user acceptance tests of the overall functionalities of system services may be conducted to ensure that all business processes and functionalities in the new data environment are operating as expected.
In conclusion, the framework for efficient data migration is a complex procedure requiring meticulous planning, execution, and validation. By adhering to the framework mentioned above and employing a skilled and experienced team, organizations can migrate their data to a new system without experiencing data loss or corruption.

3.5. Data Collection

The demographic data collection of the actual data migration, as presented in Table 6, contains eight departments, including public relations (PR), purchasing order (PO), sale order (SO), human resources (HR), goods receipt (GR), delivery notes (DN), job order (JO), and hand over (HO). The minimum number of tables is 16 (human resources), and the maximum number of tables is 45 (purchasing order). Human resources (HR) are the category of the most minor concern, with 30 tables and 49,200 records accounting for 13.36 percent of the total data, making it the largest category in terms of both tables and records. Purchasing order (PO) is the least problematic category, with 13 tables and 17,810 records comprising 5.93% of the total data, making it the category with the most tables and records. These descriptive statistics can be used to effectively plan and execute the data migration process, ensuring data integrity and accuracy throughout the migration.

3.6. Data Analysis and Results

To determine the success rate of a tested data migration, developer teams must verify the data model and collect the relevant data migration results based on a practical, specific scenario. Each iteration of data migration would involve analyzing the number of successful detail evaluations for the migration. Every iteration of data migration would compute the number of successful and unsuccessful migrations, as well as any other relevant metrics that define success or failure in the tested data of the model. After collecting the necessary data, the following formula can be used to calculate the success rate:
(Number of Successful Migrations/Total Number of Migrations) × 100 = Success Rate
A table of migration results is used to monitor the success rate of individual data entities during a migration. The success rate can be determined based on whether each entity successfully migrated or encountered problems.
In our comprehensive analysis of different migration solutions presented in Table 6, this study previously assessed three methodologies: SQL server integration services (SSIS), replication, and incremental migration—manual data pumping.
The purported compatibility of these solutions with a range of source and target databases, such as MS SQL, Oracle, MySQL, and PostgreSQL, was asserted. It is imperative to acknowledge that the verification and validation stages of the migration process presented substantial challenges for each of the three approaches, ultimately resulting in their inability to succeed. These approaches demonstrate the significance of successfully implementing rigorous verification and validation procedures to migrate complex data management systems. The importance of careful planning and implementation is highlighted by our research, particularly concerning migration techniques. The previously tested migration approach highlights the significance of maintaining data integrity and validating.
Table 7 thoroughly examines data migration outcomes in different departments using three distinct methodologies: trickle migration, zero-downtime migration, and the proposed hybrid layering framework. The trickle migration method ensures that the same number of tables and records are migrated from the source data system for all departments. In contrast, the zero-downtime migration method fails to migrate the source data entirely. Despite that, both systems experience a significant number of migration failures, resulting in lengthy periods of inactivity, with an average of four failures per organizational unit. Zero-downtime can negatively affect the continuity of operations and the ability to access data. In contrast, the proposed hybrid layering framework emerges as a more sophisticated methodology, demonstrating operational continuity across all organizational units. This methodology ensures continuous access to data throughout the migration process, reducing the likelihood of operational disruptions. In addition, it guarantees the retention of the same number of tables and records as observed in the original system, indicating accurate data migration. The analysis demonstrates that the proposed hybrid layering framework effectively achieves data validity and high availability without causing operational disruptions, thereby establishing it as the preferred option for organizations executing complex data transfers.
Table 8 shows the evaluation metrics of the migration results confirming the successful implementation of the hybrid layering data migration technique for various departments utilizing the proposed framework. The high success rates (SR) and outstanding performance in metrics described in Table 5, such as compatibility, transformation, integrity, and data completeness, demonstrate the efficacy of the migration framework strategy. The table displays the percentage values for each metric, indicating to what extent the migration process achieved the desired results. The findings of the data table contribute valuable insights into the field of data migration, providing organizations with crucial information for planning and carrying out successful migration projects.
A complex interaction of operational, financial, and strategic factors determines the number of migration rounds. A successful migration requires meticulous planning, execution, and adaptation to minimize disruptions and ensure the seamless adoption of new systems and technologies within an organization. The results demonstrated that the most excellent effectiveness is typically observed when different departments undergo between two and five rounds of migration. The disparity in the number of migration rounds can be attributed to various factors and organizational considerations, with the complexity of the migration process playing a crucial role. Departments with intricate systems or complex data structures may require more rounds to facilitate a seamless transition. Moreover, the size and scope of a department’s operations are essential considerations. Larger departments or those responsible for mission-critical functions frequently require additional migration rounds to ensure the successful migration of all operational aspects without causing service interruptions. The availability of resources, including personnel, budget allocation, and technological infrastructure, substantially affects the rate and number of migration rounds. Departments with abundant resources can execute migrations more efficiently and thoroughly. Integrating each migration round, testing, and validation procedure contributes to round number variation. Comprehensive testing, essential for ensuring the accuracy and reliability of data migration, may extend the overall process, leading to additional rounds. Risk mitigation also plays a significant role in the determination of migration rounds. Especially if their operations are mission-critical, prudent departments may opt for a phased approach, distributing migration efforts across multiple rounds to minimize risks. Technical dependencies that span interconnections between systems or data within an organization exert influence. For seamless integration, complex technical interdependencies may necessitate a phased migration strategy.
Priorities within an organization are a crucial factor. Departments may have varying goals and timelines. Some organizations prioritize rapid migration to adopt new technologies, whereas others have alternative strategic objectives. The changing technological landscape necessitates migration adjustments. As technology evolves, departments may need to realign their systems and data to take advantage of emerging capabilities, which can necessitate multiple rounds of migration.
The results of each department’s data migration are depicted in Figure 6 from the viewpoints of incremental data migration rounds and system compatibility for migrated data consistency and integrity percentage rates. Figure 7 illustrates each department’s data migration outcomes regarding system compatibility, migrated data accuracy and error rates, and transformation-based data completeness rates. Figure 8 represents each department’s data migration outcomes in terms of transformation percentages for data matching ratio, integrity-based data consistency, and performance-based data accuracy.
The data migration of the public relations (PR) department was conducted in three rounds, and all three rounds were 100 percent successful. The incremental migration strategy was utilized, ensuring data consistency and integrity (DCR: 100%, DIS: 100%) and successfully migrating data to the target data (DAI: 100%). The performance of the migrated system was also satisfactory (ER: 0%), and the data completeness percentage indicated that all data were migrated accurately (DCP: 100%). Similarly, the purchasing order (PO) department underwent two migration rounds that were both 100 percent successful. The migration process maintained compatibility (DCR: 100%) and data integrity (DIS: 100%) while ensuring precise data transformation (DAI: 100%). Both the performance of the migrated system (ER: 0%) and the data integrity (DCP: 100%) were flawless.
The sales order (SO) department underwent four migration rounds, with each migration round achieving 100 percent success. Throughout the migration, compatibility (DCR: 100%) and data transformation (DAI: 100%) were maintained as a result of the incremental migration process. Data integrity was also maintained (DIS: 100%), and the performance of the migrated system revealed no errors or discrepancies (ER: 0%).
The human resources (HR) department underwent five migration rounds, each 100 percent successful. The migration process maintained compatibility (DCR: 100%) and data integrity (DIS: 100%) while ensuring precise data transformation (DAI: 100%). The flawless performance of the migrated system (ER: 0%) indicates that no errors were present in the migrated data.
The goods receipt (GR) department’s data migration was completed in five rounds, with each round achieving a 100 percent success rate. The migration procedure successfully maintained compatibility (DCR: 100%) and data integrity (DIS: 100%) while transforming the data precisely (DAI: 100%). The performance of the migrated system (ER: 0%) and data completeness (DCP: 100%) provided additional evidence of the migration’s success.
The delivery notes (DN), job order (JO), hand over (HO), warehouse stock (WS), and other departments all had identically high success rates of 100%. All of these departments successfully completed their rounds of migration, maintaining compatibility, data integrity, and accurate transformation while demonstrating outstanding performance and data completion.

4. Discussion

To improve the effectiveness of data management on data migration utilizing the hybrid layering framework, the impact of data migration on sustainable, efficient business, technology, analysis, data migration techniques, and accuracy evaluation will be detailed in the discussion section.

4.1. Data Migration’s Impact on Efficient Business and Technology

The data migration from commercial to open-source data management systems significantly impacted the organization’s business and technological characteristics. The target data system provides enhanced performance, scalability, and durability, resulting in improved data management processing and storage capabilities, supported by [46,56]. With the target data management system’s support for multiple platforms, such as Windows, Linux, and macOS, the organization gains flexibility and reduced reliance on a particular Microsoft operating system. The target data management system also adapts to shifting technology landscapes for improved data manipulation needs. According to several studies [29,56,57], migration improved data accessibility and scalability, enabling faster information retrieval for decision-making processes. The migration also improved data integration and analysis, enabling more informed business strategies to be implemented using various open-source applications. Nonetheless, the migration process encountered some initial difficulties, resulting in a brief period of downtime. Despite the minor disruptions, the overall effect on business operations was positive, as the advantages of the migrated system outweighed the temporary inconvenience.
Data migration to an open-source data management system can substantially benefit an organization’s business operations. The target data management system enabled organizations to utilize open-source technology. The data management system’s robust and efficient data retrieval capabilities enabled faster access to vital information, allowing employees to make timely, well-informed decisions. This increased data accessibility enhanced the organization’s productivity and adaptability. Additionally, the migration improved data integration and analysis. The migration process afforded the chance to reevaluate and optimize data structures, resulting in more organized data and more insightful data model analysis. In addition, the data migration project resulted in increasingly tailored data security and compliance checks. The target data management system’s advanced security features and encryption support assisted in bolstering data protection and ensuring compliance with industry regulations. This enhanced data security instilled greater confidence among customers and stakeholders, fostering stronger relationships and strengthening the organization’s reputation. Due to the fact that the target data management system is an open-source data management system, the company gained access to a larger, more vibrant, and active community of developers and contributors. The community-driven ecosystem provided continuous support, frequent updates, and abundant resources, ensuring long-term stability and growth while reducing the organization’s technology expenditures.

4.2. Hybrid Layering Data Migration’s Efficient Techniques

Depending on data migration techniques, the impact of data migration on a business’s overall operations and objectives will vary. The migration objectives have achieved the intended business goals, such as increased efficiency, cost savings, enhanced data analysis capabilities, and improved decision-making processes. The business impact must be assessed via both quantitative and qualitative measures. After migration, the new data management system is closely monitored to identify and address any potential issues or obstacles. Regular and timely analysis of the migrated data management system provides users with a reliable data servicing system and addresses any post-migration concerns expeditiously. The effectiveness of the support evaluates and resolves the problem-solving procedure to ensure a smooth transition to the new data management system. Document the entire data migration process, including any obstacles encountered, lessons learned, and effective practices implemented, to prevent future problems from arising and to facilitate their prompt resolution. These techniques are a valuable resource for future migrations and the ongoing enhancement of migration procedures.
Several recommendations can be made to improve the accuracy and efficiency of practical data migrations based on evaluating and comparing the migration techniques supported by [20,51,58,59,60]. Organizations can adopt a hybrid layering strategy by combining the advantages of zero-downtime migration for continuous access and trickle migration for gradually expedited migration completion. During the planning phase, a more thorough evaluation of the system’s requirements and data structure model should be conducted to identify potential obstacles and ensure greater preparedness. The framework could also benefit from incorporating automated testing and validation processes to improve data accuracy and reduce migration-related manual errors automatically. Moreover, collaboration and communication between members of the migration team and stakeholders are essential for resolving any issues that may arise during the migration process. Regular monitoring and reporting should be a top priority for users and database administrators to detect and resolve discrepancies expeditiously.
The data migration project was also met with obstacles. The migration process necessitated meticulous planning and execution to avoid potential business disruptions, as stated in [49,61]. Early on, the scope and purpose of the data migration should be defined. The source data’s model, tables, attributes, constraints, and functionalities are essentially defined for the target data. This procedure facilitates the identification of the precise set of tables required for the migration. In the source data, the relationships between tables are examined to identify the primary key, foreign key, and constraints and to comprehend how data are linked across tables. The analysis process aids in determining which tables that are interconnected and vital to the core business processes should be moved together to preserve data integrity. Prioritize necessary tables to ensure that data tables and essential functionalities are present in the target data. Each migrated table must be analyzed for the same volume of source data. Large tables with numerous records should necessitate additional migration planning and optimization. To ensure a smooth migration, a migration process should prioritize the transfer of smaller tables and implement an incremental migration with zero-downtime. This technique decreases the possibility of errors and downtime during the migration process. Involving key stakeholders, application developers, data analysts, and end-users in selecting data tables is essential for providing input and understanding the valuable data necessary for making informed decisions. A data migration test of the selected group of tables requires a simulation to ensure data consistency and accuracy prior to the actual migration. This validation will assist in identifying potential issues and ensure the data are transfer-ready. A comprehensive back-up and recovery strategy protects data during the migration process. A back-up ensures that data can be restored to their original state in case of unforeseen issues. The successful implementation of data reconciliation techniques led to identifying and resolving data inconsistencies among expert team members that would have otherwise gone unnoticed. This proactive approach to data validation enhanced the data migration’s dependability and quality.

4.3. Successful Data Migration of Data Validity and High Availability Sustainability

This study demonstrates the most critical concerns for data migration using the proposed framework to ensure a high level of practical evaluation during the migration process. The data migration’s effects on data compatibility [19,46], transformation [47], integrity [48], performance optimization [19,21,49], licensing and cost [21,49], accuracy [21,50,51], dependability [48,49], transparency and transformation processing [15,51,52], and persistent operations [15,51,52,62] supported one of the primary benefits. In addition, novel and effective evaluation metrics were utilized to evaluate the migration’s success regarding data integrity, consistency, and accuracy. The accuracy of the migrated data was quantified using metrics such as data consistency rate (DCR), data integrity score (DIS), data accuracy index (DAI), error rate (ER), data completeness percentage (DCP), data matching ratio (DMR), data consistency index (DCI), and data accuracy percentage (DAP). The accuracy evaluation revealed that the migration process maintained data consistency and integrity with commendable results. The DCR and DCI scores indicated few discrepancies between the source and target data, indicating a high data consistency. The DAI and DMR metrics demonstrated accurate data transformation, guaranteeing that the data in the migrated system accurately reflected the original Microsoft SQL server database. The low ER and DAP values also indicated a low error rate, indicating a successful migration with a high level of precision.
The accuracy evaluation ensured that the migrated data in the open-source database accurately reflected the desired state with minimal deviations. Data accuracy and consistency are crucial for any organization to assess direct impacts on decision making, data analysis, and overall business operations [54,57]. By conducting a rigorous evaluation of accuracy, the organization could confirm the integrity and consistency of the migrated data, thereby instilling confidence in the new data management system’s dependability. In addition, the accuracy evaluation revealed potential issues or inconsistencies during the migration process. Identifying and resolving migrated issues prevented costly data errors and potential business process disruptions. The significance of the accuracy evaluation extended beyond the migration project, laying the groundwork for future data-driven initiatives and dependable data management. Innovative testing methodologies and techniques were incorporated into the accuracy evaluation process to ensure comprehensive data validation.

4.4. The Hybrid Layering Framework Applying in the Case of Non-Relational Data

The proposed framework for data migration is adaptable and can be applied to non-relational data for numerous persuasive reasons. Understanding the system’s data models, schema, tables, attributes, data constraints, and records is the initial step of the framework. The analysis easily extends to non-relational data, such as NoSQL, with unique data structures. Analysis comprehension is essential for migration success and emphasizes the development of a transformation strategy to facilitate the transfer of data from the source to the target data management system. Document-oriented, key-value, column-family, and graph data are examples of non-relational data, whereas relational databases employ structured data paradigms. Thus, the framework’s adaptability enables transformation strategies to be tailored to the unique data models of non-relational databases.
The proposed framework also requires the active participation of stakeholders, including database administrators, users, and the chief information officer. Participation of stakeholders is essential regardless of the data type. Non-relational databases are utilized in a variety of specialized settings. Therefore, soliciting stakeholder input guarantees that the migration meets these complex requirements. The framework relies on a comprehensive back-up and recovery plan to maintain data integrity during migration. This requirement highlights the importance of data protection during data migration for all data types, including transactional databases. The framework’s execution phase includes data migration and extraction, transformation, and loading (ETL) procedures. Due to differences in data models, the ETL process for non-relational databases may differ, but data migration, validation, and verification ensure data accuracy and completeness. The framework stresses the importance of data validation and verification for migration. The technique’s concern is the significance of data quality during migration for relational and non-relational databases. In addition to these essential features, the framework permits meticulous customization to meet the specific requirements of non-relational databases. The flexibility allows organizations to tailor the framework to their specific requirements, thereby enhancing the efficiency and effectiveness of data migrations in non-relational database environments. The framework’s adaptability makes it a potent instrument for managing and executing data migrations in non-relational databases. With careful customization and adherence to its principles, organizations can confidently and accurately migrate data regardless of the database paradigm.

5. Conclusions

This study presents a comprehensive framework for accurate and successful data migration with applied efficiency. The framework’s design is based on five practical procedures: system transformation, data migration verification, functional/performance accuracy tests, system tuning, and system delivery. Through meticulous planning, validation, and evaluation, the framework confirms that organizations undergoing technologically efficient transformations can have an effortless migration experience.

5.1. Theoretical Contributions

The hybrid layering data migration framework contributes to the alignment of evaluation criteria with hardware, software, peopleware, and data, which are the primary sources of data management. The proposed framework enables organizations to make informed decisions and optimize the performance along with the dependability of the data migration technique, particularly data migration, by considering computer system components. Innovative evaluation metrics, including data consistency rate, data integrity score, and data accuracy index, provide quantitative measures to assess the accuracy of the migration process, thereby improving data-driven decision making for real-world applications. In addition, the proposed framework reveals data migration processes that can be used to migrate data in any data management system and showcases the framework’s efficacy in managing data migration tasks and ensuring compatibility transformation between source and target data.

5.2. Practical Implications

Adoption of the proposed data migration framework offers a number of practical benefits. The migration to the open-source data management system enables customization, enhanced performance, scalability, and data processing. Multiple platforms are supported by open-source data services, and open-source technology promotes greater flexibility and reduces reliance on specific operating systems, resulting in lower technology investment costs. Data migration presents an opportunity to enhance the integration and analysis of advanced data technologies, enabling more informed business strategies and better decisions during digital transformation. Additionally, the improved data security and compliance forms inspire confidence among stakeholders and enhance the organization’s reputation. In this study, the proposed framework provides a practical and systematic method for ensuring the success and accuracy of the data migration process. By considering the impact of data migration on sustainable business and technology, any organization can successfully apply the principles of the proposed data migration framework to improve the data migration process, which includes phases such as planning, data extraction, data transformation, data loading, data verification, and application validation. This study also emphasizes the importance of collaboration and communication between stakeholders, application developers, and database administrators for successful and sustainable data migration.
The study’s findings confirm the general applicability of the proposed framework. At the same time, specific efficient practices may vary based on project requirements and the specifics of the migration. Determining the probability of success sustainability for a data migration is difficult since it depends on several variables, such as the complexity of the data, the size of the data, the migration strategy, the migration team’s expertise, and the availability of resources. Data migration success rates can vary from project to project. With proper planning, adherence to actual demonstrating practices, and exhaustive testing, however, the likelihood of a successful migration can be significantly increased. It is essential to employ seasoned professionals with knowledge of source and open-source data management systems. They can evaluate the migration’s specific requirements, analyze the database schema and data, and design an appropriate migration strategy. In addition, performing a comprehensive assessment and analysis of the data and their dependencies, performing extensive testing and validation of the migrated data, and having robust back-up and recovery mechanisms can increase the likelihood of a successful migration. It is also essential to note that thorough planning, testing, and contingency plans should be in place to mitigate any potential risks and minimize the impact of unanticipated problems during the migration process. In the end, the success rate of a data migration depends on each project’s specific conditions and variables. Organizations can increase the probability of a sustainable successful migration by employing a well-defined methodology, utilizing experienced professionals, and conducting exhaustive testing and validation.

5.3. Limitations and Future Work

Despite the evaluation merits of the proposed framework, the data migration process in real-world applications may face certain challenges. The initial migration phase necessitated meticulous planning and execution to prevent operational disruptions. In addition, the migration process can become complex, requiring careful consideration and optimization, particularly for large tables with numerous records and structures of data models, including data types, tables, attributes, data types, keys, and constraints. During the data migration process, the results must support data consistency and accuracy, necessitating proactive data validation techniques. While the framework addressed the limitations to a great extent, continuous monitoring and reporting are still essential for addressing post-migration issues.
The applicability of the proposed framework can be extended to other data management systems. The effectiveness of additional data migration can be compared across various data migration scenarios or additional data models, including non-tabular databases such as NoSQL databases. Automated testing and validation processes can further improve data precision and reduce manual errors. Furthermore, hybrid migration strategies, which combine the benefits of different migration techniques, can be investigated to expedite the migration process with minimal disruptions. The long-term effects of data migration factors on organizational performance, data analysis capabilities, and business operations can also be the subject of further studies.

Author Contributions

Conceptualization, P.N., S.P. and M.R.; methodology, N.S., M.R., S.P. and P.N.; software evaluation and modeling, N.S., M.R. and S.P.; validation, P.N., M.R. and S.P.; formal analysis, P.N., M.R. and S.P.; investigation, N.S., P.N. and M.R.; resources, N.S. and S.P.; data curation, P.N., M.R., N.S. and S.P.; writing—original draft preparation, P.N., M.R. and S.P.; writing—review and editing, P.N., M.R. and S.P.; visualization, P.N., M.R. and S.P.; supervision, P.N., M.R. and S.P.; project administration, P.N., M.R., S.P. and N.S. All authors have read and agreed to the published version of the manuscript.

Funding

This research received no external funding.

Institutional Review Board Statement

Not applicable.

Informed Consent Statement

Not applicable.

Data Availability Statement

Not applicable.

Conflicts of Interest

The authors declare no conflict of interest.

References

  1. Wongvilaisakul, W.; Netinant, P.; Rukhiran, M. Dynamic multi-criteria decision making of graduate admission recommender system: AHP and fuzzy AHP approaches. Sustainability 2023, 15, 9758. [Google Scholar] [CrossRef]
  2. Sánchez-García, E.; Marco-Lajara, B.; Seva-Larrosa, P.; Martínez Falcó, J. Driving innovation by managing entrepreneurial orientation, cooperation and learning for the sustainability of companies in the energy sector. Sustainability 2022, 14, 16978. [Google Scholar] [CrossRef]
  3. Rukhiran, M.; Netinant, P. A practical model from multidimensional layering: Personal finance information framework using mobile software interface operations. J. Inf. Commun. Technol. 2020, 19, 321–349. [Google Scholar] [CrossRef]
  4. Xiong, F.; Xie, M.; Zhao, L.; Li, C.; Fan, X. Recognition and evaluation of data as intangible assets. SAGE Open 2022, 12, 21582440221094600. [Google Scholar] [CrossRef]
  5. Fleckenstein, M.; Fellows, L. Overview of data management frameworks. In Modern Data Strategy; Springer International Publishing: Cham, Switzerland, 2018; pp. 55–59. [Google Scholar] [CrossRef]
  6. Yang, Z. Replacing Oracle DBMS: A Feasibility Study. Master’s Thesis, Tampere University, Tampere, Finland, November 2020. [Google Scholar]
  7. Antanasijević, D.; Lolić, T.; Stefanović, D.; Ristić, S. The challenge of an extraction-transformation-loading tool selection. In Proceedings of the XIV International SAUM Conference on Systems, Automatic Control and Measurements, Niš, Serbia, 14–16 November 2018. [Google Scholar]
  8. Madhikerrni, M.; Främling., K. Data discovery method for extract-transform-load. In Proceedings of the 10th International Conference on Mechanical and Intelligent Manufacturing Technologies, Cape Town, South Africa, 15–17 February 2019. [Google Scholar] [CrossRef]
  9. Opara-Martins, J.; Sahandi, R.; Tian, F. Critical analysis of vendor lock-in and its impact on cloud computing migration: A business perspective. J. Cloud Comput. 2016, 5, 4. [Google Scholar] [CrossRef]
  10. Cao, R.; Iansiti, M. Digital transformation, data architecture, and legacy systems. J. Digit. Econ. 2022, 1, 1–19. [Google Scholar] [CrossRef]
  11. Cavalcanti, D.R.; Oliveira, T.; Santini, F.O. Drivers of digital transformation adoption: A weight and meta-analysis. Heliyon 2022, 8, e08911. [Google Scholar] [CrossRef] [PubMed]
  12. Banimfreg, B.H. A comprehensive review and conceptual framework for cloud computing adoption in bioinformatics. Healthc. Anal. 2023, 3, 100190. [Google Scholar] [CrossRef]
  13. Chawla, N.; Kumar, D.; Sharma, D. Improving cost for data migration in cloud computing using genetic algorithm. Int. J. Softw. Innov. 2020, 8, 69–81. [Google Scholar] [CrossRef]
  14. Ansar, M.; Ashraf, M.W.; Fatima, M. Data migration in cloud: A systematic review. Am. Sci. Res. J. Eng. Technol. Sci. 2018, 48, 73–89. [Google Scholar]
  15. Azeroual, O.; Jha, M. Without data quality, there is no data migration. Big Data Cogn. Comput. 2021, 5, 24. [Google Scholar] [CrossRef]
  16. Jaumard, B.; Pouya, H. Migration plan with minimum overall migration time or cost. J. Opt. Commun. Netw. 2018, 10, 1–13. [Google Scholar] [CrossRef]
  17. Yang, Y.; Mao, B.; Jiang, H.; Yang, Y.; Luo, H.; Wu, S. SnapMig: Accelerating VM live storage migration by leveraging the existing VM snapshots in the cloud. IEEE Trans. Parallel Distrib. Syst. 2018, 29, 1416–1427. [Google Scholar] [CrossRef]
  18. Tošić, A.; Vičič, J.; Burnard, M.; Mrissa, M. A blockchain protocol for real-time application migration on the edge. Sensors 2023, 23, 4448. [Google Scholar] [CrossRef] [PubMed]
  19. Bouhamed, M.M.; Chaoui, A.; Nouara, R.; Diaz, G.; Dembri, A. Reducing the number of migrated instances during business process change: A graph rewriting approach. J. King Saud Univ. Comput. Inf. 2022, 34, 7720–7734. [Google Scholar] [CrossRef]
  20. Eyada, M.M.; Saber, W.; Genidy, M.M.; Amer, F. Performance evaluation of IoT data management using MongoDB versus MySQL databases in different cloud environments. IEEE Access 2020, 8, 110656–110668. [Google Scholar] [CrossRef]
  21. Ellison, M.; Calinescu, R.; Paige, R.F. Evaluating cloud database migration options using workload models. J. Cloud Comput. 2018, 7, 6. [Google Scholar] [CrossRef]
  22. Luca, M.; Barlacchi, G.; Oliver, N.; Lepri, B. Leveraging mobile phone data for migration flows. In Data Science for Migration and Mobility Studies, 1st ed.; Korkmaz, E.E., Salah, A.A., Eds.; Oxford University Press: Oxford, UK, 2021; pp. 1–14. [Google Scholar] [CrossRef]
  23. Sibgatullina, A.I.; Yakupov, A.S. Development a data validation module to satisfy the retention policy metric. Russ. Digit. Libr. J. 2022, 25, 159–178. [Google Scholar] [CrossRef]
  24. Prasanna, C.; Subha, R.; Sreemathy, J.; Aravindh Ramanathan, P.; Jainaveen, M. Data validation and migration-a theoretical perspective. In Proceedings of the 7th International Conference on Advanced Computing and Communication Systems, Coimbatore, India, 19–20 March 2021. [Google Scholar] [CrossRef]
  25. Yang, C.; Zhao, F.; Tao, X.; Wang, Y. Publicly verifiable outsourced data migration scheme supporting efficient integrity checking. J. Netw. Comput. Appl. 2021, 192, 103184. [Google Scholar] [CrossRef]
  26. Latha, M.; Kumar, P. Analysis on data migration strategies in heterogeneous databases. Int. J. Adv. Res. Sci. Commun. Technol. 2021, 6, 210–214. [Google Scholar] [CrossRef]
  27. McGill, M.M.; Sexton, S.; Peterfreund, A.; Praetzellis, M. Efficient, effective, and ethical education research data management and sustainability. In Proceedings of the 52nd ACM Technical Symposium on Computer Science Education, Virtual Event, USA, 13–20 March 2021. [Google Scholar] [CrossRef]
  28. Bento, P.; Neto, M.; Corte-Real, N. How data governance frameworks can leverage data-driven decision making: A sustainable approach for data governance in organizations. In Proceedings of the 2022 17th Iberian Conference on Information Systems and Technologies, Madrid, Spain, 22–25 June 2022. [Google Scholar] [CrossRef]
  29. Dehury, C.K.; Srirama, S.N.; Chhetri, T.R. CCoDaMiC: A framework for coherent coordination of data migration and computation platforms. Future Gener. Comput. Syst. 2020, 109, 1–16. [Google Scholar] [CrossRef]
  30. Transition from Legacy Databases to a Modern Data Architecture. Available online: https://www.devprojournal.com/technology-trends/open-source/transition-from-legacy-databases-to-a-modern-data-architecture (accessed on 13 July 2023).
  31. Hussein, A.A. Data migration need, strategy, challenges, methodology, categories, risks, uses with cloud computing, and improvements in its using with cloud using suggested proposed model (DMig 1). J. Inf. Secur. 2021, 12, 17–103. [Google Scholar] [CrossRef]
  32. Mackita, M.; Shin, S.; Choe, T. ERMOCTAVE: A risk management framework for IT systems which adopt cloud computing. Future Internet 2019, 11, 195. [Google Scholar] [CrossRef]
  33. Trisnawaty, N.W.; Hidayanto, A.N.; Ruldeviyani, Y. Database and application migration in the financial services industry sector in the acquisition environment and environmental science. In Proceedings of the 9th Engineering International Conference, Semarang, Indonesia, 24 September 2020. [Google Scholar] [CrossRef]
  34. Barrios, P.; Loison, F.; Danjou, C.; Eynard, B. PLM Migration in the era of big data and IoT: Analysis of information system and data topology. In Proceedings of the 17th IFIP International Conference on Product Lifecycle Management, Rapperswil, Switzerland, 5–8 July 2020. [Google Scholar] [CrossRef]
  35. Nyeint, K.A.; Soe, K.M. Database migration based on Trickle migrations approach. Natl. J. Parallel Soft Comput. 2019, 1, 81–86. [Google Scholar]
  36. Mateus, B.G.; Martinez, M.; Kolski, C. Learning migration models for supporting incremental language migrations of software applications. Inf. Softw. Technol. 2023, 153, 107082. [Google Scholar] [CrossRef]
  37. Types of Data Migration. Available online: https://www.codemotion.com/magazine/devops/cloud/migrating-data-to-the-cloud-a-practical-guide (accessed on 29 July 2023).
  38. Gundall, M.; Stegmann, J.; Reichardt, M.; Schotten, H.D. Downtime Optimized Live Migration of Industrial Real-Time Control Services. In Proceedings of the IEEE 31st International Symposium on Industrial Electronics (ISIE), Anchorage, AK, USA, 1–3 June 2022. [Google Scholar] [CrossRef]
  39. Fernandex, J.B. Database Schema Migration in Highly Available Services. Master’s Thesis, KTH Royal Institute of Technology, Stockholm, Sweden, 20 January 2021. [Google Scholar]
  40. Rukhiran, M.; Buaroong, S.; Netinant, P. Software development for educational information services using multilayering semantics adaptation. Int. J. Serv. Sci. Manag. Eng. Technol. 2022, 13, 1–27. [Google Scholar] [CrossRef]
  41. Romhányi, Á.; Vámossy, Z. Benefits of Layered Software Architecture in Machine Learning Applications. In Proceedings of the International Conference on Image Processing and Vision Engineering, Online, 28–30 April 2021; SCITEPRESS—Science and Technology Publications: Setubal, Portugal, 2021. [Google Scholar] [CrossRef]
  42. Zhuxian, W.; Xingmin, G.; Peng, F. The application of layering technology in computer software development. In Proceedings of the 2017 International Conference on Robots & Intelligent System, Huaian, China, 15–16 October 2017. [Google Scholar] [CrossRef]
  43. Li, X. Application effect of layering technology in computer software development. J. Comput. Sci. Res. 2018, 1, 1. [Google Scholar] [CrossRef]
  44. Ramli, S.; Redzuan, I.; Mamood, I.H.; Mohd Zainudin, N.; Hasbullah, N.A.; Wook, M. National sport institute case: Automated data migration using Talend open studio with ‘Trickle approach’. In Advances in Visual Informatics; IVIC 2021, Lecture Notes in Computer Science; Zaman, H.B., Smeaton, A.F., Shih, T.K., Velastin, S., Terutoshi, T., Jørgensen, B.N., Aris, H., Ibrahim, N., Eds.; Springer: Cham, Switzerland, 2021; Volume 13051, pp. 214–223. [Google Scholar] [CrossRef]
  45. Singh, A. Data Migration from Relational Database to MongoDB Using XAMPP and NoSQL. Available online: https://papers.ssrn.com/sol3/papers.cfm?abstract_id=3372802 (accessed on 13 July 2023).
  46. Ceresnak, R.; Matiasko, K.; Dudas, A. Influencing migration processes by real-time data. In Proceedings of the 28th Conference of Open Innovations Association, Moscow, Russia, 27–29 January 2021. [Google Scholar] [CrossRef]
  47. Blagaić, S.; Babac, M.B. Application for data migration with complete data integrity. Int. J. Syst. Syst. Eng. 2022, 12, 405–432. [Google Scholar] [CrossRef]
  48. Yang, C.; Tao, X.; Wang, S.; Zhao, F. Data integrity checking supporting reliable data migration in cloud storage. In Proceedings of the 15th International Conference on Wireless Algorithms, Systems, and Applications, Qingdao, China, 13–15 September 2020. [Google Scholar] [CrossRef]
  49. Rinken, S.; Ortega, J.L. Leveraging the web for migration studies: Data sources and data extraction. In Migration Research in a Digitized World, IMISCOE Research Series; Pötzschke, S., Rinken, S., Eds.; Springer: Cham, Switzerland, 2022; pp. 129–148. [Google Scholar] [CrossRef]
  50. Petrasch, R.J.; Petrasch, R.P. Data integration and interoperability: Towards a model-driven and pattern-oriented approach. Modelling 2022, 3, 105–126. [Google Scholar] [CrossRef]
  51. Ramzan, S.; Bajwa, I.S.; Ramzan, B.; Anwar, W. Intelligent data engineering for migration to NoSQL based secure environments. IEEE Access 2019, 7, 69042–69057. [Google Scholar] [CrossRef]
  52. Wang, X. Design and implementation strategy of data migration system based on Hadoop platform. J. Phys. 2021, 2010, 012082. [Google Scholar] [CrossRef]
  53. Poudel, M.; Sarode, R.P.; Watanobe, Y.; Mozgovoy, M.; Bhalla, S. Processing analytical queries over Polystore system for a large astronomy data repository. Appl. Sci. 2022, 12, 2663. [Google Scholar] [CrossRef]
  54. Rukhiran, M.; Wong-In, S.; Netinant, P. IoT-Based biometric recognition systems in education for identity verification services: Quality assessment approach. IEEE Access 2023, 11, 22767–22787. [Google Scholar] [CrossRef]
  55. Cheng, Z.; Wang, L.; Cheng, Y.; Chen, G. Automated and intelligent data migration strategy in high energy physical storage systems. J. Phys. 2020, 1525, 012042. [Google Scholar] [CrossRef]
  56. Martinho, N.; Almeida, J.-P.d.; Simões, N.E.; Sá-Marques, A. UrbanWater: Integrating EPANET 2 in a PostgreSQL/PostGIS-based geospatial database management system. ISPRS Int. J. Geo-Inf. 2020, 9, 613. [Google Scholar] [CrossRef]
  57. Chang, Q.; Nazir, S.; Li, X. Decision-making and computational modeling of big data for sustaining influential usage. Sci. Program. 2022, 2022, 2099710. [Google Scholar] [CrossRef]
  58. Ibrahim, M.; Imran, M.; Jamil, F.; Lee, Y.; Kim, D. EAMA: Efficient adaptive migration algorithm for cloud data centers (CDCs). Symmetry 2021, 13, 690. [Google Scholar] [CrossRef]
  59. Kaur, A.; Kumar, S.; Gupta, D.; Hamid, Y.; Hamdi, M.; Ksibi, A.; Elmannai, H.; Saini, S. Algorithmic approach to virtual machine migration in cloud computing with updated SESA algorithm. Sensors 2023, 23, 6117. [Google Scholar] [CrossRef] [PubMed]
  60. Deshmukh, O. Data Migration Techniques across DBMS by Using Metadata. Int. J. Res. Appl. Sci. Eng. Technol. 2022, 10, 457–463. [Google Scholar] [CrossRef]
  61. Taherdoost, H. Change Management. In E-Business Essentials; Springer Nature: Cham, Switzerland, 2023; pp. 319–349. [Google Scholar]
  62. Singh, G.; Singh, P.; Hedabou, M.; Masud, M.; Alshamrani, S.S. A Predictive Checkpoint Technique for Iterative Phase of Container Migration. Sustainability 2022, 14, 6538. [Google Scholar] [CrossRef]
Figure 1. Overview of the methodological steps for the development of the data migration framework.
Figure 1. Overview of the methodological steps for the development of the data migration framework.
Sustainability 15 15034 g001
Figure 2. A hybrid layering framework for sustainable data migration.
Figure 2. A hybrid layering framework for sustainable data migration.
Sustainability 15 15034 g002
Figure 3. Entity–relationship diagram for a sample of a customer order data scenario.
Figure 3. Entity–relationship diagram for a sample of a customer order data scenario.
Sustainability 15 15034 g003
Figure 4. Entity–relationship diagram for a sample of a purchasing order data scenario.
Figure 4. Entity–relationship diagram for a sample of a purchasing order data scenario.
Sustainability 15 15034 g004
Figure 5. An implementation of hybrid layering data migration framework based on Python.
Figure 5. An implementation of hybrid layering data migration framework based on Python.
Sustainability 15 15034 g005
Figure 6. Results of migration rounds, data consistency rate, and data integrity rate.
Figure 6. Results of migration rounds, data consistency rate, and data integrity rate.
Sustainability 15 15034 g006
Figure 7. Results of data accuracy index, error rate, and data completeness percentage.
Figure 7. Results of data accuracy index, error rate, and data completeness percentage.
Sustainability 15 15034 g007
Figure 8. Results of data matching percentage, data consistency index, and data accurate percentage.
Figure 8. Results of data matching percentage, data consistency index, and data accurate percentage.
Sustainability 15 15034 g008
Table 1. Comparison of data migration techniques in the preprocessing phase.
Table 1. Comparison of data migration techniques in the preprocessing phase.
Methodology StepsBig Bang Data MigrationTrickle Data MigrationZero-Downtime Data MigrationProposed Hybrid Data Migration Framework
Research and AnalysisConduct an in-depth, comprehensive analysis of the current data environment and project specifications.Assess the part data characteristics and project objectives to determine and meet suitability gradually.Evaluate the data infrastructure and project goals to ascertain feasibility without any failure.Conduct a thorough analysis of the current data environment for the entire organization, considering factors such as data complexity, volume, and business continuity requirements. Evaluate the need for continuous system availability during migration and the feasibility of incremental data transfers to reduce downtime.
Selection of Migration TechniqueChoose big bang migration for low-complexity, low-volume, and short-downtime-tolerance projects.Optimize trickle migration for projects with moderate complexity and data volume, allowing gradual data transfer to minimize downtime.Select zero-downtime migration for projects requiring continuous system availability with complex data and a high tolerance for zero downtime.Utilize a combination of trickle and zero-downtime migration strategies to strike a balance between gradual data transfer and continuous system availability. Choose this strategy for projects with complex data, a high tolerance for zero downtime, and the requirement of seamless data integration without protracted interruptions in mission-critical operations.
Planning and PreparationDevelop a detailed migration plan outlining tasks, timelines, and resource allocation.Plan and prepare for incremental data transfers, ensuring coordination and data integrity across phases.Create a comprehensive migration plan, focusing on replication and synchronization to maintain system availability during the migration.Develop a comprehensive migration plan incorporating the incremental data transfer method of trickle migration and the continuous system availability strategy of zero-downtime migration. Assign resources and establish deadlines to ensure efficient execution. Prepare a comprehensive back-up and recovery plan to protect vital data during migration.
Data Profiling and PreprocessingThe data are profiled to identify potential issues and are preprocessed to ensure compatibility with the target data management system.Conduct data profiling to understand data complexities and incremental preprocessing of data for seamless integration.Conduct data profiling to ensure data compatibility and fault-tolerant preprocessing of data for synchronization with minimal disruption.Conduct exhaustive data profiling to comprehend the complexities and nuances of data during the trickle and zero-downtime migration phases. The data must be preprocessed to ensure compatibility with the target data system and maintain data consistency during the gradual data transfer, thereby ensuring rollback capability for data recovery.
Table 2. Comparison of data migration techniques in the post-processing phase.
Table 2. Comparison of data migration techniques in the post-processing phase.
Methodology StepsBig Bang Data MigrationTrickle Data MigrationZero-Downtime Data MigrationProposed Hybrid Data Migration Framework
Testing and ValidationAfter data migration, establish a dedicated testing environment to verify data accuracy and consistency.Perform rigorous testing at each migration phase to validate data integrity and identify inconsistencies.Incrementally implement thorough testing, simulating real-time scenarios to ensure data consistency and system functionality without impacting users.Establish a dedicated testing environment to verify the efficacy of the combined migration strategies. Perform rigorous testing at each successive phase, ensuring data integrity, consistency, and system functionality without interfering with ongoing operations. Conduct real-time simulations to evaluate system performance and ensure availability throughout the migration.
Execution of MigrationTransferring the entire dataset to the target data in a single migration operation.Migrate data incrementally, moving subsets of data in controlled stages while ensuring data consistency.Implement real-time data replication and synchronization mechanisms to achieve continuous availability and data consistency during migration.To incrementally transfer subsets of data, execute the migration in controlled stages utilizing the trickle method. Implement real-time data replication and synchronization mechanisms, similar to zero-downtime migration, to ensure continuous data availability throughout the entire migration process. Maintain data accuracy and system stability through vigilant migration monitoring and the prompt resolution of any problems.
Post-Migration EvaluationEvaluate the success of the migration process, addressing any post-migration issues that may arise.Analyze the effectiveness of the migration, considering data consistency and user feedback to refine the process.Assess the migration’s performance, address post-migration issues, and gather stakeholder feedback to optimize the technique.Consider data consistency, system performance, and user feedback in evaluating the success of the combined migration strategy. Evaluate the efficacy of the trickle and zero-downtime techniques in achieving the project’s goals. To refine the methodology for future migrations and optimize data management strategies, migration result is necessary to gather input from stakeholders and end-users.
Documentation and Knowledge TransferDocument the entire migration process, challenges faced, and lessons learned for future reference.Create comprehensive documentation of the migration steps and knowledge transfer sessions for IT teams and end-users.Document the migration methodology, share knowledge and insights gained, and provide training to ensure a seamless transition to the new system.For future reference, document the entire migration process, including the combined trickle and zero-downtime techniques, obstacles encountered, and lessons learned. Create detailed documentation outlining the methodology and knowledge transfer sessions for IT teams and end-users to ensure a smooth transition to the new data environment.
Table 3. Key criteria of data management for evaluating effective data migration sustainability.
Table 3. Key criteria of data management for evaluating effective data migration sustainability.
CriteriaDescription
Compatibility
[19,46]
Compatibility refers to the ability to successfully transfer data and functionality from one data management system to another without losing functionality, compromising data integrity, or requiring significant changes to the application code. This factor pertains to the assurance that the target data management system has the necessary capabilities to manage data competently and meet application requirements.
Transformation
[47]
Transformation involves modifying or converting data, schema, or other components from the source data to a format suitable for the target data. Adjustments are made to ensure compatibility, data integrity, and optimal performance in the new data environment.
Integrity
[48]
The integrity of data migration refers to the maintenance of data precision, consistency, and dependability throughout the migration process, including constraints, triggers, and referential integrity between source and target data. It ensures that the data is migrated from the source to the target data without loss or corruption.
Performance
Optimization [19,21,49]
Performance optimization in data migration is the process of improving the responsiveness and efficiency of a data management system during and after the migration process. It requires identifying and implementing techniques to improve query execution, data retrieval, and overall system performance in the new data environment.
Licensing and Cost [21,49]Licensing and cost considerations in data migration involve evaluating and managing the financial facets of migrating from one data management system to another. Understanding the licensing models, costs, and potential financial implications of the source and target data management systems is required.
Accuracy
[21,50,51]
Accuracy refers to the exactness and accuracy of the data being transferred from the source data to the target data. It involves ensuring that the migrated data represents the original data accurately and maintains its integrity throughout the migration process.
Reliability
[48,49]
In data migration, reliability refers to the dependability and uniformity of the migration process and the resulting data management system. It entails ensuring that the migration is carried out successfully, with minimal errors or interruptions, and that the migrated data functions dependably regarding data integrity, availability, and performance.
Transparency and Transformation
Processing [15,51,52]
Transparency in data migration refers to the visibility and clarity of the migration procedure. In contrast, transformation processing refers to the data and structure modifications and conversions performed during migration. Transparency provides clear and comprehensive visibility into the migration process, enabling stakeholders to comprehend and track the migration’s progress, actions, and potential consequences.
Table 5. Effectiveness and accuracy evaluation metrics and formula of data migration framework.
Table 5. Effectiveness and accuracy evaluation metrics and formula of data migration framework.
CategoryEvaluation CriteriaFormulaDescription
CompatibilityData Consistency Rate (DCR)(Number of Consistent Records/Total Number of Records) × 100Measures the level of consistency between the migrated data and the original source data.
Data Integrity Rate (DIS)(Total Number of Correct Values/Total Number of Migrated
Values) × 100
Assesses the accuracy and completeness of the migrated data to ensure data integrity is maintained.
Data Accuracy
Index (DAI)
(1 − (|Total Number of Inaccurate Values|/Total Number of
Migrated Values)) × 100
Provides an index to measure the overall accuracy of the migrated data.
Error Rate (ER)(Number of Error Records/Total Number of Records) × 100Indicates the percentage of data records that contain errors or discrepancies in the migrated data.
TransformationData Completeness Percentage (DCP)(Number of Complete Records/Total Number of Records) × 100Measures the degree to which the migrated data are complete and reflects the entirety of the source data.
Data Matching
Ratio (DMR)
(Number of Matched Records/
Total Number of Records) × 100
Evaluates the accuracy of data matching during the migration process, ensuring correct record associations.
IntegrityData Consistency Index (DCI)(1 − (|Total Number of Inconsistent Values|/Total Number of
Migrated Values)) × 100
Provides an index that reflects the consistency of data values between the source and target data.
Performance OptimizationData Accuracy
Percentage (DAP)
(Number of Correct Values/Total Number of Migrated Values) × 100Represents the overall accuracy of the migrated data.
Effectiveness and Accuracy EvaluationPrecision(True Positives)/(True Positives + False Positives)Measures the accuracy of the positive predictions made during the migration.
Recall(True Positives)/(True Positives + False Negatives)Measures the percentage of true positive matches found.
Table 6. The unsuccessful test results of data migration approaches.
Table 6. The unsuccessful test results of data migration approaches.
Support SourceSupport TargetSolutionsApproachResults
Data–Entity relationshipData–Entity relationshipSQL server integration services (SSIS)Incremental, SQL scripts, built-in features.The system functions cannot be validated and verified despite the successful migration. The system was inoperable.
ReplicationIncremental, SQL Scripts, PostgreSQL foreign data wrapper (FDW) to read data from SQL server tables and write it into PostgreSQL.The system functions cannot be validated and verified despite the successful migration. The system was inoperable.
Incremental migration—Manual data pumpingIncremental SQL Scripts migrate data in smaller batches or based on manual-specific criteria.The system functions cannot be validated and verified despite the successful migration. The system was inoperable.
Table 7. Evaluation results of data migration approaches for data validity and high availability.
Table 7. Evaluation results of data migration approaches for data validity and high availability.
DepartmentData SourceData Target
Trickle MigrationZero Downtime
Migration
Proposed Hybrid Layering Framework
TablesRecordsTablesRecordsDowntimeTablesRecordsTablesRecordsDowntime
Public Relation (PR)1918,9401918,9403Failure
occurs
-1918,9400
Purchasing Order (PO)1317,8101317,8102Failure
occurs
-1317,8100
Sale Order (SO)2469,0302469,0305Failure
occurs
-2469,0300
Human Resource (HR)3049,2003049,2004Failure
occurs
-3049,2000
Good Receipt (GR)2916,7002916,7004Failure
occurs
-2916,7000
Delivery Notes (DN)2312,1902312,1903Failure
occurs
-2312,1900
Job Order (JO)2669,0302669,0304Failure
occurs
-2669,0300
Hand Over (HO)2423702423704Failure
occurs
-2423700
Warehouse Stock (WS)2328,7602328,7603Failure
occurs
-2328,7600
Others1116,5201116,52001116,5201116,5200
Total222300,550222300,550291116,520222300,5500
Table 8. Evaluation results of successful data migration for data validity and high availability.
Table 8. Evaluation results of successful data migration for data validity and high availability.
DepartmentMigration
Rounds
Incremental
Migration
CompatibilityTransformationIntegrityPerformanceSuccess Rate
(SR)
DCRDISDAIERDCPDMRDCIDAP
PR132%53%100%100%0%53%53%100%100%100%
264%74%100%100%0%74%74%100%100%100%
3100%100%100%100%0%100%100%100%100%100%
PO154%68%100%100%0%68%68%100%100%100%
2100%100%100%100%0%100%100%100%100%100%
SO125%29%100%100%0%29%29%100%100%100%
250%52%100%100%0%52%52%100%100%100%
375%73%100%100%0%73%73%100%100%100%
4100%100%100%100%0%100%100%100%100%100%
HR120%15%100%100%0%15%15%100%100%100%
240%27%100%100%0%27%27%100%100%100%
360%63%100%100%0%63%63%100%100%100%
480%78%100%100%0%78%78%100%100%100%
5100%100%100%100%0%100%100%100%100%100%
GR114%17%100%100%0%17%17%100%100%100%
235%33%100%100%0%33%33%100%100%100%
352%58%100%100%0%58%58%100%100%100%
476%77%100%100%0%77%77%100%100%100%
5100%100%100%100%0%100%100%100%100%100%
DN126%34%100%100%0%34%34%100%100%100%
257%56%100%100%0%56%56%100%100%100%
383%74%100%100%0%74%74%100%100%100%
4100%100%100%100%0%100%100%100%100%100%
JO127%38%100%100%0%38%38%100%100%100%
254%59%100%100%0%59%59%100%100%100%
377%74%100%100%0%74%74%100%100%100%
4100%100%100%100%0%100%100%100%100%100%
HO154%67%100%100%0%67%67%100%100%100%
2100%100%100%100%0%100%100%100%100%100%
WS135%26%100%100%0%26%26%100%100%100%
274%56%100%100%0%56%56%100%100%100%
3100%100%100%100%0%100%100%100%100%100%
Others1100%100%100%100%0%100%100%100%100%100%
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

Share and Cite

MDPI and ACS Style

Netinant, P.; Saengsuwan, N.; Rukhiran, M.; Pukdesree, S. Enhancing Data Management Strategies with a Hybrid Layering Framework in Assessing Data Validation and High Availability Sustainability. Sustainability 2023, 15, 15034. https://doi.org/10.3390/su152015034

AMA Style

Netinant P, Saengsuwan N, Rukhiran M, Pukdesree S. Enhancing Data Management Strategies with a Hybrid Layering Framework in Assessing Data Validation and High Availability Sustainability. Sustainability. 2023; 15(20):15034. https://doi.org/10.3390/su152015034

Chicago/Turabian Style

Netinant, Paniti, Nattapat Saengsuwan, Meennapa Rukhiran, and Sorapak Pukdesree. 2023. "Enhancing Data Management Strategies with a Hybrid Layering Framework in Assessing Data Validation and High Availability Sustainability" Sustainability 15, no. 20: 15034. https://doi.org/10.3390/su152015034

APA Style

Netinant, P., Saengsuwan, N., Rukhiran, M., & Pukdesree, S. (2023). Enhancing Data Management Strategies with a Hybrid Layering Framework in Assessing Data Validation and High Availability Sustainability. Sustainability, 15(20), 15034. https://doi.org/10.3390/su152015034

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop