Review Reports

Round 1

Reviewer 1 Report

This manuscript has been well written but I can see there is lack of citation of recent work, Authors are suggested to cite recent work as below;

Perceived information pollution: conceptualization, measurement, and nomological validity

A multi-dimensional construct of perceived information pollution in the era of rife infollution

Authors are also recommended to add separate sections about theoretical and practical implications in the revised draft.

Good wishes,

Author Response

1-This manuscript has been well written but I can see there is lack of citation of recent work, Authors are suggested to cite recent work as below;

There are 14 references from 2018 to 2021. Thanks,

2- Perceived information pollution: conceptualization, measurement, and nomological validity

No action required.

3- A multi-dimensional construct of perceived information pollution in the era of rife infollution

No action required

4- Authors are also recommended to add separate sections about theoretical and practical implications in the revised draft.

Discussion section contains firstly theoretical and secondly practical considerations.

Reviewer 2 Report

Please see the attachment.

Comments for author File: Comments.pdf

Author Response

1- The novelty and motivation of the work is totally unclear. You need to provide a detailed summary of the state of the art and discuss their limitations which could potentially be the motivation for the work.

 

The novelty and importance are included in the introduction and in the state of the art section it has been included the existing similar works in the field of BI and process mining and highlighted why our proposal covers the existing deficiencies that are avoiding many companies to use their own unstructured data to identify their hidden knowledge assets, essential to deploy a digital transformation in any company.

 

2- In general, the article contains very standard definitions which are available apparently at every process mining blogs, tutorials, books, etc. These are very basic and I couldn't find anything new from this article. This is only an implementation of well-established techniques. It should be simplified and shortened.

 

The next information has been added after Figure 1, just to clarify that C4PM is a new methodology, based on existing concepts, but for the first time linked and described in the form of a process.

“C4PM is a new methodology that is applied as a previous step to Process Mining for those companies whose processes do not have a very high level of digitalization, but wish to obtain the benefits of Process Mining, and which allows the pre-processing of unstructured data available to most companies and which are tremendously valuable for discovering, with the appropriate treatment, intangible assets of the organisation whose existence is unknown to it. The identification and subsequent appropriate reuse of these intangible assets, which are pieces of organisational knowledge, and which have emerged with the use of C4PM, is essential to make the organisation more productive and sustainable, as it has been proven that the reuse of knowledge improves the productivity and sustainability of businesses. Thanks to C4PM, Process Mining and all its benefits are available to all organisations that need to automate their processes, simply by having access to a minimum of unstructured information that can be digitally analysed by C4PM.”

 

3- It needs to be proofread by a native English speaker to increased readability and clarity.

Following your recommendation, a Native English Speaker has reviewed the paper.

4- How to use the e-mail dataset of Enron company in process mining is not sufficiently discussed. For this purpose, it should be critical whether the data are suitable for this analysis. Whether the Enron e-mail data set is a suitable data set for process mining has not been sufficiently explained and studied.

 

This has been clarified in section 4 since 4^th paragraph.

 

5- Figure 4, Figure 5, Figure 6 are not readable, they need to be rearranged by making them legible.

 

Figures have been updated in the paper. Thanks.

 

6- Another negative aspect of the article is the analysis of the Enron company, which grew very quickly in a short time and went bankrupt in a sensational way. This situation is not seen as a suitable case study to guide companies to go digital.

 

This has been explained in section 4, 9^th paragraph as a complement to the previous reviewer comment.

 

We would like to highlight that the validation of the C4PM has been done using the ENRON dataset because it is the only one that contains public unstructured data registering the conversations in a real organizational environment. We do not want to present ENRON as a good example of digital company, it is not to guide companies to digital transformation per se, but rather to contribute to it and serve as a model for analysis of unstructured data that every company possesses, but is unaware of the pre-existing value in these emails, which can be turned into valuable knowledge assets after the application of C4PM and the analysis of these emails.

 

7- It is not explained how the Valuable Assets listed in Table 3 are determined.

 

This has been explained after figure 11.

 

The intangible assets identified came up after analysing with the Python jupyter notebook platform, the emails of 16 employees out of the 150 contained in the dataset, and extracting the most frequent words.

The 16 employees selected correspond to: 'allen-p','arnold-j','arora-h','causholli-m','corman-s','crandell-s','cuilla-m','dasovich-j','davis-d','dean-c',

             'delainey-d','derrick-j','donoho-l','donohoe-t','dorland-c','ermis-f'.

The total number of words analysed per email for each of the 16 employees is 962.385.

 

8- It is unclear how the knowledge assets identified for the two departments were determined under Chapter 4.1. under the heading of assets discovered for the Accounting Department. This is perceived as the information listed independently of the study. How such information is extracted from Enron company emails should be convincingly explained.

 

This has been explained after figure 11, as a complement of the previous comment of the reviewer.

 

The assets are subsequently determined after analysing the text of each of the 16 employees' emails out of the 150 that make up the total dataset, using the Python jupyter notebook platform for this purpose. In this analysis, the most frequent words used by each employee in their emails are identified, and within this set, those words that infer some type of knowledge asset are identified, which are finally classified according to their nature, within each area involved.

It has been included in the paper a table (table 3) that summarized the most frequent used words, and its relationship with the assets identified that are listed and described in section 4.1 and 4.2.

This result shows how C4PM is effective and efficient to one of the most difficult tasks to be developed while auditing a company as the fist step to guide it to its digital transformation, this task is the discovery of the company intangible assets, which represents its knowledge and that will represent the pillars over which the digital transformation project will be developed.

 

9- No study-specific results are presented. In the fourth chapter, which is the last chapter, general purpose discussions are made in different dimensions. Therefore, it is unclear and insufficient what readers will gain from this study.

 

This has been explained in section 5.

 

After applying C4PM as shown in the case study presented, it can be observed that the amount of assets identified just using the text of the employee emails is huge, so, any organization, can use their emails dataset to identify those assets that being very important for the organization, could be simple invisible and unknown for it. Discovering knowledge assets is essential for any organization in the path to develop its sustainable, productive and competitive digital transformation, and to use Process Mining successfully.

Reviewer 3 Report

The article is interesting, highly informative and timely. 

The process (steps) required prior to the application of data mining are well described and provide a good tool for the eventual businesses who would wish to employ the proposed C4PM method.

It would be good to provide some explanation of why the case of Enron was selected. First of all, the company went bankrupt, so the level of information available you assess as Low- which in a way negates the previously described steps?

The question of access to employees' emails deserves at least a short reference to. There is a legal issue as to the right of access, where the opinions are very different: from full free access of the employer to the issue of protection of personal space. So prior to the use of this source, the company needs to make sure it is legally entitled to do so.  This should be acknowledged.

At places, the sentences are formulated in such a complex (long) manner that it is difficult to follow the arguments. Please, check and revise. One such example is line 544... some automatically process...??

Author Response

1- It would be good to provide some explanation of why the case of Enron was selected. First of all, the company went bankrupt, so the level of information available you assess as Low- which in a way negates the previously described steps?

 

This has been clarified in section 4.

The ENRON case was selected because this company was a reference in the Energy sector, apart from having many other products that made it a diversified business, so it has been possible to find a lot of relevant information about the functioning of the organization.

ENRON was subjected to a series of investigations by the energy regulatory commission, who during the investigation published a set of emails of the company's employees on its website for public use. This dataset containing unstructured data has been formatted by different groups of researchers in order to improve its integrity so that it can be analysed in different research works such as the C4PM validation.

 

2- The question of access to employees' emails deserves at least a short reference to. There is a legal issue as to the right of access, where the opinions are very different: from full free access of the employer to the issue of protection of personal space. So prior to the use of this source, the company needs to make sure it is legally entitled to do so.  This should be acknowledged.

 

This has been clarified in section 4 since 4th paragraph.

 

We proceeded to review the state of the art in search of unstructured data to apply the last step of C4PM, and it was decided to use data sources such as email, as it is one of the main tools that every company has and through which employees transfer a large volume of data, which usually represent relevant corporate information. In addition, with process mining it is not yet possible to work with this type of unstructured data, so applying C4PM will facilitate the identification of knowledge assets that were not considered by the company until that moment. Therefore, these newly identified knowledge assets can be critical for the business, for example, organisational processes, which can be analysed under the principles of process mining.

In this sense, it was observed that several groups of researchers were conducting a series of investigations with the email data of Enron employees, therefore, they were unstructured and very significant data that were suitable for the purposes of applying natural language processing techniques in this research.

The dataset used to test the current C4PM proposal, was collected and prepared by the CALO Project (A Cognitive Assistant that Learns and Organizes). CALO was an output of the PAL (Personal Assistant that Learns) framework from DARPA, a massive collaboration project led by SRI International (https://pal.sri.com/ ). It contains data from about 150 users, mostly senior management of Enron, organized into folders. The corpus contains a total of about 0.5M messages. This data was originally made public, and posted to the web, by the Federal Energy Regulatory Commission during its investigation.

The email dataset was later purchased by Leslie Kaelbling at MIT, and turned out to have a number of integrity problems. A number of folks at SRI, notably Melinda Gervasio, worked hard to correct these problems, and it is thanks to them that the dataset is available. The dataset here does not include attachments, and some messages have been deleted "as part of a redaction effort due to requests from affected employees". Invalid email addresses were converted to something of the form user@enron.com whenever possible (i.e., recipient is specified in some parse-able format like "Doe, John" or "Mary K. Smith") and to no_address@enron.com when no recipient was specified.

This dataset has been distributed as a resource for researchers who are interested in improving current email tools, or understanding how email is currently used. This data is valuable because it is the only substantial collection of "real" email that is public, and particularly appropriate for the purpose of the C4PM validation, because C4PM requires no-structured data. The dataset version used in this paper is from May 7th, 2015 Version of dataset (about 1.7Gb, tarred and gzipped) [19] [20].

 

3- At places, the sentences are formulated in such a complex (long) manner that it is difficult to follow the arguments. Please, check and revise. One such example is line 544... some automatically process...??

A native English speaker has reviewed the paper.

Round 2

Reviewer 2 Report

My Comments and Suggestions for the Authors are given below:

The article was messy and not well organized. The same topics and basic information are explained in an unnecessarily long way in Introduction, Section 1, and Section 2. The article should be significantly shortened and reorganized concisely and apprehensibly.

The authors present a comparative table of sixteen different process mining software tools in Table 1. When the table is examined, it is understood that almost all tools have similar structures for process mining. Based on this comparison, the authors claim or argue that they make a sustainable contribution to process mining, especially with the use of structured and unstructured data. However, the article seems to focus on text mining rather than its contribution to process mining. The data sets used in the article could not be brought into the scope of process analysis and process mining. Within the scope of the revision, a new table of the most frequently used words and identified assets by the employee has been prepared in Table 3. The 60 words most frequently used by all 16 employees whose e-mails were analyzed are highlighted in bold. The work done in the article is essentially exactly text mining. How the identified assets on the right side of the table were obtained is still not clearly explained.

Based on this, how the assets defined in Sections 4.1 and 4.2 are determined is not fully understood and cannot be underlined. As a result, the original contribution of the study to process mining remains limited and insufficient. As a result, the original contribution of the study as a sustainable complement to process mining methodologies remains limited and inadequate.

Comments for author File: Comments.pdf

Author Response

1- The article was messy and not well organized. The same topics and basic information are explained in an unnecessarily long way in Introduction, Section 1, and Section 2. The article should be significantly shortened and reorganized concisely and apprehensibly.

Following your recommendations, it has been done. Thanks.

 

2- The authors present a comparative table of sixteen different process mining software tools in Table 1. When the table is examined, it is understood that almost all tools have similar structures for process mining. Based on this comparison, the authors claim or argue that they make a sustainable contribution to process mining, especially with the use of structured and unstructured data. However, the article seems to focus on text mining rather than its contribution to process mining.

 

Thanks for the comment we have explained that in the paper.

 

The main problem with process mining is that it is being able to deal with processes that are already digitalized, so in digitally immature companies (most of the existing companies around the world) process mining cannot be applied. Besides, for many companies, process mining tools are not affordable. With C4PM we allow those less digitally matured companies, to identify their intangible assets, and those intangible assets make more efficient the finding of important processes that are working in the company and that are susceptible entries for the process mining methodologies.  

3 - The data sets used in the article could not be brought into the scope of process analysis and process mining.

Thanks for the comment we have explained that in the paper.

Neither process analysis nor process mining can work with unstructured or semi-structured data, which are the kind of data that most of the existing organizations have. This paper is not making a contribution to process mining, but it is providing a complement, so C4PM can identify intangible assets, which can lead to the identification of processes that can potentially be input for process mining. Covering in this way a very difficult activity which is, the discovery of the things that the company doesn’t know that has or is able to do.

4- Within the scope of the revision, a new table of the most frequently used words and identified assets by the employee has been prepared in Table 3. The 60 words most frequently used by all 16 employees whose e-mails were analyzed are highlighted in bold. The work done in the article is essentially exactly text mining.

Thanks for the comment we have explained this in the paper.

C4PM is based on three steps. Just part of the third step is based on text mining. The rest of the process is based on the manual identification of potential knowledge assets based on the documents that companies own and aligned with the company's business goal. The text mining performed over emails has been quite useful to identify the most frequent words that pointed out potential intangible assets, and digging into them, can be discovered processes that can be potentially, an input to process mining.

5- How the identified assets on the right side of the table were obtained is still not clearly explained. Based on this, how the assets defined in Sections 4.1 and 4.2 are determined is not fully understood and cannot be underlined.

Thanks for the comment we have explained this in the paper.

“Most frequent words” provide a source of very useful information. Once they have been identified, the software engineer can be more efficient in following the leads provided by those words, and can, in a more efficient way, dig into the organization to identify procedures and processes and then check their digital maturity to realize whether or not the identified process can be an entry to the process mining methodology. Identified Assets column in table 3 has been obtained by digging into the most frequent words by analyzing the public documentation of the Enron company.

6- As a result, the original contribution of the study to process mining remains limited and insufficient. As a result, the original contribution of the study as a sustainable complement to process mining methodologies remains limited and inadequate.

Thanks for the comment we have explained that in the paper.

In view of the existing deficiencies in the field of BI and in the current process mining tools to identify intangible assets hidden in structured of semi-structured data and aligned with the companies´ business goals, the method that we propose in this paper can cover both deficiencies and effectively creating a source of data that can be useful as an entry to process mining methodologies without the use of existing process mining tools, and is affordable for companies with less digital maturity.

With C4PM, IT professionals have an effective, holistic, systemic, iterative, and GIA-driven method to identify the information and knowledge assets that will ultimately support digital processes.

C4PM is a new method that can be enacted by companies of any size or sector, and is useful for pre-processing unstructured data that are available to most companies. Properly processed, these data are extremely valuable for discovering an organization’s hidden intangible assets that can lead to the discovery of companies processes that can be a potential input for process mining methodologies. Besides, the identification, and appropriate reuse, of these intangible assets, that is, pieces of organizational knowledge, discovered by applying C4PM, leads to a more productive and sustainable organization. Indeed, it has been proven that knowledge reuse improves the productivity and sustainability of businesses on the basis of “do it taking advantage of what you have”. In sum, information is power, an organization that uses mechanisms to reuse its own information and discover its intangible knowledge assets can materialize the power of information.

C4PM can output a list of all the critical intangible business assets discovered based on the existing (automated or non-automated) organizational processes. These organizational processes will be susceptible to being properly detailed according to the knowledge of the people performing the activities conforming to the processes. The output of C4PM provides potential input for process mining and reduces noisy data used in the processes analyzed by process mining discovery software tools. By linking C4PM with process mining, we ensure that the implementation of the process mining project is properly aligned with company's strategic and business objectives from the very beginning. As a result, the process mining project can be implemented more efficiently, because it will be driven by the intangible assets identified with C4PM.

Round 3

Reviewer 2 Report

Although the authors stated that they prepared the article with the aim of being a sustainable complement to process mining methodologies within the framework of the C4PM method, the results obtained do not fully support this idea. From the email datasets of Enron, used as an example in the case study, it is understood that the identified assets in the last column of Table 3 and the defined assets described in Sections 4.1 and 4.2 are more suitable for text mining. The data analysis performed in the case study could not establish a sufficiently robust relationship that supports the process mining.

Author Response

In view of the reviewer comment, we think that maybe the title of the paper should be changed, avoiding focusing just on process mining (PM), provided that the output of the proposed method can be an entry into process mining or business intelligence (BI) projects. The key to the current proposal is that it can be used as a previous step for BI or PM to discover intangible assets aligned with organizations business goals, that both BI and PM are unable to discover per se.

 

The data analysis performed in the case study has shown the capability of the proposal to identify hidden intangible assets aligned with the company business goals, which identification is essential for any sustainable transformation to be done. We know that it is not supporting process mining because it is not the intention of the work done. The proposal is intended to output a set of intangible assets (knowledge assets) that can point out potential processes that can be an entry to process mining. Without the identification of those intangible assets, process mining can operate just on a narrow set of processes for which the organization has already logs of information, but is totally blind to any other process less mature.

 

The main problem with process mining is that it is being able to deal with processes that are already digitalized, so in digitally immature companies (most of the existing companies around the world) process mining cannot be applied. Besides, for many companies, process mining tools are not affordable.

C4PM can identify intangible assets, from unstructured data, which can lead to the identification of processes that can potentially be input for process mining. Covering in this way a very difficult activity which is, the discovery of the things that the company doesn’t know that has or is able to do, its knowledge assets, that neither Business Intelligence nor Process Mining Methodologies are able to discover.

Unstructured or semi-structured data, properly processed, are extremely valuable for discovering an organization’s hidden intangible assets that can lead to the discovery of companies processes that can be a potential input for process mining methodologies. Besides, the identification, and appropriate reuse, of these intangible assets, that is, pieces of organizational knowledge, discovered by applying C4PM, leads to a more productive and sustainable organization, because, it has been proven that knowledge reuse improves the productivity and sustainability of businesses on the basis of “do it taking advantage of what you know”. In sum, information is power, an organization that uses mechanisms to reuse its own information and discover its intangible knowledge assets can materialize the power of information, being more productive and sustainable.

The output of C4PM provides potential input for process mining and reduces noisy data used in the processes analyzed by process mining discovery software tools. By linking C4PM with process mining, we ensure that the implementation of the process mining project to be done is properly aligned with the company's strategy and business objectives from the very beginning. As a result, the process mining project can be implemented more efficiently, because it will be driven by the intangible assets identified with C4PM.