Change Point Detection Using Penalized Multidegree Splines

Round 1

Reviewer 1 Report

Overall impression of the manuscript is good.

There are at the beginning cited a series of textbooks without providing section/page. The manuscript were prepared for diversity and transferred to axioms. Such a jump. anyway, judging by the references the suitability of the manuscript for the journal is in question. Nothing listed at references from mathematics or from axioms.

"Author Contributions: These authors contributed equally to this work.. All authors have read and agreed to the published version of the manuscript." - even so, the type of the contribution of each author must be specified.

"Since the input variable is a date and not numeric data, 144 sample sizes between 0 and 1 were extracted at equal intervals for the proposed method QP implementation." - please provide an example of a sample of size 1 and of a sample of size 0.

"The data consists of 144 observations. This has to do with the subprime mortgage crisis. This crisis, which took place between 2007 and 2008, refers to a series of economic crises that started with the bankruptcy of the largest mortgage lenders in the United States and caused a credit crunch not only in the United States but also in the international financial market." - please list an example of an observation.

"This data obtained from Investing.com shows the exchange rate change of the ISK per the USD from 2004-01 to 2015-12, which was measured as a monthly average. " - please list an example.

"Again, our estimator gave the best results. " - please specify your estimator.

"In this section, we illustrate the performance of the proposed method on the simulated examples. We generate the predictor as sample size n sequences between [0, 1]. εi is generated from N(0, σ2) for i = 1, . . . , n and σ is set 0.25 for all examples." - where is εi in your examples, please?

The authors needs to mention the software in which they implemented the "
Algorithm 1: Coordinate Descent Algorithm (CDA"

"Hence we use Bayesian Information Criterion (BIC) [17], which is the model selection criteria as defined as follows:" - BIC is one possible alternative and should be mentioned that are also other information measures in current use (DOI 10.3390/w12082075, 10.1100/tsw.2009.131, 10.3390/e22121400). 

Connection with axioms and mathematics must be improved.

Author Response

We are thankful for the constructive critiques. We present detailed responses to the reviewer's set of comments with respect to the subject manuscript. All of the reviewer's points have been addressed. We hope you find this manuscript publishable in this form.

Please see the attachment.

Author Response File: Author Response.pdf

Reviewer 2 Report

The authors propose change point detection through fitting the data with a linear combination of a polynomial and a truncated power spline. They develop a coordinate descent algorithm to estimate the coefficients and compare it with quadratic program solvers. They fit synthetic and real data with the proposed function and several comparison methods, and evaluate the capabilities of the proposed method.

My biggest concern with the submitted manuscript is that a research article has to have a point, while this manuscript has two, and both are quite weak. The first original contribution seems to be the coordinate descent algorithm, but it may be just a small adaptation of methods already used and published - no context information is provided. The other one is the proposed estimator (fitting function), which is compared with some standard alternatives in a not very convincing way.

In the chapter 3. Implementation, particularly in 3.1. Coordinate descent algorithm, it is not clearly specified what is new and what is taken from the literature (there is a lack of cited references). The distinction between the optimization problem and the optimization algorithm is not clearly made, but it should be. The optimization problem consists of the objective function and the constraints, the optimization algorithm serves to find the solution of the given optimization problem. Coordinate descent algorithm is an optimization algorithm. Quadratic program is a type of optimization problem. You solve the QP using the algorithms in quadprog package of R.

The lines from 110 on seem to imply that the derived quadratic program is equivalent to the original optimization problem from Eq. (2) which is solved using CDA. Please specify explicitly whether this is so (or isn't so). It is very important to know whether you are comparing optimization methods through solving the same optimization problem with them, or comparing different optimization problems, different definitions of what a spline that fits well is. I cannot interpret Figure 1 without it - are you fitting a good red and bad blue function to the data, or are you fitting the same function twice, just doing a poor job in the blue case?

The comparison of your method with the others in 4.1. Simulation seems to give an unfair advantage to your method, as the functions being fitted are themselves linear combinations of a polynomial and a truncated power spline. They are of the same form as the fitting function. I don't think that FT, TF, SS, and ST use the same family of functions for fitting. Please comment on that, add some discussion to the text. The problem with 4.2. Real data analysis is that we do not really know where the change point is. On the graph, it sure looks like it is September 2009, as the biggest monthly jump is there - one may even say that it is obvious. But why use such a sophisticated method to find the obvious? How would your method perform in non-obvious cases, and what do these tests tell you about it? Do you envision a possible use case for your method? Discussion on these questions would greatly increase the value of the article.

The instructions for authors https://www.mdpi.com/journal/axioms/instructions#suppmaterials require "Data Availability Statements provide details regarding where data supporting reported results can be found, including links to publicly archived datasets analyzed or generated during the study" and "For work where novel computer code was developed, authors should release the code either by depositing in a recognized, public repository or uploading as supplementary information to the publication. The name and version of all software used should be clearly indicated." And also "Give the name and version of any software used". The goal is to enable the reader to easily repeat, verify, and continue your work, so the "Not applicable" excuse you provide is not acceptable. You should provide your software code in such a form that the reader can download it, run it, and re-generate the results. Provide also the simulated data or the code used for generating it together with the parameters (preferably both), and provide a proper reference to the data from Investing.com. Clarify the versions of R and the libraries that you used so the reader knows what to run your code with .

A related issue is that R and its packages (including quadprog) should be properly cited and listed in the references.

I see that the name of the journal "Diversity" repeats a lot throughout the manuscript. Have you forgotten to change the template or have you actually submitted the manuscript to Diversity too (which you most definitely shouldn't)?

In line 12, you mention a "relationship between two variables". In general, it isn't necessary that there are two, it is just your particular choice. Please clarify it.

In line 13, you mention "the two relationships". What do you mean? Please clarify.

The references [5] and [6] are to whole books, which is not acceptable for supporting such claims. Please let the reader know which chapter (or page) of the book provides the result you are using.

Also regarding references, "DOI numbers (Digital Object Identifier) are not mandatory but highly encouraged."

Line 31 mentions a curve that is "almost perpendicular and parallel". How can it be both at the same time? And what are the "overlapping intervals" you are talking about (I don't know it after reading the article)? Please make the text clearer and more relevant to the point you are making in this manuscript, don't be just listing unrelated facts.

In 33-34, you mention that "the objective function to be optimized is mainly a convex or concave function." The disturbing word is "mainly" - is it or isn't it? Are you doing convex optimization or not, or are you not sure? Convex optimization is a small and very important subset of general optimization, so if you are doing optimization, you should be very certain whether it is convex or not.

Particularly in the paragraph from line 44 on, you are using [numbers] as if they were words in sentences, which is distracting. I suggest saying something like "Osborne et al. [9] propose ..." instead of just "[9] propose ..."

In line 80 (and elsewhere), you promise comparison of CDA and QP, which does not make sense, as one is optimization method and one is optimization problem (I'm sure one can use CDA to solve a QP, and you may actually be doing it, while quadprog "implements the dual method of Goldfarb and Idnani (1982, 1983) for solving quadratic programming problems" as stated in https://cran.r-project.org/web/packages/quadprog/quadprog.pdf). Also, why cover both "algorithms" in detail? Cover in detail what is original. And tell us what the original contribution is, please.

In line 163, the sentence ending in "it couldn't" seems to be missing something as it seems to be meaningless.

The knots t are not really discussed. How do you place them? Equidistantly or are they optimized?

Even though the amount and the annoyingness of the comments may not show it, I am very supportive of publishing the work presented in the manuscript. I think the study was performed well and the results can be informative, even if they are not revolutionary. However, if the article is to be meaningful, the results have to be provided in a way that is easy to understand, repeat and reuse, and with enough insightful discussion. Nobody will care about some random results you obtained when trying out some random things; someone may care a lot about a method they can reuse in their work, but they will need enough context to notice that it is relevant and enough detail (preferably the computer code as per the instructions of the journal) to be able to reuse it.

Author Response

We are thankful for the constructive critiques. We present detailed responses to the reviewer's set of comments with respect to the subject manuscript. All of the reviewer's points have been addressed. We hope you find this manuscript publishable in this form.

Please see the attachment.

Author Response File: Author Response.pdf

Round 2

Reviewer 1 Report

I am not satisfied nor with the changes and the current version of the manuscript. In my opinion the manuscript needs another round of review or , more exact, a revision & resubmission.

I don't see the reason for the redundant references "See 16 [1], page 118-180 of [2], chapter 2 of [3], chapter 1 of [4]." and the authors did not bothered to justify why the reader should go to each of those refs.

The authors should revise "belongs to an infinite-dimensional parametric space" which is now in conflict with the newly introduced eq.1.

Regarding eq.1, the authors must argue why the distribution is not specified for εi.

For the not numbered eq. in between eq.1 and eq.2 the range for k is omitted.

According to usual notations, for the norms the index is in upper position  (
$ \ell^1$) usual notations, for the norms the index is in upper position while for distances is in lower position ($ \ell_1$). The authors must revise their notations (l.87, l.90, l.91).

It is an error in the statement: "Algorithm 1 represents the implementation of the proposed CDA with the software program R" - 'the software program R' DOES NOT IMPLEMENTS ALGORITHMS, only programs, and yours is not an R program!

In the not numbered equation at the beginning of 3.1.3, the dimensionality of 1, X, ..., X^d is not given.

The story between lines 121 and 122 needs references.

In the eqs defining u_+ and u_- please consider to provide non-overlapping intervals.

One equation is moving out of the page.

Please consider numbering of all relevant equations.

The meaning of Dmat, dvec, bvec, and Amat is not given.

In the text the need for the large Figure 1 is not given. Why the authors has given a large number of representations with different number of interior knots? The justification must appear in the text, otherwise the Figure 1 must be reduced to only one representation.

I am sure that the authors may want to enrich their cited literature.

Please consider making all requested changes and/or additionally checking of the rest of the manuscript for the sake of shortening the reviewing process.

Author Response

We are thankful for the constructive critiques. We present detailed responses to the reviewer's set of comments with respect to the subject manuscript. All of the reviewer's points have been addressed. We hope you find this manuscript publishable in this form. We also made English language editing in this revision.

Please se the attachment.

Author Response File: Author Response.pdf

Reviewer 2 Report

You have done a very good job revising the manuscript, now I find it much more "user-friendly". I still cannot get rid of the impression that the presented examples are designed to suit your method better than FL, TF, SS, and ST (can SS really fit non-continuous functions?) but I'm not qualified to claim that it is so. If it is not the case, your results are truly amazing!!!

The only remaining suggestions I can give you (or ideas I can share) are:
- Revise the abstract, as it has not been updated to reflect the new reality of the manuscript. It is worth doing it, as the abstract is the most read part of the article, and the manuscript is good!
- Line 31: instead of "basis are highly correlated", it should probably say "basis functions are highly correlated".
- Line 144: the mention of "global minimum" worries me, as such phrasing typically implies that the method converges to a "wrong" local minimum. If the optimization problem being solved is convex, there is only one local minimum that is also the global minimum and can be called simply "minimum". Please double-check that your optimization problem is convex, ad if it is so, remove the "global".
- The supplementary file is great, as it actually works right away and produces a graph! However, it doesn't work perfectly for me: if I run "Rscript code.R", I get the error "could not find function "qp_sim"" - does some package have to be loaded? If you can solve the issue, that would be great. I don't think the provided code calculates the criteria for FL, TF, SS, and ST - if you can add that easily enough, I'm sure it would be appreciated by the readers as well.

Author Response

We are thankful for the constructive critiques. We present detailed responses to the reviewer's set of comments with respect to the subject manuscript. All of the reviewer's points have been addressed. We hope you find this manuscript publishable in this form. We also made English language editing in this revision.

Please see the attachment.

Author Response File: Author Response.pdf

Round 3

Reviewer 1 Report

The authors made some progress with their manuscript. 

The authors still needs to support and argue the connection of their manuscript with the Axioms journal.

As it is now have a very little in common with the journal.

Conclusions are very ambiguous about who with who is compared and what sort of result is extracted from the comparison.

Author Response

We are thankful for the constructive critiques. We present detailed responses to the reviewer’s set of comments with respect to the subject manuscript. All of the reviewer’s points have been addressed. We hope you find this manuscript publishable in this form.

Please see the attachment.

Author Response File: Author Response.pdf