Trip Purpose Imputation Using GPS Trajectories with Machine Learning

Round 1

Reviewer 1 Report

This paper discusses a trip purpose imputation technique using several machine learning algorithms. I have a few comments as follows.

• The evaluation of machine learning techniques is quite weak as only one single dataset was used.
• All machine learning algorithms are the existing ones; thus, the originality of this work is still questionable.
• Figure 3 is not clear to me. The caption should be made clearer while the map itself is not too informative. 
• The selection of base classifiers for constructing an ensemble is not clear. Why those classifiers were chosen? There are no criteria for selecting them. 

Author Response

Thanks very much for your comments. Below are our point-to-point replies.

This paper discusses a trip purpose imputation technique using several machine learning algorithms. I have a few comments as follows.

Comment 1: The evaluation of machine learning techniques is quite weak as only one single dataset was used.

Answer 1 : Thanks. We agree that it would be better to have multiple datasets to evaluate the model performance. However, it is difficult to get comparable labelled datasets with similar temporal and spatial resolution/size in the transport domain. Futhermore, considering that our dataset contains a large number of participants and recorded activities, and that we evaluate our models in different ways (e.g. comparing different algorithms and splitting the training datasets using different criteria), we would argue that the evalution of our approach supports the conclusions made in the paper .

Comment 2: All machine learning algorithms are the existing ones; thus, the originality of this work is still questionable.

Answer 2 : Thanks. Indeed, this paper does not propose a new algorithm. However, as addressed in the paper, one of our innovative points that is related to algorithms is the combined use of multiple algorithms and the ensemble filter to account for mislabelling by participants with respect to this particular domain problem of activity purpose labelling. This method has not been used in the transport research field despite the issue of mislabeling being an important one, which can be attibuted to the smaller datasets in previous studies.

Comment 3: Figure 3 is not clear to me. The caption should be made clearer while the map itself is not too informative. 

Answer 3 : Thanks. We have modified the caption to make it clearer. In case any remaining ambiguity, please indicate which element needs to be improved and we can make additional changes.

Comment 4: The selection of base classifiers for constructing an ensemble is not clear. Why those classifiers were chosen? There are no criteria for selecting them.

Answer 4 : Thanks for this comment. These algorithms are selected because of better performance out of a series of algorithms available in the R language. More than 20 algorithms which are mainly provided by the caret package (https://topepo.github.io/caret/index.html) were tested in our preliminary study and we found the four chosen algorithms are superior to others in our case.  We have now clarified this point in the last paragraph of Section 3.2.

Reviewer 2 Report

This study imputed trip purposes using machine learning based on GPS trajectory data in Switzerland. Multiple data mining techniques and classification algorithms were adopted to evaluate their performance and applicability. Overall, the paper is interesting and novel. I only have several questions and suggestions:

• The title of Section 2 should be changed to “Literature review”.
• What fields and information of data are collected from the 3689 Swiss participants? It is suggested to present some sample data on the GPS trajectories to show the data structure.
• The Cluster-based features should be further elaborated. How did the authors cluster the activities?
• Why did the authors choose a random forest algorithm as the baseline? It should be further explained.
• Did the authors consider applying the neural networks to this dataset?
• Do the curves in Fig.5 represent the average accuracy of the four classification algorithms? It should be clarified.

Author Response

Thanks very much for your comments. Below are our point-to-point replies.

This study imputed trip purposes using machine learning based on GPS trajectory data in Switzerland. Multiple data mining techniques and classification algorithms were adopted to evaluate their performance and applicability. Overall, the paper is interesting and novel. I only have several questions and suggestions:

Comment 1: The title of Section 2 should be changed to “Literature review”.

Answer 1 : Thanks. It has been changed.

Comment 2: What fields and information of data are collected from the 3689 Swiss participants? It is suggested to present some sample data on the GPS trajectories to show the data structure.

Answer 2 : Thanks. The data collected from the participants only include two parts: 1) GPS trajectories. They are composed of a series of longitude-latitude pairs with timing information.  (Such data are processed by the Motion Tag app: If a participant stay in one place for more than 5 minutes, this period is regarded as having an activity.) 2) The socio-demographic information, which are displayed in the first column of Table 2. Some sample data are not presented mainly because the tracks of daily life of people are quite intuitive and do not provide much relevant information regarding our targeted research questions. If it is still suggested to have such example figure, we would happily add one.

Comment 3: The Cluster-based features should be further elaborated. How did the authors cluster the activities?

Answer 3 : Thanks for reminding us, it is a very good point. The cluster-based features are extracted using the hierarchical clustering, which is described in detail in the second paragraph of Section 3.2.

Comment 4: Why did the authors choose a random forest algorithm as the baseline? It should be further explained.

Answer 4 : Thanks. It is indicated in Section 2.3 that: “Because of the good performance of random forests compared to other methods demonstrated by numerous studies [32-34], we employed it as a starting point for analysis.” Besides, such superior performance is also reflected through our analysis. So we argue that random forest is quite robust in terms of such task.

Comment 5: Did the authors consider applying the neural networks to this dataset?

Answer 5 : Thanks. Yes, we also tried neural networks for this dataset. We have to admit that neural networks/deep learning are quite versatile in many studies, including the one on trip mode imputation cited in our paper [citation 7]. However, in our case, the data structures are much simpler, the use of neural networks did not give satisfactory results as expected. Maybe more efforts is required to have better results with neural networks, but we do not regard it as our major research direction.

Comment 6: Do the curves in Fig.5 represent the average accuracy of the four classification algorithms? It should be clarified.

Answer 6 : We agree that it should be clarified. The accuracy rate is calculated based on the classification results which are voted by the four algorithms.

Reviewer 3 Report

The manuscript is well written and presented.

Minor comments:

• it is understood that the unlabeled data (in table 1) are not used. Can you please confirm and clarify in the text?
• the fact that not considering the activity-based features results in better performance for some classes (figure 2) deserve some explanation. Do you feel that those data not correlated with the travel purpose? Do you fell there might be a significant number of wrong labels?

Author Response

Thanks very much for your comments. Below are our point-to-point replies.

The manuscript is well written and presented.
Minor comments:

Comment 1: it is understood that the unlabeled data (in table 1) are not used. Can you please confirm and clarify in the text?
Answer 1 : Thanks. Yes, it is true and we mentioned this aspect in the first sentence of Section 4.1: “The performance of random forests can be measured through OOB error rates without splitting the training and test dataset and implementing cross-validation, so we use only labelled data as training data in this subsection for supervised machine learning.”
To make it clearer, we also now mention it in the Section 3.1 Material: “Considering solely the 91% of all activities that are within Switzerland, it amounts to 1.82 million activities above a time threshold of 5 minutes, of which 43% is labelled by participants and used in our following experiments.”

Comment 2: the fact that not considering the activity-based features results in better performance for some classes (figure 2) deserve some explanation. Do you feel that those data not correlated with the travel purpose? 
Answer 2 : Thanks. It is really an interesting phenomenon that without activity information, the model performs better. It is clarified in the text:
“It is interesting that the elimination of activity information gives similar or even slightly better results compared to using all features, which might result from the intercorrelation or interaction among features.
However, the activity information indeed improves the model performance when only cluster-based information is used (not shown), which means the activity information is related to the travel purpose.”
We suspect that such phenomenon is related to information saturation where added information does not provide additional performance improvement, but we do not go deep into this direction.

Comment 3: Do you fell there might be a significant number of wrong labels?
Answer 3: Thanks. The wrong labels are a focus of this study. It is hard to quantify such phenomenon or identify the cases which are wrongly labelled. Therefore, it has until now not been investigated in the transport research field. 
In this study, we used ensemble filters that adopt an ensemble of classifiers to eliminate the mislabelled training data that cannot be correctly classified by all or part of the classifiers using n-fold cross-validation.
Although it is not perfect and might be biased by the selected classification algorithms, such research is a significant step towards more realistic trip purpose imputation and better usage of the collected data.