Deformable image registration (DIR) [1
], i.e., the process of searching for the optimal non-linear transformation to align two images, plays an increasingly important role in radiotherapy [2
], with applications ranging from radiotherapy planning [3
], dose accumulation [4
], contour propagation [5
], to radiotherapy response monitoring [6
One of the challenges of DIR in clinical practice concerns the parameter choices that need to be made for each registration instance, since the success of most registration methods depends on setting a variety of parameters well (e.g., the weights in the cost function, the number of registration levels, the control point grid spacing). The weights of the cost function are especially important, as they determine the trade-off between all the objectives of interest, including the effect of regularization of the DIR problem, which is necessary, as DIR is inherently ill-posed [7
Optimizing these parameters for each individual DIR instance is challenging in clinical practice. Having a class solution for DIR, i.e., a configuration of parameters for which DIR performs well on all instances of a DIR problem, would facilitate wide-scale clinical application. Although several approaches have been proposed [8
], still, often, parameters are manually tuned for each case of the DIR problem separately via trial-and-error adaptations, followed by visual inspection of the registration outcome.
Finding a class solution may, however, be impossible for challenging DIR cases, as it was shown that preferred parameter settings can vary greatly even for different instances of the same type of registration problem [9
]. Different parameter configurations lead to different DIR outcomes, due to different trade-offs between the objectives of interest in DIR, such as similarity between the images and the amount of deformation.
The fact that different trade-offs may be preferable for different instances of a DIR problem indicates that DIR, although currently typically solved as a single-objective optimization problem, where all objectives of interest are combined in one cost function, is inherently a multi-objective optimization problem. In multi-objective optimization, the objectives of interest are not combined into one cost function, but optimized simultaneously, resulting in a set of DIR outcomes that represent high-quality trade-offs between the objectives of interest. If the multi-objective problem is solved to optimality, then this set of DIR outcomes is called the Pareto front. In cases where the Pareto front is not known (such as in the case of DIR), it can be said that a non-dominated front is obtained, i.e., a front where no solution is better in both objectives than any other solution in this front. Recently, it was shown that using a patient-specific multi-objective optimization approach can be useful for challenging DIR cases, for which manual tuning becomes far more complex [10
]. This multi-objective, patient-specific approach can also be considered an online tuning approach: online in the sense that here, the tuning algorithm is run to search for the best parameter settings for the DIR algorithm for a specific patient. This approach removes the challenge of manual parameter tuning and provides the expert with a non-dominated front of DIR outcomes. Such a solution can be navigated to find a preferred DIR outcome insightfully. However, it can be computationally expensive, as it involves running a search algorithm on top of a DIR algorithm every time a DIR case needs to be solved. Furthermore, the more parameters to be tuned, the more time consuming this process becomes [12
In this work, we consider an alternative approach for efficient parameter tuning: an evolutionary multi-objective machine learning approach that computes, in an offline training phase, a multi-objective class solution, i.e., a set of parameter configurations, that, when used on any instance of a DIR problem, yields DIR outcomes that approximate the non-dominated front. To this end, we re-design the parameter optimization problem, and we evaluate the performance of the evolutionary machine learning approach by performing a comparison with the method presented in [10
]. Since computing the multi-objective class solution needs to be performed only once, it is then sufficient for a new DIR instance to run multiple DIRs straightforwardly using the class solutions to obtain a navigable set of DIR outcomes for that instance. These runs can be executed in parallel, resulting in a large reduction in computation time, compared to a patient-specific multi-objective tuning approach.
We developed and tested our evolutionary multi-objective machine learning approach on two breast magnetic resonance (MR) DIR problems of different levels of complexity. The first one is prone-prone breast MR DIR of patients undergoing pre-operative partial breast radiotherapy. Registering images of the patients before and after therapy can be used to monitor therapy response. The second DIR problem is prone-supine breast MR DIR (in this work, data from healthy volunteers are used), which presents a significantly higher challenge due to the very large deformation occurring between the two positions (i.e., lying face down versus lying face up). Prone-supine DIR can be used to map pre-surgical diagnostic imaging to the radiotherapy planning geometry, e.g., for radiotherapy after breast-conserving surgery.
We presented an evolutionary multi-objective class-solution learning approach for DIR, which we applied to breast MR DIR problems of increasing difficulty. We used our approach to learn weights typically used in DIR software, but the approach was general and could straightforwardly be used to consider other parameters, as well. The multi-objective class-solution approach presented in this work kept the benefits of a multi-objective perspective [10
] and was more efficient than multi-objective patient-specific parameter optimization, by allowing parallel computation of DIRs without any further tuning or optimization, thereby much more so facilitating the use of multi-objective DIR in clinical practice.
Although the concept of a class solution implies the existence of a unique solution that solves all instances of a particular problem well, we considered a set of class solutions, i.e., a multi-objective class solution, to be more appropriate for the problem of DIR. The idea that a single trade-off can work for all DIR cases of a certain type comes from a single-objective optimization perspective, but even among registration problems of the same type, parameter settings that give the desired results can differ [10
]. A multi-objective optimization approach instead has the ability to capture high-quality trade-offs among which the preferred outcome is likely present. Moreover, selecting this outcome is insightful and easy [28
]. It was also observed that what the users thought was a high-quality solution coincided with a mean TRE (very) close to the minimum mean TRE obtained, indicating that the results obtained by our method can be considered reliable.
The evaluation of the quality of the DIR outcomes, in the absence of a ground truth, relied on the annotated landmarks, which has known limitations (it may not be representative of the quality of the deformation throughout the entire volume). A realistic phantom-based study could potentially circumvent this, but such an elaborate study is out of the scope of this work. We further saw that when the solutions were visually inspected by observers, what the users thought was a high-quality solution coincided with a mean TRE (very) close to the minimum mean TRE obtained [28
], indicating that the mean TRE was representative of the overall quality of the solutions.
Since EAs are state-of-the-art for multi-objective black-box optimization and given the limited amount of data present for the DIR problems studied in this work, the choice of an EA as a machine learning technique, as opposed to more widely-used machine learning approaches such as, e.g., convolutional neural networks, seems reasonable. Using the EA to tackle this problem, even in the presence of very limited data, was possible since the problem concerned a limited number of parameters (i.e., a low-dimensional manifold) to be learned and the learning of point sets. The EA employed in this work was specifically chosen as it was designed for continuous, multi-objective problems, and it has been shown to perform well for those problems [25
]. Given that the bulk of the computation was in the DIRs during meta-optimization and given the performance of this EA on different benchmark problems compared to other EAs, we do not expect that choosing a different EA as a meta-optimizer would yield significant differences in terms of computation time.
The results of this approach, which may be considered an evolutionary multi-objective machine learning approach [31
], indicate that deriving a multi-objective class solution for DIR is feasible, even with a limited amount of data, for DIR problem classes that are solvable and for which the variation within the class is not too large, as in the case of prone-prone breast MR image registration. It may be feasible even for quite challenging DIR problems, as in the case of the prone-supine DIR problems, provided that the underlying DIR method, on which our approach was ultimately dependent, can actually solve the DIR problem, and the variation in (quality of) the images and their content was limited.
Further, for our approach to work properly, it was necessary to perform a classification of prone-supine DIR instances, into two groups, one with larger and one with smaller deformations. It may be challenging to always classify correctly the prone-supine DIR instance at hand. One solution could be to pre-process the data. For instance, classification could be performed by quantifying breast volume (e.g., with breast cup size) and/or based on age. DIR cases with small breast volumes (and therefore with limited deformation present) and of young patients, which tend to have breasts with more fibroglandular tissue (therefore with more information present in the images), are most likely easier to solve.
Further research could involve using the approach presented in this work to tune multiple parameters, with very few parameters subsequently tuned patient-specifically, to perhaps obtain even higher quality solutions for challenging DIR problems. In the case of tuning multiple parameters, the computational benefit of using a class solution approach instead of a patient-specific approach increases. Moreover, a so-called adaptive steering mechanism in the EA could be incorporated in order to obtain more solutions in the region of the non-dominated front that is of interest [32
]. Lastly, we note that the number of DIRs to be performed and the total online computation time can potentially be reduced by selecting a diverse, but front-spanning subset of the multi-objective class solution of a limited, pre-defined size.