3D Ultrasound Mosaic of the Whole Shoulder: A Feasibility Study

Abstract

A protocol is proposed to acquire a tomographic ultrasound (US) scan of the musculoskeletal (MSK) anatomy in the rotator cuff region. Current clinical US imaging techniques are hindered by occlusions and a narrow field of view and require expert acquisition and interpretation. There is limited literature on 3D US image registration of the shoulder or volumetric reconstruction of the full shoulder complex. We believe that a clinically accurate US volume reconstruction of the entire shoulder can aid in pre-operative surgical planning and reduce the complexity of US interpretation. The protocol was used in generating data for deep learning model training to automatically register US mosaics in real-time. An in vivo 3D US tomographic reconstruction of the entire rotator cuff region was produced by registering 53 sequential 3D US volumes acquired by an MSK sonographer. Anatomical surface thicknesses and distances in the US mosaic were compared to their corresponding MRI measurements as the ground truth. The humeral head surface was marginally thicker in the reconstructed US mosaic than its original thickness observed in a single US volume by 0.65 mm. The humeral head diameter and acromiohumeral distance (ACHD) matched with their measured MRI distances with a reconstruction error of 0 mm and 1.2 mm, respectively. Furthermore, the demonstration of 20 relevant MSK structures was independently graded between 1 and 5 by two sonographers, with higher grades indicating poorer demonstration. The average demonstration grade for each anatomy was as follows: bones = 2, muscles = 3, tendons = 3, ligaments = 4–5 and labrum = 4–5. There was a substantial agreement between sonographers (Cohen’s Weighted kappa of 0.71) on the demonstration of the structures, and they both independently deemed the mosaic clinically acceptable for the visualisation of the bony anatomy. Ligaments and the labrum were poorly observed due to anatomy size, location and inaccessibility in a static scan, and artefact build-up from the registration and compounding approaches.

1. Introduction

Ultrasound imaging (US) is one of the most widely used medical imaging techniques and has been extensively applied in surgery guidance and therapy planning [1,2]. Superior image quality and soft tissue contrast enable the demonstration of structures that cannot be visualised in other modalities, such as computed tomography (CT) [3]. US is also the only diagnostic imaging modality in the literature without reported patient side-effects to date [4]. US systems are cost-effective, portable and have real-time image acquisition capabilities and virtually unlimited resolution [5]. However, it is believed that US, especially for quantitative use, is not being fully exploited in clinical practice [6].

US suffers from several limitations as it cannot image beyond interfaces between materials with large acoustic impedance differences, for example, through bones or air pockets [1,5,7]. US is typically a monodirectional modality, which means that, once the probe has been positioned, it can scan structures only along a specific direction [8,9,10]. Conventional US systems require manual, laborious and expert scan acquisition and interpretation [11,12]. Experienced clinical sonographers routinely perform US examinations using free-hand probes, and adjust their technique, including probe pressure and position, according to the anatomy of interest, while maintaining stable contact. Moreover, the limited field of view (FOV) of conventional US probes adds to the examination duration, as it means multiple probe positions are required to cover the entire area of interest [11]. The lack of ergonomic design in US systems affects both sonographer fatigue and scan quality, with up to 85% of sonographers reportedly scanning in pain resulting from musculoskeletal disorders [13]. Works in the literature attempted to improve the FOV limitation using image registration approaches or by introducing 3D imaging, also known as 4D US if in real-time [14]. These advancements made US the only diagnostic modality that allows volumetric real-time imaging, enabling various medical applications in complex environments, including operating theatres [3]. However, the FOV of 3D US probes remains a significant limitation, and scanning still requires skilled and experienced users; thus, it is often considered an operator-dependent modality [15].

US is clinically used to diagnose shoulder impingement, shoulder instability and rotator cuff disorders [8]. The shoulder US protocols that sonographers use focus on the biceps tendon, subscapularis tendon, supraspinatus tendon, infraspinatus tendon, glenoid labrum, suprascapular notch, acromioclavicular joint and subacromial–subdeltoid bursa [8,9,10]. For each of these landmarks, multiple scan angles are typically employed to minimise the issues reported in the previous paragraphs and ensure that artefacts are properly identified and distinguished from pathology [8,9,10]. Moreover, when assessing the subacromial impingement, dynamic US examination becomes necessary [8,9,10]. This is achieved by having the sonographer monitor the subacromial–subdeltoid bursa while the patient is performing a sequence of shoulder movements [8,9,10].

There are literature reports on the US mosaic reconstruction of the liver and kidneys by using an optically tracked 3D US probe and experimenting with various registration techniques [16]. Ref. [17] registered multiple 3D cardiac US volumes using US probe tracking in order to compare various US volume compounding techniques (volume merging techniques) and assessed the resultant combined volumes with the help of 3D US experienced clinicians. Another example is the use of high-refresh-rate 3D US and its registration to MRI data in specific applications, such as the reconstruction of the intra-articular anatomy of the knee for arthroscopy [18]. Similarly, [19,20,21,22] registered intraoperative 3D US data to preoperative MRI and CT data for assisting and automating surgeries. But the literature on the registration of 3D US images of the shoulder or volumetric reconstruction of the rotator cuff is very limited. US volume registration can be automated [23]. However, registrations typically rely on segmented US images, other imaging modalities and tracking systems which impose limitations on the US image acquisition and add labour to the post-process operations. The amount of work required to segment the external modalities and US images as well as perform the registrations is typically significant [17,19,20,22,24,25,26,27,28]. This work investigates the feasibility of employing manual 3D US-US registration to reconstruct a single clinically viable US volume (a mosaic) of the entire rotator cuff region, which could be used to reduce the clinical competency needed to acquire and interpret shoulder US data and offer a CT/MRI-like view of the anatomical region. This study utilises expert sonographers and clinicians and corresponding participant shoulder MRI data to validate the clinical usefulness and reconstruction accuracy of the mosaic. This way, US can be more viable for pre-operative and intraoperative surgical planning and becomes more display-friendly for patients being examined and novice US learners. This work is preparatory for the ultimate goal of training a deep learning algorithm to automatically register US shoulder mosaics in real-time. The main significance of this study lies in the newfound understanding that by combining information from multiple 3D US volumes, all the necessary structures in the shoulder can be effectively visualised. This insight suggests that with improvements implemented to the hardware and algorithms utilising the newly discovered challenges, once sufficient US mosaic data have been collected for multiple participants, a deep learning-based 3D US registration algorithm may be trained to reconstruct 3D US shoulder mosaics in real-time, addressing all the previously discussed gaps.

2. Materials and Methods

2.1. Data Acquisition and Ultrasound System Parameters

An in vivo sonography was performed on a healthy volunteer’s right shoulder at the US research lab at the Queensland University of Technology (QUT) (Brisbane, Australia). The subject was a 30-year-old female, standing 1.70 m tall and weighing 58 kg, with a BMI of 20.1. She had no history of shoulder operations or injuries, and there were no distinctive features about her shoulder. Ethics approval for data collection was granted by the QUT Human Research Ethics Committee (No. 1700001110). The US data were acquired using a Philips EPIQ7 US system and a VL13-5 3D US probe (Philips Medical Systems, Andover, MA, USA), a mechanically swept 3D linear probe with an FOV of 38 mm × 30 degrees of sweep and a frequency range of (5–13) MHz. The US system parameters were set by a certified sonographer to a shoulder preset, with a penetration depth of 4 cm, wide imaging, XRES image processing and SonoCT compound imaging. The US volume resolution was (512 × 403 × 256) voxels with spacings of 0.1229 mm × 0.107 mm × 0.2405 mm. A corresponding shoulder and upper arm MRI of the same subject was also already available from a previous study [29] using a 3-Tesla MR scanner (Ingenia, Koninklijke Philips N.V., Best, The Netherlands) at voxel dimensions of 0.4 × 0.4 × 0.8 mm. From this, MRI bone and muscle segmentation data were derived and included in this study. Bone segmentations comprised the humerus, scapula and clavicle. Muscle segmentations included trapezius, supraspinatus, infraspinatus, subscapularis, triceps, rhomboid, latissimus, pectoralis major, biceps and deltoid.

2.2. Shoulder Sonography Protocol

The scans were performed by an experienced clinical sonographer and the procedure was divided into three main collections of US volumes (series): inferior, anterosuperior, and posterosuperior (series 1, 2 and 3, respectively). The three imaging series were developed to scan the whole region with contiguous US probe imaging of each subregion, while at the same time avoiding major superficial bony anatomy obstructions. The inferior aspect of the shoulder could be scanned by continually moving the probe on a horizontal plane from the central anterior aspect to the central posterior aspect through the lateral side of the shoulder, as shown in Figure 1. However, the US probe’s limited FOV (38 mm × 30°), and the large anatomical area of the superior aspect of the shoulder, resulted in this region being divided into the anterosuperior and posterosuperior scan series. Scanning the clavicle directly was avoided due to the possible acoustic coupling loss when the probe is in contact with superficial bony structures. However, the bone was captured through probe rotations and steering in series 3 and some parts of series 1. Figure 1 depicts the trajectory followed by the probe for each series, and Table 1 shows the number of volumes acquired, probe direction and starting and ending landmarks for each series. The trajectory was plotted by analysing the video recording of the sonography.

Figure 1. Probe trajectory during sonography for all 3 shoulder series. The (left) and (right) images show the anterior and posterior views of the shoulder, respectively. Bony anatomy is labelled in black, and muscles are labelled in white. The yellow, blue and green probe footprints and arrows correspond, respectively, to the probe imaging trajectory of series 1, 2 and 3. The two footprints for each colour represent the start and end position of the probe in the US imaging of the corresponding series.

Table 1. Details of the sonography process, including the starting and ending probe positions, the trajectory and the number of volumes acquired.

Series	Region	Starting Landmark	Ending Landmark	Probe Trajectory	Number of Volumes
Series 1 (Part 1)	Anteroinferior	Medial-Clavicle	ACJ	Medial to Lateral	16
Series 1 (Part 2)	Posteroinferior		Medial-Scapula Spine	Lateral to Medial	15
Series 2	Posterosuperior	Scapula	ACJ	Medial to Lateral	11
Series 3	Anterosuperior	Medial Clavicle	ACJ	Medial to Lateral	11

The subject was asked to expose her right shoulder, adopt a seated erect orientation and retain a still position for 25 min. A back-supported seat was offered to facilitate the subject’s endurance for the duration of the scan. A horizontal dotted path was drawn around the subject’s shoulder to highlight the probe’s trajectory and mark the region of interest. The path followed the starting and ending landmarks mentioned in Table 1. Sufficient water-based scanning gel was applied in advance along the whole path to ensure continuous scanning. More gel was added during imaging if needed. As illustrated by the orientations of the coloured probes in Figure 1, the sonographer then followed the drawn line using a transverse probe orientation, maintaining a horizontal trajectory and a large overlay between consecutive volumes, as demonstrated in Figure 2. In areas without adjacent bony structures such as the posterosuperior region, the probe was slightly tilted towards deep bony anatomies, which are easier to identify with respect to the position of the scanned volume. A video of the whole sonography process was captured using a smartphone to aid the subsequent registration process by allowing the visualisation of the position and orientation of each volume in space relative to the subject’s shoulder and recording the time taken for the experiment.

Figure 2. An example of manual registration of a pair of consecutive US volumes by identifying and overlaying common features in the ImFusion software (https://www.imfusion.com/, accessed on 20 May 2022). In this case, the common feature is the coracoid process. The three view panels from the top left to bottom left show the transverse (red), sagittal (green) and coronal (blue) plane representations of the volumes. The bottom-right panel corresponds to the 3D volumetric display of the two selected volumes.

2.3. Registration Protocol

Table 2 details the procedure for creating a complete ultrasound view of a rotator cuff by combining the acquired volumes into a US mosaic using US-US registration and aligning it with a corresponding MRI using US-MRI registration.

Table 2. Protocol for creating a complete rotator cuff US mosaic, overlayed by a corresponding MRI, using manual US-US and US-MRI registration techniques. Each procedure step has a description and rationale.

Protocol Step	Description	Rationale
US volume Setup	The 53 acquired rotator cuff volumes were imported into ImFusion (ImFusion GmbH, Munich, Germany) in a common-world coordinate frame.
Pairwise Registration	Common features (anatomical landmarks, mainly bones) in consecutive volume pairs were manually identified and matched by an operator based on their clinical knowledge.	A similar approach was successfully employed by [18] for 3D US images of the knee. Attempt to improve on [16]’s simple compounding technique.
Transformation	The first volume in each pair was fixed while the second was rigidly transformed (translated and rotated) with respect to it to align common features. Figure 2 shows an example of manual registration of a pair of consecutive volumes using common features.	Recommended by [30] for 3D and 4D US registration of bony anatomy with potential for submillimeter accuracy since bones are non-deformable structures. Non-rigid transformation was avoided as it may degrade image quality, and it is a very complex and potentially error-prone operation [17].
Quadruple-wise Registration	After aligning 2 volume pairs, all 4 neighbouring volumes were reviewed. Necessary transformation adjustments were performed by linking the adjusted volume to all following volumes to update their relative transformations.	To check for overall consistency and identify more anatomical structures to verify volume overlays.
Preceding steps were repeated for the whole series
Volume Compounding	Registered volumes were compounded (merged) in sets of 4 using maximum volume compounding, ultimately resulting in a single compounded volume for the whole series.	This compounding method was used as it offers an improved contrast in the US volumes which is often essential for distinguishing and assessing MSK structures as reported by [17].
Preceding step was performed 3 times, once for each series
Superior Series Compounding	Compounded posterosuperior and anterosuperior volumes were registered to each other, forming a superior shoulder volume data set.	Finding the initial configuration of the superior and inferior shoulder series was challenging.
MRI Setup	The 2 volume data sets and the MRI were both imported into ImFusion, visualizing the two modalities under a common world frame.	Comparing the position of the outlined bones and muscles in the MRI can aid in identifying the US structures in question.
US-MRI Registration	The bony structures in each of the 2 US volume data sets were registered to the contours of their corresponding bone segmentations of the MRI.	MRI-based bone segmentations offered a starting point for the registration of the 2 noncontiguous volume data sets (inferior and superior shoulder).
Full shoulder Series Registration and Compounding	After the initial registration, the bone and soft tissue anatomy in the 2 US volume data sets were registered with each other, following the previous steps.
Procedure was revised by 3 different operators including the sonographer

2.4. Mosaic Assessment Protocol (Expert Analysis of Structure Demonstration)

The US mosaic was assessed for clinical viability of visualising musculoskeletal structures in the rotator cuff region for conservative management and surgical applications. Since US diagnosis is mainly performed qualitatively and relies heavily on the operator’s expertise, causing potential inter-operator variability [3], the assessment procedure involved two clinicians and two sonographers. The evaluated MSK structures were selected by consulting two shoulder surgeons. The surgeons listed 20 structures, including bony structures, muscles (and bursa), tendons, ligaments and joints. Regarding bony anatomy, the following structures were assessed: humerus, scapula, clavicle, acromioclavicular (ACJ), ACHD, bicipital groove and spinoglenoid notch. The following tendons were involved: biceps, subscapularis, supraspinatus, infraspinatus and teres minor tendons. Muscles included the infraspinatus, teres minor/major, supraspinatus, trapezius, deltoid and subacromial/subdeltoid bursa. As for ligaments and other anatomy, the demonstration of the coracoacromial (CAL), coracohumeral (CHL), glenohumeral (GHL) and post-glenohumeral labrum/joint was evaluated. Subsequently, two MKS-specialised sonographers with more than 15 years of experience in clinical scanning evaluated the US mosaic. One of the sonographers revised the registration of each pair of US volumes, as well as the registration of the 3 US series and the MRI. The other sonographer was completely blind to the registration process and only participated in the evaluation of the resultant 3D US shoulder mosaic. The evaluation process comprised grading the obtained 3D US shoulder mosaic based on the demonstration of all 20 structures between 1 and 5. Grade 1: a well-demonstrated structure; 2: adequately demonstrated; 3: partially demonstrated; 4: structure is not demonstrated, but the region where the structure is located is seen; and 5: structure is not demonstrated and region is not seen. The two sonographers independently graded the US mosaic three times: first, in its original state, second, with MRI bone segmentation overlay; and lastly, with the MRI muscle segmentation overlay. This was to evaluate the amount of information required from external modalities to identify anatomical structures in the US mosaic. Sonographers were requested to comment on each grade whenever relevant, justifying the grade offered and explaining how clearer a structure becomes with an overlay or how it could be better demonstrated using a different shoulder position. Finally, the grades provided by the two sonographers were compiled and assessed for inter-rater agreement using JASP 0.18.0.0, an open-source statistical software [31]. Cohen’s Weighted kappa was the metric chosen as it is commonly utilised for assessing the agreement between two raters on ordinal categorical data [32].

2.5. Mosaic Assessment Protocol (Quantitative Analysis)

An accurately reconstructed shoulder US mosaic should preserve the anatomical geometry of the rotator cuff [16]. The reconstruction accuracy calculations involved evaluating structure thicknesses and distance measures.

The first assessment compared the thickness of the humeral head surface in a single US volume to the reconstructed thickness in the US mosaic by segmenting the anatomy in both US data sets. Due to the irregular shape of the segmented humeral head surface, the anatomical thickness was determined by averaging multiple evenly distributed caliper measurements across the humeral head. For each of the three planes, 6 measurements were averaged for the single US volume and 12 measurements were averaged for the US mosaic. The difference between both resulting mean thicknesses represented the reconstruction error.

The second assessment of the US mosaic’s reconstruction accuracy replicated the evaluation conducted by [16] for the abdominal organ US reconstructions and involved comparing the distances between readily recognisable features in the US mosaic against their true distances from the MRI. However, due to the rotational offset between the two modalities in this study, selected distances included the humeral diameter for spanning a single non-deformable bone and the ACHD for its easily identifiable tips located in the acromion and humeral head and for its presumed minimal impact from the rotational offset, due to the presence of the supraspinatus tendon and rotational offset direction.

3. Results

3.1. Registration and Sonography Protocols

Table 3 shows the results and findings for each element of the sonography and registration protocols.

Table 3. Overview of results and findings of the sonography and registration protocols.

Element	Result	Findings
Time	25 min
Mosaic coverage	The whole rotator cuff region	Excluding a minor gap at the superolateral aspect of the shoulder, directly inferior to the ACJ, unobservable in the anteriror view of Figure 3 and Figure 4, yet mildly higlighted in the superolateral view of Figure 5.
Number of registered volumes	53 3D contiguous volumes	Figure 3 shows the anterior view of the US mosaic.
Bones detected	Series 1, 2 and 3 detected the humerus, clavicle and scapula, respectively	The clavicle was not demonstrated in the inferior shoulder, except for the first volume.
Patient movements	Minor for soft tissue Did not notably affect bony anatomy	Perfect alignment of soft tissue, especially inter-series, was challenging. Minor tissue displacements were detected live on the US system’s display during the scans.
Registration (Anatomy)	Priority was offered to bones over soft tissue	In many cases, when registration of the bone anatomy was achieved, the soft tissue alignment wasn’t perfect and vice versa.
Registration (Regions)	Registering anterior images was significantly easier than posterior counterparts	Due to easily identifiable bony structures. Inferolateral volumes were most challenging for relatively poorer probe contact and presence of anatomical landmarks.
MRI bony overlay (Positive observations)	Improved visual demonstration of bony anatomy	Visual inspection of Figure 3 shows a satisfying alignment of the humerus across all planes and the acromion in the sagittal plane.
MRI bony overlay (Negative observations)	Slight malalignments between US mosaic and the MRI bones mostly in form of rotational offsets	When scrolling through the transverse plane, the coracoid appears in the US mosaic before it is outlined in the corresponding MRI, and the opposite occurs in the coronal demonstration of part of the clavicle as indicated by the arrows in Figure 4.
MRI muscle overlay	The offset was more eminent as shown in Figure 5.	Discontinuities observed between the US mosaic and MRI muscle overlay because of mild soft tissue and muscle plane distortion caused by positional variation of the subject between modalities.

Figure 5. An (a) anterior (top) and (b) superolateral (bottom) view of the US mosaic overlaying the bony segmentations of the MRI.

Figure 4. Anterior view of the US mosaic overlaying the bony segmentations of the MRI, focusing the 3 planes on the humerus.

Figure 3. Anterior view of the 53 3D US volumes overlaying each other, the US mosaic of the rotator cuff region of a subject’s shoulder. The three view panels from the top left to bottom left show the transverse, sagittal and coronal plane representations of the volumes. The bottom-right panel corresponds to the 3D volumetric display of the US mosaic.

Table 3. Overview of results and findings of the sonography and registration protocols.

Element	Result	Findings
Time	25 min
Mosaic coverage	The whole rotator cuff region	Excluding a minor gap at the superolateral aspect of the shoulder, directly inferior to the ACJ, unobservable in the anteriror view of Figure 3 and Figure 4, yet mildly higlighted in the superolateral view of Figure 5.
Number of registered volumes	53 3D contiguous volumes	Figure 3 shows the anterior view of the US mosaic.
Bones detected	Series 1, 2 and 3 detected the humerus, clavicle and scapula, respectively	The clavicle was not demonstrated in the inferior shoulder, except for the first volume.
Patient movements	Minor for soft tissue Did not notably affect bony anatomy	Perfect alignment of soft tissue, especially inter-series, was challenging. Minor tissue displacements were detected live on the US system’s display during the scans.
Registration (Anatomy)	Priority was offered to bones over soft tissue	In many cases, when registration of the bone anatomy was achieved, the soft tissue alignment wasn’t perfect and vice versa.
Registration (Regions)	Registering anterior images was significantly easier than posterior counterparts	Due to easily identifiable bony structures. Inferolateral volumes were most challenging for relatively poorer probe contact and presence of anatomical landmarks.
MRI bony overlay (Positive observations)	Improved visual demonstration of bony anatomy	Visual inspection of Figure 3 shows a satisfying alignment of the humerus across all planes and the acromion in the sagittal plane.
MRI bony overlay (Negative observations)	Slight malalignments between US mosaic and the MRI bones mostly in form of rotational offsets	When scrolling through the transverse plane, the coracoid appears in the US mosaic before it is outlined in the corresponding MRI, and the opposite occurs in the coronal demonstration of part of the clavicle as indicated by the arrows in Figure 4.
MRI muscle overlay	The offset was more eminent as shown in Figure 5.	Discontinuities observed between the US mosaic and MRI muscle overlay because of mild soft tissue and muscle plane distortion caused by positional variation of the subject between modalities.

3.2. Mosaic Assessment (Expert Analysis of Structure Demonstration)

Table 4 shows the evaluation of the 20 clinically relevant structures in the US mosaic by the first clinical sonographer. The sonographer’s final average grade for each anatomy with the MRI overlays was as follows: bones = 2, muscles = 3, tendons = 3, ligaments = 4 and post-glenohumeral labrum/joint = 4. Table 5 illustrates the evaluation by the second sonographer that was only involved in the assessment process. The corresponding final grades were as follows: bones = 2, muscles = 3, tendons = 3, ligaments = 5 and labrum = 5. Visual demonstrations of all 20 structures in the US mosaic in addition to MRI bone and muscle segmentation overlays are illustrated in Appendix A, Table A1. There was a substantial agreement between the two sonographers on the demonstration of the 20 structures as shown in Table 6.

Table 4. Mosaic assessment (grade and comments) by the first clinical sonographer through grading the 20 rotator cuff structures listed by two experienced shoulder surgeons for each of the following stages: US mosaic standalone (US), US mosaic overlaying MRI bony segmentations (MRI) and US mosaic overlaying MRI muscle segmentation (Muscle).

Sonographer 1 Assessment
Anatomy	US Grade	US Comments	MRI Grade	MRI Comments	Muscle Grade	Muscle Comments
Bony Anatomy
Acromioclavicular (ACJ)	3	Partially Demonstrated	1	Anteroposterior alignment of the ACJ = 1	Did not influence grades
Acromiohumeral Distance (ACHD)	3	Partially Demonstrated	1	Well demonstrated
Bicipital Groove	2	Adequately Demonstrated	2	Adequately Demonstrated
Spinoglenoid Notch	4	Region Seen	4	AP: seen Sup-inf: not seen
Bony Surface (Humerus (H), Scapula (S) and Clavicle (C))	4 C:4 H:2 S:4	Region Seen	3 C: 3 H:1 S:3	Clavicle: limited length seen Humerus: some humeral surface malalignment Scapula: lateral aspect of spine obstructed by overlying anatomy
Tendons
Biceps	2	Adequately Demonstrated	2	Better seen if arm rotated externally	Did not influence grades
Subscapularis	3	Partially Demonstrated	3	Some bony artefact shadow is inevitable
Supraspinatus	3	Partially Demonstrated	3	Better seen with arm in internal rotation
Infraspinatus	3	Partially Demonstrated	3	Difficult to assess/distinguish from IST in this position. Better seen with arm in internal rotation
Teres Minor	3	Partially Demonstrated	3	Better seen if arm rotated internally
Muscles
Infraspinatus	3	Partially Demonstrated	3	Difficult to say whether the entire muscle is included—extensive muscle	Did not influence grades
Teres Minor/Major	4	Region seen (Partially)	4	Difficult to delineate from IS muscle
Supraspinatus	1	Well Demonstrated	1
Trapezius	3	Region Seen (Partially)	3	Limited section seen
Deltoid	3	Partially Demonstrated	3	Some interruption by artefacts laterally
Subacromial/subdeltoid Bursa	2	Adequately Demonstrated	2	Some regions of bony shadowing/artefact
Ligaments
Coracoacromial (CAL)	4	Region seen	4	Best demonstrated ligament	Did not influence grades
Coracohumeral (CHL)	4	Region seen	4	The region around the bicipital groove
Glenohumeral (GHL)	4	Region seen	4	Best seen with arm in a different position
Other
Post-Glenohumeral Labrum/Joint	4	Region seen	4	Better seen with arm in a different position	Did not influence grades

Table 5. Mosaic assessment (grade and comments) by the second clinical sonographer through grading the 20 rotator cuff structures listed by two experienced shoulder surgeons for each of the following stages: US mosaic standalone (US), US mosaic overlaying MRI bony segmentations (MRI) and US mosaic overlaying MRI muscle segmentation (Muscle).

Sonographer 2 Assessment
Anatomy	US Grade	US Comments	MRI Grade	MRI Comments	Muscle Grade	Muscle Comments
Bony Anatomy
ACJ	2	Best seen in A	2	Bone contour easily seen	2
ACHD	3	Humeral head not continuous at region of measurement	1	Easily measured with MRI bone outline	1
Bicipital Groove	3	Seen best on F slice	3	Bone does not line up with US groove. US humerus is more internally rotated	3
Spinoglenoid Notch	3		3	Same to no improvement	3	More confusing to find
Bony Surface (Humerus, Scapula and Clavicle)	3	Not aligned so difficult to see Humerus neck-3 Humeral head-3 Scapula Acromium-3 Scapula Coracoid-3 Spine of Scap-3 Clavicle-4	3	Mild non-alignment of coracoids Humerus neck-2 Humeral head-3 Scapula Acromium-3 Scapula Coracoid-2 Spine of Scap-3 Clavicle-4	3	Some are more difficult to find due to the overlay of all the muscles Humerus neck-2 Humeral head-3 Scapula Acromium-3 Scapula Coracoid-3 Spine of Scap-2 Clavicle-4
Tendons
Biceps	4	Poorly seen.	4	Axial/transverse in groove, but not longitudinal	3	Easier to identify, but not adequately seen
Subscapularis	3	Central aspect seen best, limited assessment of footplate	2	Humerus rotation slightly limits alignment, otherwise good.	2	More difficult with muscle overlay, but still visible
Supraspinatus	4	Poorly seen	3	Definitely need the bones to help identify region	3	Easiest, but still not adequately seen
Infraspinatus	5	Not identified.	3	Massive improvement with bones	3	More difficult with bones
Teres Minor	5	Not identified.	5	Not identified	5
Muscles
Infraspinatus	4		3		3
Teres Minor/Major	5		5	Not identified	4
Supraspinatus	3		2	Definitely easier	2
Trapezius	3		3	No change	3
Deltoid	4		4		3	More visible, not fully
Subacromial/subdeltoid Bursa	2		2	No change	2
Ligaments
CAL	3		3	Malalignment of coracoid	3	More difficult to find with bone overlay
CHL	5	Not identified	5		5
GHL	5	Not identified	5		5
Other
Post-Glenohumeral Labrum/Joint	5	Not identified	5		5

Table 6. Cohen’s Weighted kappa.

			95% CI
Ratings	Weighted Kappa	SE	Lower	Upper
Average kappa	0.701
SONOGRAPHER 1-SONOGRAPHER 2	0.701	0.090	0.526	0.877

Totals of 20 shoulder structures and 2 raters/sonographers. Confidence intervals are asymptotic.

3.3. Mosaic Assessment (Quantitative Analysis)

In the case of reconstruction error evaluation via structural thickness, Table 7 depicts the determined mean thickness for the segmented humeral head surface in both the US mosaic and in a single US volume across all three planes along with the difference in measurements (reconstruction error) for each plane. It also presents the average overall thickness of the humeral head surface considering all planes and the variation between the US mosaic and a single US volume. Overall, the humeral head surface was thicker in the US mosaic compared to an individual volume by 0.649 mm. Further information on the thickness measurements acquired for the assessment are illustrated in Appendix A, Figure A1 and Figure A2.

Table 7. Mean humeral head surface thickness in the US mosaic and in a single US volume.

Mean Values (mm)	Transverse Plane	Sagittal Plane	Coronal Plane	Overall
Thickness in mosaic	2.675	2.481666667	2.755	2.637222
Thickness in a single volume	1.686666667	1.758333333	2.52	1.988333
Reconstruction Error (Thickness)	0.988333333	0.723333333	0.235	0.648889

Regarding evaluating the reconstruction error by anatomical distance measures, Figure 6 illustrates the aligned US mosaic and MRI segmentations of the humeral head, focusing on the humeral head diameter measured between two identifiable points in the MRI segmentation. The MRI segmentation of the structure was within the thickness range of the US mosaic segmentation on all planes with a mild misalignment on the sagittal plane. Figure 7 shows the corresponding distance examined in the US mosaic. As shown in the two figures, the corresponding humeral head diameters were identical in both modalities at 43.3 mm, resulting in a reconstruction error of 0 mm. As for the inter-anatomical distance, Figure 8 shows the distance between the tip of the acromion and the humeral head, measured as 14 mm in the US mosaic and 15.2 mm in the MRI, respectively, corresponding to a reconstruction error of 1.2 mm.

Figure 6. Aligned MRI and US mosaic segmentation of the humeral head: measured diameter between identified tips in the humeral head MRI segmentation.

Figure 7. Measured diameter between the 2 points in the humeral head US mosaic segmentation corresponding to the MRI segmentation.

Figure 8. Measured corresponding distances between the 2 points, in the acromion and the humeral head, in the US mosaic (left) and MRI segmentation (right) across the transverse plane.

4. Discussion

A total of 51 3D US images of a healthy subject’s shoulder were acquired from all shoulder directions (anteroinferior, posteroinferior, anterosuperior and posterosuperior), and registered and combined into a single, complete ultrasound volume of the entire rotator cuff, a US shoulder mosaic. The US mosaic was also registered with an MRI that is segmented for bones and muscles. Visual comparison of the two modalities based on anatomical positions deemed the registration satisfactory. The anatomical geometry was preserved in the US mosaic when compared to the MRI or a single US volume. The humeral head surface was found to be only marginally thicker overall, by 0.65 mm, when segmented in the US mosaic than in a single US volume due to built-up artefacts and malalignments in the image registrations. The thickness of the humeral head surfaces significantly matched in both US data sets on the coronal plane, differing by only 0.235 mm. However, the reconstruction error was most notable across the transverse plane with the humeral head surface being 0.988 mm thicker in the US mosaic. For distance measures, the humeral head diameter perfectly matched in the MRI and US segmentation and the ACHD exhibited a marginal deviation in the US mosaic by 1.2 mm. The performance of the manual registration protocol developed in terms of preserving the anatomical distance measurements showed a significantly better performance than the block-based rigid-body registration used in [16], with a mean error of 2.7 mm. This is despite their approach being assessed on a foetus phantom with a US mosaic consisting of only 10 volumes. Further evaluation of the US mosaic in this study was performed by two expert MSK clinical sonographers, who assessed it as a standalone model, and were subsequently provided with the registered MRI bone segmentations, followed by the muscle segmentations. Evaluated anatomical structures included anatomies that are typically imaged in a shoulder US examination [8,9,10] as well as structures listed as surgery-relevant by two experienced shoulder surgeons. Having access to the MRI bone segmentation improved only the bone grades for the first sonographer. For the second sonographer, it helped better demonstrate the ACHD; infraspinatus and supraspinatus muscles; and subscapularis, supraspinatus and infraspinatus tendons. The further overlay of the MRI muscle segmentations had negligible impact on the structure grades for both sonographers, possibly because identifying soft tissue is the conventional way of using US for experienced sonographers. Sonographers, on average, agreed on the demonstration of all five assessed anatomies in the US mosaic when provided with the MRI bone segmentations (graded bones = 2, muscles = 3, tendons = 3, ligaments = (4–5) and labrum = (4–5)), with a substantial agreement between their individual structure ratings: Cohen’s weighted kappa of 0.7 (95% CI: 0.526–0.877). There was consensus on the clinical viability of visualising the bony anatomy. They were the best demonstrated anatomy, likely because they are rigid and were considered as reference structures for the rigid alignment between volumes. The sonographers also agreed on the partial demonstration of the muscles and tendons and the failure to identify most of the ligaments and labrum. The partial demonstration of tendons was partly due to the subject’s physique as the sonographers found the scanned tendons small and difficult to examine. They noted a clear visualisation of the ACHD, ACJ, subacromial/subdeltoid bursa and supraspinatus. The CAL was the only demonstrated ligament since multiple structures were concealed by the acromion, considering the limitations of a static US mosaic due to the subject’s stationary, centralised arm position. The results suggest that with future development to the technology and optimisation of the registration and compounding methods used, the US mosaic may be employed as an educational or research tool for the shoulder, as it reduces the complexity of imaging and interpreting the relationship between MSK anatomies by producing a comprehensive 3D imaging display similar to that produced by MRI and CT scanners. The latter advantage and the adequately demonstrated bony anatomy in the mosaic suggest that it may potentially be used for assessing various bone-related conditions, including fractures [33], dislocations [34] and joint space narrowing [33] (especially given that the ACHD was well demonstrated). The well-demonstrated anteroposterior alignment of the ACJ, identified by the first sonographer, is not easily achieved in routine 2D US and is used to evaluate the ACJ’s mobility and laxity, inferring a sprain or rupture of the ACJ ligaments depending on the level of bony separation, and determining whether a patient requires surgery or conservative treatment [35]. The partial demonstration of muscles, especially the well-demonstrated supraspinatus, indicates that the mosaic may potentially be employed to assess muscle atrophy, tear and contraction by visualising areas of muscle discontinuity as well as change in muscle size and shape [33]. Further studies shall evaluate the effectiveness of the US shoulder mosaic in detecting and diagnosing these conditions.

In terms of limitations and future work, this feasibility study is preparatory for the ultimate goal of constructing a real-time mosaic of the whole shoulder. Due to the time, cost and resources required to reconstruct and evaluate the US mosaic, this study was limited to a single participant to assess the protocol’s feasibility, explore challenges and determine improvements necessary prior to involving multiple, diverse participants and patients. Directly comparing the MRI to the US mosaic as a way of better validating the registration was not feasible, due to the different acquisition protocols of the two modalities. While the US volumes were acquired for a seated upright subject, the subject was in a supine position with cushions placed inferior to the shoulder when the MRI was taken. As a result, the cushions slightly rotated the arms anteriorly during the MRI and the supine position changed the pressure that was exerted on the anatomical structures compared to the seated position during US imaging. Moreover, similar to [17]’s findings on compounding echocardiography volumes using maximum compounding, the technique facilitated improved contrast for distinguishing and assessing MSK structures. However, what was not reported in [17]’s study was the accumulation of registration errors and artefacts produced, which can cover relevant anatomical image information and downgrade the visibility of certain shoulder structures. Employing an alternative compounding method such as mean compounding does not resolve the issue, as it may produce blurry US volumes which are difficult to assess [17]. Misalignments similar to those obtained by [16] using the block-based rigid-body registration of optically tracked 3D volumes were identified despite manually registering the volumes based on the clinical experience of a registered sonographer in addition to two separate revisions by two different operators. This may suggest that rigid registration approaches are not ideal if quantitative clinical applications are intended. Further studies shall investigate the use of [16]’s block-based warping registration approach in conjunction with manual expert intervention to reduce the build-up of alignment errors and artefacts and to create a higher-quality US mosaic, which is easier to visualise and assess. Additionally, one of US’s greatest advantages is the ability to dynamically assess rotator cuff anatomy in real-time; such an advantage is dismissed in a static mosaic. It is believed that an automatic, real-time version of the US shoulder mosaic shall demonstrate structures that were concealed in the static mosaic and facilitate many more clinical use capabilities. Ongoing research aims to explore approaches involving multiple 3D US probes, an algorithm to localise the probe and a deep learning-based algorithm to automatically register the acquired 3D US volumes in real-time. The resulting automatic, real-time mosaic shall significantly aid in pre-operative and intraoperative applications.

5. Conclusions

In this study, we proved that it is feasible to reconstruct a US mosaic of a full in vivo rotator cuff using manual registration of overlapping 3D US images. Anatomical structures including bones, muscles, tendons, ligaments and joints in the US mosaic were assessed and their visibility was graded by two experienced clinical sonographers. The sonographers agreed on the clinical viability of visualising the bony anatomy and the potential of other MSK anatomies with further optimisation of the registration, compounding and imaging approaches. An MRI segmented for bones aided in the identification of anatomy, and muscle segmentations were unnecessary. We also proved that anatomical geometries were preserved in the reconstructed mosaic by comparing thicknesses of known surfaces and distances between anatomical structures inside the US mosaic and validating the measurements against an original US volume and the MRI data. The results suggest that there is potential for the US mosaic to be used as an educational tool for assessing rotator cuff conditions.