Next Article in Journal
2D Strain Analysis in Myocarditis—Can We Be Any Closer to Diagnose the Acute Phase of the Disease?
Previous Article in Journal
The Efficacy and Safety of EUS-Guided Gallbladder Drainage as a Bridge to Surgery for Patients with Acute Cholecystitis
 
 
Search for Articles:
Title / Keyword
Author / Affiliation / Email
Journal
Article Type
 
 
Advanced Search
 
Section
Special Issue
Volume
Issue
Number
Page
 
Journals
JCM
Volume 12
Issue 8
10.3390/jcm12082776
jcm-logo
Submit to this Journal Review for this Journal Propose a Special Issue
► Article Menu

Article Menu

share Share announcement Help format_quote Cite question_answer Discuss in SciProfiles
first_page
settings
Order Article Reprints
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Systematic Review

The Influence of Extracerebral Tissue on Continuous Wave Near-Infrared Spectroscopy in Adults: A Systematic Review of In Vivo Studies

by
Nick Eleveld
1,*,
Diana C. Esquivel-Franco
1,
Gea Drost
1,2,
Anthony R. Absalom
3,
Clark J. Zeebregts
4,
Jean-Paul P. M. de Vries
4,
Jan Willem J. Elting
1 and
Natasha M. Maurits
1
1
Department of Neurology, University Medical Centre Groningen, University of Groningen, Postbus 30001, 9700 RB Groningen, The Netherlands
2
Department of Neurosurgery, University Medical Centre Groningen, University of Groningen, Postbus 30001, 9700 RB Groningen, The Netherlands
3
Department of Anaesthesiology, University Medical Centre Groningen, University of Groningen, Postbus 30001, 9700 RB Groningen, The Netherlands
4
Department of Surgery, Division of Vascular Surgery, University Medical Centre Groningen, University of Groningen, Postbus 30001, 9700 RB Groningen, The Netherlands
*
Author to whom correspondence should be addressed.
J. Clin. Med. 2023, 12(8), 2776; https://doi.org/10.3390/jcm12082776
Submission received: 22 February 2023 / Revised: 4 April 2023 / Accepted: 5 April 2023 / Published: 8 April 2023
(This article belongs to the Section Clinical Neurology)
Browse Figures
Versions Notes

Abstract

:
Near-infrared spectroscopy (NIRS) is a non-invasive technique for measuring regional tissue haemoglobin (Hb) concentrations and oxygen saturation (rSO2). It may be used to monitor cerebral perfusion and oxygenation in patients at risk of cerebral ischemia or hypoxia, for example, during cardiothoracic or carotid surgery. However, extracerebral tissue (mainly scalp and skull tissue) influences NIRS measurements, and the extent of this influence is not clear. Thus, before more widespread use of NIRS as an intraoperative monitoring modality is warranted, this issue needs to be better understood. We therefore conducted a systematic review of published in vivo studies of the influence of extracerebral tissue on NIRS measurements in the adult population. Studies that used reference techniques for the perfusion of the intra- and extracerebral tissues or that selectively altered the intra- or extracerebral perfusion were included. Thirty-four articles met the inclusion criteria and were of sufficient quality. In 14 articles, Hb concentrations were compared directly with measurements from reference techniques, using correlation coefficients. When the intracerebral perfusion was altered, the correlations between Hb concentrations and intracerebral reference technique measurements ranged between |r| = 0.45–0.88. When the extracerebral perfusion was altered, correlations between Hb concentrations and extracerebral reference technique measurements ranged between |r| = 0.22–0.93. In studies without selective perfusion modification, correlations of Hb with intra- and extracerebral reference technique measurements were generally lower (|r| < 0.52). Five articles studied rSO2. There were varying correlations of rSO2 with both intra- and extracerebral reference technique measurements (intracerebral: |r| = 0.18–0.77, extracerebral: |r| = 0.13–0.81). Regarding study quality, details on the domains, participant selection and flow and timing were often unclear. We conclude that extracerebral tissue indeed influences NIRS measurements, although the evidence (i.e., correlation) for this influence varies considerably across the assessed studies. These results are strongly affected by the study protocols and analysis techniques used. Studies employing multiple protocols and reference techniques for both intra- and extracerebral tissues are therefore needed. To quantitatively compare NIRS with intra- and extracerebral reference techniques, we recommend applying a complete regression analysis. The current uncertainty regarding the influence of extracerebral tissue remains a hurdle in the clinical implementation of NIRS for intraoperative monitoring. The protocol was pre-registered in PROSPERO (CRD42020199053).

Graphical Abstract

1. Introduction

Cerebral near-infrared spectroscopy (NIRS) is a non-invasive technique used to measure and monitor tissue haemoglobin (Hb) concentrations and the oxygen saturation (rSO2) of the brain [1,2,3,4]. It is a promising technique for monitoring adequacy of cerebral perfusion and tissue oxygenation in patients that are at risk of peri-operative cerebral ischemia or hypoxia, such as those undergoing cardiothoracic or carotid surgery [3,5].
For cerebral NIRS, a light source on the scalp is used to emit light in the near-infrared (NIR) frequency range into the head. Only a small percentage of the emitted light is received by a detector on the scalp a few centimetres away because of the scattering and reflection of NIR light by human (brain) tissue. Light of specific frequencies can also be absorbed if there are sufficient concentrations of chromophores along the light path. Oxygenated (OxyHb) and deoxygenated haemoglobin (HHb) are both chromophores, but they have distinct NIR light absorption profiles. By using light of multiple wavelengths, and application of the Beer–Lambert law, which relates attenuation of light to the properties of the material through which it is travelling, OxyHb and HHb concentrations can be estimated. This allows us to calculate the regional tissue oxygen saturation (rSO2) as the ratio of OxyHb to total haemoglobin (tHb, sum of OxyHb and HHb) [6].
In patients undergoing procedures that pose a risk to cerebral perfusion, changes in cerebral tissue oxygenation are possible while systemic oxygenation is unchanged. As clinicians may use NIRS-based measurements during surgical procedures to inform clinical management decisions, it is important to know to what extent the obtained measurements reflect purely cerebral physiology [6,7].
When NIRS technology is used for cerebral measurements, an important challenge is that the light reaching the detector optode has also passed through extracerebral tissues. These tissues, which include scalp, skull, dura mater, and cerebrospinal fluid, can all influence the amount of light received by the detector, as they can reflect and scatter the light. They also contain molecules such as haemoglobin and myoglobin that can absorb NIR light [8,9,10,11]. Simulation studies suggest that extracerebral tissues have a large influence on NIRS measurements [9]. In these studies, the results varied according to the location on the head and source–detector separation (SDS); however, of the total amount of emitted light that was absorbed prior to reaching the detector optode, only 6 to 20% had been absorbed by the cortical grey matter. Most of the attenuation or absorption occurred during passage through the scalp and skull. However, these studies used computer simulations that assume certain anatomical and optical properties of the head tissues, which may not always be valid in vivo [9].
Despite these disadvantages, cerebral NIRS has continued to be used extensively in scientific and clinical practice [3,12,13]. To better extract the intracerebral (grey and white matter) NIRS components, manufacturers have developed NIRS devices with multiple sources and detectors and digital filtering techniques [4]. It could therefore be that the influence of extracerebral tissue is less pronounced in practice than in theory.
Surprisingly, even though NIRS devices have been used for decades, there is no consensus on the degree to which extracerebral tissue influences cerebral NIRS measurements. Moreover, there is currently no unified method to determine how these tissue layers influence NIRS measurements. Thus, before more widespread use of NIRS as an intraoperative monitoring modality is warranted, these issues need to be better understood. We therefore performed a systematic review of the literature, assessed several study protocols, determined methodological quality, and extracted quantitative results. We discuss the current methodological challenges and provide practical recommendations.

2. Materials and Methods

A systematic review was conducted following the Preferred Reporting Items for Systematic Reviews and Meta-Analysis of Diagnostic Test Accuracy Studies (PRISMA-DTA) Statement [14]. PRISMA-DTA checklists are provided as supplementary materials. The protocol was registered in the International Prospective Register of Systematic Reviews (PROSPERO) database, under registration number CRD42020199053.

2.1. Research Question

The research question was designed following the Population, Exposure, Outcome (PEO) format: ‘What is the influence of extracerebral tissue (Exposure) on cerebral near-infrared spectroscopy (Outcome) in adult humans (Population)?’

2.2. Search Strategy

A broad search strategy was used to generate a comprehensive overview of all published literature investigating the influence of extracerebral tissue on NIRS. Specifically, clinical and non-clinical human adult studies, and pre-clinical studies involving large adult animals as a model for humans, were included. Four research databases were used: Scopus, Embase, Web of Science, and PubMed.

2.3. Search Concepts

Two search concepts were used to screen potentially relevant studies: ‘extracerebral tissue’ and ‘near-infrared spectroscopy’. Titles, abstracts, keywords, and (if applicable) Medical Subject Headings (MESH) or Emtree terms were included in the search strategy. The search was conducted on 2 February 2022. No restriction on the year of publication was applied. The complete literature search strategy can be found in File S1. The references of included articles were screened for overlooked articles.

2.4. Inclusion Criteria

Eligibility was determined based on three elements corresponding to the PEO research question and two elements regarding study quality and methodology, as described in detail in Table 1. Population: our scope was limited to the adult population, since adults have a thicker scalp and skull than children [15,16], making NIRS measurements in adults more susceptible to interference by extracerebral tissue. Animal studies that can be considered a reasonable model for adult human cerebral NIRS were also included, as these studies often employ comprehensive, invasive monitoring techniques and study protocols. Outcome measure: we limited our scope to continuous wave and frequency-domain NIRS devices, as they are most commonly used in research and clinical practice. Exposure: we only included studies that specifically investigated the influence of intracerebral or extracerebral tissue layers on NIRS measurements. Studies that investigate the influence of systemic hemodynamic changes, for example, during functional NIRS studies, are therefore not included in the current review. Moreover, the included studies had to use reference techniques that measure blood flow, Hb concentrations or the tissue oxygen saturation of both the intra- and extracerebral tissue, or they had to apply a protocol in which the intra- or extracerebral tissue perfusion was selectively altered.
Articles were screened on title and abstract first, and subsequently on full text. Two researchers (NE, DE) screened the articles, and a consistency check was performed on 100 articles. For all articles, uncertainty regarding any inclusion criterion was discussed in a consensus meeting until agreement was reached.

2.5. Critical Appraisal

The Quality Assessment of Diagnostic Accuracy Studies 2 (QUADAS-2) tool was used to appraise the quality of included studies [17]. It was extended with the critical appraisal domain ‘Perfusion Modification Protocol’, and some questions were modified to fit the scope of our review. Details of the modified QUADAS-2 criteria are provided in Table S1.

2.6. Software and Statistical Analysis

Screening, data extraction, and critical appraisal were performed by two authors (NE, DE) in CADIMA [18]. Quantitative synthesis was performed in MATLAB (MATLAB (R2019b), The MathWorks Inc., Natick, MA, USA). The code can be found in File S2. Correlation figures were created using a Python 3.4 script adapted from that used by Bernaldo de Quiros and co-workers [19]. If the required data were only represented graphically in an article, the quantitative results were obtained from the graph using Webplotdigitizer [20].
If an article reported results from different NIRS devices, source–detector configurations, NIRS indices (Hb indices or rSO2), or they used different experimental protocols; these were identified as individual sub-studies and analysed in this review as such.
Group characteristics were expressed by median and inter-quartile range (IQR). Relations between NIRS and reference technique measurements were expressed as Pearson’s or Spearman’s correlation (‘r’), if available. If correlations were Fisher Z-transformed, these were transformed back.

3. Results

3.1. Literature Screening

An overview of the screening process is provided in Figure 1. The initial search yielded 3047 articles. A further 54 potential articles were identified by screening the references of the included articles. The interrater agreement was 84%, κ = 0.65, which is considered substantial agreement [21]. After duplicates were removed, 1390 remaining records were screened on title and abstract, after which 1112 records were excluded. The full texts of 278 articles were assessed for eligibility. Of these, 237 articles were excluded for the reasons summarized in Figure 1 (some articles were excluded based on multiple criteria): study population (n = 3), NIRS-index (n = 47), influence extracerebral tissue not investigated (n = 159), no empirical peer-reviewed research (n = 50), or no in vivo study methodology (n = 14). Eight additional duplicate studies were found that had not been discovered during duplicate removal due to slight differences in title spelling. Lastly, one study only reported the measurement setup, but provided no results.
Forty-one articles were included in the review. These comprised 116 sub-studies with different NIRS devices, source–detector configurations, NIRS indices, or experimental protocols.

3.2. General Characteristics

An extensive overview of all study characteristics, including device type, study protocol, and results is provided in Table S2 and Table S3. Only studies with human subjects met all inclusion criteria. A total of 85 sub-studies focused on healthy participants and 31 on a patient population. Population size varied between 4 and 160 participants (median [IQR]: 12 [10,11,12,13,14,15,16,17,18,19,20,21]). All but one sub-study investigated continuous wave NIRS. Frequency-domain NIRS was assessed in one article, but this article had to be excluded post hoc because of poor study quality (as explained under Critical Appraisal). NIRS sensors were mostly placed on the forehead (n = 109), but a minority (n = 7) investigated other locations (occipital lobe, parietal lobe, or specifically covering somatosensory or motor cortex). The SDS ranged between 0.7 and 6 cm. For studies examining Hb concentrations, the majority used an SDS of 3 cm (median, IQR: 3, [3,4,5]). For the calculation of oxygen saturation indices (all included in this review as ‘rSO2’: regional tissue oxygen saturation), often, two or more source–detector pairs with a different SDS were used (generally, combinations of 3, 4, and 5 cm). For 23 sub-studies, no information on SDS was available.

3.3. Study Methodology

None of the authors directly quantified the influence of signal attenuation by the extracerebral tissue layers on the NIRS measurements, analogous to the simulation studies. Instead, the influence of the extracerebral or intracerebral tissue layers was estimated indirectly by either (1) comparing (changes in) NIRS measurements with (changes in) the measurements of intra- or extracerebral reference techniques, or (2) quantifying the change in NIRS measurements when the intra- or extracerebral tissue perfusion was selectively altered.
In total, 95 sub-studies used a methodology in which the intra and/or extracerebral perfusion was selectively altered. Intracerebral perfusion modifications were achieved by hyper-/hypocapnia (n = 34) or occlusion of the internal carotid artery (ICA) (n = 10). Extracerebral perfusion modification protocols involved scalp cuff inflation/release (n = 32), external carotid artery (ECA) occlusion (n = 7), phenylephrine infusion (n = 5), and several others (n = 7 in total). A total of 21 sub-studies applied a methodology that we classified as non-selective intra- or extracerebral perfusion modification. These were mostly cognitive or motor task studies that have a high risk of also causing systemic hemodynamic changes, particularly changes in arterial blood pressure [4].
In 68 sub-studies, an intracerebral reference technique was used; this was mostly transcranial Doppler ultrasound (TCD) (n = 52) or functional magnetic resonance imaging (fMRI) (n = 12). In one article, multiple intracerebral reference techniques were employed [23]. With TCD, the cerebral blood flow velocity (CBFv) was determined non-invasively with ultrasound in one of the large cerebral arteries, often the middle cerebral artery (MCA). With fMRI, the relative change in OxyHb and HHb concentrations in a tissue volume can be measured because of the differences in the magnetic susceptibility of the two molecules [24].
A total of 63 sub-studies used an extracerebral reference technique. The majority employed laser Doppler flowmetry (LDF) (n = 49), scalp pulse oximetry or photoplethysmography (PPG) (n = 15), or fMRI of the scalp and skull tissues (n = 5). Ten sub-studies applied multiple extracerebral reference techniques. LDF measures the perfusion in the microcirculation of the upper skin layers. For scalp pulse oximetry, the arterial oxygen saturation of the scalp tissue is estimated using a technique analogous to pulse oximetry performed on the finger.

3.4. Critical Appraisal

Critical appraisal scores are summarized in Figure 2. Details are provided in Table S4.
The domains of participant selection and flow and timing showed a high or unclear risk of bias (RoB) in most articles. Specifically, participant selection was often not clearly described, and it was not always evident if a case–control design had been avoided. Exclusions were generally not clearly documented. Methods to prevent or correct artifacts were often not described. The sampling frequency of the assessed NIRS devices was mostly low or not described. No studies used a blinded study design.
If applicable, reference techniques and the perfusion modification methodology were often described well. In general, the index test (NIRS measurement) was also well described. The method of calculating Hb concentrations was mostly pre-specified. It involved application of the modified Beer–Lambert law [25], which is a standard method, but some studies used custom calculations. For rSO2, mostly commercially available devices were used without software changes.
Concerns regarding the applicability of the studies to the research question were low in general. Most authors included healthy participants or patients without suspected cerebrovascular diseases, which represent the general population. A minority of studies specifically included patients with cerebrovascular pathology, in whom intra- and extracerebral perfusion can deviate from that of a healthy population [26]. Some studies used manipulation of the intra- or extracerebral perfusion to calculate the investigated NIRS index, namely through clamping of the carotid artery or inflation of a cuff around the scalp. The requirement of such a protocol limits applicability to a broad population.
In total, 7 articles comprising 11 sub-studies scored a high or unclear RoB in more than 50% of QUADAS-2 domains. These articles were excluded post hoc. This cut-off for study quality was chosen because weighing individual QUADAS-2 domains is difficult, and most studies (105 sub-studies in 34 articles) would remain for data synthesis. Details are provided in Table S5 and Table S6.

3.5. Comparison of Haemoglobin Concentrations and Reference Techniques

A total of 14 articles comprising 25 sub-studies compared within-subject changes in Hb concentrations (OxyHb, HHb, tHb, or Hbdiff [difference between OxyHb and HHb]) with changes in reference technique measurements [27,28,29,30,31,32,33,34,35,36,37,38,39,40]. They all expressed these relations with correlation coefficients. The correlations are illustrated in Figure 3. Study quality is reflected by circle size, itself representing the percentage of the QUADAS-2 domains classified as low RoB or applicability concerns.
Six authors used intracerebral perfusion modification protocols (acetazolamide infusion, hypercapnia, ICA clamping) and exclusively compared NIRS with intracerebral reference techniques [27,28,29,30,31,32]. Changes in OxyHb, Hbdiff, and tHb all correlated positively with changes detected by TCD and fMRI, ranging from r = 0.45 to r = 0.88. Changes in HHb were negatively correlated with changes measured with TCD and fMRI (r = −0.44 to r = −0.62). The studies using ICA clamping scored lower on study quality than studies applying hypercapnia protocols.
One author compared tHb with an intracerebral technique (TCD) and two extracerebral (NIRS on ipsilateral cheek and scalp PPG) techniques during three study protocols: hypocapnia (cerebral hypoperfusion), Valsalva manoeuvre (intracerebral hypoperfusion and extracerebral hyperperfusion), and head-up tilt (intracerebral and extracerebral hypoperfusion). During all protocols, the correlation with extracerebral techniques was higher (r = 0.37 to r = 0.7) than with intracerebral techniques (r = −0.14 to r = −0.32) [40].
Three authors applied selective extracerebral perfusion modification protocols (head-cuff inflation, teeth-clenching, skin pressure) and used only extracerebral reference techniques as comparison [33,34,35]. With cuff inflation to restrict flow to the scalp underneath the NIRS sensor, positive correlations between OxyHb and LDF-derived scalp blood flow were seen (r = 0.46 to r = 0.73) [33]. In another article, teeth-clenching was applied to change temporal muscle oxygenation with a NIRS device covering this muscle. The correlation between OxyHb and HHb with short-channel (10 mm SDS) NIRS measurements was low (OxyHb: r = 0.21; HHb: r = −0.14) [34]. Lastly, one article used the application of skin pressure in a unique protocol [35]. Changes in OxyHb and HHb were assessed in a verbal fluency protocol with and without pressure to the scalp on the measurement location. Without pressure, a clear increase in OxyHb was seen in the NIRS sensors covering the forehead. However, there were also concurrent increases in scalp perfusion as measured with both LDF and short-channel (5 mm SDS) NIRS close to the long-channel (3.0 mm SDS) NIRS optodes. A correlation of r = 0.89 between long-channel NIRS OxyHb and LDF-based scalp blood flow was found. Moreover, the correlation between long-channel OxyHb and short-channel OxyHb was r = 0.93. When the verbal fluency protocol was applied with manual pressure to the scalp, OxyHb changes were absent, as were the changes in scalp blood flow and short-channel OxyHb concentration. This may indicate that the task-related increase in OxyHb could be largely explained by changes in scalp perfusion, as measured with LDF and short-channel NIRS [35]. The authors provided no results for HHb.
Four articles comprising seven sub-studies compared changes in Hb with both intra- and extracerebral reference techniques during cognitive or motor task protocols [36,37,38,39]. These tasks were classified as no selective intracerebral or extracerebral perfusion modification. Correlations of NIRS measurements with those acquired from intra- and extracerebral reference techniques were lower than with a selective perfusion modification protocol (illustrated in Figure 3 by the paler circle colours). Absolute values ranged from |r| = 0.0 to |r| = 0.5. However, the positive correlations with OxyHb concentrations accompanied by negative correlations with HHb measurements were preserved, except for the study by Heinzel and colleagues, which found positive correlations between NIRS and fMRI measurement results for both OxyHb and HHb [39]. Study quality was generally lower for these studies.

3.5.1. Extracerebral Correction Methods

Kirkpatrick and colleagues determined the correlation between TCD-based CBFv and two calculations of Hbdiff upon clamping of the internal carotid artery [32]. For one calculation, Hbdiff was corrected for changes in extracerebral perfusion by subtracting the (extrapolated) change in Hbdiff when the external carotid artery was clamped. They found a higher correlation with changes in CBFv when extracerebral correction was applied (Hbdiff: r = 0.64, Hbdiff with correction: r = 0.73).
Funane and colleagues identified ‘deep’ and ‘shallow’ OxyHb and HHb components [36]. To identify the components, groups of adjacent short (15 mm SDS) and long (30 mm SDS) NIRS channels were assessed with independent component analysis. The absolute correlation with both the intracerebral (fMRI) and extracerebral (LDF) reference technique measurements was only available for the ‘deep’ OxyHb and HHb signals, and ranged between |r| = 0.0 and |r| = 0.3.

3.5.2. Source–Detector Separation (SDS)

SDS ranged between 1.5 and 6 cm (Figure 3, second column). Given the large variety of methodologies used, stratification for SDS was not performed in this review. However, changes in OxyHb for three SDS’s were compared with LDF measurements of scalp blood flow during scalp cuff inflation by Hirasawa and colleagues [33]. In contrast to the authors’ hypothesis, the correlation between OxyHb and scalp blood flow increased for a larger SDS between the NIRS optodes (1.5 cm: r = 0.47, 2.25 cm: r = 0.73, 3.0 cm: r = 0.73).

3.6. Comparison of Oxygen Saturation Indices and Reference Techniques

Five articles with six sub-studies compared within-subject changes in rSO2 with changes in reference technique measurements [23,41,42,43,44]. Correlation coefficients are shown in Figure 4. Although a high correlation was found between rSO2 and TCD-based CBFv upon acetazolamide infusion (r = 0.77) [41], a lower correlation with CBFv (r = 0.23) was seen during hypercapnia, which was even lower (r = 0.18) when a head-cuff was inflated to restrict perfusion to the scalp of the forehead [42]. rSO2 showed a good correlation with an invasive measurement of brain tissue oxygen (PbrO2) (r = 0.5) during hypercapnia induced by changes in ventilatory settings [43]. Upon clamping of the ICA, the correlation of changes in rSO2 with TCD-based CBFv was higher than for LDF-based scalp flow measurements (r = 0.56 for TCD and 0.13 for LDF) [44]. Lastly, high correlations of rSO2 with both duplex ECA conductance (blood flow velocity divided by mean arterial pressure) and scalp blood flow were seen during phenylephrine infusion, r = 0.64 and r = 0.81, respectively [23].

3.7. Studies without Within-Subject Comparisons with Reference Techniques

In 66 sub-studies in 16 articles, no reference techniques were used, or NIRS measurements were compared with those from reference techniques only on a group level and not on an individual level [40,45,46,47,48,49,50,51,52,52,53,54,55,56,57,58,59]. The results are described in detail in File S3.

4. Discussion

4.1. Main Findings

To assess the potential of NIRS as a routine intraoperative neuromonitoring modality, we here first investigated how extracerebral tissue influences NIRS measurements, and the extent of this influence, by means of a systematic review. We synthesized the results of 41 articles. NIRS measurements correlated with both intracerebral and extracerebral reference technique measurements. Although this indicates that NIRS measurements are influenced by extracerebral tissue, we could not quantify this influence.

4.2. Interpretation of the Results

Studies with selective intracerebral perfusion modification report high correlations (|r| = 0.23 to 0.88) between NIRS and intracerebral reference techniques. This provides evidence for the (partially) cerebral origin of the NIRS signal. However, studies using extracerebral perfusion modification showed high correlations with extracerebral reference techniques (|r| = 0.14 to 0.93). This indicates that the NIRS signal is influenced by extracerebral tissue as well. Given the high correlations with extracerebral measurements (LDF, fMRI, short-channel NIRS, Duplex ECA), it is evident that extracerebral tissue influences NIRS measurements, although a quantification of this influence was not possible based on the current literature. This applies to both Hb concentrations and rSO2.
Five articles compared Hb concentrations with reference techniques for both tissue layers. Four found higher correlations with intracerebral techniques during cognitive or motor tasks. One article reported higher correlations with extracerebral refence techniques using several study designs [36,37,38,39,40]. For rSO2, only one study performed intra- and extracerebral measurements. They found a higher correlation between NIRS measurements and intracerebral measurements during ICA clamping [44].

4.3. The Problem of Quantifying Extracerebral Influence

In computational simulation studies, the influence of extracerebral tissue on NIRS measurements can be quantified exactly [9]. In in vivo human studies, this has not yet been possible. Instead, the studies in this review used correlations with intra- or extracerebral reference techniques as an indirect proxy for the intra- or extracerebral influence on NIRS.
Although correlations between NIRS and reference techniques provide evidence for the influence that the intra- and extracerebral tissue have on NIRS measurements, they are insufficient to quantify this influence. If the influence of extracerebral tissue on the NIRS signal is small, changes in extracerebral perfusion would lead to relatively small changes in the NIRS measurement; however, this does not necessarily entail a low correlation. To quantify the influence of extracerebral tissue, the slope of the linear regression line through the datapoints is required, which none of the authors reported. Future research on extracerebral influence should therefore incorporate a complete regression analysis with correlation and slope.

4.4. Methodological Recommendations for Investigations of Extracerebral Influence

Future studies could incorporate the following methodological recommendations: (1) clearly describe participant selection and preferably study a homogenous population regarding (collateral) cerebral perfusion; (2) describe the calculation of the NIRS indices, including correction methods for extracerebral contamination; (3) apply at least one reference technique per tissue layer; (4) provide a complete description of the relation between reference techniques and NIRS measurements, including correlation coefficients, statistical significance, and regression parameters; (5) include both absolute and relative changes in NIRS indices to allow comparison between devices; (6) describe exclusions and artifact handling to prevent bias; and (7) use a standardized study protocol with both selective perfusion modification and a baseline measurement. Selective perfusion modification should only be omitted (for example, in rest) if a high signal-to-noise ratio can be expected. Otherwise, low correlations between NIRS measurements and intra- and extracerebral reference technique measurements can be expected. We advise NIRS manufacturers to validate their NIRS devices following the above recommendations to demonstrate the extent to which different tissue layers contribute to NIRS signals.

4.5. Implications for NIRS in Clinical Practice

NIRS devices have been used in clinical practice for many years, in particular to monitor cerebral oxygenation during cardiothoracic surgery [3,60]. Low NIRS-derived rSO2 values have been associated with poor clinical outcomes, and rSO2 optimisation management strategies may improve outcome [61,62,63,64,65,66]. The evidence for the clinical utility of NIRS measurements during cardiothoracic surgery may, however, be the result of collinearity between extracerebral and intracerebral perfusion; both skin and cerebral perfusion can be altered by systematic hemodynamic changes that may affect outcome. The question thus remains whether selective alterations in cerebral oxygen saturation will be captured with NIRS. Based on the literature discussed here, there is no indisputable evidence that most of the NIRS signal originates from intracerebral tissue. We conclude that the current uncertainty regarding the influence of extracerebral tissue on the NIRS signal remains a hurdle in the clinical implementation of NIRS as a routine intraoperative neuromonitoring modality.

4.6. Limitations

Although this review was performed according to the PRISMA-DTA guidelines [14] and with standardised critical appraisal, several factors limit our ability to draw strong conclusions. First, study quality varied widely; details regarding participant selection and the domain flow and timing were unclear in most articles. The number of participants was generally small (n < 20 for 75% of studies), which limits the generalisability of the obtained results (correlations). This furthermore limits the possibility for correction of confounders or a subgroup analysis.
Second, studies did not follow a standardized research protocol. There was large heterogeneity in study designs and outcome measurements. We therefore could not perform a quantitative (meta)analysis of the results.
Third, each of the employed reference techniques for blood flow, Hb concentration or tissue oxygen saturation in the intracerebral or extracerebral tissue layers has advantages when serving as the ‘gold standard’ for Hb concentrations and rSO2 as obtained with NIRS. The suitability of a reference technique depends primarily on the similarity between the measured quantities. Short-channel NIRS measures the same quantity as NIRS measurements with a longer SDS, but it is unclear how CBFv (TCD), scalp tissue blood flow (LDF), or OxyHb and HHb concentrations obtained with fMRI relate to NIRS-based Hb concentrations or rSO2. Reference techniques further differ in spatial resolution. fMRI offers the best spatial resolution, especially since the location of the NIRS sensors can be mapped onto the structural MRI image [36,37,38,39]. The spatial resolution of TCD is lower, as it only provides a measurement of blood flow velocity in a large cerebral blood vessel, generally the MCA. Lastly, the NIRS devices and the reference techniques strongly differ in their temporal resolution. Tissue Hb concentrations vary over time, and for a proper comparison between techniques, each technique should be able to capture the time-dependent phenomena. Cardiovascular signals may contain frequencies up to 25 Hz, implying a required sampling frequency of at least 50 Hz [67]. fMRI has a temporal resolution < 0.5 Hz, whereas LDF and TCD can reach sampling frequencies > 50 Hz. Regrettably, the sampling frequency has not been disclosed for most commercial NIRS devices.
Fourth, not all newer NIRS devices have been investigated in studies meeting our inclusion criteria. Studies that investigate the influence of extracerebral tissue on NIRS measurements from the newer devices are warranted. Furthermore, only one study specifically reported the difference between NIRS indices with and without correction for extracerebral tissue influence [32], even though several correction methods have been developed [4]. The effect of these algorithms on the extracerebral contamination of NIRS signals thus remains unclear.
Fifth, in this review, we limited our scope to continuous wave and frequency-domain NIRS, as these techniques are most commonly used in clinical practice and research. Future studies could include time-domain NIRS-measurements, for which the influence of extracerebral tissue can be lower [68].
Sixth, our initial screening criteria allowed the inclusion of large animal studies that could have been used as a model for adult human anatomy and physiology. This could have been seen as a limitation, as the validity of animal studies as a model for adult humans remains unclear. None of the authors of the animal studies identified in the initial search provided evidence of the validity of their model (porcine or ovine) for the influence of extracerebral tissue on NIRS signals in humans. In any event, during the full-text screening stage, these animal studies were all excluded because they did not meet all inclusion criteria [69,70,71,72,73,74,75]. The findings and conclusions we report are therefore not influenced by the results of animal studies.

5. Conclusions

Extracerebral tissue influences cerebral NIRS measurements, but there is currently insufficient evidence to quantify this influence based on the available in vivo studies. The current uncertainty regarding the influence of extracerebral tissue remains a hurdle in the clinical implementation of NIRS for intraoperative monitoring.

Supplementary Materials

The following supporting information can be downloaded at: https://www.mdpi.com/article/10.3390/jcm12082776/s1, Table S1: Critical appraisal strategy, Table S2: Detailed study characteristics haemoglobin (Hb) studies, Table S3: Detailed study characteristics oxygen saturation indices (rSO2), Table S4: Critical appraisal classification results, Table S5: Studies excluded post hoc based on study quality, Table S6: Detailed study characteristics of studies excluded post hoc, File S1: Literature search strategy, File S2: Data Synthesis Script, File S3: Results for studies without within-subject comparisons with reference techniques, File S4: PRISMA DTA Abstract Checklist, File S5: PRISMA DTA Checklist. References [76,77,78,79,80,81,82] are cited in the Supplementary Materials.

Author Contributions

Conceptualisation, N.E., G.D., A.R.A., J.W.J.E. and N.M.M.; methodology, N.E., D.C.E.-F., G.D., A.R.A., C.J.Z., J.-P.P.M.d.V., J.W.J.E. and N.M.M.; software, N.E. and D.C.E.-F.; validation, N.E., D.C.E.-F., G.D., A.R.A., J.W.J.E. and N.M.M.; formal analysis, N.E., D.C.E.-F., J.W.J.E. and N.M.M.; investigation, N.E. and D.C.E.-F.; resources, N.E.; data curation, N.E. and D.C.E.-F.; writing—original draft preparation, N.E., J.W.J.E. and N.M.M.; writing—review and editing, N.E., D.C.E.-F., G.D., A.R.A., C.J.Z., J.-P.P.M.d.V., J.W.J.E. and N.M.M.; visualization, N.E., G.D., A.R.A., J.W.J.E. and N.M.M.; supervision, N.M.M.; project administration, N.E.; funding acquisition, N.E.; All authors have read and agreed to the published version of the manuscript.

Funding

This research was funded by a personal grant for NE from the Graduate School of Medical Sciences of the University of Groningen.

Institutional Review Board Statement

Not applicable.

Informed Consent Statement

Not applicable.

Data Availability Statement

The data presented in this study are available in Tables S2–S6. The code used for data synthesis is available in File S2.

Conflicts of Interest

A.R.A. is an editorial board member of the British Journal of Anaesthesia. His research group/department has received reimbursement for consultancy work and/or sponsor-initiated phase 1 studies from The Medicines Company, Rigel Inc., Paion, Terumo, BD (Carefusion), Philips, Ever Pharma and Johnson & Johnson (Janssen Pharmaceutica). The funders had no role in the design of the study; in the collection, analyses, or interpretation of data; in the writing of the manuscript; or in the decision to publish the results.

References

  1. Ferrari, M.; Quaresima, V. A Brief Review on the History of Human Functional Near-Infrared Spectroscopy (FNIRS) Development and Fields of Application. Neuroimage 2012, 63, 921–935. [Google Scholar] [CrossRef] [PubMed]
  2. Obrig, H.; Villringer, A. Beyond the Visible—Imaging the Human Brain with Light. J. Cereb. Blood Flow Metab. 2003, 23, 1–18. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  3. Scheeren, T.W.; Schober, P.; Schwarte, L.A. Monitoring Tissue Oxygenation by near Infrared Spectroscopy (NIRS): Background and Current Applications. J. Clin. Monit. Comput. 2012, 26, 279–287. [Google Scholar] [CrossRef] [Green Version]
  4. Scholkmann, F.; Kleiser, S.; Metz, A.J.; Zimmermann, R.; Mata Pavia, J.; Wolf, U.; Wolf, M. A Review on Continuous Wave Functional Near-Infrared Spectroscopy and Imaging Instrumentation and Methodology. Neuroimage 2014, 85, 6–27. [Google Scholar] [CrossRef]
  5. Khan, J.M.; McInnis, C.L.; Ross-White, A.; Day, A.G.; Norman, P.A.; Boyd, J.G. Overview and Diagnostic Accuracy of Near Infrared Spectroscopy in Carotid Endarterectomy: A Systematic Review and Meta-Analysis. Eur. J. Vasc. Endovasc. Surg. 2021, 62, 695–704. [Google Scholar] [CrossRef] [PubMed]
  6. Murkin, J.M.; Arango, M. Near-Infrared Spectroscopy as an Index of Brain and Tissue Oxygenation. Br. J. Anaesth. 2009, 103, i3–i13. [Google Scholar] [CrossRef] [Green Version]
  7. Edmonds, H.L.; Ganzel, B.L.; Austin, E.H. Cerebral Oximetry for Cardiac and Vascular Surgery. Semin. Cardiothorac. Vasc. Anesth. 2004, 8, 147–166. [Google Scholar] [CrossRef]
  8. Haeussinger, F.B.; Heinzel, S.; Hahn, T.; Schecklmann, M.; Ehlis, A.-C.; Fallgatter, A.J. Simulation of Near-Infrared Light Absorption Considering Individual Head and Prefrontal Cortex Anatomy: Implications for Optical Neuroimaging. PLoS ONE 2011, 6, e26377. [Google Scholar] [CrossRef] [Green Version]
  9. Strangman, G.E.; Zhang, Q.; Li, Z. Scalp and Skull Influence on near Infrared Photon Propagation in the Colin27 Brain Template. Neuroimage 2014, 85 Pt 1, 136–149. [Google Scholar] [CrossRef]
  10. Zhang, Y.; Liu, X.; Wang, Q.; Liu, D.; Yang, C.; Sun, J. Influence of Extracerebral Layers on Estimates of Optical Properties with Continuous Wave Near Infrared Spectroscopy: Analysis Based on Multi-Layered Brain Tissue Architecture and Monte Carlo Simulation. Comput. Assist. Surg. 2019, 24, 144–150. [Google Scholar] [CrossRef]
  11. Afshari, A.; Ghassemi, P.; Lin, J.; Halprin, M.; Wang, J.; Mendoza, G.; Weininger, S.; Pfefer, T.J. Cerebral Oximetry Performance Testing with a 3D-Printed Vascular Array Phantom. Biomed. Opt. Express 2019, 10, 3731–3746. [Google Scholar] [CrossRef]
  12. Boas, D.A.; Elwell, C.E.; Ferrari, M.; Taga, G. Twenty years of functional near-infrared spectroscopy: Introduction for the special issue. NeuroImage 2014, 85, 1–5. [Google Scholar] [CrossRef] [PubMed]
  13. Pinti, P.; Tachtsidis, I.; Hamilton, A.; Hirsch, J.; Aichelburg, C.; Gilbert, S.; Burgess, P.W. The Present and Future Use of Functional Near-Infrared Spectroscopy (FNIRS) for Cognitive Neuroscience. Ann. N. Y. Acad. Sci. 2020, 1464, 5–29. [Google Scholar] [CrossRef] [PubMed]
  14. McInnes, M.D.F.; Moher, D.; Thombs, B.D.; McGrath, T.A.; Bossuyt, P.M.; Clifford, T.; Cohen, J.F.; Deeks, J.J.; Gatsonis, C.; Hooft, L.; et al. Preferred Reporting Items for a Systematic Review and Meta-Analysis of Diagnostic Test Accuracy Studies The PRISMA-DTA Statement. JAMA J. Am. Med. Assoc. 2018, 319, 388–396. [Google Scholar] [CrossRef]
  15. Sharma, S.D.; Park, E.; Purcell, P.L.; Gordon, K.A.; Papsin, B.C.; Cushing, S.L. Age-Related Variability in Pediatric Scalp Thickness: Implications for Auditory Prostheses. Int. J. Pediatr. Otorhinolaryngol. 2020, 130, 109853. [Google Scholar] [CrossRef]
  16. Smith, K.; Politte, D.; Reiker, G.; Nolan, T.S.; Hildebolt, C.; Mattson, C.; Tucker, D.; Prior, F.; Turovets, S.; Larson-Prior, L.J. Automated Measurement of Pediatric Cranial Bone Thickness and Density from Clinical Computed Tomography. Annu. Int. Conf. IEEE Eng. Med. Biol. Soc. 2012, 2012, 4462–4465. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  17. Whiting, P.F.; Rutjes, A.W.S.; Westwood, M.E.; Mallett, S.; Deeks, J.J.; Reitsma, J.B.; Leeflang, M.M.G.; Sterne, J.A.C.; Bossuyt, P.M.M. QUADAS-2: A Revised Tool for the Quality Assessment of Diagnostic Accuracy Studies. Ann. Intern. Med. 2011, 155, 529–536. [Google Scholar] [CrossRef] [PubMed]
  18. Kohl, C.; McIntosh, E.J.; Unger, S.; Haddaway, N.R.; Kecke, S.; Schiemann, J.; Wilhelm, R. Online Tools Supporting the Conduct and Reporting of Systematic Reviews and Systematic Maps: A Case Study on CADIMA and Review of Existing Tools. Environ. Evid. 2018, 7, 8. [Google Scholar] [CrossRef] [Green Version]
  19. de Quirós, M.B.; Douma, E.H.; van den Akker-Scheek, I.; Lamoth, C.J.C.; Maurits, N.M. Quantification of Movement in Stroke Patients under Free Living Conditions Using Wearable Sensors: A Systematic Review. Sensors 2022, 22, 1050. [Google Scholar] [CrossRef]
  20. Rohatgi, A. Webplotdigitizer, Version 4.6; Pacifica, CA, USA. September 2022. Available online: https://automeris.io/WebPlotDigitizer/citation.html (accessed on 4 April 2023).
  21. McHugh, M.L. Interrater Reliability: The Kappa Statistic. Biochem. Med. 2012, 22, 276–282. [Google Scholar] [CrossRef]
  22. Moher, D.; Liberati, A.; Tetzlaff, J.; Altman, D.G. Preferred Reporting Items for Systematic Reviews and Meta-Analyses: The PRISMA Statement. PLoS Med. 2009, 6, e1000097. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  23. Ogoh, S.; Sato, K.; Okazaki, K.; Miyamoto, T.; Secher, F.; Sørensen, H.; Rasmussen, P.; Secher, N.H. A Decrease in Spatially Resolved Near-Infrared Spectroscopy-Determined Frontal Lobe Tissue Oxygenation by Phenylephrine Reflects Reduced Skin Blood Flow. Anesth. Analg. 2014, 118, 823–829. [Google Scholar] [CrossRef] [PubMed]
  24. Logothetis, N.K.; Wandell, B.A. Interpreting the BOLD Signal. Annu. Rev. Physiol. 2004, 66, 735–769. [Google Scholar] [CrossRef] [PubMed]
  25. Delpy, D.T.; Cope, M.; Van Der Zee, P.; Arridge, S.; Wray, S.; Wyatt, J. Estimation of Optical Pathlength through Tissue from Direct Time of Flight Measurement. Phys. Med. Biol. 1988, 33, 1433–1442. [Google Scholar] [CrossRef] [Green Version]
  26. Liebeskind, D.S. Collateral Circulation. Stroke 2003, 34, 2279–2284. [Google Scholar] [CrossRef] [Green Version]
  27. Totaro, R.; Barattelli, G.; Quaresima, V.; Carolei, A.; Ferrari, M. Evaluation of Potential Factors Affecting the Measurement of Cerebrovascular Reactivity by Near-Infrared Spectroscopy. Clin. Sci. 1998, 95, 497–504. [Google Scholar] [CrossRef]
  28. Yang, H.C.; Liang, Z.; Vike, N.L.; Lee, T.; Rispoli, J.V.; Nauman, E.A.; Talavage, T.M.; Tong, Y. Characterizing Near-Infrared Spectroscopy Signal under Hypercapnia. J. Biophotonics 2020, 13, e202000173. [Google Scholar] [CrossRef]
  29. Holzschuh, M.; Woertgen, C.; Metz, C.; Brawanski, A. Comparison of Changes in Cerebral Blood Flow and Cerebral Oxygen Saturation Measured by Near Infrared Spectroscopy (NIRS) after Acetazolamide. Acta Neurochir. (Wien) 1997, 139, 58–62. [Google Scholar] [CrossRef]
  30. Smielewski, P.; Czosnyka, M.; Pickard, J.D.; Kirkpatrick, P. Assessment of Cerebrovascular Reactivity in Patients with Carotid Artery Disease Using Near-Infrared Spectroscopy. Acta Neurochir. Suppl. 1998, 1998, 263–265. [Google Scholar] [CrossRef]
  31. Smielewski, P.; Kirkpatrick, P.; Minhas, P.; Pickard, J.D.; Czosnyka, M. Can Cerebrovascular Reactivity Be Measured with Near-Infrared Spectroscopy? Stroke 1995, 26, 2285–2292. [Google Scholar] [CrossRef]
  32. Kirkpatrick, P.J.; Lam, J.; Al-Rawi, P.; Smielewski, P.; Czosnyka, M. Defining Thresholds for Critical Ischemia by Using Near-Infrared Spectroscopy in the Adult Brain. J. Neurosurg. 1998, 89, 389–394. [Google Scholar] [CrossRef] [PubMed]
  33. Hirasawa, A.; Yanagisawa, S.; Tanaka, N.; Funane, T.; Kiguchi, M.; Sørensen, H.; Secher, N.H.; Ogoh, S. Influence of Skin Blood Flow and Source-Detector Distance on near-Infrared Spectroscopy-Determined Cerebral Oxygenation in Humans. Clin. Physiol. Funct. Imaging 2015, 35, 237–244. [Google Scholar] [CrossRef]
  34. Schecklmann, M.; Mann, A.; Langguth, B.; Ehlis, A.C.; Fallgatter, A.J.; Haeussinger, F.B. The Temporal Muscle of the Head Can Cause Artifacts in Optical Imaging Studies with Functional Near-Infrared Spectroscopy. Front. Hum. Neurosci. 2017, 11, 456. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  35. Takahashi, T.; Takikawa, Y.; Kawagoe, R.; Shibuya, S.; Iwano, T.; Kitazawa, S. Influence of Skin Blood Flow on Near-Infrared Spectroscopy Signals Measured on the Forehead during a Verbal Fluency Task. Neuroimage 2011, 57, 991–1002. [Google Scholar] [CrossRef] [PubMed]
  36. Funane, T.; Sato, H.; Yahata, N.; Takizawa, R.; Nishimura, Y.; Kinoshita, A.; Katura, T.; Atsumori, H.; Fukuda, M.; Kasai, K.; et al. Concurrent FNIRS-FMRI Measurement to Validate a Method for Separating Deep and Shallow FNIRS Signals by Using Multidistance Optodes. Neurophotonics 2015, 2, 015003. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  37. Haeussinger, F.B.; Dresler, T.; Heinzel, S.; Schecklmann, M.; Fallgatter, A.J.; Ehlis, A.C. Reconstructing Functional Near-Infrared Spectroscopy (FNIRS) Signals Impaired by Extra-Cranial Confounds: An Easy-to-Use Filter Method. Neuroimage 2014, 95, 69–79. [Google Scholar] [CrossRef]
  38. Sato, H.; Yahata, N.; Funane, T.; Takizawa, R.; Katura, T.; Atsumori, H.; Nishimura, Y.; Kinoshita, A.; Kiguchi, M.; Koizumi, H.; et al. A NIRS-FMRI Investigation of Prefrontal Cortex Activity during a Working Memory Task. Neuroimage 2013, 83, 158–173. [Google Scholar] [CrossRef]
  39. Heinzel, S.; Haeussinger, F.B.; Hahn, T.; Ehlis, A.C.; Plichta, M.M.; Fallgatter, A.J. Variability of (Functional) Hemodynamics as Measured with Simultaneous FNIRS and FMRI during Intertemporal Choice. Neuroimage 2013, 71, 125–134. [Google Scholar] [CrossRef]
  40. Canova, D.; Roatta, S.; Bosone, D.; Micieli, G. Inconsistent Detection of Changes in Cerebral Blood Volume by near Infrared Spectroscopy in Standard Clinical Tests. J. Appl. Physiol. 2011, 110, 1646–1655. [Google Scholar] [CrossRef]
  41. Tachtsidis, I.; Tisdall, M.; Delpy, D.T.; Smith, M.; Elwell, C.E. Measurement of Cerebral Tissue Oxygenation in Young Healthy Volunteers during Acetazolamide Provocation: A Transcranial Doppler and near-Infrared Spectroscopy Investigation. Adv. Exp. Med. Biol. 2008, 614, 389–396. [Google Scholar] [CrossRef]
  42. Germon, T.J.; Young, A.E.R.; Manara, A.R.; Nelson, R.J. Extracerebral Absorption of near Infrared Light Influences the Detection of Increased Cerebral Oxygenation Monitored by near Infrared Spectroscopy. J. Neurol. Neurosurg. Psychiatry 1995, 58, 477–479. [Google Scholar] [CrossRef] [PubMed]
  43. Picton, P.; Vlisides, P.E.; Teig, M.K.; Heth, J.A.; Orringer, D.; Brooks, J.; McKinney, A.; Mentz, G.; Mashour, G.A. Correlation between Brain Tissue Oxygen Tension and Regional Cerebral Oximetry in Uninjured Human Brain under Conditions of Changing Ventilation Strategy. J. Clin. Monit. Comput. 2022, 36, 1227–1232. [Google Scholar] [CrossRef] [PubMed]
  44. Al-Rawi, P.G.; Smielewski, P.; Kirkpatrick, P.J. Evaluation of a Near-Infrared Spectrometer (NIRO 300) for the Detection of Intracranial Oxygenation Changes in the Adult Head. Stroke 2001, 32, 2492–2499. [Google Scholar] [CrossRef]
  45. Yoshitani, K.; Kawaguchi, M.; Tatsumi, K.; Kitaguchi, K.; Furuya, H. A Comparison of the INVOS 4100 and the NIRO 300 Near-Infrared Spectrophotometers. Anesth. Analg. 2002, 94, 586–590. [Google Scholar] [CrossRef]
  46. Henson, L.C.; Temp, J.A.; Ward, D.S. Accuracy of a Cerebral Oximeter in Healthy Volunteers under Conditions of Isocapnic Hypoxia. Anesthesiology 1998, 88, 58–65. [Google Scholar] [CrossRef] [PubMed]
  47. Grubhofer, G.; Tonninger, W.; Keznickl, P.; Skyllouriotis, P.; Ehrlich, M.; Hiesmayr, M.; Lassnigg, A. A Comparison of the Monitors INVOS 3100 and NIRO 500 in Detecting Changes in Cerebral Oxygenation. Acta Anaesthesiol. Scand. 1999, 43, 470–475. [Google Scholar] [CrossRef]
  48. Samra, S.K.; Stanley, J.C.; Zelenock, G.B.; Dorje, P. An Assessment of Contributions Made by Extracranial Tissues during Cerebral Oximetry. J. Neurosurg. Anesth. 1999, 11, 1–5. [Google Scholar] [CrossRef]
  49. Cho, H.; Nemoto, E.M.; Yonas, H.; Balzer, J.; Sclabassi, R.J. Cerebral Monitoring by Means of Oximetry and Somatosensory Evoked Potentials during Carotid Endarterectomy. J. Neurosurg. 1998, 89, 533–538. [Google Scholar] [CrossRef]
  50. Kato, S.; Yoshitani, K.; Kubota, Y.; Inatomi, Y.; Ohnishi, Y. Effect of Posture and Extracranial Contamination on Results of Cerebral Oximetry by Near-Infrared Spectroscopy. J. Anesth. 2017, 31, 103–110. [Google Scholar] [CrossRef]
  51. Germon, T.J.; Kane, N.M.; Manara, A.R.; Nelson, R.J. Near-Infrared Spectroscopy in Adults: Effects of Extracranial Ischaemia and Intracranial Hypoxia on Estimation of Cerebral Oxygenation. Br. J. Anaesth. 1994, 73, 503–506. [Google Scholar] [CrossRef]
  52. Greenberg, S.; Murphy, G.; Shear, T.; Patel, A.; Simpson, A.; Szokol, J.; Avram, M.J.; Vender, J. Extracranial Contamination in the INVOS 5100C versus the FORE-SIGHT ELITE Cerebral Oximeter: A Prospective Observational Crossover Study in Volunteers. Can. J. Anesth. 2016, 63, 24–30, Erratum in Can. J. Anesth. 2016, 63, 512. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  53. Davie, S.N.; Grocott, H.P. Impact of Extracranial Contamination on Regional Cerebral Oxygen Saturation: A Comparison of Three Cerebral Oximetry Technologies. Anesthesiology 2012, 116, 834–840. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  54. Sørensen, H.; Rasmussen, P.; Siebenmann, C.; Zaar, M.; Hvidtfeldt, M.; Ogoh, S.; Sato, K.; Kohl-Bareis, M.; Secher, N.H.; Lundby, C. Extra-Cerebral Oxygenation Influence on near-Infrared-Spectroscopy-Determined Frontal Lobe Oxygenation in Healthy Volunteers: A Comparison between INVOS-4100 and NIRO-200NX. Clin. Physiol. Funct. Imaging 2015, 35, 177–184. [Google Scholar] [CrossRef] [PubMed]
  55. Germon, T.J.; Evans, P.D.; Barnett, N.J.; Wall, P.; Manara, A.R.; Nelson, R.J. Cerebral near Infrared Spectroscopy: Emitter-Detector Separation Must Be Increased. Br. J. Anaesth. 1999, 82, 831–837. [Google Scholar] [CrossRef] [PubMed]
  56. Germon, T.J.; Evans, P.D.; Manara, A.R.; Barnett, N.J.; Wall, P.; Nelson, R.J. Sensitivity of near Infrared Spectroscopy to Cerebral and Extra-Cerebral Oxygenation Changes Is Determined Dy Emitter-Detector Separation. J. Clin. Monit. Comput. 1998, 14, 353–360. [Google Scholar] [CrossRef]
  57. Tateishi, A.; Maekawa, T.; Soejima, Y.; Sadamitsu, D.; Yamamoto, M.; Matsushita, M.; Nakashima, K. Qualitative Comparison of Carbon Dioxide-Induced Change in Cerebral near-Infrared Spectroscopy versus Jugular Venous Oxygen Saturation in Adults with Acute Brain Disease. Crit. Care Med. 1995, 23, 1734–1738. [Google Scholar] [CrossRef]
  58. Hirasawa, A.; Kaneko, T.; Tanaka, N.; Funane, T.; Kiguchi, M.; Sørensen, H.; Secher, N.H.; Ogoh, S. Near-Infrared Spectroscopy Determined Cerebral Oxygenation with Eliminated Skin Blood Flow in Young Males. J. Clin. Monit. Comput. 2016, 30, 243–250. [Google Scholar] [CrossRef]
  59. Moerman, A.T.; Vandenheuvel, M.; Tuybens, P.J.; Van Gompel, C.; De Hert, S.G. Incongruous Effect of Phenylephrine on Changes in Cerebral Blood Volume Measured by Near-Infrared Spectroscopy (NIRS) Indicating Extracranial Contamination. J. Clin. Monit. Comput. 2021, 36, 745–750. [Google Scholar] [CrossRef]
  60. Moerman, A.; Wouters, P. Near-Infrared Spectroscopy (NIRS) Monitoring in Contemporary Anesthesia and Critical Care. Acta Anaesthesiol. Belg 2010, 61, 185–194. [Google Scholar]
  61. Yao, F.S.F.; Tseng, C.C.A.; Ho, C.Y.A.; Levin, S.K.; Illner, P. Cerebral Oxygen Desaturation Is Associated with Early Postoperative Neuropsychological Dysfunction in Patients Undergoing Cardiac Surgery. J. Cardiothorac. Vasc. Anesth. 2004, 18, 552–558. [Google Scholar] [CrossRef]
  62. Fischer, G.W.; Lin, H.M.; Krol, M.; Galati, M.F.; Luozzo, G.; Griepp, R.B.; Reich, D.L. Noninvasive Cerebral Oxygenation May Predict Outcome in Patients Undergoing Aortic Arch Surgery. J. Thorac. Cardiovasc. Surg. 2011, 141, 815–821. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  63. Goldman, S.; Sutter, F.; Ferdinand, F.; Trace, C. Optimizing Intraoperative Cerebral Oxygen Delivery Using Noninvasive Cerebral Oximetry Decreases the Incidence of Stroke for Cardiac Surgical Patients. Heart Surg. Forum 2004, 7, 376–381. [Google Scholar] [CrossRef] [PubMed]
  64. Orihashi, K.; Sueda, T.; Okada, K.; Imai, K. Near-Infrared Spectroscopy for Monitoring Cerebral Ischemia during Selective Cerebral Perfusion. Eur. J. Cardio-Thorac. Surg. 2004, 26, 907–911. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  65. Olsson, C.; Thelin, S. Regional Cerebral Saturation Monitoring with Near-Infrared Spectroscopy during Selective Antegrade Cerebral Perfusion: Diagnostic Performance and Relationship to Postoperative Stroke. J. Thorac. Cardiovasc. Surg. 2006, 131, 371–379. [Google Scholar] [CrossRef] [Green Version]
  66. Higami, T.; Kozawa, S.; Asada, T.; Obo, H.; Gan, K.; Iwahashi, K.; Nohara, H. Retrograde Cerebral Perfusion versus Selective Cerebral Perfusion as Evaluated by Cerebral Oxygen Saturation during Aortic Arch Reconstruction. Ann. Thorac. Surg. 1999, 67, 1091–1096. [Google Scholar] [CrossRef]
  67. Lam, S.; Liu, H.; Jian, Z.; Settels, J.; Bohringer, C. Intraoperative Invasive Blood Pressure Monitoring and the Potential Pitfalls of Invasively Measured Systolic Blood Pressure. Cureus 2021, 13, e17610. [Google Scholar] [CrossRef]
  68. Lange, F.; Tachtsidis, I. Clinical Brain Monitoring with Time Domain NIRS: A Review and Future Perspectives. Appl. Sci. 2019, 9, 1612. [Google Scholar] [CrossRef] [Green Version]
  69. Bein, B.; Cavus, E.; Stadlbauer, K.H.; Tonner, P.H.; Steinfath, M.; Scholz, J.; Dörges, V. Monitoring of Cerebral Oxygenation with near Infrared Spectroscopy and Tissue Oxygen Partial Pressure during Cardiopulmonary Resuscitation in Pigs. Eur. J. Anaesthesiol. 2006, 23, 501–509. [Google Scholar] [CrossRef]
  70. Nosrati, R.; Lin, S.; Ramadeen, A.; Monjazebi, D.; Dorian, P.; Toronov, V. Cerebral Hemodynamics and Metabolism during Cardiac Arrest and Cardiopulmonary Resuscitation Using Hyperspectral near Infrared Spectroscopy. Circ. J. 2017, 81, 879–887. [Google Scholar] [CrossRef] [Green Version]
  71. Dixon, B.; Turner, R.; Christou, C. Assessment of a Non-Invasive Brain Oximeter in a Sheep Model of Acute Brain Injury. Med. Devices Evid. Res. 2019, 12, 479–487. [Google Scholar] [CrossRef] [Green Version]
  72. Elliott, J.T.; Diop, M.; Tichauer, K.M.; Lee, Y.T.; St. Lawrence, K. Measurement of Cerebral Blood Flow in the Adult Pig by Depth-Resolved Broadband near-Infrared Spectroscopy. In Proceedings of the Journal of Cerebral Blood Flow & Metabolism; Elliott, J.T., Ed.; Imaging Division, Lawson Health Research Institute: London, ON, Canada, 2009; Volume 29, pp. S394–S406. [Google Scholar]
  73. Nosrati, R.; Ramadeen, A.; Hu, X.; Woldemichael, E.; Kim, S.; Dorian, P.; Toronov, V. Simultaneous Measurement of Cerebral and Muscle Tissue Parameters during Cardiac Arrest and Cardiopulmonary Resuscitation. In Proceedings of the Optical Techniques in Neurosurgery, Neurophotonics, and Optogenetics II; SPIE: San Francisco, CA, USA, 2015; Volume 9305, p. 93051G. [Google Scholar]
  74. Gerega, A.; Lachowska, M.; Morrison, L.; St. Lawrence, K.; Liebert, A. Multi-Wavelength Time-Resolved NIRS Measurements in Pigs During Inflow and Washout of ICG: Assessment of Extracerebral Signal Contamination. In Proceedings of the Biophotonics Congress: Biomedical Optics Congress 2018 (Microscopy/Translational/Brain/OTS); OSA: Washington, DC, USA, 2018; p. JTh3A.62. [Google Scholar]
  75. Lotfabadi, S.S.; Toronov, V.; Ramadeen, A.; Hu, X.; Kim, S.; Dorian, P.; Hare, G.M.T. Quantitative Assessment of Brain Tissue Oxygenation in Porcine Models of Cardiac Arrest and Cardiopulmonary Resuscitation Using Hyperspectral Near-Infrared Spectroscopy. In Proceedings of the Optical Techniques in Neurosurgery, Neurophotonics, and Optogenetics; SPIE: San Francisco, CA, USA, 2014; Volume 8928. [Google Scholar]
  76. Zarei, M.; Ansari, M.A.; Zare, K. The Temporal Confounding Effects of Extra-Cerebral Contamination Factors on the Hemodynamic Signal Measured by Functional near-Infrared Spectroscopy. J. Lasers Med. Sci. 2019, 10, S73–S81. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  77. Smielewski, P.; Czosnyka, M.; Pickard, J.D.; Kirkpatrick, P. Clinical Evaluation of Near-Infrared Spectroscopy for Testing Cerebrovascular Reactivity in Patients with Carotid Artery Disease. Stroke 1997, 28, 331–338. [Google Scholar] [CrossRef]
  78. Takeda, N.; Fujita, K.; Katayama, S.; Tamaki, N. Cerebral Oximetry for the Detection of Cerebral Ischemia during Temporary Carotid Artery Occlusion. Neurol. Med. Chir. 2000, 40, 557–563. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  79. Steinbrink, J.M.; Kempf, F.; Schreiber, S.; Uludag, K.; Kohl, M.; Valdueza, J.M.; Villringer, A.; Obrig, H. Functional Brain Imaging by CW-NIRS Coregistered by Blood Flow Monitors. In Proceedings of the Photon Migration and Diffuse-Light Imaging; SPIE: San Francisco, CA, USA, 2003; Volume 5138. [Google Scholar]
  80. Toronov, V.Y.; Webb, A.; Choi, J.H.; Wolf, M.; Gratton, E.; Hueber, D.M. Simultaneous Functional Magnetic Resonance and Near-Infrared Imaging of Adult Human Brain. In Proceedings of the Optical Tomography and Spectroscopy of Tissue IV; SPIE: San Francisco, CA, USA, 2001; Volume 4250, pp. 380–382. [Google Scholar]
  81. Duncan, L.A.; Ruckley, C.V.; Wildsmith, J.A. Cerebral Oximetry: A Useful Monitor during Carotid Artery Surgery. Anaesthesia 1995, 50, 1041–1045. [Google Scholar] [CrossRef]
  82. Lam, J.M.K.K.; Smielewski, P.; Al-Rawi, P.; Griffiths, P.; Pickard, J.D.; Kirkpatrick, P.J. Internal and External Carotid Contributions to Near-Infrared Spectroscopy during Carotid Endarterectomy. Stroke 1997, 28, 906–911. [Google Scholar] [CrossRef] [PubMed]
Jcm 12 02776 g001 550
Figure 1. Screening process. Screening followed the Preferred Reporting Items for Systematic Reviews and Meta-Analysis of Diagnostic Test Accuracy Studies (PRISMA-DTA) Statement [14]. Exclusion of full text articles was possible based on multiple criteria. Sub-studies of sufficient quality are those for which more than 50% of the QUADAS-2 domains are assessed as ‘Low Risk of Bias’. NIRS, near-infrared spectroscopy; QUADAS-2, Quality Assessment of Diagnostic Accuracy Studies, revised second version; WoS, Web of Science. Figure adapted from Moher and colleagues, with permission [22].
Figure 1. Screening process. Screening followed the Preferred Reporting Items for Systematic Reviews and Meta-Analysis of Diagnostic Test Accuracy Studies (PRISMA-DTA) Statement [14]. Exclusion of full text articles was possible based on multiple criteria. Sub-studies of sufficient quality are those for which more than 50% of the QUADAS-2 domains are assessed as ‘Low Risk of Bias’. NIRS, near-infrared spectroscopy; QUADAS-2, Quality Assessment of Diagnostic Accuracy Studies, revised second version; WoS, Web of Science. Figure adapted from Moher and colleagues, with permission [22].
Jcm 12 02776 g001
Jcm 12 02776 g002 550
Figure 2. Critical appraisal results. Left panel: the proportion of 116 sub-studies that scored a low, high, or unclear risk of bias for each modified QUADAS-2 domain. Right panel: the proportion of 116 sub-studies for which there were low, high, or unclear concerns regarding applicability for each modified QUADAS-2 domain. Figure following the recommendations of Whiting and colleagues [17]. Detailed classification results are provided in Table S4. QUADAS-2, Quality Assessment of Diagnostic Accuracy Studies, revised second version.
Figure 2. Critical appraisal results. Left panel: the proportion of 116 sub-studies that scored a low, high, or unclear risk of bias for each modified QUADAS-2 domain. Right panel: the proportion of 116 sub-studies for which there were low, high, or unclear concerns regarding applicability for each modified QUADAS-2 domain. Figure following the recommendations of Whiting and colleagues [17]. Detailed classification results are provided in Table S4. QUADAS-2, Quality Assessment of Diagnostic Accuracy Studies, revised second version.
Jcm 12 02776 g002
Jcm 12 02776 g003 550
Figure 3. Haemoglobin concentrations; correlation between reference techniques and NIRS indices per study. Rows represent individual sub-studies. Columns have been grouped into intracerebral and extracerebral reference techniques and per NIRS index (OxyHb, HHb, Hbdiff, tHb). Circle size represents the percentage of CA domains categorized as ‘Low’ risk of bias or applicability concerns. Colour represents the correlation, with hot colours expressing positive correlation and cold colours negative correlation. Correlation is also expressed numerically in the circle. Source–detector separation (SDS) is shown in the second column. *: NIRS-index was corrected for extracerebral contamination. Extra, extracerebral; fMRI, functional magnetic resonance imaging; FT, finger tapping task, Hb, haemoglobin; Hbdiff, difference between OxyHb and HHb; HHb, deoxygenated Hb; intra, intracerebral; ITC, intertemporal choice task; LDF, laser doppler flowmetry; NIRS, near-infrared spectroscopy; Not spec., no specifically intra-or extracerebral perfusion modification; OxyHb, oxygenated Hb; PPG, photoplethysmography; RoB, risk of bias; SC, short-channel; SDS, source–detector separation; TCD, transcranial doppler; tHb, total Hb; VFT, verbal fluency task; VM, Valsalva manoeuvre; WMT, working memory task. Studies are [27,28,29,30,31,32,33,34,35,36,37,38,39,40]. Figure inspired by Bernaldo-de-Quiros and co-workers [19].
Figure 3. Haemoglobin concentrations; correlation between reference techniques and NIRS indices per study. Rows represent individual sub-studies. Columns have been grouped into intracerebral and extracerebral reference techniques and per NIRS index (OxyHb, HHb, Hbdiff, tHb). Circle size represents the percentage of CA domains categorized as ‘Low’ risk of bias or applicability concerns. Colour represents the correlation, with hot colours expressing positive correlation and cold colours negative correlation. Correlation is also expressed numerically in the circle. Source–detector separation (SDS) is shown in the second column. *: NIRS-index was corrected for extracerebral contamination. Extra, extracerebral; fMRI, functional magnetic resonance imaging; FT, finger tapping task, Hb, haemoglobin; Hbdiff, difference between OxyHb and HHb; HHb, deoxygenated Hb; intra, intracerebral; ITC, intertemporal choice task; LDF, laser doppler flowmetry; NIRS, near-infrared spectroscopy; Not spec., no specifically intra-or extracerebral perfusion modification; OxyHb, oxygenated Hb; PPG, photoplethysmography; RoB, risk of bias; SC, short-channel; SDS, source–detector separation; TCD, transcranial doppler; tHb, total Hb; VFT, verbal fluency task; VM, Valsalva manoeuvre; WMT, working memory task. Studies are [27,28,29,30,31,32,33,34,35,36,37,38,39,40]. Figure inspired by Bernaldo-de-Quiros and co-workers [19].
Jcm 12 02776 g003
Jcm 12 02776 g004 550
Figure 4. Oxygen saturation indices (rSO2): Correlation between reference techniques and NIRS-indices per study. Rows represent individual sub-studies. Columns have been grouped into intracerebral and extracerebral reference techniques. Circle size represents the percentage of CA domains categorized as ‘Low’ risk of bias or applicability concerns. Colour represents correlation, with hot colours expressing positive correlation and cold colours negative correlation. Correlation is also expressed numerically in the circle. Duplex ECA, duplex sonography of the external carotid artery; ICA, internal carotid artery; LDF, laser Doppler flowmetry; PbrO2, brain tissue oxygen; RoB, risk of bias; TCD, transcranial Doppler. Studies are [23,41,42,43,44]. Figure inspired by Bernaldo-de-Quiros and co-workers [19].
Figure 4. Oxygen saturation indices (rSO2): Correlation between reference techniques and NIRS-indices per study. Rows represent individual sub-studies. Columns have been grouped into intracerebral and extracerebral reference techniques. Circle size represents the percentage of CA domains categorized as ‘Low’ risk of bias or applicability concerns. Colour represents correlation, with hot colours expressing positive correlation and cold colours negative correlation. Correlation is also expressed numerically in the circle. Duplex ECA, duplex sonography of the external carotid artery; ICA, internal carotid artery; LDF, laser Doppler flowmetry; PbrO2, brain tissue oxygen; RoB, risk of bias; TCD, transcranial Doppler. Studies are [23,41,42,43,44]. Figure inspired by Bernaldo-de-Quiros and co-workers [19].
Jcm 12 02776 g004
Table
Table 1. Inclusion criteria for title and abstract, and full text screening. Studies were assessed in five domains: population, outcome measure, exposure, study quality and type, and study methodology. Hb, haemoglobin; HHb, deoxygenated haemoglobin; OxyHb, oxygenated haemoglobin; rSO2, regional tissue saturation of oxygen; StO2, tissue saturation of oxygen; TOI, tissue oxygenation index; TSI, tissue saturation index.
Table 1. Inclusion criteria for title and abstract, and full text screening. Studies were assessed in five domains: population, outcome measure, exposure, study quality and type, and study methodology. Hb, haemoglobin; HHb, deoxygenated haemoglobin; OxyHb, oxygenated haemoglobin; rSO2, regional tissue saturation of oxygen; StO2, tissue saturation of oxygen; TOI, tissue oxygenation index; TSI, tissue saturation index.
DomainScreening Criteria
PopulationHuman adults (>18 years) or adult large animal models
Outcome measureCerebral continuous wave or frequency-domain near-infrared spectroscopy:
OxyHb, HHb, Total Hb (tHb), Hbdiff (difference between OxyHb and HHb), or a tissue oxygen saturation index (rSO2). Other manufacturer-specific oxygenation indices are allowed, e.g., TOI, TSI, StO2.
ExposureInfluence of extracerebral tissue (scalp, skull, cerebrospinal fluid, or dura mater) was an explicit goal of the research, i.e., it is a primary or secondary research question, or primary or secondary research goalANDReference techniques for both the intra- and extracerebral tissue that measure blood flow, Hb concentrations or tissue oxygen saturation
OR
A perfusion modification protocol selective for the intra- or extracerebral tissue
Study quality and typeEmpirical peer-reviewed research presenting original data. No reviews, book chapters, commentaries, conference abstracts without a full text, or case reports
Study methodologyOnly in vivo studies; no computational modelling, phantom, or tissue sample studies
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

Share and Cite

MDPI and ACS Style

Eleveld, N.; Esquivel-Franco, D.C.; Drost, G.; Absalom, A.R.; Zeebregts, C.J.; de Vries, J.-P.P.M.; Elting, J.W.J.; Maurits, N.M. The Influence of Extracerebral Tissue on Continuous Wave Near-Infrared Spectroscopy in Adults: A Systematic Review of In Vivo Studies. J. Clin. Med. 2023, 12, 2776. https://doi.org/10.3390/jcm12082776

AMA Style

Eleveld N, Esquivel-Franco DC, Drost G, Absalom AR, Zeebregts CJ, de Vries J-PPM, Elting JWJ, Maurits NM. The Influence of Extracerebral Tissue on Continuous Wave Near-Infrared Spectroscopy in Adults: A Systematic Review of In Vivo Studies. Journal of Clinical Medicine. 2023; 12(8):2776. https://doi.org/10.3390/jcm12082776

Chicago/Turabian Style

Eleveld, Nick, Diana C. Esquivel-Franco, Gea Drost, Anthony R. Absalom, Clark J. Zeebregts, Jean-Paul P. M. de Vries, Jan Willem J. Elting, and Natasha M. Maurits. 2023. "The Influence of Extracerebral Tissue on Continuous Wave Near-Infrared Spectroscopy in Adults: A Systematic Review of In Vivo Studies" Journal of Clinical Medicine 12, no. 8: 2776. https://doi.org/10.3390/jcm12082776

APA Style

Eleveld, N., Esquivel-Franco, D. C., Drost, G., Absalom, A. R., Zeebregts, C. J., de Vries, J.-P. P. M., Elting, J. W. J., & Maurits, N. M. (2023). The Influence of Extracerebral Tissue on Continuous Wave Near-Infrared Spectroscopy in Adults: A Systematic Review of In Vivo Studies. Journal of Clinical Medicine, 12(8), 2776. https://doi.org/10.3390/jcm12082776

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop