From Health Technology Assessment to Health Technology Sustainability
Department of Management, Sapienza University of Rome, 00161 Rome, Italy
*
Author to whom correspondence should be addressed.
Sustainability 2018, 10(12), 4748; https://doi.org/10.3390/su10124748
Submission received: 30 October 2018
/
Revised: 3 December 2018
/
Accepted: 11 December 2018
/
Published: 12 December 2018
(This article belongs to the Special Issue Sustainability for Healthcare)
Abstract
:This paper aims to propose a methodological lens to the assessment of technological innovations in healthcare based on the principles of social, economic, and political sustainability. Starting from the consideration of a lack of a unified interpretative framework of health technology assessment, using a content analysis of the relevant literature on the topic, we identified both the scientific perspectives adopted by the scholars and the most widely discussed topics. Consequently, the less explored scientific areas were framed, and, therefore, those more susceptible to further investigation came to light. The result is an overall picture which highlights the absence of unified and generally accepted approaches to evaluation, together with the lack of awareness on the fact that the multiplicity of methods adopted is essentially connected to the multiplicity of innovations, for each of which a method (or a set of methods) of preferable evaluation can be prefigured. Based on these observations, we propose a general reference framework for evaluation, based on the Viable Systems Approach (vSa), and a schematic outline of the connections between the complexity of innovations and the evaluation methodologies.
1. Introduction
The increasing attention to the treatment and care of patients makes central the role of healthcare technologies, as they are in constant development [1]. Indeed, technology has the potential to enhance efficiency and effectiveness in multiple areas of healthcare, and represents a great opportunity to improve the patient care and to increase the efficiency and effectiveness of all healthcare stakeholders, including policy makers, regulatory authorities, payers, and physicians [2]. Remote sensors, robotics, genomics, stem cells, and the availability of big data are just some of the innovations in this sector [3]. Only considering artificial intelligence, there are already numerous examples of how it can enhance the medical profession. Robot-assisted surgery, virtual nursing assistants, and technologies are able to identify changes in the usual behavior of patients; it also opens a wide debate on ethical and social challenges [4]. However, technology alone cannot improve the whole healthcare system, and a successful implementation requires a complete understanding “a priori” of the technology capabilities and its application [5]. Moreover, the economic and health policy issues increasingly influence healthcare technology solutions, with the aim to keep the healthcare quality high and the costs under control, especially regarding the pharmaceutical sector [3].
Considering the complexity of the healthcare context, the Health Technology Assessment (HTA) has been developed as a set of techniques able to support decisions regarding health policy, pushed by two driving forces: one relates to the more and more serious budget constraints due to the recession; the other relates to the increasingly demanding policy makers and funders, who require greater evidence for new and existing therapies [6].
Adopted primarily in the USA in the 1970s [7,8] and rapidly spread in Europe in the following decades, HTAs are currently being introduced in most countries of the world [3]. During the 1970s, the focus was to summarize the evidence concerning studies on the cost effectiveness of health interventions. Afterwards, at the end of the 1990s, HTA widened its focus from the evaluation of only large, expensive and machine-based technologies also to smaller technologies and healthcare, addressing broader issues (organizational, social implications, and ethics) [7]. Currently, the health technology assessment processes critically evaluate reimbursement submissions of pharmaceuticals, simple medical devices, and complex medical devices, such as hospital technologies [9], vaccines, procedures, health services, and public health interventions. Therefore, they are becoming an important tool to support health policy decisions in many countries [9].
Nowadays, the importance of HTAs has increased further, also spreading in Central and Eastern European countries. HTA activities have a national focus associated with the Ministry of Health and, by influencing health policy documents of the European Commission, it seems likely that, in the future, the HTA will be institutionalized somehow as part of the EU’s activities. The task of applying HTA techniques for the evaluation of new health technologies is devolved to the different government support bodies (e.g., NICE in the UK, IQWIG in Germany, and AGENAS in Italy).
Thus, HTAs act to define a policy research approach that examines the short and long-term social, economic, and political consequences of the application or use of technology [10], evaluating in a multidisciplinary way, medical technology about efficacy, safety, feasibility, cost, cost-effectiveness, and indications for use [11].
Although many attempts have been carried out, a unified approach has not yet been defined, and the institutions may fail to optimize their future technology acquisition, thus purely considering the medical innovation as an increasing financial risk [12,13]. Furthermore, several biomedical technologies approved are adopted based on limited evidence of safety and effectiveness [3,5,14]. Indeed, these innovative therapies, as digital innovations and revolutionary technologies (such as 3D bioprinting), present not only opportunities but also a complex set of technical, ethical, and financial challenges. Healthcare consumers are also inclined to be more sensitive to medical treatments costs, whether they value more or less the information in reducing future health uncertainty [15]. In any case, considering the real potential of the new technologies to improve the healthcare system, a new approach is required, able to support decision-makers to modernize the pre-existing framework and to exploit new technologies in a more efficient, effective and sustainable way [2] while also protecting patients, spending resources more wisely, and fostering the “right” type of innovation in the future [3].
Currently, cost effectiveness analysis is the most frequently used methodology [16] but, as far as it supports the allocation of resources [4], it suffers from theoretical limitations that make it inappropriate in many situations, leading to contraindications [17,18].
In particular, technological innovation, as regards cost-containment measures, is an important factor for managing high and increasing health costs. In literature, the evidence suggests that enhanced health technologies generally increase rather than reduce healthcare expenditures. Indeed, nations with a greater degree of integration into the health system have relied on spending controls and global budgets to control costs [15,19].
In this direction, this paper reviews the different contributions existing in literature regarding the methodologies connected to the HTA, highlighting the alternative approaches proposed [20,21] and identifying the main weaknesses.
A lack of unified generally accepted analysis schemes still exists. The methodologies currently used are consequently unsatisfactory: the literature that criticizes the cost-effectiveness analysis (CEA) is wide, currently being the most widespread evaluation methodology. Similarly, the literature that proposes alternative approaches to evaluation is wide as well.
Starting from this, we noted the lack of a holistic vision in HTA techniques able to understand—even if not able to quantify, as in the ambition of the cost-benefit analysis—the multiplicity of subjects and effects of innovation in healthcare. In this regard, we propose a rereading of the existing evaluation methods, through the interpretative lens of the Viable Systems Approach (vSa) [22,23,24], with the aim to consider, in this complex scenario, the multiplicity of entities and effects that over time are influenced by new technologies. A systems approach to decision-making is suggested, able to consider economic, social, and political aspects together with relations and interactions between them [25].
Few contributions address the topic of healthcare in a systems perspective and together with the topic of evaluations: in this sense, vSa becomes a bridging concept by the means of which the sustainability perspective can be incorporated into the management control system of healthcare organizations [26].
Based on previous considerations, the paper is structured as follows: Section 2 introduces the methodology adopted herein for a content analysis of the literature contributions on HTA. Thereafter, the viable systems approach is presented, as a theoretical framework for the analysis of health technology assessment, in the light of sustainability issues. Finally, a re-reading of healthcare assessment methods is proposed. The paper concludes with a discussion and conclusions.
2. Materials and Methods
In order to carry out a content analysis of the relevant literature on the topic, we used the textual contents of the scientific contributions on the analysed subject/topic, derived from the set of keywords, consisting of the author keywords plus those provided by Scopus through the encoding of the abstracts and the titles of the bibliographic references, contained in each recorded contribution. We used the method of co-occurrence of the keywords as a basis for the analysis, with the aim of identifying the scientific perspectives adopted by the scholars and the most widely discussed topics and, consequently, discerning the less explored scientific areas and those more susceptible to further investigation. The method adopted herein was the content analysis [27,28], modified due to the availability of textual data already codified by third parties. This change appears to provide a number of advantages: it abstracts from the subjectivity of those who carry out the codification; it guarantees the repeatability of the analysis, as the text bases are publicly available; it calls for the progressive formulation of hypotheses according to the discovery principles of grounded theories [29,30].
Data Collection and Methodology
The multidisciplinary nature of the topic made it necessary to limit the research to the contributions that are most consistent with the objectives of our work (the “Health Technology Assessment” query on Scopus returns 4511 results without limitations, 2/3 of which in the medicine area).
Therefore, data were collected through a research on Scopus using the query TITLE-ABS-KEY (“health technology assessment”) AND (LIMIT-TO (SUBJAREA, “ECON”) OR LIMIT-TO (SUBJAREA, “SOCI”) OR LIMIT-TO (SUBJAREA, “BUSI”) OR LIMIT-TO (SUBJAREA, “DECI”)) AND LIMIT-TO (DOCTYPE, “ar”)).
The results were 212 contributions, in the subject areas of Economics, Econometrics and Finance, Social Sciences, Business, Management and Accounting and Decision Sciences. We exported the results, choosing to consider all of the available information, except the index keywords (as to only analyse the author keywords), in CSV (Excel) format.
The data collection method, being based on the Scopus database, is subject to the shortcomings of the database itself, most notably the presence of duplications. For example, one study found 12% of the records in the seven Scopus-indexed journals to be duplicate [31]. As to solve the problem of data duplication, Vuong et al. [32] introduced an open database of the scientific output of Vietnamese researchers, using scientist’s self-reports, open online sources, and cross-checking with Scopus database. Given the size of the sample in our study, it was possible to verify the presence of duplications directly in the exported CSV file from Scopus and edit it accordingly.
The adapted version of the content analysis required the use of the VosViewer software [33,34], developed with the specific purpose of constructing, displaying and making publicly available bibliometric maps. VosViewer software provides distance-based bibliometric maps, e.g., graphical representations, in which the importance of a term is represented by its size, and the distance between two terms reflects the strength of the relationships between them: the smaller the distance, the more intense the relationship that binds them.
For the type of the analysis, we used the co-occurrence model of the author keywords in fractional counting. Aiming to regroup the same keywords expressed in different ways (e.g., Health Technology Assessment, HTA) and delete the keywords not strictly relevant to the purposes of this article (for example those that indicate the specific geopolitical areas or specific pathologies) we created a thesaurus, as presented in Table A1. The resulting keywords were 1724, 160 of which repeated at least five times, allowing us to obtain the map represented in the “density visualization” mode (Figure 1) and in the “network visualization” mode (Figure 2) (keywords extracted: 104). The map has been made publicly available, as seen in the Supplementary Materials section.
We made available, in Table A2, all of the 212 contributions considered in the content analysis, displaying the weight of each contribution with regards to total link strength and citations, as to show the closeness of data points.
Table 1 shows the strength of the links of the main keywords with the keyword “Health Technology Assessment”.
Data in Table 1 essentially show two scientific areas: that of decision-making to support health policy for the design of effective and efficient Health Care Systems, in which ethical issues are very important, and that of evaluation techniques and implementation of results with methodological insights and adaptations to local situations (Cost Benefit Analysis, Economic Evaluation, Reimbursement).
3. Results
The analysis carried out highlights two fundamental aspects of the problem:
- the lack of a holistic approach which, starting from evaluation methodologies able to embrace the complex of direct and indirect effects of the introduction of an innovation in healthcare on all the actors involved, constitutes a general and commonly accepted framework of reference; and
- the lack of awareness that the multiplicity of the methods adopted derives from the multiplicity of the characteristics of the innovations, for each of which specific methods are preferable.
Regarding the first aspect, the principles of the vSa described below are certainly helpful. Regarding extremely differentiated fields of investigation, far from the ambition of realizing a taxonomy of the innovations in healthcare to each class of which linking a particular evaluation technique, the considerations of Section 3.2 below are, however, possible.
3.1. The vSa as a Framework for the Analysis of Unified Health Assessment
As emerges from the above, the fundamental Health Technology Assessment seems to be focused mainly on cost-effectiveness considerations rather than on the assessment of the benefits of technology. Considering also the lack of an interpretative framework of HTA based on the principles of social, economic, and political sustainability [25], we aim to propose a methodological lens to the analysis of unified healthcare assessment that overcomes a reductionist approach to the study of this phenomenon and its related issues. Specifically, we believe an inclusive and holistic conceptual framework is needed, able to consider the variety of dimensions, which are the subjects and the effects that, over time, are influenced by the introduction of a new technology or a new drug in healthcare context.
In this sense, as shown in Figure 3, we need to shift the focus from efficiency and effectiveness concepts to that of sustainability in the Healthcare Technology Assessment, incorporating the politico-institutional suprasystems, until now neglected [25].
Accordingly, we propose a systems view, able to highlight key elements of analysis to any system’s functioning in a social as well as economic context [35,36].
Among systems view, the one adopted herein is the one of Viable Systems Approach (vSa) [22,23,24,37]. It is a meta-level lens, with respect to the specific observed phenomena, that provides general useful interpretation schemes.
The Viable Systems Approach, starting from Stafford Beer’s Viable System Model [38], proposes several conceptual innovations, which can be summarized in the following principles [39]:
- Survival: a viable system has the aim to survive in a specific context;
- Eidos: from an ontological viewpoint, a viable system can be considered in both a structural and a systemic perspective;
- Isotropy: in terms of behavior, a viable system distinguishes an area of decision-making and one of acting;
- Acting: its aim is to reach a result, an objective, through the interaction with supra and subsystems from which the system receives, but to which it also supplies, indications and rules; and
- Exhaustiveness: external entities are also viable systems, which are components deriving from a superior level.
From the principles explained above, it emerges that, in the vSa perspective, every entity capable of action (viable system) can be observed both in its structural configuration (static perspective) and in its becoming (dynamic perspective). The static perspective pertains to the structure and describes a viable system “as it is”, allowing the identification of a physical boundary between what it is internal and external to the structure. Conversely, the dynamic perspective regards the system, understood as a “specific structure oriented to the achievement of a purpose”, namely survival [39]. The structure-system dualism generates a conceptual dichotomy between relation and interaction. The relation has a “structural”, static and objective nature, and it is configured as a physical or logical connection between the components of the structure. The interaction, instead, presupposes a relation and it is a dynamic and “systemic” concept that takes on different connotations, according to the perspective of the observer.
Furthermore, from a systems view, environment and context are two different concepts. The context is the result of a process of perception and subjective interpretation of the environment by the decision-maker of a viable system. The decision-maker filters only the entities considered relevant from the environment, thus identifying other viable systems with which to relate, the so-called supra-systems. They are systems capable of projecting their expectations on another system, which is qualified as a subsystem. The relevance depends on the ability of each suprasystem to condition the chances of survival of the considered viable system. It is also possible to measure the degree of relevance by referring to the resources released by the suprasystems, according to their ability of affecting the considered system’s survival. In this sense, within the context of reference different actors mature different expectations with respect to the same viable system.
These expectations translate into the ability of the viable system itself to ensure targets of efficiency, effectiveness, and sustainability.
- -
- Efficiency (plans): things are done in the right way.
- -
- Effectiveness (goals): the right things get done.
- -
- Sustainability (relationships): The right relationships exist with other service systems.
Bringing back these definitions to the vSa perspective, the objectives of efficiency can be measured with reference to the structure, while the effectiveness of the viable system is realized and measured with regard to its specific context of reference. Thus, it is possible to achieve sustainability objectives in relation to the general environment. From the above it derives that pursuing sustainability objectives is fundamental and functional to guaranteeing the conditions of viability and, therefore, the survival of the system [41,42].
In fact, in this case, we mean sustainability as a simultaneous achievement of economic, social and environmental sustainability, paying specific attention to the politico-institutional suprasystems, whose resources are critical for the system functioning and may consequently exert an influence on its survival and evolutionary dynamics.
Therefore, economic sustainability refers to the operating structure, where the achievement of conditions of economic-structural efficiency is a priority. Social sustainability adopts a vision oriented towards the systemic effectiveness of the context. Environmental sustainability, by incorporating the political dimension, has a broader scope and refers to long-term survival and long-term systemic balance.
In summary, in a Viable Systems Approach perspective, the search for sustainability is to be attributed to the search for the right relationships (defined as consonance) and the right interactions (defined as resonance) with the other systems within a specific context.
Thus, based on this general framework, in the next section we will propose a critical and holistic analysis for a unified health assessment, in a complex decision-making scenarios, such as HTA, that can harmonize a variety of resources, skills, expertise, interests, and expectations, reconciling the typically opposite targets of efficiency, effectiveness, and sustainability [2].
3.2. A Systems Approach to Health Evaluation Methods
The analysis in Section 2 shows that the most widespread methodologies for health assessment are limited to one-dimensional measurements of efficiency, inspired by the simple relationship between costs and results. Among them, the most common evaluation methods used for the assessment of new medical devices or drugs are the cost-effectiveness-analysis (CEA) and the cost-utility analysis (CUA) [43].
CEA defines the outcomes of an innovation in medicine according to specific dimensions (for example: delay in aggravating a syndrome, deferral of the time of surgery, effects of prevention programs, etc.).
Instead, the outcomes of the CUA are measured as health-related preferences, described as Quality Adjusted Life Years (QALYs) or Quality Adjusted Life Expected (QALE) gained; in fact, this method assumes that health is a function of length and quality of life and combines these values into a single index number.
In addition to the most commonly used methods, many other propositions can be found in literature, some of which are also sporadically used by the agencies of the different countries.
Among them, the Cost Consequence Analysis (CCA) does not attempt to summarize outcomes in a single measure (such as the QALY) or in financial terms. Instead, its outcomes are shown in their natural units (some of which may be monetary) and it is left to decision-makers to determine whether a treatment is worth being carried out. Consequently, CCA poses on the evaluator the problem of aggregating, weighing, and evaluating the components, data, and outcomes.
Apart from CUA, which summarizes the effects of an initiative in QALY, both CEA and CCA are susceptible to a further step, consisting in the treatment of their results with the multi-criteria analysis (Multiple Criteria Decision Analysis, MCDA) that is recently expanding into the field of drug and health-related assessments [44,45,46]. The definition of MCDA encompasses a wide range of different approaches. However, Garattini and Padula [47] criticize its use in HTA, because the main intrinsic limit for health policy decisions on new technologies is the lack of key information at the early stage of market approval.
The databases that can be used, regardless of the evaluation method, comprise data from a variety of sources, including, but not limited to, clinical trials and observations. It is worth noting that there are two basic approaches to economic evaluation. In trial-based studies, economic data (e.g., resource utilization, and quality of life) are collected alongside a single clinical study, usually a controlled clinical trial. In modelling studies, data from a wide range of sources (e.g., existing clinical trials, observational studies) are synthesized using an economic model [12]; the authors also conclude that the two approaches are complementary and not mutually exclusive.
A separate analysis deserves the cost benefit analysis (CBA) which is not affected by the limitations of the efficiency measurements of the cited methods, being by its nature aimed at considering the complex impact that any investment (in our case in medicine) produces on the well-being of the community that benefits from it. However, the recognized difficulties in applying the cost-benefit analysis, mostly deriving from the ambition to assign a monetary value to each of the expected effects, is amplified in the case of innovations in health systems for a series of ethical issues that its application raises, first of all the ones related to the evaluation of human life [48,49].
We believe that the QALY/CUA can be used on the condition that the assertion of every QALY having the same value regardless of the condition or the personal characteristics of the population treated (age, sex, severity of disease, level of deprivation, or other characteristics) is exceeded. Therefore, the QALY, as a direct effect of the new technology, must be quantified in a systems perspective, such as the proposed one of the vSa, with reference to:
- the indirect effects, related not only to the patient, but also to the organization that provides care; and
- the indirect costs of the disease (e.g., caregivers, etc.), on the consequences of the patient’s family entourage.
This, mainly, in order to overcome the first flaw identified by Drummond and Sculpher [12] consisting in the “omission of important costs or benefit”. Furthermore, the sustainability of the process must be considered in terms of the anthropic environment regarding the entire life cycle.
Table 2 summarizes the different evaluation methods analysed so far and the related measures.
As emerges from Table 2, the most used evaluation methods for HTA consider measures usually related to effectiveness. The proposed theoretical view of HTA, by referring to vSa principles, considers sustainability intended as the consideration of the political dimension in the assessment procedures, together with the economic and social aspects [25].
With the term political sustainability, in fact, it is to be intended the development and maintenance of the political will necessary to sustain a major policy direction in the health care system [25,50].
The vSa and its conceptualizations highlight key elements that allow the definition of a more appropriate approach to Health Technology Assessment. In fact, the general view of vSa considers the different perspectives, the priorities, and the mechanisms of influence of all the suprasystems in the HTA.
Consistently with a vSa view, the viability of the Healthcare system linked to the inclusion of social and political dimensions implies the shift from Health Technology Assessment (HTA) to Health Technology Sustainability (HTS), as there cannot be healthcare sustainability without health technology sustainability, and vice versa [51].
Accordingly, from the above several implications derive:
- -
- the consideration of all the dimensions of evaluation processes; in particular, efficiency and effectiveness are enhanced by including the sustainability perspective [2];
- -
- because of the previous point, the simultaneous consideration of all the suprasystems involved in healthcare system, both as users and as decision-makers; and
- -
- based on systems thinking, the evaluation of healthcare system in its both structural (efficiency perspective) and systemic (effectiveness and sustainability perspective) configuration.
In this regard, the proposed unified health assessment in a vSa framework presupposes the use of a new perspective that adds the political dimension to the pre-existing economic and social aspects, thus proposing an evaluation based on Cost per Unit Change expressed in terms of Quality Adjusted Life Year (QALY).
In line with the considerations proposed above, this new approach might be able to overcome the limitations of every QALY having the same value, thus quantifying the indirect effects of medical treatments, in a systems perspective.
4. Discussion, Limitations, and Future Lines of Research
The criticism of the methodological aspects of evaluating innovations in healthcare, and the continuous search for changes or alternatives to existing techniques seem to derive from the lack of awareness of the multiplicity of possible innovations with various characteristics, rather than from their intrinsic limitations. This means that for each of them it is necessary to identify the most appropriate methods among the existing ones. If, for innovations with high technological contents (e.g., high cost), and high social, economic and political impacts, a systems approach that considers the multiplicity of effects on the multiplicity of stakeholders involved is necessary, for procedural innovations of low cost and limited impact (for example procedures in the limited area of a hospital) simpler techniques are enough. This is also because evaluation procedures have costs that must be proportionate to the expected benefits. In this context, it should be noted that the cost of evaluation is mostly linked to the retrieval and systematization of data and information necessary for the evaluation itself, while only in residual part is attributable to the calculations. Whereas data and information can be used in different evaluation techniques, it is appropriate to apply them all: the result of each contributes to enrich the information flow for decision-makers.
Where replicated with reference to a larger sample (e.g., other disciplines, other areas, etc.), the analysis should be carried out employing different techniques, or a database able to exceed the limits of Scopus, connected to issues of duplications, delay in database update, and substantial costs [32].
Further limitations of present work could deal with the absence of a deeper analysis based on multivariate analysis on major components analysis, although Vosviewer Software still gives information about total link strength and citation, and closeness of data points.
Future lines of research will be focused on the number, the different nature, and the different degrees of complexity of innovation to be evaluated, as well as on the corresponding number and different nature of the decision-makers involved, whose decisions, depending on their role, produce effects on very different scales. This will imply the creation of a taxonomy of innovations, on one hand, and the analysis of roles and composition of decision-makers on the other. In fact, the possible numerous intersections between the objects (innovations) and the subjects (decision-makers) seem to be the starting point for the realization of reliable assessments, under the constraint of limited resources, and the related need to contain costs.
Supplementary Materials
The map is downloadable at the following link: https://bit.ly/2z9BCnd.
Author Contributions
Introduction: L.F.; Materials and Methods: V.P.; Results: I.F., F.I., L.F.; The Discussion and Conclusions section is the result of synergic final considerations.
Funding
This research received no external funding.
Conflicts of Interest
The authors declare no conflict of interest.
Appendix A
Table A1.
Thesaurus used in the VosViewer elaboration.
| Label | Replace By |
|---|---|
| Adolescent | Adolescent |
| Adult | Adult |
| Aged | Aged |
| Article | |
| Assessment method | Assessment method |
| Australia | |
| Biomedical technology | Health technology assessment |
| Biomedical technology assessment | Health technology assessment |
| Brazil | |
| Budget | Budget |
| Canada | |
| Child | Child |
| China | |
| Clinical decision-making | Clinical decision-making |
| Clinical practice | Clinical practice |
| Comparative study | Comparative study |
| Conceptual framework | Conceptual framework |
| Controlled study | Controlled study |
| Cost benefit analysis | Cost benefit analysis |
| Cost control | Cost control |
| Cost effectiveness analysis | Cost effectiveness analysis |
| Cost utility analysis | Cost utility analysis |
| Cost-benefit analysis | Cost benefit analysis |
| Cost-effectiveness | Cost effectiveness analysis |
| Cost-effectiveness analysis | Cost effectiveness analysis |
| Decision-making | decision-making |
| Decision-making, organizational | decision-making |
| Decision support system | Decision support system |
| Decision support techniques | Decision support system |
| Decision-making | decision-making |
| Delivery of health care | Delivery of health care |
| Devices | Devices |
| Diffusion of innovation | Diffusion of innovation |
| Documentation | Documentation |
| Drug | Drug |
| Drug cost | Drug costs |
| Drug costs | Drug costs |
| Drug efficacy | Drug efficacy |
| Drug industry | Drug industry |
| Drug manufacture | Drug manufacture |
| Drug marketing | Drug marketing |
| Drug policy | Drug policy |
| Drug safety | Drug safety |
| Economic aspect | Economic aspect |
| Economic evaluation | Economic evaluation |
| Economics | Economics |
| England | |
| Ethics | Ethics |
| Europe | |
| Evidence based medicine | Evidence based medicine |
| Evidence-based medicine | Evidence based medicine |
| Female | Female |
| Financial management | Financial management |
| Forecasting | Forecasting |
| France | |
| Funding | Funding |
| Generic drug | Generic drug |
| Germany | |
| Government | Government |
| Great britain | |
| Gross national product | Gross national product |
| Health care | Health care |
| Health care access | Health care access |
| Health care cost | Health care costs |
| Health care costs | Health care costs |
| Health care delivery | Health care delivery |
| Health care financing | Health care financing |
| Health care organization | Health care organization |
| Health care planning | Health care planning |
| Health care policy | Health care policy |
| Health care quality | Health care quality |
| Health care reform | Health care reform |
| Health care system | Health care system |
| Health care utilization | Health care utilization |
| Health economics | Health economics |
| Health impact assessment | Health impact assessment |
| Health insurance | Health insurance |
| Health policy | Health care policy |
| Health service | Health services |
| Health services | Health services |
| Health status | Health status |
| Health survey | Health survey |
| Health technology assessment | Health technology assessment |
| Health technology assessment (hta) | Health technology assessment |
| Health technology assessments | Health technology assessment |
| Hta | Health technology assessment |
| Human | |
| Humans | |
| Hungary | |
| Information processing | Information processing |
| Innovation | Innovation |
| Insurance | Insurance |
| Interview | |
| Law | Law |
| Literature | |
| Major clinical study | Major clinical study |
| Male | Male |
| Management | Management |
| Medical decision-making | Medical decision-making |
| Medical device | Medical devices |
| Medical devices | Medical devices |
| Medical ethics | Ethics |
| Medical research | Medical research |
| Medical technology | Medical technology |
| Methodology | |
| Middle aged | Middle aged |
| Models, economic | |
| National health programs | National health programs |
| National health service | National health service |
| Netherlands | |
| Oncology | |
| Organization | Organization |
| Organization and management | Organization and management |
| Outcome assessment | Outcome assessment |
| Outcome assessment (health care) | Outcome assessment |
| Patient preference | Patient preference |
| Pharmaceuticals | Pharmaceuticals |
| Pharmacoeconomics | Pharmacoeconomics |
| Poland | |
| Policy | Policy |
| Policy making | Policy making |
| Practice guideline | Practice guideline |
| Prescription | Prescription |
| Pricing | Pricing |
| Priority journal | |
| Procedures | Procedures |
| Public health | Public health |
| Public health service | Public health |
| Publication | |
| Qualitative research | |
| Quality adjusted life year | Quality adjusted life year |
| Quality control | Quality control |
| Quality of life | Quality of life |
| Quality-adjusted life years | Quality adjusted life year |
| Questionnaire | |
| Randomized controlled trial (topic) | Randomized controlled trial |
| Reimbursement | Reimbursement |
| Research design | |
| Resource allocation | Resource allocation |
| Review | |
| Review literature as topic | |
| Risk assessment | Risk assessment |
| Standard | Standards |
| Standards | Standards |
| State medicine | State medicine |
| Statistical analysis | Statistical analysis |
| Statistics and numerical data | Statistical analysis |
| Sweden | |
| Systematic review | |
| Technological development | Technological development |
| Technology | Technology |
| Technology assessment | Health technology assessment |
| Technology assessment, biomedical | Health technology assessment |
| Total quality management | Total quality management |
| Treatment outcome | Treatment outcome |
| Trends | Trends |
| Uncertainty | Uncertainty |
| United kingdom | |
| United states | |
| Wellbeing | Wellbeing |
Appendix B
Table A2.
Weights and Links strength of the references.
| Id | Label | Url | x | y | Cluster | Weight <Links> | Weight <Total Link Strength> | Weight <Citations> | Weight <Norm. Citations> | Score <Pub. Year> | Score <Citations> | Score <Norm. Citations> |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 1 | rutnam (1991) | https://doi.org/10.1080/08164649.1991.9994628 | 1.0023 | 2.2198 | 16 | 0 | 0 | 1 | 1.000 | 1991 | 1 | 1.000 |
| 2 | smith (1994) | https://doi.org/10.1016/0277-9536(94)90067-1 | 1.9484 | 1.3767 | 17 | 0 | 0 | 6 | 0.947 | 1994 | 6 | 0.947 |
| 3 | france (1994) | https://doi.org/10.1016/0277-9536(94)90064-7 | 0.6613 | 2.3638 | 18 | 0 | 0 | 3 | 0.474 | 1994 | 3 | 0.474 |
| 4 | granados (1994) | https://doi.org/10.1016/0277-9536(94)90065-5 | 2.309 | −0.5175 | 19 | 0 | 0 | 10 | 1.579 | 1994 | 10 | 1.579 |
| 5 | freemantle (1995) | https://doi.org/10.1016/0277-9536(94)00272-u | 2.204 | 0.8791 | 20 | 0 | 0 | 13 | 1.000 | 1995 | 13 | 1.000 |
| 6 | reuzel (1999) | https://doi.org/10.1023/a:1009963018813 | −0.9821 | −2.3246 | 21 | 0 | 0 | 14 | 1.000 | 1999 | 14 | 1.000 |
| 7 | jones (2000) | https://doi.org/10.1057/palgrave.jmm.5040022 | −0.3431 | 0.0427 | 8 | 1 | 1 | 1 | 0.286 | 2000 | 1 | 0.286 |
| 8 | reuzel (2000) | https://doi.org/10.1177/13563890022209389 | −0.2478 | 0.0384 | 6 | 7 | 5 | 6 | 1.714 | 2000 | 6 | 1.714 |
| 9 | jones (2001) | https://doi.org/10.1057/palgrave.jmm.5040041 | −0.2421 | −0.0874 | 4 | 1 | 1 | 0 | 0.000 | 2001 | 0 | 0.000 |
| 10 | oliver (2001) | https://doi.org/10.1177/13563890122209847 | −0.2387 | 0.1697 | 5 | 2 | 1 | 16 | 2.000 | 2001 | 16 | 2.000 |
| 11 | wells (2002) | https://doi.org/10.1049/em:20020410 | −0.0999 | −0.1005 | 4 | 1 | 1 | 3 | 1.000 | 2002 | 3 | 1.000 |
| 12 | jacobs (2003) | https://doi.org/10.1177/10442073030140021001 | −0.3183 | −0.0247 | 11 | 7 | 2 | 7 | 0.240 | 2003 | 7 | 0.240 |
| 13 | aspinall (2003) | https://doi.org/10.1016/s0277-9536(02)00027-8 | 2.1339 | 1.0488 | 22 | 0 | 0 | 25 | 0.857 | 2003 | 25 | 0.857 |
| 14 | may (2003) | https://doi.org/10.1016/s0277-9536(02)00419-7 | −0.1394 | 0.1675 | 1 | 8 | 4 | 89 | 3.051 | 2003 | 89 | 3.051 |
| 15 | sloane (2003) | https://doi.org/10.1016/s0305-0548(02)00187-9 | −0.1546 | −0.0002 | 11 | 1 | 2 | 51 | 1.749 | 2003 | 51 | 1.749 |
| 16 | cohen (2003) | https://doi.org/10.1002/hec.791 | −0.2579 | −0.1468 | 2 | 3 | 3 | 2 | 0.069 | 2003 | 2 | 0.069 |
| 17 | szucs (2003) | https://doi.org/10.1057/palgrave.jcb.3040064 | −0.2471 | −0.0797 | 4 | 6 | 5 | 1 | 0.034 | 2003 | 1 | 0.034 |
| 18 | briggs (2004) | https://doi.org/10.2165/00148365-200403020-00004 | −0.3095 | −0.1748 | 3 | 11 | 8 | 29 | 1.000 | 2004 | 29 | 1.000 |
| 19 | vázquez-polo (2005) | https://doi.org/10.1002/hec.947 | −0.3572 | −0.1971 | 3 | 5 | 8 | 11 | 0.454 | 2005 | 11 | 0.454 |
| 20 | milewa (2005) | https://doi.org/10.1111/j.1467-9515.2005.00452.x | −0.3126 | 0.1284 | 5 | 14 | 10 | 26 | 1.072 | 2005 | 26 | 1.072 |
| 21 | ginnelly (2005) | https://doi.org/10.2165/00148365-200504010-00006 | −0.2972 | −0.1484 | 3 | 12 | 11 | 25 | 1.031 | 2005 | 25 | 1.031 |
| 22 | hofmann (2005) | https://doi.org/10.1007/s10202-005-0073-1 | −0.1804 | 0.1491 | 1 | 16 | 10 | 35 | 1.443 | 2005 | 35 | 1.443 |
| 23 | milewa (2006) | https://doi.org/10.1016/j.socscimed.2006.08.009 | −0.3348 | 0.1431 | 5 | 9 | 14 | 19 | 1.000 | 2006 | 19 | 1.000 |
| 24 | brown (2007) | https://doi.org/10.1057/palgrave.jmm.5050086 | 1.2452 | −2.0779 | 23 | 0 | 0 | 12 | 1.000 | 2007 | 12 | 1.000 |
| 25 | scheibler (2008) | https://doi.org/10.1016/j.zefq.2008.07.017 | −0.2634 | −0.0507 | 11 | 2 | 2 | 5 | 1.333 | 2008 | 5 | 1.333 |
| 26 | hoppe (2008) | https://doi.org/10.1111/j.1467-8691.2008.00495.x | −0.1427 | 0.2134 | 1 | 3 | 3 | 5 | 1.333 | 2008 | 5 | 1.333 |
| 27 | lehoux (2008) | https://doi.org/10.1177/1356389008090857 | −0.2077 | 0.1455 | 1 | 21 | 17 | 4 | 1.067 | 2008 | 4 | 1.067 |
| 28 | freemantle (2008) | https://doi.org/10.1007/s10198-008-0123-4 | −0.1887 | −0.2168 | 2 | 2 | 3 | 1 | 0.267 | 2008 | 1 | 0.267 |
| 29 | sacchini (2009) | https://doi.org/10.1007/s11019-009-9206-y | −0.1283 | 0.2168 | 1 | 17 | 12 | 13 | 0.944 | 2009 | 13 | 0.944 |
| 30 | göhlen (2009) | https://doi.org/10.1016/j.zefq.2009.06.015 | 1.2087 | 2.1022 | 24 | 0 | 0 | 0 | 0.000 | 2009 | 0 | 0.000 |
| 31 | vespermann (2009) | https://doi.org/10.1016/j.zefq.2009.06.008 | 2.0483 | 1.2143 | 25 | 0 | 0 | 0 | 0.000 | 2009 | 0 | 0.000 |
| 32 | wild (2009) | https://doi.org/10.1016/j.zefq.2009.06.010 | 2.3075 | 0.5343 | 26 | 0 | 0 | 0 | 0.000 | 2009 | 0 | 0.000 |
| 33 | schwarzer (2009) | https://doi.org/10.1016/j.zefq.2009.05.020 | −0.2724 | −0.1973 | 3 | 1 | 1 | 2 | 0.145 | 2009 | 2 | 0.145 |
| 34 | welton (2009) | https://doi.org/10.1111/j.1467-985x.2008.00548.x | −0.199 | −0.1982 | 2 | 4 | 5 | 62 | 4.500 | 2009 | 62 | 4.500 |
| 35 | lehoux (2009) | https://doi.org/10.1016/j.socscimed.2009.03.017 | −0.1648 | 0.1546 | 1 | 13 | 10 | 37 | 2.686 | 2009 | 37 | 2.686 |
| 36 | gammon (2009) | https://doi.org/10.1136/jme.2008.027920 | −0.1254 | 0.2055 | 1 | 11 | 4 | 7 | 0.508 | 2009 | 7 | 0.508 |
| 37 | moreno (2009) | https://doi.org/10.1080/13571510903227056 | −0.3567 | −0.1939 | 3 | 5 | 7 | 3 | 0.218 | 2009 | 3 | 0.218 |
| 38 | groop (2010) | −0.2386 | −0.1609 | 2 | 5 | 3 | 6 | 0.465 | 2010 | 6 | 0.465 | |
| 39 | strech (2010) | https://doi.org/10.1016/j.zefq.2010.03.001 | −0.288 | −0.0565 | 2 | 10 | 4 | 2 | 0.155 | 2010 | 2 | 0.155 |
| 40 | boenink (2010) | https://doi.org/10.1007/s11019-009-9223-x | −0.1443 | 0.2025 | 1 | 15 | 6 | 27 | 2.093 | 2010 | 27 | 2.093 |
| 41 | bührlen (2010) | https://doi.org/10.1016/j.zefq.2010.10.012 | −0.2486 | 0.0812 | 6 | 21 | 9 | 1 | 0.078 | 2010 | 1 | 0.078 |
| 42 | koivisto (2010) | https://doi.org/10.1332/174426410 × 482980 | −0.1949 | 0.135 | 1 | 14 | 7 | 4 | 0.310 | 2010 | 4 | 0.310 |
| 43 | torbica (2010) | https://doi.org/10.1057/jmm.2009.48 | −0.2188 | −0.0673 | 4 | 8 | 3 | 11 | 0.853 | 2010 | 11 | 0.853 |
| 44 | allen (2010) | https://doi.org/10.3109/01421590903390619 | 0.2806 | 2.4565 | 27 | 0 | 0 | 1 | 0.078 | 2010 | 1 | 0.078 |
| 45 | bridges (2010) | https://doi.org/10.1108/s0731-2199(2010)0000022005 | −0.4454 | −0.0092 | 7 | 11 | 12 | 12 | 0.930 | 2010 | 12 | 0.930 |
| 46 | woodman (2010) | https://doi.org/10.3163/1536-5050.98.2.006 | 1.5576 | 1.8341 | 28 | 0 | 0 | 8 | 0.620 | 2010 | 8 | 0.620 |
| 47 | gauvin (2010) | https://doi.org/10.1016/j.socscimed.2010.01.036 | −0.1958 | 0.1599 | 1 | 36 | 24 | 57 | 4.419 | 2010 | 57 | 4.419 |
| 48 | martin (2011) | https://doi.org/10.4067/s1726-569 × 2011000200009 | −0.1584 | 0.1717 | 1 | 22 | 9 | 1 | 0.060 | 2011 | 1 | 0.060 |
| 49 | czech (2011) | https://doi.org/10.14254/2071-789x.2011/4-1a/8 | −0.5471 | 0.0001 | 10 | 2 | 2 | 0 | 0.000 | 2011 | 0 | 0.000 |
| 50 | brousselle (2011) | https://doi.org/10.1016/j.socscimed.2011.01.008 | −0.3098 | 0.0323 | 6 | 39 | 27 | 33 | 1.976 | 2011 | 33 | 1.976 |
| 51 | drummond (2011) | https://doi.org/10.1007/s10198-010-0274-y | −0.3244 | 0.0744 | 6 | 1 | 1 | 37 | 2.216 | 2011 | 37 | 2.216 |
| 52 | jarosławski (2011) | https://doi.org/10.2165/11592960-000000000-00000 | −0.561 | 0.0202 | 10 | 2 | 1 | 6 | 0.359 | 2011 | 6 | 0.359 |
| 53 | bombard (2011) | https://doi.org/10.1016/j.socscimed.2011.04.017 | −0.1556 | 0.1892 | 1 | 31 | 21 | 35 | 2.096 | 2011 | 35 | 2.096 |
| 54 | walters (2011) | https://doi.org/10.1080/02664763.2010.545375 | −0.5061 | 0.0147 | 10 | 1 | 1 | 2 | 0.120 | 2011 | 2 | 0.120 |
| 55 | goeree (2011) | https://doi.org/10.2147/ceor.s14404 | −0.3672 | −0.051 | 4 | 3 | 5 | 36 | 2.156 | 2011 | 36 | 2.156 |
| 56 | orlewska (2011) | https://doi.org/10.1556/socec.33.2011.3.8 | −0.6368 | −0.1509 | 13 | 4 | 7 | 3 | 0.180 | 2011 | 3 | 0.180 |
| 57 | meltzer (2011) | https://doi.org/10.1016/b978-0-444-53592-4.00007-4 | −0.2879 | −0.0226 | 6 | 25 | 13 | 14 | 0.838 | 2011 | 14 | 0.838 |
| 58 | droste (2012) | https://doi.org/10.1016/j.zefq.2012.05.019 | −0.13 | 0.2185 | 1 | 10 | 2 | 2 | 0.222 | 2012 | 2 | 0.222 |
| 59 | kelly (2012) | https://doi.org/10.1057/sth.2011.21 | −0.2728 | 0.0231 | 6 | 20 | 6 | 29 | 3.222 | 2012 | 29 | 3.222 |
| 60 | boenink (2012) | https://doi.org/10.1007/s10728-011-0173-0 | −0.2148 | 0.121 | 1 | 21 | 14 | 3 | 0.333 | 2012 | 3 | 0.333 |
| 61 | jommi (2012) | https://doi.org/10.1177/1745790412440704 | −0.5812 | 0.0278 | 10 | 3 | 3 | 3 | 0.333 | 2012 | 3 | 0.333 |
| 62 | kuchenbecker (2012) | https://doi.org/10.1016/j.vhri.2012.09.009 | −0.3636 | 0.0516 | 7 | 8 | 5 | 8 | 0.889 | 2012 | 8 | 0.889 |
| 63 | augustovski (2012) | https://doi.org/10.1016/j.vhri.2012.09.007 | −0.4075 | 0.0654 | 7 | 1 | 1 | 5 | 0.556 | 2012 | 5 | 0.556 |
| 64 | vargas-zea (2012) | https://doi.org/10.1016/j.vhri.2012.09.004 | 2.358 | 0.1837 | 29 | 0 | 0 | 13 | 1.444 | 2012 | 13 | 1.444 |
| 65 | siebert (2013) | https://doi.org/10.1016/j.zefq.2013.10.020 | −0.3448 | −0.1338 | 3 | 14 | 12 | 11 | 1.133 | 2013 | 11 | 1.133 |
| 66 | perleth (2013) | https://doi.org/10.1016/j.zefq.2013.04.006 | −0.2862 | −0.0864 | 4 | 17 | 6 | 1 | 0.103 | 2013 | 1 | 0.103 |
| 67 | wild (2013) | https://doi.org/10.1016/j.zefq.2013.02.008 | −0.3427 | −0.0319 | 2 | 3 | 4 | 1 | 0.103 | 2013 | 1 | 0.103 |
| 68 | eckermann (2013) | https://doi.org/10.1016/j.socscimed.2012.10.020 | −0.3497 | −0.1116 | 3 | 21 | 7 | 7 | 0.721 | 2013 | 7 | 0.721 |
| 69 | spinner (2013) | https://doi.org/10.2147/ceor.s39624 | −0.3627 | 0.0567 | 5 | 31 | 13 | 14 | 1.442 | 2013 | 14 | 1.442 |
| 70 | smith (2013) | https://doi.org/10.1177/1745790413476876 | −0.1785 | −0.074 | 9 | 8 | 5 | 3 | 0.309 | 2013 | 3 | 0.309 |
| 71 | niewada (2013) | https://doi.org/10.1016/j.vhri.2013.05.002 | −0.3032 | 0.008 | 12 | 11 | 5 | 8 | 0.824 | 2013 | 8 | 0.824 |
| 72 | odame (2013) | https://doi.org/10.1016/j.vhri.2013.07.006 | −0.3549 | −0.0125 | 7 | 26 | 9 | 5 | 0.515 | 2013 | 5 | 0.515 |
| 73 | petrou (2013) | https://doi.org/10.1016/j.vhri.2013.06.016 | −0.2596 | −0.0383 | 4 | 4 | 3 | 13 | 1.339 | 2013 | 13 | 1.339 |
| 74 | sura (2013) | https://doi.org/10.1016/j.vhri.2013.06.012 | 1.9613 | −1.3558 | 30 | 0 | 0 | 1 | 0.103 | 2013 | 1 | 0.103 |
| 75 | kaló (2013) | https://doi.org/10.1016/j.vhri.2013.06.002 | −0.6073 | −0.1402 | 13 | 3 | 4 | 27 | 2.782 | 2013 | 27 | 2.782 |
| 76 | elsisi (2013) | https://doi.org/10.1016/j.vhri.2013.06.014 | −0.3557 | −0.0197 | 11 | 17 | 6 | 7 | 0.721 | 2013 | 7 | 0.721 |
| 77 | salvatore (2013) | https://doi.org/10.1177/1745790413498410 | −0.1837 | −0.0458 | 4 | 9 | 5 | 0 | 0.000 | 2013 | 0 | 0.000 |
| 78 | thébaut (2013) | https://doi.org/10.1016/j.socscimed.2013.10.020 | −0.2687 | −0.0791 | 2 | 7 | 6 | 3 | 0.309 | 2013 | 3 | 0.309 |
| 79 | hevér (2013) | https://doi.org/10.1556/socec.2013.0008 | −0.6075 | −0.1331 | 13 | 6 | 8 | 1 | 0.103 | 2013 | 1 | 0.103 |
| 80 | attema (2013) | https://doi.org/10.1007/s10198-013-0508-x | −0.2782 | −0.0913 | 2 | 11 | 9 | 36 | 3.709 | 2013 | 36 | 3.709 |
| 81 | ulucanlar (2013) | https://doi.org/10.1016/j.socscimed.2013.09.008 | −0.1574 | 0.1045 | 1 | 15 | 11 | 27 | 2.782 | 2013 | 27 | 2.782 |
| 82 | neyt (2014) | https://doi.org/10.3917/rpve.534.0055 | −0.2916 | −0.0018 | 6 | 18 | 2 | 0 | 0.000 | 2014 | 0 | 0.000 |
| 83 | ríos (2014a) | https://doi.org/10.1016/j.vhri.2014.02.005 | 0.6749 | −2.3566 | 15 | 1 | 4 | 0 | 0.000 | 2014 | 0 | 0.000 |
| 84 | ríos (2014b) | https://doi.org/10.1016/j.vhri.2014.08.002 | 0.6755 | −2.3563 | 15 | 1 | 4 | 1 | 0.157 | 2014 | 1 | 0.157 |
| 85 | jain (2014) | https://doi.org/10.1016/j.vhri.2014.04.006 | −0.3537 | 0.0247 | 7 | 10 | 7 | 1 | 0.157 | 2014 | 1 | 0.157 |
| 86 | gulácsi (2014) | https://doi.org/10.1007/s10198-014-0590-8 | −0.3691 | 0.0678 | 8 | 5 | 5 | 32 | 5.016 | 2014 | 32 | 5.016 |
| 87 | kennedy-martin (2014) | https://doi.org/10.1016/j.vhri.2014.03.001 | −0.3809 | 0.0229 | 8 | 17 | 8 | 5 | 0.784 | 2014 | 5 | 0.784 |
| 88 | hunger (2014) | https://doi.org/10.1007/s00038-013-0494-x | −0.399 | −0.1522 | 3 | 2 | 1 | 1 | 0.157 | 2014 | 1 | 0.157 |
| 89 | jakubiak-lasocka (2014) | https://doi.org/10.1016/j.vhri.2014.06.008 | −0.3366 | −0.0218 | 11 | 10 | 4 | 11 | 1.724 | 2014 | 11 | 1.724 |
| 90 | böhm (2014) | https://doi.org/10.1080/07036337.2013.793679 | −0.3527 | −0.0619 | 3 | 7 | 3 | 8 | 1.254 | 2014 | 8 | 1.254 |
| 91 | elias (2014) | https://doi.org/10.1016/j.zefq.2014.08.021 | −0.3841 | 0.0612 | 7 | 2 | 4 | 2 | 0.314 | 2014 | 2 | 0.314 |
| 92 | madan (2014) | https://doi.org/10.1111/rssa.12018 | −0.1944 | −0.2018 | 2 | 5 | 7 | 9 | 1.411 | 2014 | 9 | 1.411 |
| 93 | cerri (2014) | https://doi.org/10.1007/s10198-013-0514-z | −0.3005 | 0.0907 | 6 | 7 | 5 | 10 | 1.568 | 2014 | 10 | 1.568 |
| 94 | skoupá (2014) | https://doi.org/10.1016/j.vhri.2014.06.003 | −0.429 | −0.1091 | 13 | 3 | 2 | 10 | 1.568 | 2014 | 10 | 1.568 |
| 95 | gorenoi (2014) | https://doi.org/10.1016/j.zefq.2014.03.017 | −0.2066 | −0.162 | 2 | 6 | 7 | 0 | 0.000 | 2014 | 0 | 0.000 |
| 96 | mendonça (2014) | https://doi.org/10.1007/s10198-013-0522-z | 1.5818 | −1.8096 | 31 | 0 | 0 | 3 | 0.470 | 2014 | 3 | 0.470 |
| 97 | lopert (2014) | https://doi.org/10.1016/j.zefq.2014.08.020 | −0.3964 | 0.0986 | 5 | 5 | 2 | 1 | 0.157 | 2014 | 1 | 0.157 |
| 98 | daniel mullins (2014) | https://doi.org/10.1016/j.vhri.2014.02.006 | −0.3253 | −0.0512 | 4 | 11 | 13 | 4 | 0.627 | 2014 | 4 | 0.627 |
| 99 | horváth cs.z. (2014) | https://doi.org/10.1007/s10198-014-0601-9 | −0.6504 | −0.1549 | 13 | 2 | 4 | 5 | 0.784 | 2014 | 5 | 0.784 |
| 100 | tetteh (2014) | https://doi.org/10.1186/s13561-014-0026-2 | −0.3634 | −0.0993 | 3 | 6 | 2 | 1 | 0.157 | 2014 | 1 | 0.157 |
| 101 | heintz (2014) | https://doi.org/10.1016/j.zefq.2014.09.006 | −0.5044 | −2.454 | 32 | 0 | 0 | 0 | 0.000 | 2014 | 0 | 0.000 |
| 102 | abrishami (2014) | https://doi.org/10.1016/j.socscimed.2014.07.046 | −0.1706 | 0.0643 | 9 | 12 | 7 | 9 | 1.411 | 2014 | 9 | 1.411 |
| 103 | gurtner (2014) | https://doi.org/10.1097/hmr.0b013e3182993b91 | −0.2725 | −0.0189 | 11 | 30 | 9 | 9 | 1.411 | 2014 | 9 | 1.411 |
| 104 | mitton (2014) | https://doi.org/10.1007/s40258-013-0074-5 | −0.2622 | −0.0655 | 11 | 12 | 8 | 23 | 3.605 | 2014 | 23 | 3.605 |
| 105 | walzer (2014) | https://doi.org/10.2147/ceor.s53601 | 1.7058 | 1.688 | 33 | 0 | 0 | 9 | 1.411 | 2014 | 9 | 1.411 |
| 106 | rogers (2014) | https://doi.org/10.1111/j.1467-8519.2012.01980.x | −0.0998 | −0.1004 | 4 | 1 | 1 | 2 | 0.314 | 2014 | 2 | 0.314 |
| 107 | li (2014) | https://doi.org/10.1016/j.vhri.2013.04.001 | −0.3843 | 0.0386 | 8 | 6 | 3 | 1 | 0.157 | 2014 | 1 | 0.157 |
| 108 | rader (2014) | https://doi.org/10.1002/jrsm.1097 | −0.113 | 0.3162 | 14 | 1 | 1 | 14 | 2.195 | 2014 | 14 | 2.195 |
| 109 | robertson (2014) | https://doi.org/10.1002/jrsm.1102 | −0.193 | −0.182 | 2 | 5 | 2 | 5 | 0.784 | 2014 | 5 | 0.784 |
| 110 | pieper (2014) | https://doi.org/10.1002/jrsm.1107 | −0.2253 | −0.1384 | 2 | 5 | 7 | 9 | 1.411 | 2014 | 9 | 1.411 |
| 111 | siebert (2015) | https://doi.org/10.1016/j.zefq.2015.06.012 | −0.4021 | −0.1473 | 3 | 4 | 2 | 4 | 0.656 | 2015 | 4 | 0.656 |
| 112 | schnell-inderst (2015) | https://doi.org/10.1016/j.zefq.2015.06.011 | −0.1153 | −0.0927 | 4 | 11 | 6 | 10 | 1.639 | 2015 | 10 | 1.639 |
| 113 | stürzlinger (2015) | https://doi.org/10.1016/j.zefq.2015.07.002 | 1.0457 | −2.1952 | 34 | 0 | 0 | 0 | 0.000 | 2015 | 0 | 0.000 |
| 114 | ivlev (2015) | https://doi.org/10.1016/j.ejor.2015.05.075 | −0.2275 | −0.0303 | 9 | 16 | 9 | 19 | 3.115 | 2015 | 19 | 3.115 |
| 115 | wang (2015) | https://doi.org/10.1093/hrlr/ngv025 | 1.3938 | 1.9738 | 35 | 0 | 0 | 1 | 0.164 | 2015 | 1 | 0.164 |
| 116 | pfadenhauer (2015) | https://doi.org/10.1016/j.zefq.2015.01.004 | −0.1431 | 0.1065 | 1 | 5 | 3 | 18 | 2.951 | 2015 | 18 | 2.951 |
| 117 | nachtnebel (2015) | https://doi.org/10.1016/j.zefq.2015.05.012 | −0.1566 | 0.23 | 1 | 3 | 2 | 2 | 0.328 | 2015 | 2 | 0.328 |
| 118 | rao (2015) | https://doi.org/10.5912/jcb669 | −0.5469 | 0.037 | 10 | 3 | 1 | 1 | 0.164 | 2015 | 1 | 0.164 |
| 119 | cuijpers (2015) | https://doi.org/10.1016/j.techfore.2014.03.006 | 2.3655 | 0.0078 | 36 | 0 | 0 | 9 | 1.475 | 2015 | 9 | 1.475 |
| 120 | peine (2015) | https://doi.org/10.1016/j.techfore.2014.08.019 | −0.1561 | 0.1296 | 1 | 7 | 6 | 4 | 0.656 | 2015 | 4 | 0.656 |
| 121 | cook (2015) | https://doi.org/10.1016/j.vhri.2015.03.013 | −0.3957 | −0.0039 | 8 | 4 | 4 | 1 | 0.164 | 2015 | 1 | 0.164 |
| 122 | lopes (2015) | https://doi.org/10.1016/j.socscimed.2015.04.021 | −0.2795 | 0.1043 | 5 | 34 | 15 | 5 | 0.820 | 2015 | 5 | 0.820 |
| 123 | rocchi (2015) | https://doi.org/10.2147/ceor.s82549 | −2.1628 | 1.3879 | 37 | 0 | 0 | 5 | 0.820 | 2015 | 5 | 0.820 |
| 124 | kolominsky-rabas (2015) | https://doi.org/10.1016/j.techfore.2013.12.005 | −0.1868 | −0.0957 | 9 | 16 | 12 | 12 | 1.967 | 2015 | 12 | 1.967 |
| 125 | griffiths (2015) | https://doi.org/10.2147/ceor.s87462 | −0.3785 | 0.0421 | 5 | 25 | 8 | 7 | 1.148 | 2015 | 7 | 1.148 |
| 126 | dranitsaris (2015) | https://doi.org/10.1007/s40258-014-0130-9 | −0.4767 | 0.0379 | 10 | 19 | 6 | 7 | 1.148 | 2015 | 7 | 1.148 |
| 127 | petrou (2015) | https://doi.org/10.1007/s40258-015-0191-4 | −0.2727 | −0.0473 | 7 | 3 | 3 | 0 | 0.000 | 2015 | 0 | 0.000 |
| 128 | winnette (2015) | https://doi.org/10.1016/j.vhri.2015.03.008 | −0.4524 | 0.0066 | 7 | 3 | 3 | 1 | 0.164 | 2015 | 1 | 0.164 |
| 129 | bitencourt (2015) | https://doi.org/10.1016/j.vhri.2015.08.002 | 2.1415 | −1.0297 | 38 | 0 | 0 | 3 | 0.492 | 2015 | 3 | 0.492 |
| 130 | brazier (2015) | https://doi.org/10.1007/s40258-015-0194-1 | −0.2521 | −0.0961 | 2 | 6 | 7 | 13 | 2.131 | 2015 | 13 | 2.131 |
| 131 | sacchini (2016) | −0.1442 | 0.2087 | 1 | 23 | 20 | 0 | 0.000 | 2016 | 0 | 0.000 | |
| 132 | wortley (2016) | https://doi.org/10.1108/jhom-08-2015-0119 | −0.2717 | 0.0556 | 5 | 38 | 22 | 1 | 0.309 | 2016 | 1 | 0.309 |
| 133 | peregrin (2016) | https://doi.org/10.1504/ijbsr.2016.075746 | −0.1731 | −0.0022 | 11 | 10 | 8 | 0 | 0.000 | 2016 | 0 | 0.000 |
| 134 | petrillo (2016) | https://doi.org/10.1504/ijmcdm.2016.077878 | −0.2622 | 0.0267 | 11 | 10 | 11 | 1 | 0.309 | 2016 | 1 | 0.309 |
| 135 | manelli (2016) | −0.2554 | 0.0602 | 6 | 20 | 5 | 0 | 0.000 | 2016 | 0 | 0.000 | |
| 136 | brown (2016) | https://doi.org/10.1177/0306312715609699 | −0.4162 | 0.1269 | 5 | 2 | 2 | 7 | 2.162 | 2016 | 7 | 2.162 |
| 137 | babigumira (2016) | https://doi.org/10.1111/jphs.12120 | −0.3355 | 0.0036 | 7 | 6 | 7 | 2 | 0.618 | 2016 | 2 | 0.618 |
| 138 | lysdahl (2016) | https://doi.org/10.1186/s12910-016-0099-z | −0.1371 | 0.1632 | 1 | 18 | 11 | 2 | 0.618 | 2016 | 2 | 0.618 |
| 139 | koh (2016) | https://doi.org/10.1016/j.vhri.2015.06.004 | 1.4224 | −1.9477 | 39 | 0 | 0 | 6 | 1.853 | 2016 | 6 | 1.853 |
| 140 | dang (2016) | https://doi.org/10.1016/j.vhri.2015.11.005 | −0.3974 | 0.0107 | 7 | 13 | 8 | 4 | 1.235 | 2016 | 4 | 1.235 |
| 141 | dilokthornsakul (2016) | https://doi.org/10.1016/j.vhri.2015.12.003 | −0.3584 | −0.1186 | 3 | 10 | 8 | 1 | 0.309 | 2016 | 1 | 0.309 |
| 142 | mühlbacher (2016) | https://doi.org/10.1007/s40258-016-0232-7 | −0.1706 | −0.0797 | 9 | 15 | 10 | 21 | 6.485 | 2016 | 21 | 6.485 |
| 143 | assasi (2016) | https://doi.org/10.1186/s12910-016-0118-0 | −0.138 | 0.2677 | 14 | 20 | 18 | 1 | 0.309 | 2016 | 1 | 0.309 |
| 144 | sullivan (2016) | https://doi.org/10.1007/s10198-015-0720-y | 2.2618 | 0.7075 | 40 | 0 | 0 | 6 | 1.853 | 2016 | 6 | 1.853 |
| 145 | thompson (2016) | https://doi.org/10.2147/ceor.s96616 | −0.5617 | 0.0341 | 10 | 5 | 6 | 0 | 0.000 | 2016 | 0 | 0.000 |
| 146 | radu (2016) | https://doi.org/10.1016/j.vhri.2016.07.006 | −0.3752 | 0.0754 | 8 | 1 | 1 | 4 | 1.235 | 2016 | 4 | 1.235 |
| 147 | grundy (2016) | https://doi.org/10.1016/j.socscimed.2016.07.042 | −0.252 | −0.0145 | 7 | 8 | 9 | 1 | 0.309 | 2016 | 1 | 0.309 |
| 148 | panayidou (2016) | https://doi.org/10.1002/jrsm.1202 | −0.274 | −0.1742 | 3 | 4 | 8 | 5 | 1.544 | 2016 | 5 | 1.544 |
| 149 | tsiachristas (2016) | https://doi.org/10.5334/ijic.2472 | −0.2259 | −0.0147 | 12 | 27 | 8 | 4 | 1.235 | 2016 | 4 | 1.235 |
| 150 | meyer (2016) | https://doi.org/10.1016/j.zefq.2016.07.011 | −0.1628 | −0.1172 | 9 | 1 | 1 | 0 | 0.000 | 2016 | 0 | 0.000 |
| 151 | janssen (2016) | https://doi.org/10.2147/ppa.s122319 | −0.1999 | −0.0711 | 2 | 15 | 7 | 2 | 0.618 | 2016 | 2 | 0.618 |
| 152 | ducey (2017) | https://doi.org/10.1332/174426415 × 14443053123024 | −0.1692 | 0.1577 | 1 | 28 | 22 | 0 | 0.000 | 2017 | 0 | 0.000 |
| 153 | callea (2017) | https://doi.org/10.1016/j.socscimed.2016.11.038 | −0.1549 | −0.025 | 4 | 7 | 6 | 4 | 1.892 | 2017 | 4 | 1.892 |
| 154 | wright (2017) | https://doi.org/10.1111/1758-5899.12215 | −0.4475 | −0.0246 | 10 | 14 | 15 | 0 | 0.000 | 2017 | 0 | 0.000 |
| 155 | mühlbacher (2017) | https://doi.org/10.1007/s10198-016-0763-8 | −0.1563 | −0.0791 | 9 | 5 | 9 | 8 | 3.784 | 2017 | 8 | 3.784 |
| 156 | blome (2017) | https://doi.org/10.1007/s10198-016-0765-6 | −0.4683 | −0.0188 | 10 | 2 | 2 | 0 | 0.000 | 2017 | 0 | 0.000 |
| 157 | castro (2017) | https://doi.org/10.1111/1758-5899.12333 | −0.3354 | 0.0339 | 8 | 21 | 9 | 1 | 0.473 | 2017 | 1 | 0.473 |
| 158 | mossman (2017) | https://doi.org/10.1111/1758-5899.12221 | −0.2374 | 0.0693 | 5 | 4 | 2 | 0 | 0.000 | 2017 | 0 | 0.000 |
| 159 | kanavos (2017) | https://doi.org/10.1111/1758-5899.12386 | −0.5595 | 0.0198 | 10 | 8 | 12 | 3 | 1.419 | 2017 | 3 | 1.419 |
| 160 | hensher (2017) | https://doi.org/10.1016/j.socscimed.2017.01.020 | −0.3613 | −0.0928 | 3 | 3 | 1 | 3 | 1.419 | 2017 | 3 | 1.419 |
| 161 | thijssen (2017) | https://doi.org/10.1016/j.jedc.2017.01.016 | −0.1679 | −0.0085 | 9 | 3 | 3 | 1 | 0.473 | 2017 | 1 | 0.473 |
| 162 | jakubczyk (2017) | https://doi.org/10.1007/s10479-015-1910-9 | −0.3707 | −0.1213 | 3 | 21 | 13 | 5 | 2.365 | 2017 | 5 | 2.365 |
| 163 | markiewicz (2017) | https://doi.org/10.5912/jcb780 | −0.1928 | −0.0632 | 9 | 21 | 6 | 1 | 0.473 | 2017 | 1 | 0.473 |
| 164 | gyalrong-steur (2017) | https://doi.org/10.1016/j.zefq.2017.01.002 | 1.8356 | 1.536 | 41 | 0 | 0 | 0 | 0.000 | 2017 | 0 | 0.000 |
| 165 | hofmann (2017) | https://doi.org/10.1007/s11948-016-9791-0 | −0.1316 | 0.2436 | 1 | 11 | 6 | 2 | 0.946 | 2017 | 2 | 0.946 |
| 166 | greer (2017) | https://doi.org/10.1057/cep.2016.6 | −0.3111 | 0.0516 | 8 | 23 | 8 | 2 | 0.946 | 2017 | 2 | 0.946 |
| 167 | nicod (2017) | https://doi.org/10.1007/s10198-016-0823-0 | −0.4591 | 0.0102 | 10 | 28 | 18 | 12 | 5.676 | 2017 | 12 | 5.676 |
| 168 | rautenberg (2017) | https://doi.org/10.2147/ceor.s140902 | −0.367 | −0.1346 | 3 | 1 | 1 | 1 | 0.473 | 2017 | 1 | 0.473 |
| 169 | cowles (2017) | https://doi.org/10.1007/s40258-017-0309-y | −0.3315 | 0.0075 | 8 | 15 | 7 | 2 | 0.946 | 2017 | 2 | 0.946 |
| 170 | angelis (2017) | https://doi.org/10.1016/j.socscimed.2017.06.024 | −0.2626 | −0.0563 | 12 | 31 | 24 | 7 | 3.311 | 2017 | 7 | 3.311 |
| 171 | inotai (2017) | https://doi.org/10.1016/j.vhri.2017.06.003 | −0.5388 | −0.1163 | 13 | 8 | 7 | 2 | 0.946 | 2017 | 2 | 0.946 |
| 172 | brixner (2017) | https://doi.org/10.1016/j.vhri.2017.02.001 | −0.2531 | −0.0401 | 12 | 17 | 16 | 3 | 1.419 | 2017 | 3 | 1.419 |
| 173 | skoupá (2017) | https://doi.org/10.1016/j.vhri.2017.08.002 | 1.7247 | −1.6649 | 42 | 0 | 0 | 1 | 0.473 | 2017 | 1 | 0.473 |
| 174 | yagudina (2017) | https://doi.org/10.1016/j.vhri.2017.07.006 | −0.4697 | 0.0632 | 5 | 1 | 1 | 1 | 0.473 | 2017 | 1 | 0.473 |
| 175 | dimova (2017) | https://doi.org/10.1016/j.vhri.2017.08.001 | 2.2091 | −0.8607 | 43 | 0 | 0 | 2 | 0.946 | 2017 | 2 | 0.946 |
| 176 | culig (2017) | https://doi.org/10.1016/j.vhri.2017.07.005 | −0.4386 | 2.4633 | 44 | 0 | 0 | 1 | 0.473 | 2017 | 1 | 0.473 |
| 177 | jahnz-różyk (2017) | https://doi.org/10.1016/j.vhri.2017.07.001 | 2.3408 | −0.3436 | 45 | 0 | 0 | 3 | 1.419 | 2017 | 3 | 1.419 |
| 178 | silins (2017) | https://doi.org/10.1016/j.vhri.2017.08.006 | −1.7813 | −1.8328 | 46 | 0 | 0 | 2 | 0.946 | 2017 | 2 | 0.946 |
| 179 | chambers (2017) | −0.4242 | 0.1111 | 5 | 6 | 3 | 1 | 0.473 | 2017 | 1 | 0.473 | |
| 180 | knott (2017) | https://doi.org/10.1016/j.socscimed.2017.08.033 | −0.2469 | −0.1106 | 2 | 1 | 2 | 0 | 0.000 | 2017 | 0 | 0.000 |
| 181 | mertz (2017) | https://doi.org/10.1016/j.zefq.2017.07.010 | −0.1115 | 0.3164 | 14 | 1 | 3 | 0 | 0.000 | 2017 | 0 | 0.000 |
| 182 | donin (2017) | https://doi.org/10.3846/16111699.2017.1409798 | −0.1875 | −0.0608 | 9 | 7 | 9 | 1 | 0.473 | 2017 | 1 | 0.473 |
| 183 | rawson (2017) | https://doi.org/10.2147/ceor.s144695 | −0.2916 | 0.1078 | 6 | 4 | 7 | 2 | 0.946 | 2017 | 2 | 0.946 |
| 184 | rosselli (2017) | https://doi.org/10.1016/j.vhri.2017.02.004 | −0.3833 | 0.0498 | 7 | 28 | 13 | 1 | 0.473 | 2017 | 1 | 0.473 |
| 185 | kibel (2017) | https://doi.org/10.1016/j.socscimed.2017.11.024 | −0.2257 | −0.0132 | 2 | 10 | 6 | 1 | 0.473 | 2017 | 1 | 0.473 |
| 186 | paolucci (2017) | https://doi.org/10.1007/s40258-017-0349-3 | −0.2708 | −0.0197 | 12 | 14 | 9 | 1 | 0.473 | 2017 | 1 | 0.473 |
| 187 | angelis (2018) | https://doi.org/10.1007/s10198-017-0871-0 | −0.3359 | 0.0181 | 12 | 42 | 37 | 5 | 6.500 | 2018 | 5 | 6.500 |
| 188 | klímová (2018) | https://doi.org/10.15240/tul/001/2018-1-008 | −0.202 | −0.0593 | 9 | 21 | 7 | 0 | 0.000 | 2018 | 0 | 0.000 |
| 189 | castro (2018) | https://doi.org/10.1590/1807-57622016.0549 | −0.197 | 0.0503 | 5 | 27 | 16 | 0 | 0.000 | 2018 | 0 | 0.000 |
| 190 | chen (2018a) | https://doi.org/10.5582/bst.2018.01038 | −0.3881 | 0.0496 | 8 | 7 | 3 | 0 | 0.000 | 2018 | 0 | 0.000 |
| 191 | löblová (2018) | https://doi.org/10.1111/psj.12213 | −0.3518 | 0.0459 | 8 | 18 | 7 | 3 | 3.900 | 2018 | 3 | 3.900 |
| 192 | wong (2018) | https://doi.org/10.1007/s40258-017-0339-5 | −0.4439 | 0.0517 | 5 | 16 | 11 | 0 | 0.000 | 2018 | 0 | 0.000 |
| 193 | fierlbeck (2018) | https://doi.org/10.1111/capa.12253 | −0.2639 | 0.0924 | 6 | 16 | 15 | 0 | 0.000 | 2018 | 0 | 0.000 |
| 194 | rehfuess (2018) | https://doi.org/10.1002/jrsm.1254 | −0.1636 | 0.0294 | 12 | 5 | 4 | 5 | 6.500 | 2018 | 5 | 6.500 |
| 195 | nord (2018) | https://doi.org/10.1007/s10198-017-0882-x | −0.3253 | −0.0257 | 6 | 8 | 4 | 2 | 2.600 | 2018 | 2 | 2.600 |
| 196 | yi (2018) | https://doi.org/10.2147/tcrm.s163190 | −0.197 | −0.162 | 2 | 5 | 2 | 0 | 0.000 | 2018 | 0 | 0.000 |
| 197 | zhen (2018) | https://doi.org/10.1016/j.vhri.2018.01.010 | −0.3836 | 0.0407 | 8 | 10 | 6 | 0 | 0.000 | 2018 | 0 | 0.000 |
| 198 | chen (2018b) | https://doi.org/10.1016/j.techfore.2018.01.033 | −0.1611 | −0.0277 | 9 | 2 | 2 | 1 | 1.300 | 2018 | 1 | 1.300 |
| 199 | chen (2018c) | https://doi.org/10.1016/j.vhri.2018.03.004 | 2.2668 | −0.6904 | 47 | 0 | 0 | 0 | 0.000 | 2018 | 0 | 0.000 |
| 200 | thornton snider (2018) | https://doi.org/10.1515/fhep-2016-0014 | 1.8509 | −1.5132 | 48 | 0 | 0 | 0 | 0.000 | 2018 | 0 | 0.000 |
| 201 | olofsson (2018) | https://doi.org/10.1007/s10198-017-0922-6 | −0.3437 | −0.0018 | 6 | 7 | 4 | 1 | 1.300 | 2018 | 1 | 1.300 |
| 202 | kyle (2018) | https://doi.org/10.1007/s11151-018-9639-7 | −1.0899 | 2.2796 | 49 | 0 | 0 | 1 | 1.300 | 2018 | 1 | 1.300 |
| 203 | al rabayah (2018) | https://doi.org/10.1111/jphs.12241 | −0.3556 | −0.0638 | 13 | 17 | 5 | 0 | 0.000 | 2018 | 0 | 0.000 |
| 204 | zegeye (2018) | https://doi.org/10.1016/j.vhri.2018.07.001 | −0.3092 | −0.0138 | 7 | 6 | 6 | 0 | 0.000 | 2018 | 0 | 0.000 |
| 205 | brixner (2018) | https://doi.org/10.1016/j.vhri.2018.01.003 | −0.2265 | −0.0518 | 12 | 9 | 7 | 2 | 2.600 | 2018 | 2 | 2.600 |
| 206 | radu (2018) | https://doi.org/10.1016/j.vhri.2017.11.003 | 2.3392 | 0.3594 | 50 | 0 | 0 | 0 | 0.000 | 2018 | 0 | 0.000 |
| 207 | prasolov (2018) | https://doi.org/10.1016/j.vhri.2018.04.002 | 2.3583 | −0.1679 | 51 | 0 | 0 | 0 | 0.000 | 2018 | 0 | 0.000 |
| 208 | yfantopoulos (2018) | https://doi.org/10.1016/j.vhri.2018.06.006 | −0.1402 | −0.0268 | 4 | 1 | 1 | 0 | 0.000 | 2018 | 0 | 0.000 |
| 209 | mägi (2018) | https://doi.org/10.1016/j.vhri.2017.10.001 | 2.0584 | −1.1946 | 52 | 0 | 0 | 0 | 0.000 | 2018 | 0 | 0.000 |
| 210 | palozzi (2018) | https://doi.org/10.3390/su10103550 | −0.2471 | 0.0048 | 4 | 31 | 16 | 0 | 0.000 | 2018 | 0 | 0.000 |
| 211 | espinoza (2018) | https://doi.org/10.1016/j.vhri.2018.07.003 | −0.2227 | −0.0464 | 12 | 10 | 3 | 0 | 0.000 | 2018 | 0 | 0.000 |
| 212 | calderón (2018) | https://doi.org/10.1016/j.vhri.2018.01.011 | −0.1967 | −0.1695 | 2 | 5 | 3 | 0 | 0.000 | 2018 | 0 | 0.000 |
References
- Smith, C. New technology continues to invade healthcare: What are the strategic implications/outcomes? Nurs. Adm. Q. 2004, 28, 92–98. [Google Scholar] [CrossRef] [PubMed]
- Saviano, M.; Bassano, C.; Calabrese, M. A VSA-SS approach to healthcare service systems the triple target of efficiency, effectiveness and sustainability. Serv. Sci. 2010, 2, 41–61. [Google Scholar] [CrossRef]
- OECD. New Health Technologies Managing Access, Value and Sustainability: Managing Access, Value and Sustainability; OECD Publishing: Paris, France, 2017. [Google Scholar]
- Russell, S.; Hauert, S.; Altman, R.; Veloso, M. Robotics: Ethics of artificial intelligence. Nature 2015, 521, 415–418. [Google Scholar] [PubMed] [Green Version]
- Thielst, C.B. The future of healthcare technology. J. Healthc. Manag. 2007, 52, 7. [Google Scholar] [CrossRef] [PubMed]
- Gulácsi, L.; Orlewska, E.; Péntek, M. Health economics and health technology assessment in Central and Eastern Europe: A dose of reality. Eur. J. Health Econ. 2012, 13, 525–531. [Google Scholar] [CrossRef] [PubMed]
- Banta, D. The development of health technology assessment. Health Policy 2003, 63, 121–132. [Google Scholar] [CrossRef]
- Banta, D.; Jonsson, E. History of HTA: Introduction. Int. J. Technol. Assess. Health Care 2009, 25, 1–6. [Google Scholar] [CrossRef]
- Inotai, A.; Csanadi, M.; Harsanyi, A.; Nemeth, B. Drug Policy in Central Eastern Europe—Hungary. Value Health Reg. Issues 2017, 13, 16–22. [Google Scholar] [CrossRef]
- Draborg, E.; Gyrd-Hansen, D.; Poulsen, P.B.; Horder, M. International comparison of the definition and the practical application of health technology assessment. Int. J. Technol. Assess. Health Care 2005, 21, 89–95. [Google Scholar] [CrossRef]
- Herndon, J.H.; Hwang, R.; Bozic, K. Healthcare technology and technology assessment. Eur. Spine J. 2007, 16, 1293–1302. [Google Scholar] [CrossRef]
- Drummond, M.; Sculpher, M. Common methodological flaws in economic evaluations. Med. Care 2005, 43, II5–II14. [Google Scholar] [CrossRef]
- Abrishami, P.; Boer, A.; Horstman, K. Understanding the adoption dynamics of medical innovations: Affordances of the da Vinci robot in the Netherlands. Soc. Sci. Med. 2014, 117, 125–133. [Google Scholar] [CrossRef] [PubMed]
- Kern, S.E.; Jaron, D. Healthcare technology, economics, and policy: An evolving balance. IEEE Eng. Med. Biol. Mag. 2003, 22, 16–19. [Google Scholar] [CrossRef] [PubMed]
- Vuong, Q.H.; Ho, T.M.; Vuong, T.T. Healthcare consumers’ sensitivity to costs: A reflection on behavioural economics from an emerging market. Palgrave Commun. 2018, 4, 70. [Google Scholar] [CrossRef]
- Hansjürgens, B. Economic valuation through cost-benefit analysis–possibilities and limitations. Toxicology 2004, 205, 241–252. [Google Scholar] [CrossRef] [PubMed]
- Goeree, R.; He, J.; O’Reilly, D.; Tarride, J.E.; Xie, F.; Lim, M.; Burke, N. Transferability of health technology assessments and economic evaluations: A systematic review of approaches for assessment and application. ClinicoEcon. Outcomes Res. 2011, 3, 89–104. [Google Scholar] [CrossRef] [PubMed]
- Yagudina, R.I.; Kulikov, A.U.; Serpik, V.G.; Ugrekhelidze, D.T. Concept of Combining Cost-Effectiveness Analysis and Budget Impact Analysis in Health Care Decision Making. Value Health Reg. Issues 2017, 13, 61–66. [Google Scholar] [CrossRef]
- Bodenheimer, T. High and rising health care costs. Part 2: Technologic innovation. Ann. Intern. Med. 2005, 142, 932–937. [Google Scholar] [CrossRef]
- Groop, J.; Reijonsaari, K.; Lillrank, P. Applying the theory of constraints to health technology assessment. Int. J. Adv. Life Sci. 2010, 2, 115–124. [Google Scholar]
- Brousselle, A.; Lessard, C. Economic evaluation to inform health care decision-making: Promise, pitfalls and a proposal for an alternative path. Soc. Sci. Med. 2011, 72, 832–839. [Google Scholar] [CrossRef]
- Golinelli, G.M. L’approccio Sistemico al Governo Dell’impresa, Volume 1; CEDAM: Padova, Italy, 2000. [Google Scholar]
- Golinelli, G.M. Viable Systems Approach (VSA): Governing Business Dynamics; CEDAM: Padua, Italy, 2010. [Google Scholar]
- Barile, S. Management Sistemico Vitale; Giappichelli: Lyon, France, 2009. [Google Scholar]
- Borgonovi, E.; Compagni, A. Sustaining universal health coverage: The interaction of social, political, and economic sustainability. Value Health 2013, 16, S34–S38. [Google Scholar] [CrossRef] [PubMed]
- Saviano, M.; Bassano, C.; Piciocchi, P.; Di Nauta, P.; Lettieri, M. Monitoring viability and sustainability in healthcare organizations. Sustainability 2018, 10, 3548. [Google Scholar] [CrossRef]
- Berelson, B. Content Analysis in Communication Research; Free Press: Glencoe, IL, USA, 1952. [Google Scholar]
- Krippendorff, K. Content Analysis: An Introduction to Its Methodology, 2nd ed.; Sage: Thousand Oaks, CA, USA, 2004; ISBN 9780761915454. [Google Scholar]
- Glaser, B.G.; Strauss, A.L. The Discovery of Grounded Theory: Strategies for Qualitative Research; Aldine Pub.: Chicago, IL, USA, 1967. [Google Scholar]
- Goulding, C. Grounded Theory: A Practical Guide for Management, Business and Market Researchers; Sage: London, UK, 2002. [Google Scholar]
- Valderrama-Zurián, J.C.; Aguilar-Moya, R.; Melero-Fuentes, D.; Aleixandre-Benavent, R. A systematic analysis of duplicate records in Scopus. J. Inf. 2015, 9, 570–576. [Google Scholar] [CrossRef] [Green Version]
- Vuong, Q.H.; La, V.P.; Vuong, T.T.; Ho, M.T.; Nguyen, H.K.; Nguyen, V.H.; Pham, H.H.; Ho, M.T. An open database of productivity in Vietnam’s social sciences and humanities for public use. Sci. Data 2018, 5, 180188. [Google Scholar] [CrossRef] [PubMed]
- van Eck, N.J.; Waltman, L. Software survey: VOSviewer, a computer program for bibliometric mapping. Scientometrics 2010, 84, 523–538. [Google Scholar] [CrossRef] [PubMed]
- van Eck, N.J.; Waltman, L. Visualizing bibliometric networks. In Measuring Scholarly Impact; Springer: Berlin, Germany, 2014; pp. 285–320. [Google Scholar]
- Badinelli, R.; Barile, S.; Ng, I.; Polese, F.; Saviano, M.; Di Nauta, P. Viable service systems and decision making in service management. J. Serv. Manag. 2012, 23, 498–526. [Google Scholar] [CrossRef] [Green Version]
- Barile, S.; Saviano, M. Foundations of systems thinking: The structure-systems paradigm. In Contributions to Theoretical and Practical Advances in Management. A Viable Systems Approach (vSa); Barile, S., Ed.; International Printing: Avellino, Italy, 2011. [Google Scholar]
- Barile, S. Management Sistemico Vitale: Decisioni e Scelte in Ambito Complesso; International Printing Srl Editore: Avellino, Italy, 2011. [Google Scholar]
- Beer, S. Brain of the Firm: The Managerial Cybernetics of Organization; Allen Lane the Penguin Press: London, UK, 1972. [Google Scholar]
- Barile, S.; Sancetta, G.; Saviano, M. Management. Il Modello Sistemico e le Decisioni Manageriali, Volume I; Giappichelli: Turin, Italy, 2015. [Google Scholar]
- Spohrer, J.; Maglio, P.P.; Bailey, J.; Gruhl, D. Steps toward a science of service systems. Computer 2007, 40, 71–77. [Google Scholar] [CrossRef]
- Barile, S.; Saviano, M.; Iandolo, F.; Calabrese, M. The viable systems approach and its contribution to the analysis of sustainable business behaviors. Syst. Res. Behav. Sci. 2014, 31, 683–695. [Google Scholar] [CrossRef]
- Quattrociocchi, B.; Iandolo, F.; Fulco, I.; Calabrese, M. Capitolo III Efficienza, efficacia e sostenibilità. Il contributo dell’Approccio Sistemico Vitale (ASV) all’orientamento dei comportamenti d’impresa. In Il Controllo Manageriale e gli Indicatori di Performance Dentro e Fuori le Organizzazioni: Alcuni Contributi di Studio; Edizioni Nuova Cultura: Rome, Italy, 2018. [Google Scholar]
- Klímová, B.; Marešová, P. Economic methods used in health technology assessment. Econ. Manag. 2018, 21, 116–126. [Google Scholar] [CrossRef] [Green Version]
- Angelis, A.; Kanavos, P. Multiple Criteria Decision Analysis (MCDA) for evaluating new medicines in Health Technology Assessment and beyond: The Advance Value Framework. Soc. Sci. Med. 2017, 188, 137–156. [Google Scholar] [CrossRef] [Green Version]
- Ivlev, I.; Kneppo, P.; Bartak, M. Multicriteria decision analysis: A multifaceted approach to medical equipment management. Technol. Econ. Dev. Econ. 2014, 20, 576–589. [Google Scholar] [CrossRef]
- Jakubiak-Lasocka, J.; Jakubczyk, M. Cost-effectiveness versus Cost-Utility Analyses: What Are the Motives Behind Using Each and How Do Their Results Differ?—A Polish Example. Value Health Reg. Issues 2014, 4, 66–74. [Google Scholar] [CrossRef] [PubMed]
- Garattini, L.; Padula, A. Multiple Criteria Decision Analysis in Health Technology Assessment for Drugs: Just Another Illusion? Appl. Health Econ. Health Policy 2018, 16, 1–4. [Google Scholar] [CrossRef] [PubMed]
- Assasi, N.; Tarride, J.E.; O’Reilly, D.; Schwartz, L. Steps toward improving ethical evaluation in health technology assessment: A proposed framework. BMC Med. Ethics 2016, 17, 34. [Google Scholar] [CrossRef] [PubMed]
- Stein, K.; Fry, A.; Round, A.; Milne, R.; Brazier, J. What value health? A review of health state values used in early technology assessments for NICE. Appl. Health Econ. Health Policy 2005, 4, 219–228. [Google Scholar] [CrossRef] [PubMed]
- Canadian Public Health Association. Sustainability and Equity: Primary Health Care in Developing Countries; Canadian Public Health Association: Ottawa, ON, Canada, 1990. [Google Scholar]
- Tagliente, I.; Solvoll, T.; Murgia, F.; Bella, S. Telemonitoring in cystic fibrosis: A 4-year assessment and simulation for the next 6 years. Interact. J. Med. Res. 2016, 5, e11. [Google Scholar] [CrossRef] [PubMed]
Figure 1.
Density visualization.
Figure 2.
Network Visualization.
Figure 3.
The shift in perspective in health technology assessment.
Table 1.
Keywords’ link strength in relation to “Health Technology Assessment”.
| Keywords | Link Strength |
|---|---|
| Decision-making | 7.97 |
| Health Care Policy | 5.68 |
| Cost Effectiveness Analysis | 4.82 |
| Evidence | 4.28 |
| Ethics | 4.22 |
| Medical Technologies | 3.99 |
| Health Care Systems | 3.81 |
| Cost Benefit Analysis | 3.74 |
| Economic Evaluation | 3.63 |
| Reimbursement | 3.55 |
| Procedures | 3.44 |
| Innovation | 3.1 |
| Economics | 2.84 |
| Health Economics | 2.47 |
| Pharmaeconomics | 2.14 |
| Health Care Costs | 2.11 |
| Public Health | 2.02 |
| Drug costs | 1.91 |
| Medical Decision-making | 1.77 |
| Standards | 1.75 |
Table 2.
Evaluation methods.
| Methods | Measures |
|---|---|
| Cost Effectiveness Analysis (CEA) | Cost per unit change in output (e.g., cost per unit of social housing) |
| Cost Consequence Analysis (CCA) (or “Balance Sheet”) | Listing of all major costs and outcomes in natural units |
| Cost Utility Analysis (CUA) | Cost per unit change in Quality Adjusted Life Year (QALY) |
| Cost Benefit Analysis (CBA) | Value all outcomes in a common unit (e.g., monetary units) |
© 2018 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).
Share and Cite
MDPI and ACS Style
Iandolo, F.; Vito, P.; Fulco, I.; Loia, F. From Health Technology Assessment to Health Technology Sustainability. Sustainability 2018, 10, 4748. https://doi.org/10.3390/su10124748
AMA Style
Iandolo F, Vito P, Fulco I, Loia F. From Health Technology Assessment to Health Technology Sustainability. Sustainability. 2018; 10(12):4748. https://doi.org/10.3390/su10124748
Chicago/Turabian StyleIandolo, Francesca, Pietro Vito, Irene Fulco, and Francesca Loia. 2018. "From Health Technology Assessment to Health Technology Sustainability" Sustainability 10, no. 12: 4748. https://doi.org/10.3390/su10124748
Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.
