Next Article in Journal
An Evaluation of Input–Output Value for Sustainability in a Chinese Steel Production System Based on Emergy Analysis
Next Article in Special Issue
Pursuing Sustainability for Healthcare through Digital Platforms
Previous Article in Journal
Characteristics and Influencing Factors of Spatial Differentiation of Urban Black and Odorous Waters in China
Previous Article in Special Issue
Socio-Cultural Sustainability of Private Healthcare Providers in an Indian Slum Setting: A Bottom-of-the-Pyramid Perspective
Article Menu
Issue 12 (December) cover image

Export Article

Sustainability 2018, 10(12), 4748; https://doi.org/10.3390/su10124748

Article
From Health Technology Assessment to Health Technology Sustainability
Department of Management, Sapienza University of Rome, 00161 Rome, Italy
*
Author to whom correspondence should be addressed.
Received: 30 October 2018 / Accepted: 11 December 2018 / Published: 12 December 2018

Abstract

:
This paper aims to propose a methodological lens to the assessment of technological innovations in healthcare based on the principles of social, economic, and political sustainability. Starting from the consideration of a lack of a unified interpretative framework of health technology assessment, using a content analysis of the relevant literature on the topic, we identified both the scientific perspectives adopted by the scholars and the most widely discussed topics. Consequently, the less explored scientific areas were framed, and, therefore, those more susceptible to further investigation came to light. The result is an overall picture which highlights the absence of unified and generally accepted approaches to evaluation, together with the lack of awareness on the fact that the multiplicity of methods adopted is essentially connected to the multiplicity of innovations, for each of which a method (or a set of methods) of preferable evaluation can be prefigured. Based on these observations, we propose a general reference framework for evaluation, based on the Viable Systems Approach (vSa), and a schematic outline of the connections between the complexity of innovations and the evaluation methodologies.
Keywords:
health technology assessment; evaluation methods; viable systems approach; healthcare; sustainability

1. Introduction

The increasing attention to the treatment and care of patients makes central the role of healthcare technologies, as they are in constant development [1]. Indeed, technology has the potential to enhance efficiency and effectiveness in multiple areas of healthcare, and represents a great opportunity to improve the patient care and to increase the efficiency and effectiveness of all healthcare stakeholders, including policy makers, regulatory authorities, payers, and physicians [2]. Remote sensors, robotics, genomics, stem cells, and the availability of big data are just some of the innovations in this sector [3]. Only considering artificial intelligence, there are already numerous examples of how it can enhance the medical profession. Robot-assisted surgery, virtual nursing assistants, and technologies are able to identify changes in the usual behavior of patients; it also opens a wide debate on ethical and social challenges [4]. However, technology alone cannot improve the whole healthcare system, and a successful implementation requires a complete understanding “a priori” of the technology capabilities and its application [5]. Moreover, the economic and health policy issues increasingly influence healthcare technology solutions, with the aim to keep the healthcare quality high and the costs under control, especially regarding the pharmaceutical sector [3].
Considering the complexity of the healthcare context, the Health Technology Assessment (HTA) has been developed as a set of techniques able to support decisions regarding health policy, pushed by two driving forces: one relates to the more and more serious budget constraints due to the recession; the other relates to the increasingly demanding policy makers and funders, who require greater evidence for new and existing therapies [6].
Adopted primarily in the USA in the 1970s [7,8] and rapidly spread in Europe in the following decades, HTAs are currently being introduced in most countries of the world [3]. During the 1970s, the focus was to summarize the evidence concerning studies on the cost effectiveness of health interventions. Afterwards, at the end of the 1990s, HTA widened its focus from the evaluation of only large, expensive and machine-based technologies also to smaller technologies and healthcare, addressing broader issues (organizational, social implications, and ethics) [7]. Currently, the health technology assessment processes critically evaluate reimbursement submissions of pharmaceuticals, simple medical devices, and complex medical devices, such as hospital technologies [9], vaccines, procedures, health services, and public health interventions. Therefore, they are becoming an important tool to support health policy decisions in many countries [9].
Nowadays, the importance of HTAs has increased further, also spreading in Central and Eastern European countries. HTA activities have a national focus associated with the Ministry of Health and, by influencing health policy documents of the European Commission, it seems likely that, in the future, the HTA will be institutionalized somehow as part of the EU’s activities. The task of applying HTA techniques for the evaluation of new health technologies is devolved to the different government support bodies (e.g., NICE in the UK, IQWIG in Germany, and AGENAS in Italy).
Thus, HTAs act to define a policy research approach that examines the short and long-term social, economic, and political consequences of the application or use of technology [10], evaluating in a multidisciplinary way, medical technology about efficacy, safety, feasibility, cost, cost-effectiveness, and indications for use [11].
Although many attempts have been carried out, a unified approach has not yet been defined, and the institutions may fail to optimize their future technology acquisition, thus purely considering the medical innovation as an increasing financial risk [12,13]. Furthermore, several biomedical technologies approved are adopted based on limited evidence of safety and effectiveness [3,5,14]. Indeed, these innovative therapies, as digital innovations and revolutionary technologies (such as 3D bioprinting), present not only opportunities but also a complex set of technical, ethical, and financial challenges. Healthcare consumers are also inclined to be more sensitive to medical treatments costs, whether they value more or less the information in reducing future health uncertainty [15]. In any case, considering the real potential of the new technologies to improve the healthcare system, a new approach is required, able to support decision-makers to modernize the pre-existing framework and to exploit new technologies in a more efficient, effective and sustainable way [2] while also protecting patients, spending resources more wisely, and fostering the “right” type of innovation in the future [3].
Currently, cost effectiveness analysis is the most frequently used methodology [16] but, as far as it supports the allocation of resources [4], it suffers from theoretical limitations that make it inappropriate in many situations, leading to contraindications [17,18].
In particular, technological innovation, as regards cost-containment measures, is an important factor for managing high and increasing health costs. In literature, the evidence suggests that enhanced health technologies generally increase rather than reduce healthcare expenditures. Indeed, nations with a greater degree of integration into the health system have relied on spending controls and global budgets to control costs [15,19].
In this direction, this paper reviews the different contributions existing in literature regarding the methodologies connected to the HTA, highlighting the alternative approaches proposed [20,21] and identifying the main weaknesses.
A lack of unified generally accepted analysis schemes still exists. The methodologies currently used are consequently unsatisfactory: the literature that criticizes the cost-effectiveness analysis (CEA) is wide, currently being the most widespread evaluation methodology. Similarly, the literature that proposes alternative approaches to evaluation is wide as well.
Starting from this, we noted the lack of a holistic vision in HTA techniques able to understand—even if not able to quantify, as in the ambition of the cost-benefit analysis—the multiplicity of subjects and effects of innovation in healthcare. In this regard, we propose a rereading of the existing evaluation methods, through the interpretative lens of the Viable Systems Approach (vSa) [22,23,24], with the aim to consider, in this complex scenario, the multiplicity of entities and effects that over time are influenced by new technologies. A systems approach to decision-making is suggested, able to consider economic, social, and political aspects together with relations and interactions between them [25].
Few contributions address the topic of healthcare in a systems perspective and together with the topic of evaluations: in this sense, vSa becomes a bridging concept by the means of which the sustainability perspective can be incorporated into the management control system of healthcare organizations [26].
Based on previous considerations, the paper is structured as follows: Section 2 introduces the methodology adopted herein for a content analysis of the literature contributions on HTA. Thereafter, the viable systems approach is presented, as a theoretical framework for the analysis of health technology assessment, in the light of sustainability issues. Finally, a re-reading of healthcare assessment methods is proposed. The paper concludes with a discussion and conclusions.

2. Materials and Methods

In order to carry out a content analysis of the relevant literature on the topic, we used the textual contents of the scientific contributions on the analysed subject/topic, derived from the set of keywords, consisting of the author keywords plus those provided by Scopus through the encoding of the abstracts and the titles of the bibliographic references, contained in each recorded contribution. We used the method of co-occurrence of the keywords as a basis for the analysis, with the aim of identifying the scientific perspectives adopted by the scholars and the most widely discussed topics and, consequently, discerning the less explored scientific areas and those more susceptible to further investigation. The method adopted herein was the content analysis [27,28], modified due to the availability of textual data already codified by third parties. This change appears to provide a number of advantages: it abstracts from the subjectivity of those who carry out the codification; it guarantees the repeatability of the analysis, as the text bases are publicly available; it calls for the progressive formulation of hypotheses according to the discovery principles of grounded theories [29,30].

Data Collection and Methodology

The multidisciplinary nature of the topic made it necessary to limit the research to the contributions that are most consistent with the objectives of our work (the “Health Technology Assessment” query on Scopus returns 4511 results without limitations, 2/3 of which in the medicine area).
Therefore, data were collected through a research on Scopus using the query TITLE-ABS-KEY (“health technology assessment”) AND (LIMIT-TO (SUBJAREA, “ECON”) OR LIMIT-TO (SUBJAREA, “SOCI”) OR LIMIT-TO (SUBJAREA, “BUSI”) OR LIMIT-TO (SUBJAREA, “DECI”)) AND LIMIT-TO (DOCTYPE, “ar”)).
The results were 212 contributions, in the subject areas of Economics, Econometrics and Finance, Social Sciences, Business, Management and Accounting and Decision Sciences. We exported the results, choosing to consider all of the available information, except the index keywords (as to only analyse the author keywords), in CSV (Excel) format.
The data collection method, being based on the Scopus database, is subject to the shortcomings of the database itself, most notably the presence of duplications. For example, one study found 12% of the records in the seven Scopus-indexed journals to be duplicate [31]. As to solve the problem of data duplication, Vuong et al. [32] introduced an open database of the scientific output of Vietnamese researchers, using scientist’s self-reports, open online sources, and cross-checking with Scopus database. Given the size of the sample in our study, it was possible to verify the presence of duplications directly in the exported CSV file from Scopus and edit it accordingly.
The adapted version of the content analysis required the use of the VosViewer software [33,34], developed with the specific purpose of constructing, displaying and making publicly available bibliometric maps. VosViewer software provides distance-based bibliometric maps, e.g., graphical representations, in which the importance of a term is represented by its size, and the distance between two terms reflects the strength of the relationships between them: the smaller the distance, the more intense the relationship that binds them.
For the type of the analysis, we used the co-occurrence model of the author keywords in fractional counting. Aiming to regroup the same keywords expressed in different ways (e.g., Health Technology Assessment, HTA) and delete the keywords not strictly relevant to the purposes of this article (for example those that indicate the specific geopolitical areas or specific pathologies) we created a thesaurus, as presented in Table A1. The resulting keywords were 1724, 160 of which repeated at least five times, allowing us to obtain the map represented in the “density visualization” mode (Figure 1) and in the “network visualization” mode (Figure 2) (keywords extracted: 104). The map has been made publicly available, as seen in the Supplementary Materials section.
We made available, in Table A2, all of the 212 contributions considered in the content analysis, displaying the weight of each contribution with regards to total link strength and citations, as to show the closeness of data points.
Table 1 shows the strength of the links of the main keywords with the keyword “Health Technology Assessment”.
Data in Table 1 essentially show two scientific areas: that of decision-making to support health policy for the design of effective and efficient Health Care Systems, in which ethical issues are very important, and that of evaluation techniques and implementation of results with methodological insights and adaptations to local situations (Cost Benefit Analysis, Economic Evaluation, Reimbursement).

3. Results

The analysis carried out highlights two fundamental aspects of the problem:
  • the lack of a holistic approach which, starting from evaluation methodologies able to embrace the complex of direct and indirect effects of the introduction of an innovation in healthcare on all the actors involved, constitutes a general and commonly accepted framework of reference; and
  • the lack of awareness that the multiplicity of the methods adopted derives from the multiplicity of the characteristics of the innovations, for each of which specific methods are preferable.
Regarding the first aspect, the principles of the vSa described below are certainly helpful. Regarding extremely differentiated fields of investigation, far from the ambition of realizing a taxonomy of the innovations in healthcare to each class of which linking a particular evaluation technique, the considerations of Section 3.2 below are, however, possible.

3.1. The vSa as a Framework for the Analysis of Unified Health Assessment

As emerges from the above, the fundamental Health Technology Assessment seems to be focused mainly on cost-effectiveness considerations rather than on the assessment of the benefits of technology. Considering also the lack of an interpretative framework of HTA based on the principles of social, economic, and political sustainability [25], we aim to propose a methodological lens to the analysis of unified healthcare assessment that overcomes a reductionist approach to the study of this phenomenon and its related issues. Specifically, we believe an inclusive and holistic conceptual framework is needed, able to consider the variety of dimensions, which are the subjects and the effects that, over time, are influenced by the introduction of a new technology or a new drug in healthcare context.
In this sense, as shown in Figure 3, we need to shift the focus from efficiency and effectiveness concepts to that of sustainability in the Healthcare Technology Assessment, incorporating the politico-institutional suprasystems, until now neglected [25].
Accordingly, we propose a systems view, able to highlight key elements of analysis to any system’s functioning in a social as well as economic context [35,36].
Among systems view, the one adopted herein is the one of Viable Systems Approach (vSa) [22,23,24,37]. It is a meta-level lens, with respect to the specific observed phenomena, that provides general useful interpretation schemes.
The Viable Systems Approach, starting from Stafford Beer’s Viable System Model [38], proposes several conceptual innovations, which can be summarized in the following principles [39]:
  • Survival: a viable system has the aim to survive in a specific context;
  • Eidos: from an ontological viewpoint, a viable system can be considered in both a structural and a systemic perspective;
  • Isotropy: in terms of behavior, a viable system distinguishes an area of decision-making and one of acting;
  • Acting: its aim is to reach a result, an objective, through the interaction with supra and subsystems from which the system receives, but to which it also supplies, indications and rules; and
  • Exhaustiveness: external entities are also viable systems, which are components deriving from a superior level.
From the principles explained above, it emerges that, in the vSa perspective, every entity capable of action (viable system) can be observed both in its structural configuration (static perspective) and in its becoming (dynamic perspective). The static perspective pertains to the structure and describes a viable system “as it is”, allowing the identification of a physical boundary between what it is internal and external to the structure. Conversely, the dynamic perspective regards the system, understood as a “specific structure oriented to the achievement of a purpose”, namely survival [39]. The structure-system dualism generates a conceptual dichotomy between relation and interaction. The relation has a “structural”, static and objective nature, and it is configured as a physical or logical connection between the components of the structure. The interaction, instead, presupposes a relation and it is a dynamic and “systemic” concept that takes on different connotations, according to the perspective of the observer.
Furthermore, from a systems view, environment and context are two different concepts. The context is the result of a process of perception and subjective interpretation of the environment by the decision-maker of a viable system. The decision-maker filters only the entities considered relevant from the environment, thus identifying other viable systems with which to relate, the so-called supra-systems. They are systems capable of projecting their expectations on another system, which is qualified as a subsystem. The relevance depends on the ability of each suprasystem to condition the chances of survival of the considered viable system. It is also possible to measure the degree of relevance by referring to the resources released by the suprasystems, according to their ability of affecting the considered system’s survival. In this sense, within the context of reference different actors mature different expectations with respect to the same viable system.
These expectations translate into the ability of the viable system itself to ensure targets of efficiency, effectiveness, and sustainability.
These three dimensions can be defined as follows [2,40]:
-
Efficiency (plans): things are done in the right way.
-
Effectiveness (goals): the right things get done.
-
Sustainability (relationships): The right relationships exist with other service systems.
Bringing back these definitions to the vSa perspective, the objectives of efficiency can be measured with reference to the structure, while the effectiveness of the viable system is realized and measured with regard to its specific context of reference. Thus, it is possible to achieve sustainability objectives in relation to the general environment. From the above it derives that pursuing sustainability objectives is fundamental and functional to guaranteeing the conditions of viability and, therefore, the survival of the system [41,42].
In fact, in this case, we mean sustainability as a simultaneous achievement of economic, social and environmental sustainability, paying specific attention to the politico-institutional suprasystems, whose resources are critical for the system functioning and may consequently exert an influence on its survival and evolutionary dynamics.
Therefore, economic sustainability refers to the operating structure, where the achievement of conditions of economic-structural efficiency is a priority. Social sustainability adopts a vision oriented towards the systemic effectiveness of the context. Environmental sustainability, by incorporating the political dimension, has a broader scope and refers to long-term survival and long-term systemic balance.
In summary, in a Viable Systems Approach perspective, the search for sustainability is to be attributed to the search for the right relationships (defined as consonance) and the right interactions (defined as resonance) with the other systems within a specific context.
Thus, based on this general framework, in the next section we will propose a critical and holistic analysis for a unified health assessment, in a complex decision-making scenarios, such as HTA, that can harmonize a variety of resources, skills, expertise, interests, and expectations, reconciling the typically opposite targets of efficiency, effectiveness, and sustainability [2].

3.2. A Systems Approach to Health Evaluation Methods

The analysis in Section 2 shows that the most widespread methodologies for health assessment are limited to one-dimensional measurements of efficiency, inspired by the simple relationship between costs and results. Among them, the most common evaluation methods used for the assessment of new medical devices or drugs are the cost-effectiveness-analysis (CEA) and the cost-utility analysis (CUA) [43].
CEA defines the outcomes of an innovation in medicine according to specific dimensions (for example: delay in aggravating a syndrome, deferral of the time of surgery, effects of prevention programs, etc.).
Instead, the outcomes of the CUA are measured as health-related preferences, described as Quality Adjusted Life Years (QALYs) or Quality Adjusted Life Expected (QALE) gained; in fact, this method assumes that health is a function of length and quality of life and combines these values into a single index number.
In addition to the most commonly used methods, many other propositions can be found in literature, some of which are also sporadically used by the agencies of the different countries.
Among them, the Cost Consequence Analysis (CCA) does not attempt to summarize outcomes in a single measure (such as the QALY) or in financial terms. Instead, its outcomes are shown in their natural units (some of which may be monetary) and it is left to decision-makers to determine whether a treatment is worth being carried out. Consequently, CCA poses on the evaluator the problem of aggregating, weighing, and evaluating the components, data, and outcomes.
Apart from CUA, which summarizes the effects of an initiative in QALY, both CEA and CCA are susceptible to a further step, consisting in the treatment of their results with the multi-criteria analysis (Multiple Criteria Decision Analysis, MCDA) that is recently expanding into the field of drug and health-related assessments [44,45,46]. The definition of MCDA encompasses a wide range of different approaches. However, Garattini and Padula [47] criticize its use in HTA, because the main intrinsic limit for health policy decisions on new technologies is the lack of key information at the early stage of market approval.
The databases that can be used, regardless of the evaluation method, comprise data from a variety of sources, including, but not limited to, clinical trials and observations. It is worth noting that there are two basic approaches to economic evaluation. In trial-based studies, economic data (e.g., resource utilization, and quality of life) are collected alongside a single clinical study, usually a controlled clinical trial. In modelling studies, data from a wide range of sources (e.g., existing clinical trials, observational studies) are synthesized using an economic model [12]; the authors also conclude that the two approaches are complementary and not mutually exclusive.
A separate analysis deserves the cost benefit analysis (CBA) which is not affected by the limitations of the efficiency measurements of the cited methods, being by its nature aimed at considering the complex impact that any investment (in our case in medicine) produces on the well-being of the community that benefits from it. However, the recognized difficulties in applying the cost-benefit analysis, mostly deriving from the ambition to assign a monetary value to each of the expected effects, is amplified in the case of innovations in health systems for a series of ethical issues that its application raises, first of all the ones related to the evaluation of human life [48,49].
We believe that the QALY/CUA can be used on the condition that the assertion of every QALY having the same value regardless of the condition or the personal characteristics of the population treated (age, sex, severity of disease, level of deprivation, or other characteristics) is exceeded. Therefore, the QALY, as a direct effect of the new technology, must be quantified in a systems perspective, such as the proposed one of the vSa, with reference to:
  • the indirect effects, related not only to the patient, but also to the organization that provides care; and
  • the indirect costs of the disease (e.g., caregivers, etc.), on the consequences of the patient’s family entourage.
This, mainly, in order to overcome the first flaw identified by Drummond and Sculpher [12] consisting in the “omission of important costs or benefit”. Furthermore, the sustainability of the process must be considered in terms of the anthropic environment regarding the entire life cycle.
Table 2 summarizes the different evaluation methods analysed so far and the related measures.
As emerges from Table 2, the most used evaluation methods for HTA consider measures usually related to effectiveness. The proposed theoretical view of HTA, by referring to vSa principles, considers sustainability intended as the consideration of the political dimension in the assessment procedures, together with the economic and social aspects [25].
With the term political sustainability, in fact, it is to be intended the development and maintenance of the political will necessary to sustain a major policy direction in the health care system [25,50].
The vSa and its conceptualizations highlight key elements that allow the definition of a more appropriate approach to Health Technology Assessment. In fact, the general view of vSa considers the different perspectives, the priorities, and the mechanisms of influence of all the suprasystems in the HTA.
Consistently with a vSa view, the viability of the Healthcare system linked to the inclusion of social and political dimensions implies the shift from Health Technology Assessment (HTA) to Health Technology Sustainability (HTS), as there cannot be healthcare sustainability without health technology sustainability, and vice versa [51].
Accordingly, from the above several implications derive:
-
the consideration of all the dimensions of evaluation processes; in particular, efficiency and effectiveness are enhanced by including the sustainability perspective [2];
-
because of the previous point, the simultaneous consideration of all the suprasystems involved in healthcare system, both as users and as decision-makers; and
-
based on systems thinking, the evaluation of healthcare system in its both structural (efficiency perspective) and systemic (effectiveness and sustainability perspective) configuration.
In this regard, the proposed unified health assessment in a vSa framework presupposes the use of a new perspective that adds the political dimension to the pre-existing economic and social aspects, thus proposing an evaluation based on Cost per Unit Change expressed in terms of Quality Adjusted Life Year (QALY).
In line with the considerations proposed above, this new approach might be able to overcome the limitations of every QALY having the same value, thus quantifying the indirect effects of medical treatments, in a systems perspective.

4. Discussion, Limitations, and Future Lines of Research

The criticism of the methodological aspects of evaluating innovations in healthcare, and the continuous search for changes or alternatives to existing techniques seem to derive from the lack of awareness of the multiplicity of possible innovations with various characteristics, rather than from their intrinsic limitations. This means that for each of them it is necessary to identify the most appropriate methods among the existing ones. If, for innovations with high technological contents (e.g., high cost), and high social, economic and political impacts, a systems approach that considers the multiplicity of effects on the multiplicity of stakeholders involved is necessary, for procedural innovations of low cost and limited impact (for example procedures in the limited area of a hospital) simpler techniques are enough. This is also because evaluation procedures have costs that must be proportionate to the expected benefits. In this context, it should be noted that the cost of evaluation is mostly linked to the retrieval and systematization of data and information necessary for the evaluation itself, while only in residual part is attributable to the calculations. Whereas data and information can be used in different evaluation techniques, it is appropriate to apply them all: the result of each contributes to enrich the information flow for decision-makers.
Where replicated with reference to a larger sample (e.g., other disciplines, other areas, etc.), the analysis should be carried out employing different techniques, or a database able to exceed the limits of Scopus, connected to issues of duplications, delay in database update, and substantial costs [32].
Further limitations of present work could deal with the absence of a deeper analysis based on multivariate analysis on major components analysis, although Vosviewer Software still gives information about total link strength and citation, and closeness of data points.
Future lines of research will be focused on the number, the different nature, and the different degrees of complexity of innovation to be evaluated, as well as on the corresponding number and different nature of the decision-makers involved, whose decisions, depending on their role, produce effects on very different scales. This will imply the creation of a taxonomy of innovations, on one hand, and the analysis of roles and composition of decision-makers on the other. In fact, the possible numerous intersections between the objects (innovations) and the subjects (decision-makers) seem to be the starting point for the realization of reliable assessments, under the constraint of limited resources, and the related need to contain costs.

Supplementary Materials

The map is downloadable at the following link: https://bit.ly/2z9BCnd.

Author Contributions

Introduction: L.F.; Materials and Methods: V.P.; Results: I.F., F.I., L.F.; The Discussion and Conclusions section is the result of synergic final considerations.

Funding

This research received no external funding.

Conflicts of Interest

The authors declare no conflict of interest.

Appendix A

Table A1. Thesaurus used in the VosViewer elaboration.
Table A1. Thesaurus used in the VosViewer elaboration.
LabelReplace By
AdolescentAdolescent
AdultAdult
AgedAged
Article
Assessment methodAssessment method
Australia
Biomedical technologyHealth technology assessment
Biomedical technology assessmentHealth technology assessment
Brazil
BudgetBudget
Canada
ChildChild
China
Clinical decision-makingClinical decision-making
Clinical practiceClinical practice
Comparative studyComparative study
Conceptual frameworkConceptual framework
Controlled studyControlled study
Cost benefit analysisCost benefit analysis
Cost controlCost control
Cost effectiveness analysisCost effectiveness analysis
Cost utility analysisCost utility analysis
Cost-benefit analysisCost benefit analysis
Cost-effectivenessCost effectiveness analysis
Cost-effectiveness analysisCost effectiveness analysis
Decision-makingdecision-making
Decision-making, organizationaldecision-making
Decision support systemDecision support system
Decision support techniquesDecision support system
Decision-makingdecision-making
Delivery of health careDelivery of health care
DevicesDevices
Diffusion of innovationDiffusion of innovation
DocumentationDocumentation
DrugDrug
Drug costDrug costs
Drug costsDrug costs
Drug efficacyDrug efficacy
Drug industryDrug industry
Drug manufactureDrug manufacture
Drug marketingDrug marketing
Drug policyDrug policy
Drug safetyDrug safety
Economic aspectEconomic aspect
Economic evaluationEconomic evaluation
EconomicsEconomics
England
EthicsEthics
Europe
Evidence based medicineEvidence based medicine
Evidence-based medicineEvidence based medicine
FemaleFemale
Financial managementFinancial management
ForecastingForecasting
France
FundingFunding
Generic drugGeneric drug
Germany
GovernmentGovernment
Great britain
Gross national productGross national product
Health careHealth care
Health care accessHealth care access
Health care costHealth care costs
Health care costsHealth care costs
Health care deliveryHealth care delivery
Health care financingHealth care financing
Health care organizationHealth care organization
Health care planningHealth care planning
Health care policyHealth care policy
Health care qualityHealth care quality
Health care reformHealth care reform
Health care systemHealth care system
Health care utilizationHealth care utilization
Health economicsHealth economics
Health impact assessmentHealth impact assessment
Health insuranceHealth insurance
Health policyHealth care policy
Health serviceHealth services
Health servicesHealth services
Health statusHealth status
Health surveyHealth survey
Health technology assessmentHealth technology assessment
Health technology assessment (hta)Health technology assessment
Health technology assessmentsHealth technology assessment
HtaHealth technology assessment
Human
Humans
Hungary
Information processingInformation processing
InnovationInnovation
InsuranceInsurance
Interview
LawLaw
Literature
Major clinical studyMajor clinical study
MaleMale
ManagementManagement
Medical decision-makingMedical decision-making
Medical deviceMedical devices
Medical devicesMedical devices
Medical ethicsEthics
Medical researchMedical research
Medical technologyMedical technology
Methodology
Middle agedMiddle aged
Models, economic
National health programsNational health programs
National health serviceNational health service
Netherlands
Oncology
OrganizationOrganization
Organization and managementOrganization and management
Outcome assessmentOutcome assessment
Outcome assessment (health care)Outcome assessment
Patient preferencePatient preference
PharmaceuticalsPharmaceuticals
PharmacoeconomicsPharmacoeconomics
Poland
PolicyPolicy
Policy makingPolicy making
Practice guidelinePractice guideline
PrescriptionPrescription
PricingPricing
Priority journal
ProceduresProcedures
Public healthPublic health
Public health servicePublic health
Publication
Qualitative research
Quality adjusted life yearQuality adjusted life year
Quality controlQuality control
Quality of lifeQuality of life
Quality-adjusted life yearsQuality adjusted life year
Questionnaire
Randomized controlled trial (topic)Randomized controlled trial
ReimbursementReimbursement
Research design
Resource allocationResource allocation
Review
Review literature as topic
Risk assessmentRisk assessment
StandardStandards
StandardsStandards
State medicineState medicine
Statistical analysisStatistical analysis
Statistics and numerical dataStatistical analysis
Sweden
Systematic review
Technological developmentTechnological development
TechnologyTechnology
Technology assessmentHealth technology assessment
Technology assessment, biomedicalHealth technology assessment
Total quality managementTotal quality management
Treatment outcomeTreatment outcome
TrendsTrends
UncertaintyUncertainty
United kingdom
United states
WellbeingWellbeing

Appendix B

Table A2. Weights and Links strength of the references.
Table A2. Weights and Links strength of the references.
IdLabelUrlxyClusterWeight <Links>Weight <Total Link Strength>Weight <Citations>Weight <Norm. Citations>Score <Pub. Year>Score <Citations>Score <Norm. Citations>
1rutnam (1991)https://doi.org/10.1080/08164649.1991.99946281.00232.2198160011.000199111.000
2smith (1994)https://doi.org/10.1016/0277-9536(94)90067-11.94841.3767170060.947199460.947
3france (1994)https://doi.org/10.1016/0277-9536(94)90064-70.66132.3638180030.474199430.474
4granados (1994)https://doi.org/10.1016/0277-9536(94)90065-52.309−0.51751900101.5791994101.579
5freemantle (1995)https://doi.org/10.1016/0277-9536(94)00272-u2.2040.87912000131.0001995131.000
6reuzel (1999)https://doi.org/10.1023/a:1009963018813−0.9821−2.32462100141.0001999141.000
7jones (2000)https://doi.org/10.1057/palgrave.jmm.5040022−0.34310.042781110.286200010.286
8reuzel (2000)https://doi.org/10.1177/13563890022209389−0.24780.038467561.714200061.714
9jones (2001)https://doi.org/10.1057/palgrave.jmm.5040041−0.2421−0.087441100.000200100.000
10oliver (2001)https://doi.org/10.1177/13563890122209847−0.23870.1697521162.0002001162.000
11wells (2002)https://doi.org/10.1049/em:20020410−0.0999−0.100541131.000200231.000
12jacobs (2003)https://doi.org/10.1177/10442073030140021001−0.3183−0.0247117270.240200370.240
13aspinall (2003)https://doi.org/10.1016/s0277-9536(02)00027-82.13391.04882200250.8572003250.857
14may (2003)https://doi.org/10.1016/s0277-9536(02)00419-7−0.13940.1675184893.0512003893.051
15sloane (2003)https://doi.org/10.1016/s0305-0548(02)00187-9−0.1546−0.00021112511.7492003511.749
16cohen (2003)https://doi.org/10.1002/hec.791−0.2579−0.146823320.069200320.069
17szucs (2003)https://doi.org/10.1057/palgrave.jcb.3040064−0.2471−0.079746510.034200310.034
18briggs (2004)https://doi.org/10.2165/00148365-200403020-00004−0.3095−0.17483118291.0002004291.000
19vázquez-polo (2005)https://doi.org/10.1002/hec.947−0.3572−0.1971358110.4542005110.454
20milewa (2005)https://doi.org/10.1111/j.1467-9515.2005.00452.x−0.31260.128451410261.0722005261.072
21ginnelly (2005)https://doi.org/10.2165/00148365-200504010-00006−0.2972−0.148431211251.0312005251.031
22hofmann (2005)https://doi.org/10.1007/s10202-005-0073-1−0.18040.149111610351.4432005351.443
23milewa (2006)https://doi.org/10.1016/j.socscimed.2006.08.009−0.33480.14315914191.0002006191.000
24brown (2007)https://doi.org/10.1057/palgrave.jmm.50500861.2452−2.07792300121.0002007121.000
25scheibler (2008)https://doi.org/10.1016/j.zefq.2008.07.017−0.2634−0.0507112251.333200851.333
26hoppe (2008)https://doi.org/10.1111/j.1467-8691.2008.00495.x−0.14270.213413351.333200851.333
27lehoux (2008)https://doi.org/10.1177/1356389008090857−0.20770.14551211741.067200841.067
28freemantle (2008)https://doi.org/10.1007/s10198-008-0123-4−0.1887−0.216822310.267200810.267
29sacchini (2009)https://doi.org/10.1007/s11019-009-9206-y−0.12830.216811712130.9442009130.944
30göhlen (2009)https://doi.org/10.1016/j.zefq.2009.06.0151.20872.1022240000.000200900.000
31vespermann (2009)https://doi.org/10.1016/j.zefq.2009.06.0082.04831.2143250000.000200900.000
32wild (2009)https://doi.org/10.1016/j.zefq.2009.06.0102.30750.5343260000.000200900.000
33schwarzer (2009)https://doi.org/10.1016/j.zefq.2009.05.020−0.2724−0.197331120.145200920.145
34welton (2009)https://doi.org/10.1111/j.1467-985x.2008.00548.x−0.199−0.1982245624.5002009624.500
35lehoux (2009)https://doi.org/10.1016/j.socscimed.2009.03.017−0.16480.154611310372.6862009372.686
36gammon (2009)https://doi.org/10.1136/jme.2008.027920−0.12540.2055111470.508200970.508
37moreno (2009)https://doi.org/10.1080/13571510903227056−0.3567−0.193935730.218200930.218
38groop (2010) −0.2386−0.160925360.465201060.465
39strech (2010)https://doi.org/10.1016/j.zefq.2010.03.001−0.288−0.0565210420.155201020.155
40boenink (2010)https://doi.org/10.1007/s11019-009-9223-x−0.14430.20251156272.0932010272.093
41bührlen (2010)https://doi.org/10.1016/j.zefq.2010.10.012−0.24860.0812621910.078201010.078
42koivisto (2010)https://doi.org/10.1332/174426410 × 482980−0.19490.135114740.310201040.310
43torbica (2010)https://doi.org/10.1057/jmm.2009.48−0.2188−0.0673483110.8532010110.853
44allen (2010)https://doi.org/10.3109/014215909033906190.28062.4565270010.078201010.078
45bridges (2010)https://doi.org/10.1108/s0731-2199(2010)0000022005−0.4454−0.009271112120.9302010120.930
46woodman (2010)https://doi.org/10.3163/1536-5050.98.2.0061.55761.8341280080.620201080.620
47gauvin (2010)https://doi.org/10.1016/j.socscimed.2010.01.036−0.19580.159913624574.4192010574.419
48martin (2011)https://doi.org/10.4067/s1726-569 × 2011000200009−0.15840.1717122910.060201110.060
49czech (2011)https://doi.org/10.14254/2071-789x.2011/4-1a/8−0.54710.0001102200.000201100.000
50brousselle (2011)https://doi.org/10.1016/j.socscimed.2011.01.008−0.30980.032363927331.9762011331.976
51drummond (2011)https://doi.org/10.1007/s10198-010-0274-y−0.32440.0744611372.2162011372.216
52jarosławski (2011)https://doi.org/10.2165/11592960-000000000-00000−0.5610.0202102160.359201160.359
53bombard (2011)https://doi.org/10.1016/j.socscimed.2011.04.017−0.15560.189213121352.0962011352.096
54walters (2011)https://doi.org/10.1080/02664763.2010.545375−0.50610.0147101120.120201120.120
55goeree (2011)https://doi.org/10.2147/ceor.s14404−0.3672−0.051435362.1562011362.156
56orlewska (2011)https://doi.org/10.1556/socec.33.2011.3.8−0.6368−0.1509134730.180201130.180
57meltzer (2011)https://doi.org/10.1016/b978-0-444-53592-4.00007-4−0.2879−0.022662513140.8382011140.838
58droste (2012)https://doi.org/10.1016/j.zefq.2012.05.019−0.130.2185110220.222201220.222
59kelly (2012)https://doi.org/10.1057/sth.2011.21−0.27280.02316206293.2222012293.222
60boenink (2012)https://doi.org/10.1007/s10728-011-0173-0−0.21480.1211211430.333201230.333
61jommi (2012)https://doi.org/10.1177/1745790412440704−0.58120.0278103330.333201230.333
62kuchenbecker (2012)https://doi.org/10.1016/j.vhri.2012.09.009−0.36360.051678580.889201280.889
63augustovski (2012)https://doi.org/10.1016/j.vhri.2012.09.007−0.40750.065471150.556201250.556
64vargas-zea (2012)https://doi.org/10.1016/j.vhri.2012.09.0042.3580.18372900131.4442012131.444
65siebert (2013)https://doi.org/10.1016/j.zefq.2013.10.020−0.3448−0.133831412111.1332013111.133
66perleth (2013)https://doi.org/10.1016/j.zefq.2013.04.006−0.2862−0.0864417610.103201310.103
67wild (2013)https://doi.org/10.1016/j.zefq.2013.02.008−0.3427−0.031923410.103201310.103
68eckermann (2013)https://doi.org/10.1016/j.socscimed.2012.10.020−0.3497−0.1116321770.721201370.721
69spinner (2013)https://doi.org/10.2147/ceor.s39624−0.36270.056753113141.4422013141.442
70smith (2013)https://doi.org/10.1177/1745790413476876−0.1785−0.07498530.309201330.309
71niewada (2013)https://doi.org/10.1016/j.vhri.2013.05.002−0.30320.0081211580.824201380.824
72odame (2013)https://doi.org/10.1016/j.vhri.2013.07.006−0.3549−0.0125726950.515201350.515
73petrou (2013)https://doi.org/10.1016/j.vhri.2013.06.016−0.2596−0.0383443131.3392013131.339
74sura (2013)https://doi.org/10.1016/j.vhri.2013.06.0121.9613−1.3558300010.103201310.103
75kaló (2013)https://doi.org/10.1016/j.vhri.2013.06.002−0.6073−0.14021334272.7822013272.782
76elsisi (2013)https://doi.org/10.1016/j.vhri.2013.06.014−0.3557−0.01971117670.721201370.721
77salvatore (2013)https://doi.org/10.1177/1745790413498410−0.1837−0.045849500.000201300.000
78thébaut (2013)https://doi.org/10.1016/j.socscimed.2013.10.020−0.2687−0.079127630.309201330.309
79hevér (2013)https://doi.org/10.1556/socec.2013.0008−0.6075−0.1331136810.103201310.103
80attema (2013)https://doi.org/10.1007/s10198-013-0508-x−0.2782−0.09132119363.7092013363.709
81ulucanlar (2013)https://doi.org/10.1016/j.socscimed.2013.09.008−0.15740.104511511272.7822013272.782
82neyt (2014)https://doi.org/10.3917/rpve.534.0055−0.2916−0.0018618200.000201400.000
83ríos (2014a)https://doi.org/10.1016/j.vhri.2014.02.0050.6749−2.3566151400.000201400.000
84ríos (2014b)https://doi.org/10.1016/j.vhri.2014.08.0020.6755−2.3563151410.157201410.157
85jain (2014)https://doi.org/10.1016/j.vhri.2014.04.006−0.35370.0247710710.157201410.157
86gulácsi (2014)https://doi.org/10.1007/s10198-014-0590-8−0.36910.0678855325.0162014325.016
87kennedy-martin (2014)https://doi.org/10.1016/j.vhri.2014.03.001−0.38090.0229817850.784201450.784
88hunger (2014)https://doi.org/10.1007/s00038-013-0494-x−0.399−0.152232110.157201410.157
89jakubiak-lasocka (2014)https://doi.org/10.1016/j.vhri.2014.06.008−0.3366−0.021811104111.7242014111.724
90böhm (2014)https://doi.org/10.1080/07036337.2013.793679−0.3527−0.061937381.254201481.254
91elias (2014)https://doi.org/10.1016/j.zefq.2014.08.021−0.38410.061272420.314201420.314
92madan (2014)https://doi.org/10.1111/rssa.12018−0.1944−0.201825791.411201491.411
93cerri (2014)https://doi.org/10.1007/s10198-013-0514-z−0.30050.0907675101.5682014101.568
94skoupá (2014)https://doi.org/10.1016/j.vhri.2014.06.003−0.429−0.10911332101.5682014101.568
95gorenoi (2014)https://doi.org/10.1016/j.zefq.2014.03.017−0.2066−0.16226700.000201400.000
96mendonça (2014)https://doi.org/10.1007/s10198-013-0522-z1.5818−1.8096310030.470201430.470
97lopert (2014)https://doi.org/10.1016/j.zefq.2014.08.020−0.39640.098655210.157201410.157
98daniel mullins (2014)https://doi.org/10.1016/j.vhri.2014.02.006−0.3253−0.05124111340.627201440.627
99horváth cs.z. (2014)https://doi.org/10.1007/s10198-014-0601-9−0.6504−0.1549132450.784201450.784
100tetteh (2014)https://doi.org/10.1186/s13561-014-0026-2−0.3634−0.099336210.157201410.157
101heintz (2014)https://doi.org/10.1016/j.zefq.2014.09.006−0.5044−2.454320000.000201400.000
102abrishami (2014)https://doi.org/10.1016/j.socscimed.2014.07.046−0.17060.0643912791.411201491.411
103gurtner (2014)https://doi.org/10.1097/hmr.0b013e3182993b91−0.2725−0.01891130991.411201491.411
104mitton (2014)https://doi.org/10.1007/s40258-013-0074-5−0.2622−0.065511128233.6052014233.605
105walzer (2014)https://doi.org/10.2147/ceor.s536011.70581.688330091.411201491.411
106rogers (2014)https://doi.org/10.1111/j.1467-8519.2012.01980.x−0.0998−0.100441120.314201420.314
107li (2014)https://doi.org/10.1016/j.vhri.2013.04.001−0.38430.038686310.157201410.157
108rader (2014)https://doi.org/10.1002/jrsm.1097−0.1130.31621411142.1952014142.195
109robertson (2014)https://doi.org/10.1002/jrsm.1102−0.193−0.18225250.784201450.784
110pieper (2014)https://doi.org/10.1002/jrsm.1107−0.2253−0.138425791.411201491.411
111siebert (2015)https://doi.org/10.1016/j.zefq.2015.06.012−0.4021−0.147334240.656201540.656
112schnell-inderst (2015)https://doi.org/10.1016/j.zefq.2015.06.011−0.1153−0.09274116101.6392015101.639
113stürzlinger (2015)https://doi.org/10.1016/j.zefq.2015.07.0021.0457−2.1952340000.000201500.000
114ivlev (2015)https://doi.org/10.1016/j.ejor.2015.05.075−0.2275−0.03039169193.1152015193.115
115wang (2015)https://doi.org/10.1093/hrlr/ngv0251.39381.9738350010.164201510.164
116pfadenhauer (2015)https://doi.org/10.1016/j.zefq.2015.01.004−0.14310.1065153182.9512015182.951
117nachtnebel (2015)https://doi.org/10.1016/j.zefq.2015.05.012−0.15660.2313220.328201520.328
118rao (2015)https://doi.org/10.5912/jcb669−0.54690.037103110.164201510.164
119cuijpers (2015)https://doi.org/10.1016/j.techfore.2014.03.0062.36550.0078360091.475201591.475
120peine (2015)https://doi.org/10.1016/j.techfore.2014.08.019−0.15610.129617640.656201540.656
121cook (2015)https://doi.org/10.1016/j.vhri.2015.03.013−0.3957−0.003984410.164201510.164
122lopes (2015)https://doi.org/10.1016/j.socscimed.2015.04.021−0.27950.10435341550.820201550.820
123rocchi (2015)https://doi.org/10.2147/ceor.s82549−2.16281.3879370050.820201550.820
124kolominsky-rabas (2015)https://doi.org/10.1016/j.techfore.2013.12.005−0.1868−0.095791612121.9672015121.967
125griffiths (2015)https://doi.org/10.2147/ceor.s87462−0.37850.0421525871.148201571.148
126dranitsaris (2015)https://doi.org/10.1007/s40258-014-0130-9−0.47670.03791019671.148201571.148
127petrou (2015)https://doi.org/10.1007/s40258-015-0191-4−0.2727−0.047373300.000201500.000
128winnette (2015)https://doi.org/10.1016/j.vhri.2015.03.008−0.45240.006673310.164201510.164
129bitencourt (2015)https://doi.org/10.1016/j.vhri.2015.08.0022.1415−1.0297380030.492201530.492
130brazier (2015)https://doi.org/10.1007/s40258-015-0194-1−0.2521−0.0961267132.1312015132.131
131sacchini (2016) −0.14420.20871232000.000201600.000
132wortley (2016)https://doi.org/10.1108/jhom-08-2015-0119−0.27170.05565382210.309201610.309
133peregrin (2016)https://doi.org/10.1504/ijbsr.2016.075746−0.1731−0.00221110800.000201600.000
134petrillo (2016)https://doi.org/10.1504/ijmcdm.2016.077878−0.26220.026711101110.309201610.309
135manelli (2016) −0.25540.0602620500.000201600.000
136brown (2016)https://doi.org/10.1177/0306312715609699−0.41620.126952272.162201672.162
137babigumira (2016)https://doi.org/10.1111/jphs.12120−0.33550.003676720.618201620.618
138lysdahl (2016)https://doi.org/10.1186/s12910-016-0099-z−0.13710.16321181120.618201620.618
139koh (2016)https://doi.org/10.1016/j.vhri.2015.06.0041.4224−1.9477390061.853201661.853
140dang (2016)https://doi.org/10.1016/j.vhri.2015.11.005−0.39740.0107713841.235201641.235
141dilokthornsakul (2016)https://doi.org/10.1016/j.vhri.2015.12.003−0.3584−0.1186310810.309201610.309
142mühlbacher (2016)https://doi.org/10.1007/s40258-016-0232-7−0.1706−0.079791510216.4852016216.485
143assasi (2016)https://doi.org/10.1186/s12910-016-0118-0−0.1380.267714201810.309201610.309
144sullivan (2016)https://doi.org/10.1007/s10198-015-0720-y2.26180.7075400061.853201661.853
145thompson (2016)https://doi.org/10.2147/ceor.s96616−0.56170.0341105600.000201600.000
146radu (2016)https://doi.org/10.1016/j.vhri.2016.07.006−0.37520.075481141.235201641.235
147grundy (2016)https://doi.org/10.1016/j.socscimed.2016.07.042−0.252−0.014578910.309201610.309
148panayidou (2016)https://doi.org/10.1002/jrsm.1202−0.274−0.174234851.544201651.544
149tsiachristas (2016)https://doi.org/10.5334/ijic.2472−0.2259−0.01471227841.235201641.235
150meyer (2016)https://doi.org/10.1016/j.zefq.2016.07.011−0.1628−0.117291100.000201600.000
151janssen (2016)https://doi.org/10.2147/ppa.s122319−0.1999−0.0711215720.618201620.618
152ducey (2017)https://doi.org/10.1332/174426415 × 14443053123024−0.16920.15771282200.000201700.000
153callea (2017)https://doi.org/10.1016/j.socscimed.2016.11.038−0.1549−0.02547641.892201741.892
154wright (2017)https://doi.org/10.1111/1758-5899.12215−0.4475−0.024610141500.000201700.000
155mühlbacher (2017)https://doi.org/10.1007/s10198-016-0763-8−0.1563−0.079195983.784201783.784
156blome (2017)https://doi.org/10.1007/s10198-016-0765-6−0.4683−0.0188102200.000201700.000
157castro (2017)https://doi.org/10.1111/1758-5899.12333−0.33540.0339821910.473201710.473
158mossman (2017)https://doi.org/10.1111/1758-5899.12221−0.23740.069354200.000201700.000
159kanavos (2017)https://doi.org/10.1111/1758-5899.12386−0.55950.01981081231.419201731.419
160hensher (2017)https://doi.org/10.1016/j.socscimed.2017.01.020−0.3613−0.092833131.419201731.419
161thijssen (2017)https://doi.org/10.1016/j.jedc.2017.01.016−0.1679−0.008593310.473201710.473
162jakubczyk (2017)https://doi.org/10.1007/s10479-015-1910-9−0.3707−0.12133211352.365201752.365
163markiewicz (2017)https://doi.org/10.5912/jcb780−0.1928−0.0632921610.473201710.473
164gyalrong-steur (2017)https://doi.org/10.1016/j.zefq.2017.01.0021.83561.536410000.000201700.000
165hofmann (2017)https://doi.org/10.1007/s11948-016-9791-0−0.13160.2436111620.946201720.946
166greer (2017)https://doi.org/10.1057/cep.2016.6−0.31110.0516823820.946201720.946
167nicod (2017)https://doi.org/10.1007/s10198-016-0823-0−0.45910.0102102818125.6762017125.676
168rautenberg (2017)https://doi.org/10.2147/ceor.s140902−0.367−0.134631110.473201710.473
169cowles (2017)https://doi.org/10.1007/s40258-017-0309-y−0.33150.0075815720.946201720.946
170angelis (2017)https://doi.org/10.1016/j.socscimed.2017.06.024−0.2626−0.056312312473.311201773.311
171inotai (2017)https://doi.org/10.1016/j.vhri.2017.06.003−0.5388−0.1163138720.946201720.946
172brixner (2017)https://doi.org/10.1016/j.vhri.2017.02.001−0.2531−0.040112171631.419201731.419
173skoupá (2017)https://doi.org/10.1016/j.vhri.2017.08.0021.7247−1.6649420010.473201710.473
174yagudina (2017)https://doi.org/10.1016/j.vhri.2017.07.006−0.46970.063251110.473201710.473
175dimova (2017)https://doi.org/10.1016/j.vhri.2017.08.0012.2091−0.8607430020.946201720.946
176culig (2017)https://doi.org/10.1016/j.vhri.2017.07.005−0.43862.4633440010.473201710.473
177jahnz-różyk (2017)https://doi.org/10.1016/j.vhri.2017.07.0012.3408−0.3436450031.419201731.419
178silins (2017)https://doi.org/10.1016/j.vhri.2017.08.006−1.7813−1.8328460020.946201720.946
179chambers (2017) −0.42420.111156310.473201710.473
180knott (2017)https://doi.org/10.1016/j.socscimed.2017.08.033−0.2469−0.110621200.000201700.000
181mertz (2017)https://doi.org/10.1016/j.zefq.2017.07.010−0.11150.3164141300.000201700.000
182donin (2017)https://doi.org/10.3846/16111699.2017.1409798−0.1875−0.060897910.473201710.473
183rawson (2017)https://doi.org/10.2147/ceor.s144695−0.29160.107864720.946201720.946
184rosselli (2017)https://doi.org/10.1016/j.vhri.2017.02.004−0.38330.04987281310.473201710.473
185kibel (2017)https://doi.org/10.1016/j.socscimed.2017.11.024−0.2257−0.0132210610.473201710.473
186paolucci (2017)https://doi.org/10.1007/s40258-017-0349-3−0.2708−0.01971214910.473201710.473
187angelis (2018)https://doi.org/10.1007/s10198-017-0871-0−0.33590.018112423756.500201856.500
188klímová (2018)https://doi.org/10.15240/tul/001/2018-1-008−0.202−0.0593921700.000201800.000
189castro (2018)https://doi.org/10.1590/1807-57622016.0549−0.1970.05035271600.000201800.000
190chen (2018a)https://doi.org/10.5582/bst.2018.01038−0.38810.049687300.000201800.000
191löblová (2018)https://doi.org/10.1111/psj.12213−0.35180.0459818733.900201833.900
192wong (2018)https://doi.org/10.1007/s40258-017-0339-5−0.44390.05175161100.000201800.000
193fierlbeck (2018)https://doi.org/10.1111/capa.12253−0.26390.09246161500.000201800.000
194rehfuess (2018)https://doi.org/10.1002/jrsm.1254−0.16360.0294125456.500201856.500
195nord (2018)https://doi.org/10.1007/s10198-017-0882-x−0.3253−0.025768422.600201822.600
196yi (2018)https://doi.org/10.2147/tcrm.s163190−0.197−0.16225200.000201800.000
197zhen (2018)https://doi.org/10.1016/j.vhri.2018.01.010−0.38360.0407810600.000201800.000
198chen (2018b)https://doi.org/10.1016/j.techfore.2018.01.033−0.1611−0.027792211.300201811.300
199chen (2018c)https://doi.org/10.1016/j.vhri.2018.03.0042.2668−0.6904470000.000201800.000
200thornton snider (2018)https://doi.org/10.1515/fhep-2016-00141.8509−1.5132480000.000201800.000
201olofsson (2018)https://doi.org/10.1007/s10198-017-0922-6−0.3437−0.001867411.300201811.300
202kyle (2018)https://doi.org/10.1007/s11151-018-9639-7−1.08992.2796490011.300201811.300
203al rabayah (2018)https://doi.org/10.1111/jphs.12241−0.3556−0.06381317500.000201800.000
204zegeye (2018)https://doi.org/10.1016/j.vhri.2018.07.001−0.3092−0.013876600.000201800.000
205brixner (2018)https://doi.org/10.1016/j.vhri.2018.01.003−0.2265−0.0518129722.600201822.600
206radu (2018)https://doi.org/10.1016/j.vhri.2017.11.0032.33920.3594500000.000201800.000
207prasolov (2018)https://doi.org/10.1016/j.vhri.2018.04.0022.3583−0.1679510000.000201800.000
208yfantopoulos (2018)https://doi.org/10.1016/j.vhri.2018.06.006−0.1402−0.026841100.000201800.000
209mägi (2018)https://doi.org/10.1016/j.vhri.2017.10.0012.0584−1.1946520000.000201800.000
210palozzi (2018)https://doi.org/10.3390/su10103550−0.24710.00484311600.000201800.000
211espinoza (2018)https://doi.org/10.1016/j.vhri.2018.07.003−0.2227−0.04641210300.000201800.000
212calderón (2018)https://doi.org/10.1016/j.vhri.2018.01.011−0.1967−0.169525300.000201800.000

References

  1. Smith, C. New technology continues to invade healthcare: What are the strategic implications/outcomes? Nurs. Adm. Q. 2004, 28, 92–98. [Google Scholar] [CrossRef] [PubMed]
  2. Saviano, M.; Bassano, C.; Calabrese, M. A VSA-SS approach to healthcare service systems the triple target of efficiency, effectiveness and sustainability. Serv. Sci. 2010, 2, 41–61. [Google Scholar] [CrossRef]
  3. OECD. New Health Technologies Managing Access, Value and Sustainability: Managing Access, Value and Sustainability; OECD Publishing: Paris, France, 2017. [Google Scholar]
  4. Russell, S.; Hauert, S.; Altman, R.; Veloso, M. Robotics: Ethics of artificial intelligence. Nature 2015, 521, 415–418. [Google Scholar] [PubMed][Green Version]
  5. Thielst, C.B. The future of healthcare technology. J. Healthc. Manag. 2007, 52, 7. [Google Scholar] [CrossRef] [PubMed]
  6. Gulácsi, L.; Orlewska, E.; Péntek, M. Health economics and health technology assessment in Central and Eastern Europe: A dose of reality. Eur. J. Health Econ. 2012, 13, 525–531. [Google Scholar] [CrossRef] [PubMed]
  7. Banta, D. The development of health technology assessment. Health Policy 2003, 63, 121–132. [Google Scholar] [CrossRef]
  8. Banta, D.; Jonsson, E. History of HTA: Introduction. Int. J. Technol. Assess. Health Care 2009, 25, 1–6. [Google Scholar] [CrossRef]
  9. Inotai, A.; Csanadi, M.; Harsanyi, A.; Nemeth, B. Drug Policy in Central Eastern Europe—Hungary. Value Health Reg. Issues 2017, 13, 16–22. [Google Scholar] [CrossRef]
  10. Draborg, E.; Gyrd-Hansen, D.; Poulsen, P.B.; Horder, M. International comparison of the definition and the practical application of health technology assessment. Int. J. Technol. Assess. Health Care 2005, 21, 89–95. [Google Scholar] [CrossRef]
  11. Herndon, J.H.; Hwang, R.; Bozic, K. Healthcare technology and technology assessment. Eur. Spine J. 2007, 16, 1293–1302. [Google Scholar] [CrossRef]
  12. Drummond, M.; Sculpher, M. Common methodological flaws in economic evaluations. Med. Care 2005, 43, II5–II14. [Google Scholar] [CrossRef]
  13. Abrishami, P.; Boer, A.; Horstman, K. Understanding the adoption dynamics of medical innovations: Affordances of the da Vinci robot in the Netherlands. Soc. Sci. Med. 2014, 117, 125–133. [Google Scholar] [CrossRef] [PubMed]
  14. Kern, S.E.; Jaron, D. Healthcare technology, economics, and policy: An evolving balance. IEEE Eng. Med. Biol. Mag. 2003, 22, 16–19. [Google Scholar] [CrossRef] [PubMed]
  15. Vuong, Q.H.; Ho, T.M.; Vuong, T.T. Healthcare consumers’ sensitivity to costs: A reflection on behavioural economics from an emerging market. Palgrave Commun. 2018, 4, 70. [Google Scholar] [CrossRef]
  16. Hansjürgens, B. Economic valuation through cost-benefit analysis–possibilities and limitations. Toxicology 2004, 205, 241–252. [Google Scholar] [CrossRef] [PubMed]
  17. Goeree, R.; He, J.; O’Reilly, D.; Tarride, J.E.; Xie, F.; Lim, M.; Burke, N. Transferability of health technology assessments and economic evaluations: A systematic review of approaches for assessment and application. ClinicoEcon. Outcomes Res. 2011, 3, 89–104. [Google Scholar] [CrossRef] [PubMed]
  18. Yagudina, R.I.; Kulikov, A.U.; Serpik, V.G.; Ugrekhelidze, D.T. Concept of Combining Cost-Effectiveness Analysis and Budget Impact Analysis in Health Care Decision Making. Value Health Reg. Issues 2017, 13, 61–66. [Google Scholar] [CrossRef]
  19. Bodenheimer, T. High and rising health care costs. Part 2: Technologic innovation. Ann. Intern. Med. 2005, 142, 932–937. [Google Scholar] [CrossRef]
  20. Groop, J.; Reijonsaari, K.; Lillrank, P. Applying the theory of constraints to health technology assessment. Int. J. Adv. Life Sci. 2010, 2, 115–124. [Google Scholar]
  21. Brousselle, A.; Lessard, C. Economic evaluation to inform health care decision-making: Promise, pitfalls and a proposal for an alternative path. Soc. Sci. Med. 2011, 72, 832–839. [Google Scholar] [CrossRef]
  22. Golinelli, G.M. L’approccio Sistemico al Governo Dell’impresa, Volume 1; CEDAM: Padova, Italy, 2000. [Google Scholar]
  23. Golinelli, G.M. Viable Systems Approach (VSA): Governing Business Dynamics; CEDAM: Padua, Italy, 2010. [Google Scholar]
  24. Barile, S. Management Sistemico Vitale; Giappichelli: Lyon, France, 2009. [Google Scholar]
  25. Borgonovi, E.; Compagni, A. Sustaining universal health coverage: The interaction of social, political, and economic sustainability. Value Health 2013, 16, S34–S38. [Google Scholar] [CrossRef] [PubMed]
  26. Saviano, M.; Bassano, C.; Piciocchi, P.; Di Nauta, P.; Lettieri, M. Monitoring viability and sustainability in healthcare organizations. Sustainability 2018, 10, 3548. [Google Scholar] [CrossRef]
  27. Berelson, B. Content Analysis in Communication Research; Free Press: Glencoe, IL, USA, 1952. [Google Scholar]
  28. Krippendorff, K. Content Analysis: An Introduction to Its Methodology, 2nd ed.; Sage: Thousand Oaks, CA, USA, 2004; ISBN 9780761915454. [Google Scholar]
  29. Glaser, B.G.; Strauss, A.L. The Discovery of Grounded Theory: Strategies for Qualitative Research; Aldine Pub.: Chicago, IL, USA, 1967. [Google Scholar]
  30. Goulding, C. Grounded Theory: A Practical Guide for Management, Business and Market Researchers; Sage: London, UK, 2002. [Google Scholar]
  31. Valderrama-Zurián, J.C.; Aguilar-Moya, R.; Melero-Fuentes, D.; Aleixandre-Benavent, R. A systematic analysis of duplicate records in Scopus. J. Inf. 2015, 9, 570–576. [Google Scholar] [CrossRef][Green Version]
  32. Vuong, Q.H.; La, V.P.; Vuong, T.T.; Ho, M.T.; Nguyen, H.K.; Nguyen, V.H.; Pham, H.H.; Ho, M.T. An open database of productivity in Vietnam’s social sciences and humanities for public use. Sci. Data 2018, 5, 180188. [Google Scholar] [CrossRef] [PubMed]
  33. van Eck, N.J.; Waltman, L. Software survey: VOSviewer, a computer program for bibliometric mapping. Scientometrics 2010, 84, 523–538. [Google Scholar] [CrossRef] [PubMed]
  34. van Eck, N.J.; Waltman, L. Visualizing bibliometric networks. In Measuring Scholarly Impact; Springer: Berlin, Germany, 2014; pp. 285–320. [Google Scholar]
  35. Badinelli, R.; Barile, S.; Ng, I.; Polese, F.; Saviano, M.; Di Nauta, P. Viable service systems and decision making in service management. J. Serv. Manag. 2012, 23, 498–526. [Google Scholar] [CrossRef][Green Version]
  36. Barile, S.; Saviano, M. Foundations of systems thinking: The structure-systems paradigm. In Contributions to Theoretical and Practical Advances in Management. A Viable Systems Approach (vSa); Barile, S., Ed.; International Printing: Avellino, Italy, 2011. [Google Scholar]
  37. Barile, S. Management Sistemico Vitale: Decisioni e Scelte in Ambito Complesso; International Printing Srl Editore: Avellino, Italy, 2011. [Google Scholar]
  38. Beer, S. Brain of the Firm: The Managerial Cybernetics of Organization; Allen Lane the Penguin Press: London, UK, 1972. [Google Scholar]
  39. Barile, S.; Sancetta, G.; Saviano, M. Management. Il Modello Sistemico e le Decisioni Manageriali, Volume I; Giappichelli: Turin, Italy, 2015. [Google Scholar]
  40. Spohrer, J.; Maglio, P.P.; Bailey, J.; Gruhl, D. Steps toward a science of service systems. Computer 2007, 40, 71–77. [Google Scholar] [CrossRef]
  41. Barile, S.; Saviano, M.; Iandolo, F.; Calabrese, M. The viable systems approach and its contribution to the analysis of sustainable business behaviors. Syst. Res. Behav. Sci. 2014, 31, 683–695. [Google Scholar] [CrossRef]
  42. Quattrociocchi, B.; Iandolo, F.; Fulco, I.; Calabrese, M. Capitolo III Efficienza, efficacia e sostenibilità. Il contributo dell’Approccio Sistemico Vitale (ASV) all’orientamento dei comportamenti d’impresa. In Il Controllo Manageriale e gli Indicatori di Performance Dentro e Fuori le Organizzazioni: Alcuni Contributi di Studio; Edizioni Nuova Cultura: Rome, Italy, 2018. [Google Scholar]
  43. Klímová, B.; Marešová, P. Economic methods used in health technology assessment. Econ. Manag. 2018, 21, 116–126. [Google Scholar] [CrossRef][Green Version]
  44. Angelis, A.; Kanavos, P. Multiple Criteria Decision Analysis (MCDA) for evaluating new medicines in Health Technology Assessment and beyond: The Advance Value Framework. Soc. Sci. Med. 2017, 188, 137–156. [Google Scholar] [CrossRef][Green Version]
  45. Ivlev, I.; Kneppo, P.; Bartak, M. Multicriteria decision analysis: A multifaceted approach to medical equipment management. Technol. Econ. Dev. Econ. 2014, 20, 576–589. [Google Scholar] [CrossRef]
  46. Jakubiak-Lasocka, J.; Jakubczyk, M. Cost-effectiveness versus Cost-Utility Analyses: What Are the Motives Behind Using Each and How Do Their Results Differ?—A Polish Example. Value Health Reg. Issues 2014, 4, 66–74. [Google Scholar] [CrossRef] [PubMed]
  47. Garattini, L.; Padula, A. Multiple Criteria Decision Analysis in Health Technology Assessment for Drugs: Just Another Illusion? Appl. Health Econ. Health Policy 2018, 16, 1–4. [Google Scholar] [CrossRef] [PubMed]
  48. Assasi, N.; Tarride, J.E.; O’Reilly, D.; Schwartz, L. Steps toward improving ethical evaluation in health technology assessment: A proposed framework. BMC Med. Ethics 2016, 17, 34. [Google Scholar] [CrossRef] [PubMed]
  49. Stein, K.; Fry, A.; Round, A.; Milne, R.; Brazier, J. What value health? A review of health state values used in early technology assessments for NICE. Appl. Health Econ. Health Policy 2005, 4, 219–228. [Google Scholar] [CrossRef] [PubMed]
  50. Canadian Public Health Association. Sustainability and Equity: Primary Health Care in Developing Countries; Canadian Public Health Association: Ottawa, ON, Canada, 1990. [Google Scholar]
  51. Tagliente, I.; Solvoll, T.; Murgia, F.; Bella, S. Telemonitoring in cystic fibrosis: A 4-year assessment and simulation for the next 6 years. Interact. J. Med. Res. 2016, 5, e11. [Google Scholar] [CrossRef] [PubMed]
Figure 1. Density visualization.
Figure 1. Density visualization.
Sustainability 10 04748 g001
Figure 2. Network Visualization.
Figure 2. Network Visualization.
Sustainability 10 04748 g002
Figure 3. The shift in perspective in health technology assessment.
Figure 3. The shift in perspective in health technology assessment.
Sustainability 10 04748 g003
Table 1. Keywords’ link strength in relation to “Health Technology Assessment”.
Table 1. Keywords’ link strength in relation to “Health Technology Assessment”.
KeywordsLink Strength
Decision-making7.97
Health Care Policy5.68
Cost Effectiveness Analysis4.82
Evidence4.28
Ethics4.22
Medical Technologies3.99
Health Care Systems3.81
Cost Benefit Analysis3.74
Economic Evaluation3.63
Reimbursement3.55
Procedures3.44
Innovation3.1
Economics2.84
Health Economics2.47
Pharmaeconomics2.14
Health Care Costs2.11
Public Health2.02
Drug costs1.91
Medical Decision-making1.77
Standards1.75
Table 2. Evaluation methods.
Table 2. Evaluation methods.
MethodsMeasures
Cost Effectiveness Analysis (CEA)Cost per unit change in output (e.g., cost per unit of social housing)
Cost Consequence Analysis (CCA) (or “Balance Sheet”)Listing of all major costs and outcomes in natural units
Cost Utility Analysis (CUA)Cost per unit change in Quality Adjusted Life Year (QALY)
Cost Benefit Analysis (CBA)Value all outcomes in a common unit (e.g., monetary units)

© 2018 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).
Sustainability EISSN 2071-1050 Published by MDPI AG, Basel, Switzerland RSS E-Mail Table of Contents Alert
Back to Top