Jul 04, 2022

Public workspaceAnalysis of the reproducibility, transparency and quality of economic studies on pharmacological interventions in health included in clinical practice guidelines in Spain 2000-2020 V.3

DOI
dx.doi.org/10.17504/protocols.io.bp2l6b395gqe/v3
  • 1Reina Sofía University Hospital/IMIBIC/University of Cordoba
Juan Ruano
Icon indicating open access to content
QR code linking to this content
DOI: dx.doi.org/10.17504/protocols.io.bp2l6b395gqe/v3
Document Citation: Juan Ruano, Pedro J Gómez Arias, Jesús Gay-Mimbrera, Macarena Aguilar-Luque, Beatriz Isla-Tejera 2022. Analysis of the reproducibility, transparency and quality of economic studies on pharmacological interventions in health included in clinical practice guidelines in Spain 2000-2020. protocols.io https://dx.doi.org/10.17504/protocols.io.bp2l6b395gqe/v3Version created by Juan Ruano
License: This is an open access document distributed under the terms of the Creative Commons Attribution License,  which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited
Created: July 04, 2022
Last Modified: July 04, 2022
Document Integer ID: 65945
Abstract
Hypothesis
The degree of transparency and reproducibility of economic health analyses, whether primary studies or systematic reviews and meta-analysis, which are used for development could condition the usefulness of clinical practice guidelines in which they are included regardless of the methodological quality of them.

Objectives
Primary objective
Analyze the relationship between the methodological quality of clinical practice guides published in Spain and the current state of reproducibility and transparency of the health economic assessment studies included in them, both primary studies or systematic reviews and meta-analysis.

Secondary objectives
  1. Evaluate the quality of CFCs produced in Spain in the period 2000-2020.
  2. Evaluate the methodological quality of the economic assessment studies included in the GPCs produced in Spain in the period 2000-2020.
  3. Describe the transparency and reproducibility of the selected economic assessment studies.
  4. Analyse whether there are differences in transparency and reproducibility of economic assessment studies depending on the quality of the GPCs that include them.
  5. Analyse whether there have been improvements in the transparency and reproducibility of economic assessment studies by comparing the GPCs containing them that have been produced in 2000-2012 vs 2013-2020.
Analysis of the reproducibility, transparency, and quality of economic studies on pharmacological interventions in health included in clinical practice guidelines in Spain 2000-2020

Análisis de la reproducibilidad, transparencia y calidad de los estudios económicos sobre intervenciones farmacológicas en salud empleados en las guías de práctica clínica en España en el periodo 2000-2020

Juan Ruano, Francisco Jose Gomez Garcia, Pedro J. Gómez Arias, Jesus Gay-Mimbrera, Macarena Aguilar-Luque, Esmeralda Parra-Peralbo, Beatriz Isla-Tejera


Review question(s)
Are there differences in reproducibility, transparency, and methodological quality of economic studies of pharmacological interventions in health included in Spanish clinical practice guidelines published in Spain in the period 2000-2020?

Types of study to be included
  • Clinical Guidelines
  • Health Economic Studies

Condition or domain being studied
There is a growing interest in promoting transparency in scientific publications that allows research conducted to be reproducible by other research groups [1]. Economic assessments help determine the efficiency of health interventions and services, making it easier for both managers and healthcare professionals to make decisions under real clinical practice conditions [2]. To do this, it is necessary that, in addition to driving the studies according to the best methodological quality standards, both the data and methodological used and the results obtained are accessible to the scientific community, that is, communicated in the publications in a transparent and truthful way [3]. It has been observed that the quality with which this information is communicated in economic health studies in Spain is insufficient [4]. These authors consider that without the submission of comprehensive and transparent reports describing how these studies were designed and conducted, it is difficult to assess the validity of their results and conclusions. Recently, this same group has published a protocol for developing a study evaluating the degree of reproducibility and transparency of a representative sample of such economic assessments [5]. Finally, it is considered that the incorporation of economic studies in the development of clinical practice guidelines should contribute to reducing the variation in care practice, the amount of ineffective care and wasted on research and resources [6], thus improving health outcomes by encouraging the use of the most cost-efficient therapies [7]. However, such guidelines would not be appropriate, even if they followed the best standards for their development if we could not externally validate such studies, which would increase uncertainty about their results.

References
[1] Ioannidis JP, Greenland S, Hlatky MA, Khoury MJ, Macleod MR, Moher D, et al. Increasing value and reducing waste in research design, conduct, and analysis. Lancet. 2014; 383(9912):166–75.
[2] Williams I, McIver S, Moore D, Bryan S. The use of economic evaluations in NHS decision-making: a review and empirical investigation. Health Technol Assess. 2008 Apr;12(7):iii, ix-x, 1-175.
[3] Rennie D, Luft HS. Pharmacoeconomic analyses: making them transparent, making them credible. JAMA. 2000;283:2158–2160.
[4] Catalá-López F, Ridao M, Alonso-Arroyo A, et al. The quality of reporting methods and results of cost-effectiveness analyses in Spain: a methodological systematic review. Syst Rev. 2016;5:6. Published 2016 Jan 7.
[5] Catalá-López F, Caulley L, Ridao M, et al. Reproducible research practices, openness and transparency in health economic evaluations: study protocol for cross-sectional comparative analysis. BMJ Open. 2020 Feb 13;10(2):e034463.
[6]Alonso-Coello P, Irfan A, Solà I, et al. The quality of clinical practice guidelines over the last two decades: a systematic review of guideline appraisal studies. Qual Saf Health Care. 2010 Dec;19(6):e58.
[7] Wallace JF, Weingarten SR, Chiou C-F, et al. The limited incorporation of economic analyses in clinical practice guidelines. J Gen Intern Med. 2002;17:201–20.

Hypothesis
The degree of transparency and reproducibility of economic health analyses, whether primary studies or systematic reviews and meta-analyses, which are used for development could condition the use of clinical practice guidelines in which they are included regardless of the methodological quality of them.

Outcome(s)
Primary outcomes
Analyze the relationship between the methodological quality of clinical practice guides published in Spain and the current state of reproducibility and transparency of the health economic assessment studies included in them, both primary studies or systematic reviews and meta-analyses.

Secondary outcomes
  1. Evaluate the quality of CFCs produced in Spain in the period 2000-2020.
  2. Evaluate the methodological quality of the economic assessment studies included in the GPCs produced in Spain in the period 2000-2020.
  3. Describe the transparency and reproducibility of the selected economic assessment studies.
  4. Analyze whether there are differences in transparency and reproducibility of economic assessment studies depending on the quality of the GPCs that include them.
  5. Analyze whether there have been improvements in the transparency and reproducibility of economic assessment studies by comparing the GPCs containing them that have been produced in 2000-2012 vs 2013-2020.


Methods
Protocol
A larger a priori protocol will be developed and forwarded for publication in the repository https://www.protocols.io/obtaining a DOI from it.

Design
A descriptive cross-sectional observational study on data from scientific publications or research-on-research study.

Search Strategy
These literature sources will be consulted.

AB
CPG repositoriesGuíaSalud
Search limitersThe free term 'Spain' and the publication filter '2000-2020' will be used as search limiters.
ContextExcluded: historical articles, abstracts of congresses, case reports, surveys, narrative reviews, narrative reports (i.e. those with a focus on understanding a concept), consensus documents, or meta-analysis performed without a systematic literature search.

Eligibility
All GPCs developed in Spain from 2000 to 2020 containing at least one reference on the analysis of pharmacological interventions, regardless of age, gender, ethnicity/race,meta-analyses or pathology of the target population, will be included. For uniformity in the analysis, no guides limited to other approaches (preventive, diagnostic) will be included.

Data extraction, (selection and coding)
Two researchers will carry out all assessments and extraction of duplicate data and in case of discrepancies the criterion of a third investigator will be used.

AB
Clinical Guidelines Appraisal of Guidelines, Research and Evaluation (AGREE) II tool.
Systematic reviews and meta-analysis including economic analysisA MeaSurement Tool to Assess systematic Reviews 2 (AMSTAR 2)
Primary Health Economic StudiesConsolidated Health Economic Evaluation Reporting Standards (CHEERS)
Tools for methodological quality assessment.

Transparency and reproducibility assessment
Transparency and reproducibility will be assessed by analyzing the degree of adequacy of each economic study with respect to the following indicators:
  1. Citation, mention, or use of the CHEERS statement;
  2. Open publication;
  3. Mention of an a priori protocol;
  4. Mention of an economic analysis plan;
  5. Mention the availability of the original data and the mode of access to it;
  6. Type of data access repository used;
  7. The data provided make it possible to reproduce the base case and/or in the analysisSystematicMeasurement, of heterogeneity and/or uncertainty;
  8. Whether the results are novel, an update, or are an attempt to replicate other studies;
  9. If they have been replicated by other subsequent studies that cite them;
  10. Sources of funding (academic, industry, others);
  11. Conflict of economic interest of the authors.

Conflicts of interest
Reviewers have no disclosures

Language
Spanish or English

Country
Spain

Organizational affiliation of the meta-research study
Instituto Maimonides Investigación Biomédica de Córdoba IMIBIC
http://www.imibic.org/

Review team
Dr Juan Ruano, Department of Dermatology, IMIBIC/Reina Sofia University Hospital/University of Cordoba, Cordoba, Spain
Dr Francisco Jose Gomez Garcia, Department of Dermatology, IMIBIC/Reina Sofia University Hospital/University of Cordoba, Cordoba, Spain
Mr Pedro J. Gómez Arias, Department of Dermatology, IMIBIC/Reina Sofia University Hospital/University of Cordoba, Cordoba, Spain
Mr Jesus Gay-Mimbrera, IMIBIC/Reina Sofia University Hospital/University of Cordoba, Cordoba, Spain
Miss Macarena Aguilar-Luque, IMIBIC/Reina Sofia University Hospital/University of Cordoba, Cordoba, Spain
MS Esmeralda Parra-Peralbo, Facultad de Ciencias Biomédicas y Salud, Universidad Europea de Madrid, Madrid, Spain
Dr Beatriz Isla-Tejera, Department of Pharmacy, IMIBIC/Reina Sofia University Hospital/University of Cordoba, Cordoba, Spain

Anticipated or actual start date
15 March 2021

Anticipated completion date
5th July 2022

Funding sources/sponsors
This protocol is Juan Ruano's final project for the Master in Health Economics and Pharmacoeconomics degree at UPF Barcelona School of Management. There is no funding from any pharmaceutical company.

Contact details for further information
Dr Juan Ruano
Edificio IMIBIC, Hospital Universitario Reina Sofía, Av. Menendez Pidal, s/n, 14004 Córdoba.
juanruanoruiz@mac.com

Data analysis
The R programming language shall be used for statistical analyses and graphical representations.
The descriptive data will be presented tabulated.

Presentation of the results
A diagram will be developed and the reporting checklist Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) will be followed.

Reproducibility
Both the R scripts and the primary data used in this job will be made public on the GitHub platform: https://github.com/info4cure/TFM_UPF_JR