Pragmatic Free Trial Meta
Pragmatic Free Trial Meta is a non-commercial, open data platform and infrastructure that supports research on pragmatic trials. It shares clean trial data and ratings using PRECIS-2, permitting multiple and varied meta-epidemiological research studies to examine the effects of treatment across trials with different levels of pragmatism as well as other design features.
Background
Pragmatic trials provide evidence from the real world that can be used to make clinical decisions. The term "pragmatic" however, is not used in a consistent manner and its definition and measurement need further clarification. Pragmatic trials must be designed to inform policy and clinical practice decisions, rather than confirm a physiological or clinical hypothesis. A pragmatic trial should also try to be as similar to the real-world clinical environment as is possible, including the recruitment of participants, setting up and design as well as the execution of the intervention, determination and analysis of outcomes as well as primary analysis. This is a major distinction between explanatory trials, as defined by Schwartz & Lellouch1 which are designed to confirm a hypothesis in a more thorough manner.
Truely pragmatic trials should not blind participants or the clinicians. This could lead to an overestimation of treatment effects. The trials that are pragmatic should also try to enroll patients from a wide range of health care settings, to ensure that their findings can be applied to the real world.
Additionally the focus of pragmatic trials should be on outcomes that are crucial to patients, such as quality of life or functional recovery. This is especially important for trials involving invasive procedures or those with potential dangerous adverse events. The CRASH trial29, for example was focused on functional outcomes to compare a two-page report with an electronic system to monitor the health of patients admitted to hospitals with chronic heart failure. Similarly, the catheter trial28 focused on urinary tract infections that are symptomatic of catheters as the primary outcome.
In addition to these aspects, pragmatic trials should minimize the trial procedures and data collection requirements in order to reduce costs. Furthermore, pragmatic trials should seek to make their results as applicable to real-world clinical practice as possible by ensuring that their primary analysis is based on the intention-to-treat method (as described in CONSORT extensions for pragmatic trials).
Despite these criteria however, a large number of RCTs with features that challenge the notion of pragmatism were incorrectly labeled pragmatic and published in journals of all kinds. you could check here can lead to false claims of pragmatism, and the term's use should be standardised. The development of the PRECIS-2 tool, which offers an objective standard for assessing practical features, is a good first step.
Methods
In a pragmatic study the goal is to inform clinical or policy decisions by showing how an intervention could be integrated into everyday routine care. This differs from explanation trials that test hypotheses about the cause-effect connection in idealized situations. Consequently, pragmatic trials may be less reliable than explanatory trials and might be more susceptible to bias in their design, conduct, and analysis. Despite their limitations, pragmatic research can provide valuable information for decision-making within the healthcare context.
The PRECIS-2 tool measures the degree of pragmatism within an RCT by assessing it on 9 domains, ranging from 1 (very explanatory) to 5 (very pragmatic). In this study, the recruit-ment, organization, flexibility in delivery and follow-up domains were awarded high scores, however, the primary outcome and the method of missing data were not at the practical limit. This indicates that a trial can be designed with effective practical features, yet not harming the quality of the trial.
However, it is difficult to judge how pragmatic a particular trial is, since the pragmatism score is not a binary characteristic; certain aspects of a study can be more pragmatic than others. Furthermore, logistical or protocol modifications made during the trial may alter its pragmatism score. Additionally, 36% of the 89 pragmatic trials discovered by Koppenaal et al were placebo-controlled, or conducted prior to approval and a majority of them were single-center. They are not close to the usual practice and can only be referred to as pragmatic if their sponsors agree that the trials are not blinded.
A common feature of pragmatic studies is that researchers try to make their findings more meaningful by analyzing subgroups of the trial sample. This can result in imbalanced analyses and lower statistical power. This increases the risk of missing or misdetecting differences in the primary outcomes. This was the case in the meta-analysis of pragmatic trials as secondary outcomes were not adjusted for differences in covariates at the time of baseline.
In addition, pragmatic studies may pose challenges to collection and interpretation of safety data. This is because adverse events are generally reported by the participants themselves and are prone to reporting delays, inaccuracies or coding errors. It is therefore important to enhance the quality of outcomes assessment in these trials, in particular by using national registry databases instead of relying on participants to report adverse events in a trial's own database.
Results
Although the definition of pragmatism does not require that all trials be 100 100% pragmatic, there are benefits to incorporating pragmatic components into clinical trials. These include:
Increased sensitivity to real-world issues, reducing the size of studies and their costs, and enabling the trial results to be more quickly transferred into real-world clinical practice (by including routine patients). However, pragmatic studies can also have drawbacks. For example, the right type of heterogeneity can help a study to generalize its results to many different patients and settings; however the wrong type of heterogeneity could reduce assay sensitiveness and consequently decrease the ability of a study to detect small treatment effects.
A variety of studies have attempted to classify pragmatic trials using different definitions and scoring methods. Schwartz and Lellouch1 created an approach to distinguish between research studies that prove a physiological or clinical hypothesis, and pragmatic trials that inform the selection of appropriate treatments in the real-world clinical setting. Their framework included nine domains that were scored on a scale of 1 to 5 with 1 indicating more lucid and 5 indicating more practical. The domains were recruitment and setting, delivery of intervention and follow-up, as well as flexible adherence and primary analysis.
The original PRECIS tool3 included similar domains and a scale of 1 to 5. Koppenaal et al10 created an adaptation to this assessment called the Pragmascope that was easier to use in systematic reviews. They found that pragmatic systematic reviews had higher average score in most domains, with lower scores in the primary analysis domain.
This distinction in the primary analysis domain can be explained by the way that most pragmatic trials analyse data. Some explanatory trials, however don't. The overall score was lower for systematic reviews that were pragmatic when the domains of organisation, flexible delivery and follow-up were merged.
It is important to remember that a study that is pragmatic does not necessarily mean a low-quality study. In fact, there is increasing numbers of clinical trials that employ the term "pragmatic" either in their title or abstract (as defined by MEDLINE but which is neither precise nor sensitive). The use of these words in abstracts and titles could indicate a greater understanding of the importance of pragmatism, however, it is not clear if this is reflected in the contents of the articles.
Conclusions
In recent years, pragmatic trials have been increasing in popularity in research because the importance of real-world evidence is increasingly recognized. They are randomized trials that compare real world care alternatives to clinical trials in development. They are conducted with populations of patients that are more similar to those who receive treatment in regular medical care. This approach can overcome the limitations of observational research, for example, the biases associated with the use of volunteers as well as the insufficient availability and codes that vary in national registers.
Pragmatic trials also have advantages, such as the ability to leverage existing data sources and a greater probability of detecting meaningful distinctions from traditional trials. However, pragmatic trials may have some limitations that limit their credibility and generalizability. The participation rates in certain trials may be lower than expected because of the healthy-volunteering effect, financial incentives or competition from other research studies. A lot of pragmatic trials are restricted by the need to enroll participants in a timely manner. Certain pragmatic trials lack controls to ensure that observed differences aren't caused by biases during the trial.
The authors of the Pragmatic Free Trial Meta identified RCTs published from 2022 to 2022 that self-described as pragmatic. They evaluated pragmatism using the PRECIS-2 tool that includes the domains eligibility criteria as well as recruitment, flexibility in adherence to interventions and follow-up. They discovered that 14 of these trials scored as highly or pragmatic sensible (i.e. scoring 5 or higher) in one or more of these domains and that the majority of them were single-center.
Trials with a high pragmatism rating tend to have broader eligibility criteria than traditional RCTs that have specific criteria that aren't likely to be found in the clinical environment, and they include populations from a wide range of hospitals. The authors argue that these traits can make the pragmatic trials more relevant and useful for daily practice, but they do not necessarily guarantee that a trial using a pragmatic approach is free from bias. In addition, the pragmatism that is present in the trial is not a predetermined characteristic and a pragmatic trial that doesn't have all the characteristics of an explanatory trial may yield reliable and relevant results.