Pragmatic Free Trial Meta
Pragmatic Free Trial Meta is a free and non-commercial open data platform and infrastructure that supports research on pragmatic trials. It shares clean trial data and ratings using PRECIS-2 permitting multiple and varied meta-epidemiological studies to evaluate the effect of treatment on trials that have different levels of pragmatism as well as other design features.
Background
Pragmatic trials provide evidence from the real world that can be used to make clinical decisions. However, the usage of the term "pragmatic" is inconsistent and its definition and assessment requires clarification. Pragmatic trials are designed to inform clinical practices and policy choices, rather than prove a physiological or clinical hypothesis. A pragmatic study should strive to be as close as is possible to actual clinical practices, including recruiting participants, setting, design, delivery and implementation of interventions, determining and analysis outcomes, and primary analyses. This is a major distinction between explanatory trials, as described by Schwartz and Lellouch1, which are designed to confirm a hypothesis in a more thorough manner.
Truly pragmatic trials should not blind participants or clinicians. This can result in an overestimation of the effect of treatment. Practical trials should also aim to recruit patients from a wide range of health care settings to ensure that the results are generalizable to the real world.
Finally the focus of pragmatic trials should be on outcomes that are crucial for patients, such as quality of life or functional recovery. This is particularly relevant for trials that involve surgical procedures that are invasive or may have serious adverse effects. The CRASH trial29 compared a two-page report with an electronic monitoring system for patients in hospitals suffering from chronic cardiac failure. The catheter trial28, on the other hand utilized symptomatic catheter-related urinary tract infections as its primary outcome.
In addition to these characteristics pragmatic trials should reduce the procedures for conducting trials and requirements for data collection to cut down on costs and time commitments. In the end these trials should strive to make their findings as relevant to real-world clinical practices as they can. This can be accomplished by ensuring that their analysis is based on the intention to treat method (as described within CONSORT extensions).
Many RCTs that do not meet the requirements for pragmatism however, they have characteristics that are contrary to pragmatism have been published in journals of various kinds and incorrectly labeled pragmatic. This can lead to false claims of pragmatism and the use of the term should be standardised. The development of the PRECIS-2 tool, which provides an objective standard for assessing pragmatic characteristics, is a good first step.
Methods
In a practical study the aim is to inform clinical or policy decisions by showing how an intervention could be integrated into routine treatment in real-world situations. Explanatory trials test hypotheses concerning the causal-effect relationship in idealized settings. In this way, pragmatic trials can have less internal validity than explanation studies and be more susceptible to biases in their design as well as analysis and conduct. Despite their limitations, pragmatic studies can be a valuable source of information for decision-making within the healthcare context.
The PRECIS-2 tool evaluates the degree of pragmatism within an RCT by assessing it on 9 domains, ranging from 1 (very explicit) to 5 (very pragmatic). In this study the areas of recruitment, organization as well as flexibility in delivery flexible adherence, and follow-up scored high. However, the main outcome and method of missing data were scored below the practical limit. This indicates that a trial can be designed with effective practical features, yet not harming the quality of the trial.
It is difficult to determine the degree of pragmatism that is present in a study because pragmatism is not a have a binary attribute. Some aspects of a research study can be more pragmatic than others. A trial's pragmatism could be affected by modifications to the protocol or the logistics during the trial. Koppenaal and colleagues discovered that 36% of the 89 pragmatic studies were placebo-controlled or conducted prior to the licensing. Most were also single-center. They are not close to the standard practice, and can only be referred to as pragmatic if the sponsors agree that these trials are not blinded.
A common aspect of pragmatic research is that researchers attempt to make their findings more relevant by studying subgroups within the trial. However, this often leads to unbalanced comparisons with a lower statistical power, thereby increasing the chance of not or misinterpreting the results of the primary outcome. In the case of the pragmatic trials included in this meta-analysis this was a serious issue because the secondary outcomes were not adjusted for differences in baseline covariates.
Furthermore, pragmatic studies may pose challenges to collection and interpretation safety data. This is due to the fact that adverse events are generally reported by the participants themselves and pragmatickr.Com are prone to reporting errors, delays, or coding variations. It is therefore crucial to improve the quality of outcomes for these trials, and ideally by using national registry databases instead of relying on participants to report adverse events in a trial's own database.
Results
Although the definition of pragmatism does not require that all trials be 100 100% pragmatic, there are some advantages of including pragmatic elements in clinical trials. These include:
Incorporating routine patients, the results of trials can be translated more quickly into clinical practice. However, pragmatic studies can also have drawbacks. For instance, the appropriate type of heterogeneity can help a trial to generalise its results to different settings and patients. However the wrong type of heterogeneity could reduce assay sensitiveness and consequently lessen the ability of a study to detect minor treatment effects.
Numerous studies have attempted to categorize pragmatic trials, with a variety of definitions and scoring systems. Schwartz and Lellouch1 created a framework for distinguishing between research studies that prove a physiological or clinical hypothesis and pragmatic trials that help in the selection of appropriate treatments in the real-world clinical setting. The framework was composed of nine domains scored on a 1-5 scale, with 1 being more lucid while 5 was more pragmatic. The domains covered recruitment, setting up, delivery of intervention, flexible adherence and primary analysis.
The initial PRECIS tool3 featured similar domains and scales from 1 to 5. Koppenaal et al10 created an adaptation to this assessment called the Pragmascope that was simpler to use in systematic reviews. They discovered that pragmatic reviews scored higher across all domains, however they scored lower in the primary analysis domain.
This difference in the analysis domain that is primary could be explained by the fact that most pragmatic trials analyze their data in an intention to treat way however some explanation trials do not. The overall score for pragmatic systematic reviews was lower when the domains of organisation, flexible delivery and follow-up were merged.
It is important to understand that the term "pragmatic trial" does not necessarily mean a poor quality trial, and there is an increasing rate of clinical trials (as defined by MEDLINE search, however this is not specific nor sensitive) which use the word 'pragmatic' in their abstract or title. These terms may signal that there is a greater appreciation of pragmatism in abstracts and titles, however it's not clear whether this is evident in content.
Conclusions
As the importance of real-world evidence becomes increasingly popular the pragmatic trial has gained traction in research. They are clinical trials that are randomized which compare real-world treatment options instead of experimental treatments in development. They have populations of patients which are more closely resembling the ones who are treated in routine medical care, they utilize comparisons that are commonplace in practice (e.g., existing medications), and they rely on participant self-report of outcomes. This approach can overcome the limitations of observational research, for example, the biases that come with the reliance on volunteers, and the limited availability and the coding differences in national registry.
Other advantages of pragmatic trials include the ability to use existing data sources, and a higher chance of detecting meaningful changes than traditional trials. However, these tests could still have limitations which undermine their reliability and generalizability. Participation rates in some trials could be lower than expected due to the health-promoting effect, financial incentives, or competition from other research studies. Practical trials are often restricted by the necessity to recruit participants in a timely manner. Practical trials aren't always equipped with controls to ensure that observed variations aren't due to biases that occur during the trial.
The authors of the Pragmatic Free Trial Meta identified RCTs published from 2022 to 2022 that self-described as pragmatism. The PRECIS-2 tool was employed to assess pragmatism. It covers areas such as eligibility criteria and flexibility in recruitment as well as adherence to interventions and follow-up. They found 14 trials scored highly pragmatic or pragmatic (i.e. scoring 5 or above) in at least one of these domains.
Trials with a high pragmatism rating tend to have higher eligibility criteria than traditional RCTs that have specific criteria that are not likely to be found in clinical practice, and they comprise patients from a wide range of hospitals. The authors suggest that these characteristics could make the pragmatic trials more relevant and relevant to everyday practice, but they do not guarantee that a pragmatic trial is completely free of bias. Moreover, the pragmatism of trials is not a fixed attribute and a pragmatic trial that doesn't possess all the characteristics of an explanatory trial may yield valid and useful results.