Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2024 Oct 30;3(1):e000932.
doi: 10.1136/bmjmed-2024-000932. eCollection 2024.

Characteristics of non-randomised studies of drug treatments: cross sectional study

Affiliations

Characteristics of non-randomised studies of drug treatments: cross sectional study

Sally Yaacoub et al. BMJ Med. .

Abstract

Objective: To examine the characteristics of comparative non-randomised studies that assess the effectiveness or safety, or both, of drug treatments.

Design: Cross sectional study.

Data sources: Medline (Ovid), for reports published from 1 June 2022 to 31 August 2022.

Eligibility criteria for selecting studies: Reports of comparative non-randomised studies that assessed the effectiveness or safety, or both, of drug treatments were included. A randomly ordered sample was screened until 200 eligible reports were found. Data on general characteristics, reporting characteristics, and time point alignment were extracted, and possible related biases, with a piloted form inspired by reporting guidelines and the target trial emulation framework.

Results: Of 462 reports of non-randomised studies identified, 262 studies were excluded (32% had no comparator and 25% did not account for confounding factors). To assess time point alignment and possible related biases, three study time points were considered: eligibility, treatment assignment, and start of follow-up. Of the 200 included reports, 70% had one possible bias, related to: inclusion of prevalent users in 24%, post-treatment eligibility criteria in 32%, immortal time periods in 42%, and classification of treatment in 23%. Reporting was incomplete, and only 2% reported all six of the key elements considered: eligibility criteria (87%), description of treatment (46%), deviations in treatment (27%), causal contrast (11%), primary outcomes (90%), and confounding factors (88%). Most studies used routinely collected data (67%), but only 7% reported using validation studies of the codes or algorithms applied to select the population. Only 7% of reports mentioned registration on a trial registry and 3% had an available protocol.

Conclusions: The findings of the study suggest that although access to real world evidence could be valuable, the robustness and transparency of non-randomised studies need to be improved.

Keywords: Drug therapy; Epidemiology; Research design.

PubMed Disclaimer

Conflict of interest statement

All authors have completed the ICMJE uniform disclose form at www.icmje.org/disclosure-of-interest/ and declare: no support from any organisation for the submitted work; no financial relationships with any organisations that might have an interest in the submitted work in the previous three years; no other relationships or activities that could appear to have influenced the submitted work.

Figures

Figure 1
Figure 1. Reporting of key study elements for each report (n=200). Each horizontal line corresponds to one included report. Top right panel: a specific colour was attributed to each of the six key study elements. The colour band shows which of these items were reported for each included report. The 200 included reports of non-randomised studies were sorted according to the total number of reported items, in decreasing order. Top left panel=distribution of total number of reported items for the 200 included reports. Bottom panel=proportion of reports that reported each element
Figure 2
Figure 2. Presence of possible biases related to time point misalignment (n=200). Possible biases for each report of non-randomised studies are summarised for bias related to: inclusion of prevalent users, post-treatment eligibility criteria, immortal time periods, and classification of treatment arms. Each spoke represents one report. The bricks are a visual representation of the possible bias related to time point misalignment: possible bias, could not assess, or no bias. Every concentric circle represents one of the biases, with bias related to inclusion of prevalent users being the furthest circle from the centre, and bias in classification of treatment arms is the central circle. The most external circle represents an overview of the possible biases for each report (at least one possible bias exists, could not assess, and when no bias exists). The histogram summarises the possible biases for the 200 reports

Similar articles

References

    1. Rothwell PM. External validity of randomised controlled trials: “to whom do the results of this trial apply?”. Lancet. 2005;365:82–93. doi: 10.1016/S0140-6736(04)17670-8. - DOI - PubMed
    1. Monti S, Grosso V, Todoerti M, et al. Randomized controlled trials and real-world data: differences and similarities to untangle literature data. Rheumatology (Sunnyvale) 2018;57:vii54–8. doi: 10.1093/rheumatology/key109. - DOI - PubMed
    1. Hemkens LG, Contopoulos-Ioannidis DG, Ioannidis JPA. Routinely collected data and comparative effectiveness evidence: promises and limitations. CMAJ . 2016;188:E158–64. doi: 10.1503/cmaj.150653. - DOI - PMC - PubMed
    1. Franklin JM, Schneeweiss S. When and How Can Real World Data Analyses Substitute for Randomized Controlled Trials? Clin Pharmacol Ther. 2017;102:924–33. doi: 10.1002/cpt.857. - DOI - PubMed
    1. Dang A. Real-World Evidence: A Primer. Pharmaceut Med. 2023;37:25–36. doi: 10.1007/s40290-022-00456-6. - DOI - PMC - PubMed

LinkOut - more resources