Objective: Recognizing that few randomized controlled trials exist in rare disease research, our objective is to elaborate on the traditional evidence hierarchy in this context. We will conceptualize the potential value added to knowledge of treatment effectiveness from alternative study designs, while highlighting risks of bias for each design.

Methods: We used a critical review of the literature to inform the development of an evidence framework for studying effectiveness of treatments for rare diseases. Seminal papers were identified in the following areas: evidence typologies for evaluating effectiveness; risk of bias assessment for observational studies; methods and any existing frameworks for evaluating treatments for rare diseases. Our literature search was designed to be iterative, with the study team meeting frequently to refine its direction. Selected papers were qualitatively synthesized to identify challenges specific to the rare disease context; ability of specific study designs to address these challenges; and risk of bias.

Preliminary Results: From our literature search and consultation with members of the research team, we have identified six main challenges in generating evidence of treatment effectiveness for rare diseases: (1) power to detect treatment effects; (2) addressing heterogeneity in treatment effects; (3) ascertainment of relevant outcomes; (4) addressing confounding without randomization/blinding; (5) ability to determine long-term treatment effects; and (6) challenges in statistical analysis. We have also identified several alternative study designs for determining treatment effectiveness in rare diseases, including both quasi-experimental and observational designs, and have determined their ability to address the above challenges. We will use these results to further develop the framework to include guidance about when a particular study design is useful and about how to minimize bias for each.

Conclusions/Next steps: This work is on going and is expected to result in a framework that will provide insights into approaches for generating and synthesizing evidence about treatment effectiveness in the context of rare diseases.