Evaluation Criteria
The benefit of each intervention for various outcomes is given a rating to describe the quality and consistency of the scientific evidence for that specific area of research.
Example: “Is acupuncture effective for post-stroke pain relief”- you will find a level of evidence indicating whether it is effective or not effective and how strong the evidence is. While researchers have proposed slightly different scaling methods for producing levels of evidence, the premise behind each is primarily the same: interventions are given a higher grade (towards 1a) as the quality of the research and consistency of the findings increases.
Here is a description of the levels of evidence:
Note: The rating does not indicate that an intervention is effective, it only indicates the strength of the evidence in the scientific literature – so you may see a 1a indicating that two or more randomized clinical trials (RCTs) have studied the question – and found the treatment in question to be not effective. To help with the clarification of this point throughout StrokEngine – in all of the topic areas we provide both a
level of evidence1a (Strong) : Well-designed meta-analysis, or 2 or more high quality RCTs (PEDro ≥ 6) showing similar findings 1b(Moderate): 1 RCT of high quality (PEDro ≥ 6) 2a (Limited): At least 1 fair quality RCT (PEDro = 4-5) 2b (Limited): At least one poor quality RCT (PEDro < 4) or well-designed non-experimental study (non-randomized controlled trial, quasi-experimental studies, cohort studies with multiple baselines, single subject series with multiple baselines, etc.) 3 (Consensus): Agreement by an expert panel or a group of professionals in the field or a number of pre-post studies all with similar results 4 (Conflicting): Conflicting evidence of 2 or more equally well-designed studies 5 (No evidence): No well-designed studies - only case studies/case descriptions or cohort studies/single subject series with no multiple baselines) and an indication of effective/not-effective.
| 1a (Strong) | Well designed Meta-AnalysisMethod in which the results of two or more studies are statistically combined. Typically used when studies have few subjects, but similar designs. By increasing the available number of subjects, more weight can be given to the findings.
, or 2 or more high quality RCT’s (PEDro ≥ 6) showing similar findings |
| 1b (Moderate) | 1 RCTA randomized controlled trial (RCT) is an experimental design in which subjects are randomly assigned to a treatment group, or to a control (no treatment or alternative treatment) group. Effects of the experimental treatment are then compared statistically to results of the control treatment to determine effectiveness. of high quality (PEDro ≥ 6) |
| 2a (Limited) | At least 1 fair quality RCTA randomized controlled trial (RCT) is an experimental design in which subjects are randomly assigned to a treatment group, or to a control (no treatment or alternative treatment) group. Effects of the experimental treatment are then compared statistically to results of the control treatment to determine effectiveness. (PEDro = 4-5) |
| 2b (Limited) | At least one poor quality RCTA randomized controlled trial (RCT) is an experimental design in which subjects are randomly assigned to a treatment group, or to a control (no treatment or alternative treatment) group. Effects of the experimental treatment are then compared statistically to results of the control treatment to determine effectiveness. (PEDro < 4) or well-designed non-experimental study (non-randomized controlled trial, quasi-experimental studies, cohort studies with multiple baselines, single subject series with multiple baselines, etc.) |
| 3 (Consensus) | Agreement by an expert panel or a group of professionals in the field or a number of pre-post studies all with similar results |
| 4 (Conflict) | Conflicting evidence of 2 or more equally well designed studies
|
| 5 (Insufficient evidence) | No well-designed studies – only case studies/case descriptions, or cohort studies/single subject series with no multiple baselines) |
Further description of the levels
Level 1a
Two randomized controlled trials (RCTA randomized controlled trial (RCT) is an experimental design in which subjects are randomly assigned to a treatment group, or to a control (no treatment or alternative treatment) group. Effects of the experimental treatment are then compared statistically to results of the control treatment to determine effectiveness.) of high quality (PEDro ≥ 6) or a meta-analysis that finds acupuncture to be effective (or not effective) in reducing pain would enable a 1a rating of acupuncture for pain relief.
Level 1b
One randomized controlled trial (RCTA randomized controlled trial (RCT) is an experimental design in which subjects are randomly assigned to a treatment group, or to a control (no treatment or alternative treatment) group. Effects of the experimental treatment are then compared statistically to results of the control treatment to determine effectiveness.) of high quality (PEDro ≥ 6) that finds acupuncture to be effective (or not effective) in reducing pain would enable a 1b rating of acupuncture for pain relief.
Level 2a and 2b
One or more fair quality RCTs (PEDro = 4-5) that found acupuncture to be effective (or not effective) in reducing pain would enable a 2a rating, while lower quality studies such as poor quality RCT’s, non-randomized trials and strong single subject designs (for example those with multiple baselines) would receive a level of evidence rating of 2b.
Level 3
A consensus by an expert panel or a group of professionals in the field or a number of pre-post studies all with similar results that acupuncture was effective or not effective would receive a level of evidence rating of 3.
Level 4
A level of evidence of 4 indicates conflicting findings of equally well-designed studies, for example, one high quality study finding acupuncture to be effective for pain reduction, the other high quality study finding it not effective.
Level 5
Finally, a level of evidence of 5 indicates that there are no well-designed experimental studies for the intervention.
Note: There are times when a synthesis of the research on an intervention has been completed by a well-respected research team whose findings conflict with another synthesis, again by a well-respected team. We provide you with these conflicting reports and make an attempt to explain why the conflict exists.
