Imagine this scenario: A study is performed to find out how effective antibiotics are for ear infections. However the antibiotic chosen for use in the study is one that we know is totally inappropriate for treating ear infections. Therefore the researchers conclude that because the infection wasn’t cleared, antibiotics aren’t an effective treatment for ear infections. Of course we can understand this is quite bogus, but if you aren’t familiar with what antibiotics are generally used for ear infections, how would you know that the study should be ignored completely?
Unfortunately, we do at times come across these types of issues in acupuncture research. Although quality standards have been developed by researchers in order to improve reporting of acupuncture trials (for example, did the authors report on the possible types of bias? Did they outline in detail what was done so it may be repeated in the future?), until recently there wasn’t a way to assess the clinical relevance of the acupuncture or other East Asian Medicine technique utilized. Thankfully a quality rating scale called NICMAN has been developed to address this issue.
The NICMAN scale was initially developed in 2011 and has been gradually refined. If the paper you are reading does not assess the quality of the acupuncture intervention using NICMAN, it’s a good idea as an acupuncturist to apply the scale to the study yourself! If you’re not an acupuncturist, ask one what they think of the points used and style of acupuncture. The other way to make sure a research paper has followed accepted and recommended reporting standards is to check on whether they have followed STRICTA or at a minimum CONSORT guidelines. STRICTA is a set of guidelines specifically for acupuncture research, and is an official extension of CONSORT, developed in collaboration with the CONSORT group. CONSORT is a set of general standards for reporting of clinical trials.
For more detailed information and links on these three different sets of guidelines, see below:
STRICTA– STandards for Reporting Interventions in Clinical Trials of Acupuncture. This is an official extension of CONSORT, developed in collaboration with the CONSORT group. First published in 2001. Click the link for the official checklist.
CONSORT– CONsolidated Standards Of Reporting Trials. Industry-wide accepted general standards for reporting of randomized controlled trials.
NICMAN– National Institute for Complementary Medicine Acupuncture Network. This is a network of acupuncture researchers in Australia who developed the standards, initially developed 2011. It is intended to be used in conjunction with STRICTA, and evaluates the quality of the acupuncture used in an acupuncture randomized controlled trial or systematic review.
Important Writings on The Various Guidelines:
This paper, published in 2017, looked at 207 acupuncture trials between 1994-2015. The researchers found an improvement in reporting of STRICTA and CONSORT items over this time period. “Journals that endorse STRICTA have a better record in terms of reporting quality. Some evidence suggests that the publication of STRICTA has had a positive impact on reporting quality.”
This paper, published in 2010, describes the process of revising the STRICTA guidelines (first developed in 2001). It describes the collaborative effort of the CONSORT group, STRICTA group as well as the Chinese Cochrane Center and the Chinese Center for Evidence-based Medicine. Each item in the revised STRICTA guidelines is described in detail.
This paper describes why NICMAN was needed in addition to STRICTA: “There are now clear standards for publications to report acupuncture parameters used in clinical studies, and the time is right to address the related but separate issue of evaluating the quality of acupuncture performed during these studies. Ensuring adequate reporting of a study may contribute to improving its reliability but does not address the quality of intervention applied (i.e., validity, standards, or adequacy).”