Editor’s note: A previous version of this article was published as a chapter by these authors in The H.E.R.B.A.L. Guide: Dietary Supplement Resources for the Clinician by Robert A. Bonakdar (ed). Philadelphia, PA: Walters Kluwer/Lippincott Williams & Wilkins, 2010 (Chapter 30; pp. 175-184). The chapter is based on a presentation to health professionals by the primary author (Betz) at the annual Evidence-Based Dietary Supplements Conference sponsored by Scripps Center for Integrative Medicine in La Jolla, CA, in 2009. It has been significantly revised, expanded, and peer reviewed for publication in HerbalGram.

INTRODUCTION

Clinical trials are conducted for several reasons, including determination of efficacy. In many areas, healthcare providers seeking information will find little or no guidance, because of a lack of published human studies in the relevant area. In other cases, publications may exist but conclusions from apparently similar studies may be contradictory, making it difficult to provide informed recommendations to patients. Although many areas of medicine share these problems, the literature of complementary and integrative medicine is relatively young and may seem especially confusing. In addition, the peer-reviewed literature is not the only available information resource, and numerous books and the Internet provide access to an overwhelming volume of information. The challenge for the practitioner is to separate fact from opinion and to make sense of areas where there are no facts, where the facts are contradictory, or where opinion poses as fact.

Methodologists and meta-analysts have identified a number of criteria for evaluation of bodies of literature and for drawing conclusions from that literature. Over the past decade, several authors have published rating scales for evaluating the quality of published clinical trials, with an eye to assessing the reliability of the conclusions drawn from those trials so that evidence-based standards of medical care may be developed. These systems include the Jadad scale¹ and the Consolidated Standards of Reporting Trials (CONSORT) Statement.² Although the raters agree that the most reliable evidence is provided by randomized, placebo-controlled, double-blind clinical trials, variations among trial designs and their descriptions still allow for nominally similar designs to vary in quality. Elements of these systems allow readers to rate the strengths of individual trial designs based on variables such as sample size, intent-to-treat (ITT) analysis, degree and adequacy of blinding (single-blind, double-blind), and randomization scheme. Linde et al³ investigated trial design in the fields of homeopathy, herbal medicines, and acupuncture. They found the quality of the trials to be highly variable, with important shortcomings in reporting and/or methodology in the majority. Major problems in most trials included the description of blinding and the reporting of dropouts and withdrawals. Unfortunately, the trial rating systems were designed with a narrow focus on the mechanics of the clinical trial itself and took for granted that the investigators who conducted the study understood the composition and characteristics of the product or formulation used in the trials and that the material had been well characterized.

Three publications highlighted the importance of product quality determination and the necessity of defining and describing the nature of interventions to be used in biomedical research. In 2005, Gagnier et al recognized that in the field of biomedical research on natural products, adequate characterization and description of the intervention were not always provided in trial publications and proposed a revised CONSORT statement that includes scoring for the quality of the description of the intervention.^4,5* As a result of the growing realization of the nature and extent of the problem, Wolsko et al performed a systematic evaluation of the literature of herbal clinical trials with the intent of quantifying the reliability of the clinical botanical literature in terms of product quality.⁶ A review of 81 studies found that only 12 of the 81 (15%) reported having performed any kind of quantitative chemical analysis on the intervention. Of these, only eight studies reported results of the analyses. The authors also reported that only 40 of the 81 botanical studies (49%) identified the plant source of the intervention by Latin binomial. Only eight (10%) of the studies identified the plant part used in the study, and only 23 (28%) of the studies described the extraction/processing method used to prepare the study intervention. In 2006, Gagnier et al published an extensive review of 206 randomized controlled trials of herbal preparations and scored them against 42 separate CONSORT checklist items.⁷ A detailed description of the results is not within the scope of this discussion, but the finding that only 54% of the reviewed trials provided a precise description of the intervention confirmed the conclusions presented in the Wolsko report.⁶ Investigators have thus far struggled to provide more complete descriptions of interventions, as they have discovered that there are few reliable published analytical methods and reference materials available for evaluating product composition.

READING THE LITERATURE

Editors and peer reviewers who are experts in clinical trials are aware of the Jadad scale and CONSORT checklist. However, few have been trained in pharmacognosy, botany, or phytochemistry. Although there is no such thing as a perfect trial, studies that are deeply flawed in the clinical area seldom successfully complete the peer-review process, at least in some of the more reliable medical, pharmacy, and nutrition journals. As pointed out by Wolsko et al and Gagnier et al, many studies that are weak in nonclinical areas can and do reach the peer-reviewed literature.^6,7

As suggested by Swanson and mandated by a policy set forth by the National Institutes of Health’s National Center for Complementary and Alternative Medicine (NCCAM), all studies on natural products should share some common characteristics.^8,9 First, the researcher should provide a rationale for selecting a particular product or ingredient and there should be a detailed description of the nature of the test article.¹⁰ If the study is about a botanical intervention, the plant should be named correctly using the currently accepted Latin binomial. Correct Latin names are actually a phrase that consists of the name of the plant genus and species followed by the specific epithet and the designation of the botanist who is credited with naming the plant. The whole phrase identifies a particular plant species, for example, Echinacea angustifolia DC. A useful and easy-to-use resource for checking to see whether or not a plant has been named correctly according to the most recent taxonomic conventions can be found on the World Economic Plants page of the United States Department of Agriculture’s Germplasm Resources Information Network website (www.ars-grin.gov/cgi-bin/npgs/html/taxecon.pl).^11† Another comprehensive and reliable taxonomic resource is The Plant List (www.plantlist.org), a web-based list of plant names produced by the Royal Botanical Gardens at Kew (England), the Missouri Botanical Garden, and several other leading botanical organizations.¹² If the study has been performed on a material that is not a commercial product, it is usually helpful (and for some journals, mandatory) to include information about when and where the plant material was collected, by whom, and where a retained portion of the plant material is stored for future reference.¹³ If a reader cannot determine the plant used in the study, then neither will a future investigator trying to repeat it.

In addition to taxonomic information, the Materials and Methods section of the trial publication should indicate the part of the plant that was used to make the clinically tested material. The phytochemistry of different plant parts can vary dramatically, and if the plant has been used in a traditional healing system, the part of the plant to be used for a particular indication is usually specified. For example, the German Commission E provides positive (i.e., approved) monographs for E. pallida (Nutt.) Nutt. root and Echinacea purpurea (L.) Moench herb, whereas the E. angustifolia herb and root/E. pallida herb monograph and the E. purpurea root monograph are negative (unapproved), due to lack of adequate clinical data available at the time (August, 1992).¹⁴

Another important detail is the way in which the test article was processed before incorporating it into a dosage form. Over the past five decades or so, there has been an evolution in the production, sale, and use of botanical products. In the 1970s, these were largely sold as sifted, cut, or powdered plant material in the form of a tablet, capsule, tea bag, or tincture. Many modern botanical products are made using sophisticated plant extracts that have been spray-dried onto a solid carrier or lyophilized (freeze-dried) into a dried powdered extract and then formed into a hard or soft capsule or tablet. Traditional extracts were usually aqueous or (later) ethanolic (ethyl alcohol), but modern products are frequently extracted with a wide range of solvents, from acetone to isopropanol to supercritical carbon dioxide. The nature of the solvent can change the final chemistry of the extract product in ways that could affect safety and efficacy as compared to traditional preparations. There are a number of advantages to these new techniques, including savings in shipping costs as a result of reduced bulk, the ability to produce dosage forms that are more uniform in their composition, and the ability to concentrate preferentially the salubrious constituents of a plant while leaving behind undesirable constituents. Unfortunately, the ability to affirm botanical identity and quality using simple and inexpensive techniques such as microscopy or chemical spot-testing is lost when ingredients are processed and traded in this manner.¹⁵ In reading the herbal literature, it is important for the reader to note whether or not the investigators indicate the extraction solvent used in the production of the material, the solvent ratio (amount of herb/amount of solvent), and the carrier onto which the extract has been spray-dried, along with the identities and amounts of other ingredients in the finished product.

This information is important not only in evaluating the trial but also for ethical reasons. The presence of peanut oil, wheat gluten, soy, sulfites, lactose, or milk protein as “inert” components of a formulation is potentially dangerous to certain segments of the population.

Along the same lines, some commercial herbal preparations have been found over the years to be adulterated with prescription pharmaceuticals.¹⁶ Not all herbs are subject to such intentional adulteration, but herbs intended for certain classes of claim (weight loss, male sexual enhancement, athletic performance enhancement, etc.) have been targets in the past. This problem is international in scope and cuts across all regulatory regimes, from dietary supplements to Natural Health Products (a Canadian regulatory category), to botanically based prescription drugs.

Investigators must be aware (and indicate that they are aware) of such possibilities and should, if necessary, identify steps taken to ensure that the clinical material is “clean.” On the topic of cleanliness, investigators should know that natural products are susceptible to contamination with pesticides and fungicides, mycotoxins, toxic elements (e.g., heavy metals), and dangerous microorganisms. Details of testing protocols (including sampling plans) used and results of these determinations should be presented in studies. In some clinical trials, investigators acquire raw ingredients from a supplier and contract for the manufacture of custom dosage forms to provide a better match with placebo or to deliver a nonstandard dose. It is even more important to have the raw materials used in such products evaluated for purity.

Finally, the investigator should indicate in the publication whether or not the material used in the trial was standardized, and if so, to what? Standardization is a technical term of art that describes a process by which levels of certain phytochemical constituents are not only measured but also adjusted so that phytochemical content is consistent from batch to batch and falls within a certain predefined range.^17,18 Included in the information about standardization should be details about the analytical method(s) used to verify that the phytochemical contents are appropriate and that the method used to make the measurements is scientifically valid and fit for its purpose. In addition, analytical data generated during product evaluation and standardization should be provided along with the source and purity of chemical calibrants used in the analysis. It also would be useful to include information on the number of lots or batches of product used in the study, and information about whether or not phytochemical content was monitored throughout the study (some phytochemicals are unstable and some products have relatively short shelf lives). If commercial products are used in trials, the same information is required. For the most part, such products must meet identity and purity standards mandated by regulators in the country of origin. However, regulatory requirements vary across nations, and it would be prudent for investigators to indicate that they have confirmed product cleanliness along with the other specifications.

Authors confronted by editors or journals that object to extensive descriptions of interventions and placebos should be prepared to educate editors and reviewers about the necessity of including the information. In many cases, all that needs to be said can be said in three sentences. In addition, most journals have the ability to make supplementary data available online. In a worst case, the author can insist that sufficient details be included and if the journal still balks, she or he can publish elsewhere. Finally, some investigators have come to realize that characterization can be published separately in greater detail in phytochemistry journals and cited in the clinical trial publication.

Additional information on the clinical trial design is usually necessary in herbal trials. For instance, justifications for the dose and dosage regimen used in the trial should be provided. Few dose-ranging studies have been performed on herbal products. Instead, investigators often have relied on label directions provided by the manufacturer or information from “traditional use,” where it exists. Manufacturers may or may not have solid reasons for recommending particular doses, and when traditional use information does exist, it may be difficult to establish comparability of traditional and modern dosage forms. If a justification is made on the basis of traditional use, the traditional use should be noted. In traditional systems of healing, herbs often were used for a multitude of indications, with different doses and modes of preparation.

Finally, adequacy of blinding and the selection of placebo must be evaluated. Herbs (including herbal extracts) generally have characteristic odors and tastes, and so-called “inert” botanicals selected as placebo may not really be biologically inert. Investigators should explain and justify their reasons for choosing a particular placebo.

CASE STUDY: ECHINACEA

Background

Echinacea products and trials are confusing because the marketplace contains multiple species, multiple plant parts (root, aerial parts), and numerous preparations (tinctures, pressed fresh aerial parts, dried herb, and mixtures thereof). The chemistry and means of standardization of all of these forms also vary considerably, so this genus provides a good case study for evaluating herbal trials in general.¹⁹

According to both Tyler and Brinker, “echinacea” was introduced to Western medicine by H.C.F. Meyer, who had learned about it from Native Americans around 1871.^20,21 The plant was commercialized by the Lloyd Brothers of Cincinnati in approximately 1885, and it remained one of the most popular medicines in the United States until the 1930s, when the advent of the sulfa drugs pushed it to the side. The “Echinacea” monograph in the 20th edition of the United States Dispensatory (USD) describes the properties and uses of Brauneria pallida (Nuttall) Britton (Sampson root, Pale Purple Coneflower) root and rhizome.²² The name E. angustifolia De Candolle is listed in the USD as a synonym, and this is the current accepted name for the species. Echinacea pallida (Nutt.) Nutt. is now considered a separate species.¹¹ This change in taxonomy was reflected in the 25th edition of the USD.²³ According to the USD monographs, echinacea was used to increase bodily resistance to infection and for boils, erysipelas, septicemia, cancer, syphilis, “other impurities of the blood,” and as an aphrodisiac. More or less as an afterthought, the USD monograph describes the root and rhizomes of B. purpurea DC Britton (Rudbeckia purpurea L., E. purpurea Moench) as having similar properties and uses. Brinker recently provided a comprehensive review of the various uses of medicinal preparations made from roots in the genus Echinacea (primarily E. angustifolia) by Eclectic physicians of the latter half of the 19th and early 20th centuries.²¹ Both the 20th (1918) and 25th (1955) editions of the USD provide a description of the chemical composition of echinacea, but the descriptions reflect the state of knowledge of organic chemistry at the time. Early attempts to discover a “physiologically active” substance were described as failures.

The reader will note several points in the above discussion. The first is that the modern use of echinacea for the prevention and treatment of upper respiratory tract infections can be easily recognized in the phrase “Echinacea is used to increase bodily resistance to infection.” The second is that botanical names are not static. What is now called E. angustifolia had a different official name in 1918 (B. pallida). What is more, the two names were used interchangeably at the time. Botanists subsequently determined that the two names (one of which was later changed to E. pallida) described two different plant species, and the 1955 USD reflects this change.²³ In conducting a literature search on a botanical, it can be useful to know the taxonomic history of the species. A third point is that the monograph identified the plant part used (root and rhizome). The monograph also provided the dosage form and dose. The dosage form is described as powdered root/rhizome or a fluid extract, and the daily dose was one gram or 0.6 to 1.8 ml, respectively.^‡

When reading the description of a clinical trial in the literature, therefore, one can start by asking whether the author of the study got the name of the plant correct. The next questions asked should be whether the author adequately described the reason(s) for selecting a particular plant part, dosage form, and dose. It is not really important that the reader know the literature, although it would be beneficial. However, it would be inexcusable if the author of the paper was unfamiliar with the literature, and this often can be determined by the astute reader if there is no rationale for dose/dosage form presented or if the rationale is inconsistent with available authoritative sources of information. For example, a study that utilizes a supercritical fluid CO₂ extract of the aerial parts of E. purpurea is clearly not derived from the healing traditions of Native Americans or 19th century Eclectic physicians, and justification for the dose/dosage form should in that case come from formal dose-ranging studies or have some other scientific basis. Many of the more recent studies on echinacea use products, doses, and dosage forms that arose from relatively recent European practice. The use of freshly pressed juice of the above-ground parts of E. purpurea is a notable example. A trial on such a material is perfectly legitimate as long as the investigator recognizes and acknowledges that the pertinent literature includes the German Commission E monographs, the German or European Pharmacopeia, and previous clinical trials conducted using specific formulations and doses. When publishing their study results, clinical investigators who use commercial products that justify dosing on product label directions should demonstrate that there has been a dialogue with the manufacturer, such that the manufacturer provides a rationale for the dose and dosage form.

A review of the peer-reviewed clinical literature surrounding echinacea shows that many of the earlier published trials were “positive.” In fact, a systematic review of 16 trials (3,396 participants) designed to evaluate echinacea for the prevention and treatment of the common cold published by Melchart et al in 2000 concluded that some preparations of echinacea may be better than placebo.²⁴ However, the authors also reported that variation in preparations studied and in methodological quality precluded quantitative meta-analysis of the studies and further stated that, although the majority of studies reported positive results, there was insufficient evidence to recommend a specific echinacea product or preparation. A subsequent negative trial by Barrett et al²⁵ (see Table 1 for brand information) seemed to contradict the conclusion about efficacy drawn by Melchart et al,²⁴ but a number of letters to the editor in response to the Barrett report as well as a commentary by Turner²⁶ pointed out the obstacles to designing definitive studies and performing systematic reviews for echinacea. Among the difficulties noted were comparability of “1 g root/day” to 300 mg of an extract administered three times daily; timing of the intervention (continuous for prevention vs. at “first onset of symptoms,” but before development of two identifiable symptoms); method of assessment (self-assessment, viral titer, parent-kept log for studies on children)^27,28; adequacy of blinding (herbs have characteristic odors and tastes)³; and nature of the test article (whether the product is standardized or not, nature of the phytochemical marker compounds). As noted by Wolsko et al,⁶ some authors fail to distinguish among species, and there are three commercially important species of echinacea in the marketplace (E. purpurea, E. angustifolia, and E. pallida).^6,29

Since 2010, results of four additional echinacea trials have been published. Studies published by Barrett et al in 2010 and 2011 were extremely well designed with excellent descriptions of the interventions.^30,31 The 2010 study was a four-arm trial with 719 patients divided into two echinacea groups (one blinded, the other unblinded) receiving a commercial E. purpurea and E. angustifolia root blend extract, a blinded placebo group, and a no-pill group (see Table 1 for brand information). The authors reported that the echinacea formulations did not have a statistically significant effect on the course or severity of the common cold compared to placebo or no-pill groups, but that there was a trend toward benefit. The authors noted several study limitations, including the possibility that the study had too few participants and was therefore underpowered, with an average of 180 subjects in each arm.³⁰ The 2011 study was a follow-on study to the 2010 study that used the data obtained in the earlier study and tested the hypothesis that individuals who believed in the benefits of echinacea would derive benefit from taking pills. The authors found that, overall, participants randomized to the no-pill group trended toward having longer and more severe illnesses than those who received pills, but that those who believed in echinacea benefits and received pills had substantially shorter and less severe colds whether pills were placebo or echinacea.³¹

The two most recent trials both were published in the open-access journal Evidence-Based Complementary and Alternative Medicine in 2012. The first, by Tiralongo et al³² evaluated a commercial standardized extract of E. purpurea and E. angustifolia root extract for prevention of respiratory and other symptoms associated with long commercial airline flights. The study was designed to measure effects on symptoms associated with air travel rather than to treat upper respiratory tract infections. The trial was well designed and the intervention was well described and well characterized. Of particular interest was the careful characterization and standardization of the alkylamide (alkamide) content of the product, as this group of compounds have been hypothesized to have anti-inflammatory properties. The authors reported that respiratory symptoms for both placebo and treatment groups increased significantly during travel periods, but that the increase of symptoms in the echinacea group was significantly lower than for the placebo group.

In contrast to the treatment trials discussed above, the most recent trial was a prevention trial and was also the largest echinacea trial to date (755 healthy subjects divided into echinacea and placebo groups), using a liquid commercial product consisting of a mix of extracts of freshly harvested roots and above-ground parts of E. purpurea (in a ratio of 5:95) (see Table 1 for brand information).³³ The study was well described and the product was well characterized and standardized to one of the main alkylamides. The authors reported that compliant prophylactic intake of the proprietary liquid extract of fresh E. purpurea over a four-month period significantly reduced the total number of cold episodes, cumulated episode days within the group, and pain-killer-mediated episodes versus placebo. There was no significant difference between adverse events reported between groups. The importance of evaluating the description of the intervention as well as the overall trial design is highlighted by this publication. A detailed description of the standardized proprietary extract was provided in this study, but the authors chose to forego use of a validated and well-accepted survey instrument for evaluation of the study outcomes. For example, the 21-item Wisconsin Upper Respiratory Symptom Survey (WURSS-21) was used in a number of the trials previously mentioned.^34,35 As a result, this study is not as valuable or definitive as it might otherwise have been.

Although the biologically active constituent(s) of echinacea remain unknown, the phytochemistry of the various species has been reasonably well described. The constituents identified to date fall into four distinct chemical classes: phenolic compounds related to caftaric acid (caftaric acid, chlorogenic acid, echinacoside, cynarin, cichoric acid), isobutylamides, polysaccharides, and glycoproteins.¹⁹ Different species and different plant parts are chemically distinct from one another, and some investigators use these variations as species identifiers. That level of detail is beyond the scope of this article. The more important concept to keep in mind is that failure to provide any details on the chemistry of the intervention means that the study is essentially irreproducible. Beyond that, the discerning reader can get a feel for the quality of the intervention (and therefore of the trial) by reading the description of the chemical content of the product. The least informative studies are those in which no chemical analysis was performed or reported. In such cases, it can be difficult to determine whether or not the test article contained echinacea and, even if one were willing to give the investigator the benefit of the doubt, such a report provides no basis for evaluating the quality of the product. Low-quality plant material and improper processing or storage all can contribute to a low-quality test material, and if the author provides no chemical information, it is difficult to say of a negative result whether the trial failed because echinacea is not efficacious or because poor-quality echinacea is not efficacious.

There are competing hypotheses about which chemical constituents are most important for the biological activity of echinacea preparations and the levels at which they should occur, and individual experts may define “quality” in terms of those constituents.²⁹ Informative studies are those that measure and report the full spectrum of constituents. Even better are those that use standardized extracts in which the chemical constituents are both measured and manipulated so that they are present at consistent levels from lot to lot and over time. Studies that do not report the full chemical spectrum are not easily interpreted or compared. If only a few of the many known constituents were reported or the levels reported were very low (compared to previously published levels), a low-quality study material may have been used. Finally, in practical terms, there is no such thing as a generic “echinacea trial.”

There are trials that evaluate specific echinacea products for specific endpoints, but in the absence of detailed information on comparability of the products, the nature of the endpoints, and the dose and justification for the dose, it is impossible to make the blanket statement that echinacea is or is not efficacious. The efficacy of different types of products may vary according to the indication and the nature of the product, but these differences must be supported by the clinical evidence. In the absence of such evidence, one can point to individual studies or a body of studies and make the judgment that a specific echinacea product(s) that has the described characteristics is or is not efficacious when evaluated under the conditions described in the trial.

Echinacea Summary

This discussion was not intended to be a systematic review, but in looking over the literature since 1998, it seems that the conclusions outlined in Melchart’s 2000 review of the echinacea trial literature need not be changed very much.²⁴ Although some poorly designed and described studies still appear in the peer-reviewed literature, the biomedical research community has learned much over the past decade. This learning process and subsequent trials have led to additional systematic reviews. In a 2006 review of the effects of echinacea preparations on the upper respiratory tract infection (URI) symptom severity and quality of life (QoL), Gillespie and Coleman stated that studies depicting echinacea’s effect on URI symptoms and QoL are contradictory, but that further research is warranted.³⁶ Another meta-analysis of studies on the effects of echinacea in the prevention of induced rhinovirus colds reported that “standardized extracts of echinacea were effective in the prevention of symptoms of the common cold after clinical inoculation, compared with placebo, but noted that further prospective, appropriately powered clinical studies are required to confirm this finding.”³⁷

The two most recent meta-analyses, a Cochrane review by Linde al³⁸ and an evaluation be Shah et al,³⁹ refine the conclusions drawn by Melchart et al,²⁴ Gillespie and Coleman,³⁶ and Schoop et al.³⁷ (see Table 1 for brand information) that “Echinacea seems like it might be better than placebo but more studies are needed” to draw conclusions about the nature and strength of the evidence. The Cochrane review by Linde and colleagues concluded that “Echinacea preparations tested in clinical trials differ greatly. There is some evidence that preparations based on the aerial parts of Echinacea purpurea might be effective for the early treatment of colds in adults but results are not fully consistent. Beneficial effects of other echinacea preparations and for preventative purposes might exist but have not been shown in independently replicated, rigorous randomized trials.”³⁸ The meta-analysis by Shah et al reported that echinacea decreased the odds of developing the common cold by 58% and the duration of the cold by one to four days.³⁹ They also reported that significant reductions were maintained in subgroup analyses limited to use of a specific brand (which was named in the analysis), concomitant supplement use, method of cold exposure (natural or challenge), Jadad scores less than three, or use of a fixed-effects model. The report concluded that an analysis of the current evidence in the literature suggests that echinacea has a benefit in decreasing the incidence and duration of the common cold. A letter by von Maxen and Schoenhoefer⁴⁰ disagreed with the conclusions of Shah et al³⁹ concerning efficacy of echinacea; however, the Shah report also highlighted the fact that large-scale randomized, prospective studies that control variables such as species, quality of the echinacea preparation, dose, method of cold induction, and objectivity of study endpoints evaluated are needed before echinacea can become standard practice for preventing and treating the common cold.

CONCLUSIONS

Other authors have addressed adequacy of blinding, randomization, sample size, etc. in evaluating trial design.^1-3 The examples provided in the preceding echinacea case study are intended to emphasize the details of trial design that are taken for granted by investigators accustomed to the world of single-agent pharmaceuticals and can be applied to any herbal clinical trial. The elements that any natural product trial report (or grant application) should contain can be broken down into a series of specifications for identity, purity, composition, and strength. These are built into the conventional drug approval process and are generally in place long before humans are exposed to new single-entity drugs. Despite frequent claims to the contrary, specifications and tests for all of the quality elements noted do exist for a number of herbs in a number of pharmacopeias, including, but not limited to, the United States Pharmacopeia, the European Pharmacopoeia, the Japanese Pharmacopoeia, the Pharmacopoeia of the People’s Republic of China, the German Pharmacopoeia, and others. If specifications exist, investigators should, if applicable, use them to define their trial materials. If they do not use them, they should be able to provide a good justification for not doing so. Also, if specifications do not exist, investigators may need to invent (and of course describe) them.

In evaluating a clinical trial report, readers should pay attention to the authors’ documentation of the source of the specifications used, the tests used to identify whether or not specifications have been met, and the test results. Use of preexisting specifications is in the authors’ interest in writing a paper, as details of testing, etc., can be provided by reference rather than by tediously describing the rationale for the specifications, the specifications themselves, the testing methods used, and results. While details of identity, purity, composition, and strength should be provided in all published natural product trials, the level of detail presented is especially important if the trial uses materials that are not from commercial sources (e.g., plant material collected or purchased by the investigator and then used in the trial with or without further processing). Information about the geographical origin of plant material and whether or not it was cultivated or wild-collected, and by whom, are important elements of identity. In addition, investigators should state whether or not voucher specimens were collected, and if so, where they are deposited.¹³ Other details, such as plant part, extract ratio, solvent composition, excipients used, chemical analysis, standardization details, and stability are all vital pieces of information necessary for evaluation of herbal clinical trial results. Because regulatory schemes are not uniform from country to country, an investigator who uses a commercial product should get as much of this information from the supplier/manufacturer as possible and include it in the clinical trial manuscript. Other aspects of the trial such as adequacy of blinding, testing for impurities (e.g., microorganisms, pesticides), inertness of the placebo, reliability of the assessment tool, and rationale for dosing are also important, and readers who keep all of these elements in mind should have little difficulty sorting through the literature.

ACKNOWLEDGMENTS

The author thanks John H. Cardellina II, PhD, and Marguerite Klein, formerly of the US National Institutes of Health Office of Dietary Supplements, for their editorial attentions; Dr. Robert Bonakdar, for encouraging the author to convert the lecture into a manuscript; and Mark Blumenthal and Ashley Lindstrom for their patience and gentle administrative ministrations.

Joseph M. Betz, PhD, earned his doctorate in pharmacognosy at the Philadelphia College of Pharmacy and Science. He worked at the US Food and Drug Administration’s Center for Food Safety and Applied Nutrition for 12 years, for two years was vice-president of Scientific and Technical Affairs at the American Herbal Products Association, and since 2001 he has worked in the Office of Dietary Supplements in the US National Institutes of Health. Dr. Betz is the recipient of the American Botanical Council’s first Norman R. Farnsworth Award for Excellence in Botanical Research and the American Society of Pharmacognosy’s Varro E. Tyler Prize for outstanding scientific contributions to the broad field of dietary supplements, with special emphasis on botanicals.

Mary L. Hardy, MD, has more than 25 years of experience melding integrative and conventional practice. She is a board certified internist and the recent past medical director of the UCLA Center for Integrative Oncology. She is a recognized leader in her field serving as a current board member of the Society for Integrative Oncology and co-leader of the Oncology Interest Group in the Consortium of Academic Health Centers of Integrative Medicine. She has extensive experience in evaluating the evidence base for complementary medicine, in part, due to her work as a research associate at the RAND Corporation.

*HerbalGram subsequently reprinted a slightly modified version of the CONSORT statement for herbal RCTs in order to broaden its availability to ABC members et al.⁵

^† In the HerbalGram style format for scientific nomenclature of plants, ABC no longer includes botanical authorities but does include family names, which ABC believes may be more valuable to readers. FDA’s dietary supplement labeling regulations require Latin binomial with the botanical authority’s name, but not the name of the plant family, i.e., if the company (labeler) chooses to not use a standardized common name (SCN) as listed in the FDA-recognized American Herbal Products Association’s Herbs of Commerce, 2d ed. (2000) or if there is no such SCN.

^‡ According to the 9th edition of the United States Pharmacopoeia (1910), “Fluidextracts are concentrated liquid preparations of vegetable drugs, containing alcohol either as a solvent or as a preservative, and bearing a uniform relation to the drug used so that one mil of the fluidextract closely represents the activity of one gramme of the air-dried and powdered drug of standard quality.”

References

1. Jadad AR, Moore RA, Carroll D, et al. Assessing the quality of reports of randomized clinical trials: is blinding necessary? Control Clin Trials. 1996;17:1–12.
2. Moher D, Schulz KF, Altman DG. The CONSORT statement: revised recommendations for improving the quality of reports of parallel-group randomized trials. Ann Int Med. 2001;134:657–662.
3. Linde K, Jonas WB, Melchart D, Willich S. The methodological quality of randomized controlled trials of homeopathy, herbal medicines and acupuncture. Int J Epidemiol. 2001; 30:526–531.
4. Gagnier JJ, Boon H, Rochon P, Moher D, Barnes J, Bombardier C. Reporting randomized, controlled trials of herbal interventions: an elaborated CONSORT statement. Ann Int Med. 2005;144:364–367.
5. Gagnier JJ, Boon H, Rochon P, Moher D, Barnes J, Bombardier C. Improving the quality of reporting randomized controlled trials evaluating herbal interventions: Implementing the CONSORT Statement. HerbalGram. 2006;71:50-56. http://cms.herbalgram.org/herbalgram/issue71/article3008.html.
6. Wolsko PM, Solondz DK, Phillips RS, Schachter SC, Eisenberg DM. Lack of herbal supplement characterization in published randomized controlled trials. Am J Med. 2005;118:1087–1093.
7. Gagnier JJ, DeMelo J, Boon H, Rochon P, Bombardier C. Quality of reporting of randomized controlled trials of herbal medicine interventions. Am J Med. 2006;119:800. e1–e11.
8. Swanson CA. Suggested guidelines for articles about botanical dietary supplements. Am J Clin Nutr. 2002;75:8-10.
9. Health and Human Services, National Institutes of Health. Biologically Active Agents Used in CAM and Placebo Materials—Policy and Guidance. Bethesda, MD: National Center for Complementary and Alternative Medicine. Available at: http://nccam.nih.gov/research/policies/ bioactive.htm. Accessed March 4, 2009.
10. Applequist WL, Miller JS. Selection and authentication of botanical materials for the development of analytical methods. Anal Bioanal Chem. 2013 May;405(13):4419-4428. doi: 10.1007/s00216-012-6595-1. Epub 2012 Dec 9.
11. USDA, ARS, National Genetic Resources Program. Germplasm Resources Information Network—(GRIN) [Online Database]. Beltsville, MD: National Germplasm Resources Laboratory. Available at: www.ars-grin.gov/cgi bin/npgs/html/index.pl?language!en. Accessed September 5, 2013.
12. The Plant List (2010). Version 1. Available at: www.theplantlist.org/. Accessed September 5, 2013.
13. Hildreth J, Hrabeta-Robinson E, Applequist W, Betz J, Miller J. Standard operating procedure for the collection and preparation of voucher plant specimens for use in the nutraceutical industry. Anal Bioanal Chem. 2007;389:13–17.
14. Blumenthal M, Goldberg A, Brinckmann J, eds. Herbal Medicine, Expanded Commission E Monographs. Newton, MA: Integrative Medicine Communications; 2000.
15. Betz JM, Fisher KD, Saldanha LG, Coates PM. The NIH analytical methods and reference materials program for dietary supplements. Anal Bioanal Chem. 2007;389:19–25.
16. Guns ES, Goldenberg SL, Brown PN. Mass spectral analysis of PC-SPES confirms the presence of diethylstilbestrol. Can J Urol. 2002;9:1684–1688.
17. Busse W. The significance of quality for efficacy and safety of herbal medicinal products. Drug Inf J. 2000;34:15–23
18. Bauer R. Quality criteria and standardization of phytopharmaceuticals: can acceptable drug standards be achieved? Drug Inf J. 1998;32:101–110.
19. Barnes J, Anderson LA, Gibbons S, Phillipson JD. Echinacea species (Echinacea angustifolia (DC.) Hell., Echinacea pallida (Nutt.) Nutt., Echinacea purpurea (L.) Moench): a review of their chemistry, pharmacology and clinical properties. J Pharm Pharmacol. 2005;57:929–954.
20. Tyler VE. Herbs of Choice—the Therapeutic Use of Phytomedicinals. Binghamton, NY: Haworth Press; 1994.
21. Brinker F. Echinacea Differences Matter: Traditional Uses of Echinacea angustifolia Root Extracts vs. Modern Clinical Trials with Echinacea purpurea Fresh Plant Extracts. HerbalGram. 2013;97:46-57.
22. Remington JP, Wood HC. The Dispensatory of the United States of America. 20th ed. Philadelphia, PA: JB Lippincott Company; 1918.
23. Osol A, Farrar GE. The Dispensatory of the United States of America. 25th ed. Philadelphia, PA: JB Lippincott Company; 1955.
24. Melchart D, Linde K, Fischer P, Kaesmayr J. Echinacea for preventing and treating the common cold. Cochrane Database Syst Rev. 2000;2:CD000530.
25. Barrett BP, Brown RL, Locken K, Maberry R, Bobula JA, D’Alessio D. Treatment of the common cold with unrefined echinacea. A randomized, double-blind, placebo-controlled trial. Ann Int Med. 2002;137:939–946.
26. Turner RB. Echinacea for the common cold: can alternative medicine be evidence-based medicine? Ann Int Med. 2002;137:1001–1002.
27. Barrett B, Brown R, Voland R, Maberry R, Turner R. Relations among questionnaire and laboratory measures of rhinovirus infection. Eur Respir J. 2006;28:358–363.
28. Whelan AM, Jurgens TM, Lord L. Evaluating the quality of randomized controlled trials that examine the efficacy of natural health products: a systematic review of critical appraisal instruments. Evid Based Complement Alternat Med. 2008;5:nem186.
29. Bergeron C, Livesey JF, Awang DVC, et al. A quantitative HPLC method for the quality assurance of Echinacea products on the North American market. Phytochem Anal. 2000;11:207–215.
30. Barrett B, Brown R, Rakel D, Mundt M, Bone K, Phyto D, Barlow S, Ewers T. Echinacea for treating the common cold. Ann Intern Med.2010;153:769-777.
31. Barrett B, Brown R, Rakel D, Rabago D, Marchand L, Scheder J, Mundt M, Thomas G, Barlow S. Placebo effects and the common cold: a randomized controlled trial. Ann Fam Med. 2011;9:312-322.
32. Tiralongo E, Lea, RA, Wee, SS, Hanna, MM, Griffiths, LR. Randomized, double blind, placebo-controlled trial of Echinacea supplements in air travelers. Evid Based Complement Alternat Med. 2012. doi:10.1155/2012/417267.
33. Jawad M, Schoop R, Suter, A, Klein, P, Eccles R. Safety and efficacy profile of Echinacea purpurea to prevent common cold episodes: a randomized, double-blind, placebo-controlled trial. Evid Based Complement Alternat Med. 2012. doi:10.1155/2012/841315.
34. Barrett B, Brown R, Mundt M, Safdar N, Dye L, Maberry R, Alt J. The Wisconsin Upper Respiratory Symptom Survey is responsive, reliable, and valid. J Clin Epidemiol. 2005;58:609-617.
35. Barrett B, Brown RL, Mundt MP, Thomas GR, Barlow SK, Highstrom AD, Bahrainian M. Validation of a short form Wisconsin Upper Respiratory Symptom Survey (WURSS-21). Health Qual Life Outcomes. 2009;7:76. doi: 10.1186/1477-7525-7-76.
36. Gillespie EL, Coleman CI. The effect of Echinacea on upper respiratory infection symptom severity and quality of life. Conn Med. 2006;70:93–97.
37. Schoop R, Klein P, Suter A, Johnston SL. Echinacea in the prevention of induced rhinovirus colds: a meta-analysis. Clin Ther. 2006;28:174–183.
38. Linde K, Barrett B, Wölkart K, Bauer R, Melchart D. Echinacea for preventing and treating the common cold. Cochrane Database Syst Rev. 2006 Jan 25;(1):CD000530.
39. Shah SA, Sander S, White CM, Rinaldi M, Coleman CI. Evaluation of echinacea for the prevention and treatment of the common cold: a meta-analysis. Lancet Infect Dis. 2007;7:473–480.
40. von Maxen A, Schoenhoefer PS. Benefit of echinacea for the prevention and treatment of the common cold? Lancet Infect Dis. 2008;8:346–347.