The Cochrane LPT analyses – can they be improved?
By Jan Tunér and Lars Hode
(From The New Laser Therapy Handbook, 2010, Prima Books AB. Originally published in Laser Therapy journal 1999)
The aim of the international Cochrane collaboration is to continuously evaluate new and old medical therapies. The basis for the systematic reviews is the recognition of Randomised Controlled Trials as the “gold standard” for scientific evaluation of small and moderate effects from treatment. A thorough search is made for the available literature and the most “qualified” studies are analysed. The purpose of the analysis is to find out whether or not there is any solid support for a specific medical treatment modality. Such analyses are published in medical journals and extended versions are quarterly updated in the Cochrane Library. Since the Cochrane reviews are influential, it is important that the persons performing them possess a thorough knowledge of all aspects of LPT.
Introduction
Five systematic reviews on the effectiveness of LPT have been published in the Cochrane Library. Three reviews have evaluated the effect of LPT for Venous ulcers [B1] Osteoarthritis[B2] and Rheumatoid arthritis [B3]. However, the Cochrane style of reviewing has been criticised [B20] for not taking into account the variability of diagnoses, treatment procedures and dosages of the included trials. The impact of the Cochrane Library in the field of medicine is profound. It is therefore essential to “evaluate the evaluation”, to find out whether or not these analyses can live up to the prestige of the Cochrane Library. The following text is a critical “analysis of the analyses”.
1. Venous ulcers
Four trials are analysed: two comparing laser with placebo, one comparing laser with non-coherent light and one comparing laser with ultraviolet light. Two studies comparing laser with placebo were reported by the same investigators [B4, B5]. In the first report a 6 mW HeNe laser was used. 4 J/cm^2 was said to be given to the ulcers. Ulcer size ranged from 3-32 cm^2. Treatment technique was not stated. Regardless of technique, it would take between 36 minutes and 6 hours to achieve the stated dose per wound and session. Using a sweep technique with a focused beam, the power density would be around 0.15 W/cm^2. If a defocused beam were used to cover the entire largest wound (32 cm^2), energy density would be around 0.00019 W/cm^2, which is extremely low - lower than the energy density of the normal illumination in a medical clinic. A dose miscalculation is probable but the authors of the study have been reluctant to reveal the parameters used. In the absence of such parameters, this study cannot be properly evaluated, but very low power density is a probable reason for its negative results. In the second study[B5] on venous ulcer, GaAs was employed. 4 mW was used for 10 minutes on ulcers ranging from 4 to 52 cm^2, regardless of ulcer size. The 4 cm^2 wound would thus receive 0.6 J/cm^2 and the largest wound 0.046 J/cm^2, not the 1.96 J/cm^2 stated by the authors. Energy densities as well as dose for larger wounds are thus low. Treatment technique is not indicated. The report states that: “the laser was held perpendicular to the surface of the wound”. This is not a sufficient description of the treatment method. There is a great difference between following the outer border of the wound (active healing area) and spreading the beam over the open wound area. The distance between diode and wound is not indicated. It is furthermore of interest to know that one of the authors have been exposed as a scientific cheater.
In summary, the energy level said to be applied in these studies must be questioned. The Cochrane evaluators have not observed the essential contradiction between the actual dose and the dose indicated by the authors. Further, in one of the four studies [6] the laser wavelength and dose are not stated in the original paper. This makes an evaluation impossible.
2. Osteoarthritis
Five trials were included out of 142 potentially relevant articles. Six abstracts are awaiting assessment after the authors have been contacted for further details. In the meantime, some comments on the analysis follows:
Bülow[B7](negative outcome) is a good study using a reasonable energy level (GaAlAs laser 22.5 J/session) for painful knee osteoarthritis. However, see discussion below about chosen outcome measurements.
Basford[B8] (negative outcome) used HeNe laser 0.007 J per point for thumb arthritis. The dose used is virtually meaningless since it is too low.
Jensen [B9] (negative outcome) used GaAs laser, 0.2 J in total, for painful knee arthrosis. Here, too, the dose used is very low and below the dose levels that have been reported to be effective.
Stelian [B10] (positive outcome) used GaAlAs laser around 11 J per session, twice daily, amounting to 22 J per knee per day, 10 consecutive days. This study has a dose that is acceptable even in the light of today’s experience, although it was published as early as 1992. The outcome of this study stands in contrast to the rather similar study by Bülow. The dose is the same, and the number of sessions is almost the same (10/9). However, Bülow applied the treatment 2-4 times a week, Stelian twice daily.
Walker [B11] (positive outcome) is a classic positive study, but the use of a HeNe laser of less than 1 mW throws doubt on this study. In our opinion it should not be used as anything but a purely historical reference.
The crucial criticism of the evaluation of the studies above is that there is no discussion about dosage! On the Jadad quality scale (1-5), Basford is given 3 and Bülow 2. However, Basford used a non-significant dosage for a finger joint while Bülow has a reasonable dose for knee osteoarthritis. Johannsen (see Rheumatoid arthritis below) used a dose above the generally accepted dosage window. In retrospect, the Bülow trial has been criticised for overlooking a significant short-term effect of active laser treatment by only testing the statistical significance at follow-up. Stelian used 55 times more energy than Jensen for knee osteoarthritis! The Jadad quality scale is applied correctly to the studies. But without inclusion of the laser parameters in the scale, evaluation tends to become a “study design beauty contest” instead of an evaluation of therapeutical significance.
In a recent re-evaluation of the Cochrane OA report Bjordal [1506] summarizes:
“This Cochrane review of LLLT for osteoarthritis contained possible data errors and trial omissions which were crucial for the direction of the conclusion. The review also lacked transparency in data selections for main results calculations and the review did not adhere to the study protocol for subgroup analyses. Our sensitivity analysis showed that the negative review conclusion could be altered to a positive conclusion without changing the criteria of the review protocol by including omitted data. Performance of lacking subgroup-analyses also revealed a highly significant dose-dependent effect from LLLT. Identification of 18 possible omissions, errors and equivocal data interpretations, revealed that 17 of these questionable selections supported the review conclusion. These findings challenge the assumption that Cochrane reviews are always reliable and unbiased. In areas of controversy, extra care must be taken to maintain validity and reliability of SRs. Thorough sensitivity analyses seem to be the most effective tool for testing how robust review conclusions really are. Transparency is needed to improve methods and reduce subjective preferences in reviews from controversial areas.”
This paper [2] has been withdrawn by the Cochrane Collaboration.
3. Rheumatoid arthritis
Eight of 191 articles met the inclusion criteria; five were Randomised Controlled Trials. Five further studies are waiting assessment pending answers from the authors.
Comments:
Johannsen [B13] (negative outcome) used 11.9 J GaAlAs laser per finger joint, which is a high dose, maybe too high.
Heussler [B14] (negative outcome) used 1.5 J GaAlAs laser per finger joint.
Walker [B15] (positive outcome) used less than 1 mW of HeNe laser. Although Johannsen used a dose 1700 times higher than Walker, both studies are “put in the same basket”. A low/high dose evaluation is actually performed but the wide gap in dosages does not justify a subgroup analysis of merely two groups.
Hall [B16] and Goats [B17] used combined coherent and non-coherent light. Combined single wavelength coherent light and multi-wavelengths non-coherent light is a poorly studied area and there is no ground for postulating that they produce the same biological effects when used in various combinations or alone.
The Meta analysis by Gam [B19] (one of the Cochrane co-authors) is referred to. This analysis did not find any effect of LPT for musculoskeletal pain. The re-evaluation of the same studies made by Bjordal [B20] found a clear effect when an analysis of the dosage and therapeutic techniques was included. This latter Meta-analysis is not mentioned. Critical comments on the Cochrane reviews are supposed to be continuosly included. Bjordal writes:
“Gam et al made a firm statement that LLLT has no effect in musculoskeletal disorders. As already indicated, this review is highly sensitive to changes in inclusion criteria and demonstrates critical errors in data synthesis, which calls for considerable caution in making any conclusions at all. The statistical pooling in this review cannot be regarded as valid because of the large variation of trial designs, treatment procedures, dosage and diagnoses. In all, the conclusion of this review was assessed as invalid, because it did not adhere strictly to the available evidence, especially for subgroups.”
Conclusions
The evaluators of the Cochrane groups have been successful in finding many of the relevant studies in the literature. Several interesting observations have been made and a skilful analysis of the design parameters has been performed. Evaluation of effects is a universal problem for all empirically developed therapies, where consensus of a clearly defined optimal dose range and adequate treatment procedure is lacking. For clinicians practicing LPT it is hard to understand why the reviewers have neglected some important factors, e.g. body area treated, size of area and dose. The methodology used seems to be that of drug studies. But drugs and LPT are quite different. Oral intake of a drug is the universal procedure. LPT can be applied in several ways, such as local irradiation, trigger point irradiation, acupuncture irradiation and irradiation over peripheral nerves. All these methods must be evaluated separately.
The biggest problem has been the fact that most of the reviews have “put into the same basket” a variety of doses, energy densities and treatment procedures. New treatment methods are often subject to trials where clinicians include all their non-responding patients, and the early laser literature is no exception. Flaws in LPT studies are not uncommon [B21]. The laser literature in the year 2000 included about 100 positive and 40 negative double blind trials [B22] on a heterogeneous sample of more than 20 different diagnoses, which vary widely in pathology, tissue involved and prognosis. Added to this are all the inadequate treatment procedures and doses that have been employed in clinical LPT trials. Under such circumstances the majority of these trials will find no effect of active treatment. Therefore, we should be very careful when viewing all the trial results together to see if they add up to an effect significantly better than placebo. Future reviews are suggested to analyse the positive studies in order to find out what parameters seem to work. Subgroup analyses are of particular importance. Dosage analysis cannot be limited to the groups “high” and “low” because of the great variations in dosage. Neither can any wavelength be put into the “high” or “low” basket. 2 J HeNe for a finger joint may be “high” while 20 J HeNe for lumbar problem may be “low”. GaAs requires lower doses than GaAlAs etc.
So what have these new Cochrane reviews brought us? Two distinct steps of progress can be identified. In the rheumatoid arthritis review, attempts have been made to evaluate effects separately for high and low dose. And secondly, they even give a (qualified) recommendation: “Low level laser therapy could be considered for treatment of rheumatoid arthritis for its short term effect and lack of side effects”.
Future directions
Both laser researchers and reviewers have common responsibilities in enhancing our understanding of LPT. The three existing Cochrane reviews on laser therapy have drawn some conclusions, to which we can comment that the literature on the evaluated indications is ambiguous, the average quality of the studies is not very high, and the number of relevant studies is rather low. It can therefore be postulated that there is still insufficient scientific support for the general use of Laser Therapy for the three indications reviewed by the Cochrane evaluators and that only moderate and short-term effects can be confirmed. However, the review methodology should include valid criteria for dose and target for laser irradiation (e.g. synovia, triggerpoints, acupuncture points, peripheral nerves, etc.). Furthermore, the effect calculations could be performed for subgroups of different doses, treatment frequencies and laser types. There is also still room for improvement of the literature search. Further, reviewers must make their own dosage calculations, not taking the doses quoted in the studies for granted. Too many of the negative laser therapy studies contain serious flaws [B21], and such flaws must be firmly investigated when evaluating laser therapy studies.
In a more recent Cochrane analysis of laser therapy Vlassov [B23] found no effect of laser therapy on tuberculosis. Owing to the incoherent literature on the subject, no dosage analyses could be performed in this study.
References:
[B1] Flemming K, Cullum N: Laser Therapy for venous leg ulcers (Cochrane review). In: The Cochrane Library, 4, 2000.
[B2] Brosseau L, Welch V, Wells G et al: Low level laser therapy (Classes I, II and III) for treating osteoarthritis. The Cochrane Library. Issue 4, 2000.(Withdrawn)
[B3] Brosseau L, Welch V, Wells G et al: Low level laser therapy (Classes I, II and III) for treating rheumatoid arthritis. The Cochrane Library. Issue 4, 2000.
[B4] Lundeberg T and Malm M: Low power HeNe laser treatment of venous leg ulcers. Annals of Plastic Surgery. 1991; 27: 537-539.
[B5] Malm M and Lundeberg T: Effect of low power gallium arsenide laser on healing of venous ulcers. Scandinavian Journal of Plastic Reconstruction and Hand Surgery. 1991; 25: 249-251.
[B6] Crous L and Malherbe C: Laser and ultraviolet light irradiation in the treatment of chronic ulcers. Physiotherapy. 1988; 44: 73-77.
[B7] Bülow P M, Jensen H and Danneskiold-Samsoe B: Low power GaAlAs laser treatment of painful osteoarthritis of the knee. Scandinavian Journal of Rehabilitation Medicine. 1994; 26: 155-159.
[B8] Basford JR, Sheffield C G, Mair S D, Ilstrup D M: Low-energy helium neon laser treatment of thumb osteoarthritis. Archive of Physical and Medical Rehabilitation. 1987; 68: 794-797.
[B9] Jensen H, Harreby M and Kjær J (1994): Infrared laser – effekt ved smertende knaearthrose? Ugeskrift for Laeger, 149: 3104-3106.
[B10] Stelian J, Gil I, Habot B et al: Improvement of pain and disability in elderly patients with degenerative osteoarthritis of the knee treated with narrow band light therapy. Journal of the American Geriatric Society. 1992; 40: 23-26.
[B11] Walker J: Relief from chronic pain by low power laser irradiation. Neuroscience Letters. 1983; 43: 339-344.
[B12] Marks R, de Palma F: Clinical efficacy of low power laser therapy in osteoarthritis. Physiotherapy Research International. 1999; 4 (2): 141-147.
[B13] Johannsen F, Hauschild B, Remvig L et al: Low energy laser therapy in rheumatoid arthritis. Scandinavian Journal of Rheumatology. 1994; 23: 145-147.
[B14] Heussler J K, Hinchey G, Margiotta E et al: A double blind randomised trial of low power laser treatment in rheumatoid arthritis. Annals of the Rheumatic Diseases. 1993; 52: 703-706.
[B15] Walker J, Akhanjee L K, Cooney M M et al: Laser therapy for pain of rheumatoid arthritis. Clinical Journal of Pain. 1987; 3: 54-59.
[B16] Hall J, Clarke AK, Elvins DM, Ring EFJ: Low level laser therapy is ineffective in the management of rheumatoid finger joints. British Journal of Rheumatology. 1994; 33: 142-147.
[B17] Goats G C, Flett E, Hunter J A, Sterling A: Low intensity laser and phototherapy for rheumatoid arthritis. Physiotherapy. 1996; 82 (5): 311-320.
[B18] Seichert N: Physiotherapy: Controlled trials and facts. In (Schlapbach P, Gerber NJ eds): Rheumatology. 1991; p. 205-217.
[B19] Gam A N, Thorsen H and Lonnberg F: The effect of low-level laser therapy on musculo-skeletal pain: a Meta-analysis. Pain. 1993; 52: 63-66.
[B20] Bjordal J M, Greve G: “What may alter the conclusions of reviews?”. Physical Therapy Reviews. 1998; 3: 121-132.
[B21] Tunér J, Hode L. It´s all in the parameters – a critical analysis of some well-known negative studies on low-level laser therapy. J Clin Lasers Med Surg. 1998; 16 (5): 245-248.
[B22] Tunér J: What is in the LLLT literature? In (Simunovic Z, ed): Lasers in Medicine and Dentistry, European Medical Laser Assn. 2000; pp 217-226. ISBN 953-6059-30-4.
[B23] Vlassov V V, Pechatnikov L M, MacLehose H G. Low level laser therapy for treating tuberculosis. Cochrane Database Syst Rev. 2002; ( 3): CD003490.