Irene de la Rosa – Methodological validity and statistical management of published data – “Comparative study of high and low intensity percutaneous electrolysis in patellar tendinopathy.”

According to Irene de la Rosa, PT, PhD. Grupo de Investigación FPSM, Universidad de Alcalá;

The study from Padron A. and Rojas S, “Comparative study of high and low intensity percutaneous electrolysis in patellar tendinopathy.” has been performed under methodological rules that does not fulfill CONSORT statements (specially due to lack of sample size calculation, randomization and masking techniques), which negatively affects to the study scientific rigor. Apart from that, statistical analysis has not been raised up properly, thus, data collected have a very doubtful validity.

CONSORT2010 Checklist

Table showing different aspects to consider when communicating a Clinical Trial, according to CONSORT rules

In table 1, final conclusions from the analysis are brought forward with respect to the NON fulfillment of CONSORT rules:

  • Lack or sample size calculation
  • NO randomization
  • Study groups are heterogeneous
  • No fulfillment of masking techniques, examiner is not blinded.
  • With no internal validity there must not be external validity, which means that is not possible to extrapolate data to the rest of population.
  • Mistakes regarding the statistical tests applied according to the non-normality of the sample.

Tabla 1. Conclusiones finales 

Hereafter, methodological biases that prevent from giving consistency to collected data from this study are shown.

In relation to the study sample:

  1. N=13 subjects (17 tendons) is not enough to draw statistical conclusions. Bibliography says minimum N to develop a pilot study is N=40 (20 per group) or to calculate sample size from SD of average (or IQR of medians) from previous studies (such as, for example, from number 12 and 15 referenced in the present study).
  2. No participants randomization to each of the study groups. Lack of randomization derives into no scientific rigor. This is a CONSORT statement fundamental and unavoidable. Could this explain that all participants with elevated BMI are in the same group, coinciding with the low intensity electrolysis application? Being the fact that overweight is a risk factor that influences Achilles tendinopathy recovery, this type of error becomes something unacceptable. Without randomization any variable can be confounding factor, that is to say, study results can be due to the effect of descriptive factors form the sample such as age, BMI, bilateral injury, etc.
  3. Study groups are heterogeneous in relation to age, in this case the selected age goes from 18 to 45. Age variable affects directly in tissue regeneration process, which could be a confounder. Likewise, it is heterogeneous with respect to injury time evolution, because the inclusion criterion about this issue only considers this when surpasses 3 months. Does a patient with a 3-month evolution process present same degenerative changes and respond in the same manner to a treatment compared to a patient with a 1-year evolution tendinopathy? Surely not.
  4. Participants presenting bilateral tendinopathy should belong to a subgroup in order to carry out a previous analysis at least.

In relation to the evaluation process:

  1. Masking techniques are another CONSORT rule that all experimental trials have to conduct in order to give minimum scientific rigor to the study results. In this trial, examiner was not blinded; to know group allocation detract study validity. So much so that, nowadays is almost impossible any study that neither implements any masking nor randomization techniques can be published. CONSORT rules provide internal validity to any study. Without any internal validity, there is no external one, which means that results cannot be extrapolated to the rest of population.
  2. There is no follow up, only pre-post evaluation. If changes are not kept at least to mid term, a treatment cannot be considered a valid option.

About trial interventions:

  1. There is an instrumental evaluation performed by 3 devices. In spite of that, there is no specification of which device has been utilized for this issue.
  2. Cryotherapy application and/or NSAIDs intake 72 hours after intervention should be taken into consideration as a possible confounding factor.

Moreover, an instrumental analysis has been conducted. It is not understood that 3 devices have been used since there are only two study groups. In any case, if someone wants to perform any study to compare all 3 devices, it needs to be, on the one hand with the minimal intensity given by each device and, on the other hand, with the maximum intensity that each devices has detected as the maximum eligible. Besides, the experiment should be launched with patellar tendon, rather than brachiorradialis muscle, because the trial is targeted to treat patellar tendinopathy.

With regard to results:

  1. Normality analysis of the sample.

First item to study is the sample normality, in other words, to know if both groups follow a Gaussian distribution (normal). To this effect, Shapirowilk study is applied (because N is below 30) for the QUANTITATIVE variables. In order to classify a sample as normal, there must be one of the following instances:

  • Big sample size
  • Average and median values nearly equal
  • SD value a quarter of the average value at the most.
  • Frequencies histogram: distributed data as Gaussian bell curve.

Even in these cases, if there is no normality study because it is unknown, with N sizes lower than 20 subjects in each group, no sample normality has to be assumed. Therefore, medians and interquartile ranges (IQRs) and no parametrical tests have to be picked in order to implement the statistical analysis.

2. Homogeneity analysis

Based on if sample is normal or not, parametrical or non-parametrical test will be used respectively. The aim is to determine if there are statistically significant differences among the different variables from the sample, because if this is affirmative, variables would become confounders. In this study, age is the only difference shown, authors have assumed sample is normal because they refer to average and SD values, which is not true.

3. Outcome variables.

Mann-Whitney test is performed in all analysis, which is a non-parametrical test (when anyone knows if sample is gaussian or not, it is mandatory to assume that it is not) for non-paired data. The justification of the election of this test is somehow “peculiar” and with no bibliographic reference. Anyway, being a non-parametric test, median and IQRs values should have been selected. Furthermore, SD is above a quarter of the average value (one of the indispensable requirements to choose the SD).

Moreover, an intragroup analysis is recommendable to be performed in experimental studies, reason why Wilcoxon test should be used (non-parametrical and paired data).

With regard to discussion:

  • A discussion cannot be developed if all statistical management is incorrect.
  • No conclusion can be drawn from this study.

Reference list

Online Assistance