Betadisper() function warning message about negative values in dissimilarity matrix. Why am I getting this message?

[daroari]

Hello everyone!

I'm stuck at a problem with the betadisper function and have no idea how to move forward. I am using the betadisper() function to test for homogeneity of group dispersions (variances). As the data argument, I introduced a dissimilarity object I calculated using the daisy() function. The distances between observations are calculated using Gower's distance method. The reason I used this method in particular is because my data is composed mixed variables (continuous and categorical):

dissimilarity <- daisy(x = data, metric = 'gower')
mod <- betadisper(d = dissimilarity, group = raw_data$status)

When I run this, I get this warning message:

> mod <- betadisper(d = dissimilarity, group = raw_data$status)
Warning message:
In betadisper(dissimilarity, raw_data$status) :
some squared distances are negative and changed to zero

I have checked if there are any negative values in the dissimilarity object by converting it to a matrix and then using the which() function and there are no negative values in it.

dissimilarity_matrix <- as.matrix(dissimilarity)
negative_values <- which(dissimilarity_matrix < 0, arr.ind = TRUE)
print(negative_values)

row col

I don't know why am I getting this warning message. Further, if the betadisper function is converting values that are allegedly negative to zero, this would mean I would be losing information in terms of differences between observations.

I would appreciate any help you could provide me for this.

Thank you!

Daniel

[gavinsimpson]

The warning should be clearer, but it isn't talking about the input dissimilarities. What it is referring to are the distances of samples to their group centroid, which for non-metric dissimilarities get split into a component associated with the positive eigenvalues of the principal coordinates analysis used internally, and a component associated with the negative eigenvalues. If the distance to centroid in this negative-eigenvalue space is larger than the distance to centroid in the positive-eigenvalue space, we reset these negative distances-to-centroids to be equal to zero.

It might help to use some of the options that avoid negative eigenvalues in principal coordinates analysis if the above description of the warning still worries you.

[daroari]

Thanks for your response Gavin.

I was able to find some further information about what you explained in the documentation (link):

What I understand from this and your explanation is that this reset would ignore the distances of those samples to their group centroid, because the overall distance to centroid will be zero, is that correct?

Is there any section of your documentation where you explain the role of these distances in the negative-eigenvalue space (imaginary) in the context of testing for homogeneity of dispersions (variances)? My knowledge in matrix algebra is limited here, unfortunately, and I don't know what would be the implications of avoiding these negative eigenvalues in PCoA.

Thank you!

[jarioksa]

Actually, the distances to the centroid are not negative, but squared distances are negative – and hence distances are imaginary. When we deal with the real space (associated with non-negative eigenvalues), some of the among point distances and distances from points to their centroids are inflated (too long) compared to the original dissimilarities, and imaginary distances in effect "draw back" these inflated distances to the original observed distances. For betadisper this is not considered with original distances, but distances of points to their centroids in the real space (ignoring imaginary space associated with negative eigenvalues).

There are various tricks of avoiding this – but beware: many of them are tricks. In betadisper you can select argument add to avoid negative eigenvalues. However, then you are no longer handling Gower dissimilarities but fabricated dissimilarities – but you avoid negative eigenvalues! In many cases using sqrt.dist removes the problem, but this will change the meaning of eigenvalues: instead of showing dissimilarities, this indeed will map you square roots of dissimilarities. What ever you do, you must be consistent. I think people rarely use betadisper alone, but it is used together with adonis2 or other distance-based methods, and if you use a trick in betadisper, you must use the same trick everywhere else when dealing with the dissimilarities.

[jarioksa]

Here an expanded explanation of the nature negative eigenvalues and low down dirty tricks to avoid the nuisance of negative eigenvalues. The panels below were produced with vegan::stressplot() function of vegan::dbrda(d ~ 1) model, where d were the Bray-Curtis dissimilarities of the Dune meadow data (dune). The horizontal axis displays the input dissimilarities, and the vertical axis the dissimilarities as calculated from the full real solution of the analysis. In terms of betadisper, horizontal axis shows the input dissimilarities, and the vertical axis the dissimilarities as the real distances appear internally in betadisper.

With metric distances (Euclidean, Hellinger etc.) all points would fall on the red line showing the equality, but with semimetric Bray-Curtis most points are somewhat above the red line. This means that the points are farther away from each other in ordination than they are in input data. The task of negative eigenvalues and imaginary distances is to reduce the among-point distances and draw points back to the exact red line. In betadisper this can mean that in some cases this may mean that negative squared distances (how much above the line points are) can dominate group means, and then you get your cryptic warning message.

Some people do not like to see negative eigenvalues, and they have suggested various tricks to get rid of them. All these three alternatives are available in vegan functions such as betadisper, adonis2 and dbrda (among others). As you see, the adjustments "fix" the problem in the sense that you won't have negative eigenvalues, and all points fall on line or a continuous smooth curve. However, they won't fall on the red line of equality and won't give you back the input dissimilarities. This also means that the analysis will not handle original dissimilarities, but something else. Some people are worried about negative eigenvalues. I am more worried of not handling the input but something else.

[daroari]

Hello Jari,

Thanks a lot for the detailed explanation.

I agree, the presence of negative eigenvalues does not worry me anymore. Handling something else different from the original dissimilarities would be more concerning.

Since sometimes negative squared distances can dominate group means, and therefore, the function resets these negative squared distances to be equal to zero, this could reduce the dispersion (variance) of the data (distances of points to their centroids in the real space ), right? I ask this because I want to know to what extend does this warning affect the statistical significance of the test of homogeneity of variances, and if I still can trust the results of it (in my case was not significant, meaning I can't reject the null hypothesis that the groups have similar variances in the principal coordinate space).

The whole point of this is to check if the homoscedasticity assumption is met, to be sure I can perform a PERMANOVA using adonis2() function without violating this assumption.

Thanks again for all the valuable help @gavinsimpson and @jarioksa .