Skip to content

Commit

Permalink
rewriting function links #421
Browse files Browse the repository at this point in the history
  • Loading branch information
melina-leite committed Sep 11, 2024
1 parent dcb0d0f commit e566a2d
Show file tree
Hide file tree
Showing 46 changed files with 363 additions and 363 deletions.
26 changes: 13 additions & 13 deletions DHARMa/R/DHARMa.R
Original file line number Diff line number Diff line change
@@ -1,17 +1,17 @@


#' @keywords internal
#' @keywords internal
#' @references vignette("DHARMa", package="DHARMa")
#' @details
#' To get started with the package, look at the vignette and start with [simulateResiduals]
#'
#'
"_PACKAGE"



#' Print simulated residuals
#'
#' @param x an object with simulated residuals created by \code{\link{simulateResiduals}}.
#' @param x an object with simulated residuals created by [simulateResiduals].
#' @param ... optional arguments for compatibility with the generic function, no function implemented.
#' @export
print.DHARMa <- function(x, ...){
Expand All @@ -24,7 +24,7 @@ print.DHARMa <- function(x, ...){

#' Return residuals of a DHARMa simulation
#'
#' @param object an object with simulated residuals created by \code{\link{simulateResiduals}}
#' @param object an object with simulated residuals created by [simulateResiduals]
#' @param quantileFunction optional - a quantile function to transform the uniform 0/1 scaling of DHARMa to another distribution
#' @param outlierValues if a quantile function with infinite support (such as dnorm) is used, residuals that are 0/1 are mapped to -Inf / Inf. outlierValues allows to convert -Inf / Inf values to an optional min / max value.
#' @param ... optional arguments for compatibility with the generic function, no function implemented
Expand Down Expand Up @@ -55,7 +55,7 @@ residuals.DHARMa <- function(object, quantileFunction = NULL, outlierValues = NU
#'
#' Returns the outliers of a DHARMa object.
#'
#' @param object an object with simulated residuals created by \code{\link{simulateResiduals}}.
#' @param object an object with simulated residuals created by [simulateResiduals].
#' @param lowerQuantile lower threshold for outliers. Default is zero = outside simulation envelope.
#' @param upperQuantile upper threshold for outliers. Default is 1 = outside simulation envelope.
#' @param return wheter to return an indices of outliers or a logical vector.
Expand All @@ -64,7 +64,7 @@ residuals.DHARMa <- function(object, quantileFunction = NULL, outlierValues = NU
#'
#' Thus, keep in mind that for a small number of simulations, outliers are mostly a technical term: these are points that are outside our simulations, but we don't know how far away they are.
#'
#' If you are seriously interested in HOW FAR outside the expected distribution a data point is, you should increase the number of simulations in \code{\link{simulateResiduals}} to be sure to get the tail of the data distribution correctly. In this case, it may make sense to adjust lowerQuantile and upperQuantile, e.g. to 0.025, 0.975, which would define outliers as values outside the central 95% of the distribution.
#' If you are seriously interested in HOW FAR outside the expected distribution a data point is, you should increase the number of simulations in [simulateResiduals] to be sure to get the tail of the data distribution correctly. In this case, it may make sense to adjust lowerQuantile and upperQuantile, e.g. to 0.025, 0.975, which would define outliers as values outside the central 95% of the distribution.
#'
#' Also, note that outliers are particularly concerning if they have a strong influence on the model fit. One could test the influence, for example, by removing them from the data, or by some meausures of leverage, e.g. generalisations for Cook's distance as in Pinho, L. G. B., Nobre, J. S., & Singer, J. M. (2015). Cook’s distance for generalized linear mixed models. Computational Statistics & Data Analysis, 82, 126–136. doi:10.1016/j.csda.2014.08.008. At the moment, however, no such function is provided in DHARMa.
#'
Expand All @@ -87,10 +87,10 @@ outliers <- function(object, lowerQuantile = 0, upperQuantile = 1, return = c("i
#' @param simulatedResponse matrix of observations simulated from the fitted model - row index for observations and colum index for simulations.
#' @param observedResponse true observations.
#' @param fittedPredictedResponse optional fitted predicted response. For Bayesian posterior predictive simulations, using the median posterior prediction as fittedPredictedResponse is recommended. If not provided, the mean simulatedResponse will be used.
#' @param integerResponse if T, noise will be added at to the residuals to maintain a uniform expectations for integer responses (such as Poisson or Binomial). Unlike in \code{\link{simulateResiduals}}, the nature of the data is not automatically detected, so this MUST be set by the user appropriately.
#' @param integerResponse if T, noise will be added at to the residuals to maintain a uniform expectations for integer responses (such as Poisson or Binomial). Unlike in [simulateResiduals], the nature of the data is not automatically detected, so this MUST be set by the user appropriately.
#' @param seed the random seed to be used within DHARMa. The default setting, recommended for most users, is keep the random seed on a fixed value 123. This means that you will always get the same randomization and thus teh same result when running the same code. NULL = no new seed is set, but previous random state will be restored after simulation. FALSE = no seed is set, and random state will not be restored. The latter two options are only recommended for simulation experiments. See vignette for details.
#' @param method the quantile randomization method used. The two options implemented at the moment are probability integral transform (PIT-) residuals (current default), and the "traditional" randomization procedure, that was used in DHARMa until version 0.3.0. For details, see \code{\link{getQuantile}}.
#' @param rotation optional rotation of the residual space to remove residual autocorrelation. See details in [simulateResiduals], section *residual auto-correlation* for an extended explanation, and [getQuantile] for syntax.
#' @param method the quantile randomization method used. The two options implemented at the moment are probability integral transform (PIT-) residuals (current default), and the "traditional" randomization procedure, that was used in DHARMa until version 0.3.0. For details, see [getQuantile].
#' @param rotation optional rotation of the residual space to remove residual autocorrelation. See details in [simulateResiduals], section *residual auto-correlation* for an extended explanation, and [getQuantile] for syntax.
#' @details The use of this function is to convert simulated residuals (e.g. from a point estimate, or Bayesian p-values) to a DHARMa object, to make use of the plotting / test functions in DHARMa.
#' @note Either scaled residuals or (simulatedResponse AND observed response) have to be provided.
#' @example inst/examples/createDharmaHelp.R
Expand All @@ -107,16 +107,16 @@ createDHARMa <- function(simulatedResponse , observedResponse , fittedPredictedR
out$integerResponse = integerResponse
out$observedResponse = observedResponse

if(!is.matrix(simulatedResponse) & !is.null(observedResponse)) stop("either scaled residuals or simulations and observations have to be provided")
if(!is.matrix(simulatedResponse) & !is.null(observedResponse)) stop("either scaled residuals or simulations and observations have to be provided.")
if(ncol(simulatedResponse) < 2) stop("simulatedResponse with less than 2 simulations provided - cannot calculate residuals on that.")

if(ncol(simulatedResponse) < 10) warning("simulatedResponse with less than 10 simulations provided. This rarely makes sense")
if(ncol(simulatedResponse) < 10) warning("simulatedResponse with less than 10 simulations provided. This rarely makes sense.")

out$nObs = length(observedResponse)

if (out$nObs < 3) stop("warning - number of observations < 3 ... this rarely makes sense")
if (out$nObs < 3) stop("warning - number of observations < 3 ... this rarely makes sense.")

if(! (out$nObs == nrow(simulatedResponse))) stop("dimensions of observedResponse and simulatedResponse do not match")
if(! (out$nObs == nrow(simulatedResponse))) stop("dimensions of observedResponse and simulatedResponse do not match.")

out$nSim = ncol(simulatedResponse)

Expand Down
16 changes: 8 additions & 8 deletions DHARMa/R/compatibility.R
Original file line number Diff line number Diff line change
Expand Up @@ -79,7 +79,7 @@ weightsWarning = "Model was fit with prior weights. These will be ignored in the
#'
#' @example inst/examples/wrappersHelp.R
#'
#' @seealso \code{\link{getRefit}}, \code{\link{getSimulations}}, \code{\link{getFixedEffects}}, \code{\link{getFitted}}
#' @seealso [getRefit], [getSimulations], [getFixedEffects], [getFitted]
#' @author Florian Hartig
#' @export
getObservedResponse <- function (object, ...) {
Expand All @@ -99,7 +99,7 @@ getObservedResponse <- function (object, ...) {
#' @return A matrix with simulations.
#' @example inst/examples/wrappersHelp.R
#'
#' @seealso \code{\link{getObservedResponse}}, \code{\link{getRefit}}, \code{\link{getFixedEffects}}, \code{\link{getFitted}}
#' @seealso [getObservedResponse], [getRefit], [getFixedEffects], [getFitted]
#'
#' @details The purpose of this function is to wrap or implement the simulate function of different model classes and thus return simulations from fitted models in a standardized way.
#'
Expand All @@ -122,7 +122,7 @@ getSimulations <- function (object, nsim = 1 , type = c("normal", "refit"), ...)
#' @param object a fitted model.
#' @param ... additional parameters.
#' @example inst/examples/wrappersHelp.R
#' @seealso \code{\link{getObservedResponse}}, \code{\link{getSimulations}}, \code{\link{getRefit}}, \code{\link{getFitted}}
#' @seealso [getObservedResponse], [getSimulations], [getRefit], [getFitted]
#' @export
getFixedEffects <- function(object, ...){
UseMethod("getFixedEffects", object)
Expand All @@ -141,7 +141,7 @@ getFixedEffects <- function(object, ...){
#'
#' @example inst/examples/wrappersHelp.R
#'
#' @seealso \code{\link{getObservedResponse}}, \code{\link{getSimulations}}, \code{\link{getFixedEffects}}
#' @seealso [getObservedResponse], [getSimulations], [getFixedEffects]
#' @author Florian Hartig
#' @export
getRefit <- function (object, newresp, ...) {
Expand All @@ -161,7 +161,7 @@ getRefit <- function (object, newresp, ...) {
#'
#' @example inst/examples/wrappersHelp.R
#'
#' @seealso \code{\link{getObservedResponse}}, \code{\link{getSimulations}}, \code{\link{getRefit}}, \code{\link{getFixedEffects}}
#' @seealso [getObservedResponse], [getSimulations], [getRefit], [getFixedEffects]
#'
#' @author Florian Hartig
#' @export
Expand All @@ -183,7 +183,7 @@ getFitted <- function (object, ...) {
#'
#' @example inst/examples/wrappersHelp.R
#'
#' @seealso \code{\link{getObservedResponse}}, \code{\link{getSimulations}}, \code{\link{getRefit}}, \code{\link{getFixedEffects}}, \code{\link{getFitted}}
#' @seealso [getObservedResponse], [getSimulations], [getRefit], [getFixedEffects], [getFitted]
#'
#' @author Florian Hartig
#' @export
Expand All @@ -205,7 +205,7 @@ getResiduals <- function (object, ...) {
#'
#' @example inst/examples/wrappersHelp.R
#'
#' @seealso \code{\link{getObservedResponse}}, \code{\link{getSimulations}}, \code{\link{getRefit}}, \code{\link{getFixedEffects}}, \code{\link{getFitted}}
#' @seealso [getObservedResponse], [getSimulations], [getRefit], [getFixedEffects], [getFitted]
#'
#' @author Florian Hartig
#' @export
Expand All @@ -220,7 +220,7 @@ getPearsonResiduals <- function (object, ...) {
#' @param object a fitted model.
#' @param ... additional parameters to be passed on.
#'
#' @seealso \code{\link{getObservedResponse}}, \code{\link{getSimulations}}, \code{\link{getRefit}}, \code{\link{getFixedEffects}}, \code{\link{getFitted}}
#' @seealso [getObservedResponse], [getSimulations], [getRefit], [getFixedEffects], [getFitted]
#'
#' @author Florian Hartig
#' @export
Expand Down
30 changes: 15 additions & 15 deletions DHARMa/R/helper.R
Original file line number Diff line number Diff line change
Expand Up @@ -40,14 +40,14 @@ DHARMa.ecdf <- function (x)
#'
#' Before DHARMa 0.3.1, a different randomization procedure was used, in which the a U(-0.5, 0.5) distribution was added on observations and simulations for discrete distributions. For a completely discrete distribution, the two procedures should deliver equivalent results, but the second method has the disadvantage that a) one has to know if the distribution is discrete (DHARMa tries to recognize this automatically), and b) that it leads to inefficiencies for some distributions such as the Tweedie, which are partly continuous, partly discrete
#' (see e.g. [issue #168](https://github.com/florianhartig/DHARMa/issues/168)).
#'
#'
#' **Rotation (optional)**
#'
#' The getQuantile function includes an additional option to rotate residuals. This option should ONLY be used when the fitted model includes a particular residuals covariance structure, such as an AR1 or a spatial CAR model.
#'
#'
#' The getQuantile function includes an additional option to rotate residuals. This option should ONLY be used when the fitted model includes a particular residuals covariance structure, such as an AR1 or a spatial CAR model.
#'
#' For these models, residuals calculated from unconditional simulations will include the specified covariance structure, which will trigger e.g. temporal autocorrelation tests and can inflate type I errors of other tests. The idea of the rotation is to rotate the residual space according to the covariance structure of the fitted model, such that the rotated residuals are conditional independent (provided the fitted model is correct).
#'
#' If the residual covariance of the fitted model at the response scale can be extracted (e.g. when fitting gls type models), it would be best to extract it and provide this covariance matrix to the rotation option. If that is not the case, providing the argument "estimated" to rotation will estimate the covariance from the data simulated by the model. This is probably without alternative for GLMMs, where the covariance at the response scale is likely not known / provided, but note, that this approximation will tend to have considerable error and may be slow to compute for high-dimensional data. If you try to estimate the rotation from simulations, you should set n as high as possible! See [testTemporalAutocorrelation] for a practical example.
#'
#' If the residual covariance of the fitted model at the response scale can be extracted (e.g. when fitting gls type models), it would be best to extract it and provide this covariance matrix to the rotation option. If that is not the case, providing the argument "estimated" to rotation will estimate the covariance from the data simulated by the model. This is probably without alternative for GLMMs, where the covariance at the response scale is likely not known / provided, but note, that this approximation will tend to have considerable error and may be slow to compute for high-dimensional data. If you try to estimate the rotation from simulations, you should set n as high as possible! See [testTemporalAutocorrelation] for a practical example.
#'
#' @references
#'
Expand All @@ -68,7 +68,7 @@ getQuantile <- function(simulations, observed, integerResponse, method = c("PIT"


if(method == "traditional"){

if(!is.null(rotation)) stop("rotation can only be used with PIT residuals")

if(integerResponse == F){
Expand All @@ -79,7 +79,7 @@ getQuantile <- function(simulations, observed, integerResponse, method = c("PIT"
if(length(values) > 0){
if (all(values%%1==0)){
integerResponse = T
message("Model family was recognized or set as continuous, but duplicate values were detected in the simulation - changing to integer residuals (see ?simulateResiduals for details)")
message("Model family was recognized or set as continuous, but duplicate values were detected in the simulation - changing to integer residuals (see ?simulateResiduals for details).")
} else {
message("Duplicate non-integer values found in the simulation. If this is because you are fitting a non-inter valued discrete response model, note that DHARMa does not perform appropriate randomization for such cases.")
}
Expand All @@ -97,28 +97,28 @@ getQuantile <- function(simulations, observed, integerResponse, method = c("PIT"
}

} else {
# optional rotation before PIT

# optional rotation before PIT
if(!is.null(rotation)){
if(is.character(rotation) && rotation == "estimated"){
covar = Matrix::nearPD(cov(t(simulations)))$mat
L = t(as.matrix(Matrix::chol(covar)))
}
}
else if(is.matrix(rotation)) L <- t(chol(rotation))
else stop("DHARMa::getQuantile - wrong argument to rotation parameter")
else stop("DHARMa::getQuantile - wrong argument to rotation parameter.")
observed <- solve(L, observed)
simulations = apply(simulations, 2, function(a) solve(L, a))
}

scaledResiduals = rep(NA, n)
for (i in 1:n){
minSim <- mean(simulations[i,] < observed[i])
maxSim <- mean(simulations[i,] <= observed[i])
minSim <- mean(simulations[i,] < observed[i])
maxSim <- mean(simulations[i,] <= observed[i])
if (minSim == maxSim) scaledResiduals[i] = minSim
else scaledResiduals[i] = runif(1, minSim, maxSim)
}
}

return(scaledResiduals)
}

Expand Down
Loading

0 comments on commit e566a2d

Please sign in to comment.