diff --git a/NEWS.md b/NEWS.md index d3e66d0..492b0cd 100644 --- a/NEWS.md +++ b/NEWS.md @@ -1,3 +1,13 @@ +## Version 1.2.0 (Aug 17, 2023) +- Help documentation added: to view the help documentation for functions `scan()`, and `bulkscan()`, type `?` in front of the functions. +- New features: + - Added the wrapper function `bulkscan()` for the three algorithms of multiple-trait scans previously named as `bulkscan_null()`, `bulkscan_null_grid()`, `bulkscan_alt_grid()`. Now, the user can simply call the common interface `bulkscan(...; method = )` by supplying with the method by the user's specific favor of computational speed or precision. Allowable inputs are string types, named as "null-exact", "null-grid", "alt-grid". The default option is "null-grid" with a loose grid of h2-step of size 0.1 (a grid of 10 values from 0.0, 0.10, ..., 0.90). + - Added the option for SVD decomposition of the kinship matrix. To use this feature, supply the option `decomp_scheme = svd`. + - Added the option in both `scan()` and `bulkscan()` functions for returning the $-log_{10}(p)$ result, where $p$ is the likelihood ratio test p-value: To use this feature, supply the option `output_pvals = true`. For more details, check the help instruction by `?scan()`, `?bulkscan()`. +- Fixed bugs: + - Fixed a bug causing compilation error in `bulkscan()` "null-grid" algorithm with REML due to a typo. + - Fixed a bug causing output dimension mismatch when using the function `scan()` for permutation testing with adjusted covariates. + ## Version 1.1.1 (July 11, 2023) - Fixed bugs: - REML option in multiple trait scan functions ("bulkscan"'s) used to lead to compilation errors. diff --git a/Project.toml b/Project.toml index a7a06c2..17f45d0 100644 --- a/Project.toml +++ b/Project.toml @@ -1,7 +1,7 @@ name = "BulkLMM" uuid = "b8d15608-0852-4141-ae38-222578e2ed7b" authors = ["Zifan Yu, Gregory Farage, Saunak Sen"] -version = "1.1.1" +version = "1.2.0" [deps] CSV = "336ed68f-0bac-5ca0-87d4-7b16caf5d00b" diff --git a/README.md b/README.md index eec89ca..10a62ec 100644 --- a/README.md +++ b/README.md @@ -184,7 +184,8 @@ kinship = round.(kinship; digits = 12); For example, to conduct genome-wide associations mapping on the 1112-th trait, we can run the function `scan()` with inputs of the trait (as -a 2D-array of one column), geno matrix, and the kinship matrix. +a 2D-array of one column), geno matrix, and the kinship matrix. Type `?scan()` for more +detailed description of the function. ```julia @@ -308,28 +309,41 @@ multi-threads](https://docs.julialang.org/en/v1/manual/multi-threading/) or switch to a multi-threaded *julia* kernel if using Jupyter notebooks. -Then, run the function `bulkscan_null()` with the matrices of -traits, genome markers, kinship. The fourth required input is the -number of parallelized tasks and we recommend it to be the number of -*julia* threads. +Then, run the function `bulkscan()` with the matrices of the +traits of interest, genome markers, and the kinship. Type `?bulkscan()` for more +detailed description of the function. -Here, we started a 16-threaded *julia* and executed the program on a -Linux server with the Intel(R) Xeon(R) Silver 4214 CPU @ 2.20GHz to -get the LOD scores for all **~35k** BXD traits: +Here, we started a 16-threaded *julia* session in julia version 1.9.2. Specific session info +is as follows: +```julia +versioninfo() +``` + + Julia Version 1.9.2 + Commit e4ee485e909 (2023-07-05 09:39 UTC) + Platform Info: + OS: Linux (x86_64-linux-gnu) + CPU: 48 × Intel(R) Xeon(R) Silver 4214 CPU @ 2.20GHz + WORD_SIZE: 64 + LIBM: libopenlibm + LLVM: libLLVM-14.0.6 (ORCJIT, cascadelake) + Threads: 17 on 48 virtual cores + Environment: + JULIA_NUM_THREADS = 16 ```julia -@time multiple_results_allTraits = bulkscan_null(pheno_processed, geno_processed, kinship; nb = Threads.nthreads()); +@time multiple_results_allTraits = bulkscan(pheno_processed, geno_processed, kinship); ``` - 82.421037 seconds (2.86 G allocations: 710.821 GiB, 41.76% gc time) + 2.112011 seconds (107.94 k allocations: 5.053 GiB, 2.59% gc time) +Please Note: the default method and modeling options for `bulkscan()` takes an approximated approach for +the best runtime performance. The user may choose to use other methods and options provided for more precision but longer runtime, following the detailed instructions in `?bulkscan()`. The output `multiple_results_allTraits` is an object containing our model results: - the matrix of LOD scores $L_{p \times m}$, where $p$ is the number of markers and $m$ is number of traits; each column corresponds to the LOD scores resulting from performing GWAS on each given trait. -- the vector of heritability estimate per trait, `h2_null_list`, obtained from fitting the null model. - -Similarly as the single trait scan function `scan()`, variance components are estimated from maximum-likelihood (ML) by default ("reml = false"). The user may choose REML for estimating by specifying in the input "reml = true". +- variance components (heritability) results will be returned in various formats depending on the specific method and other options by the user. For more details, enter `?bulkscan()`. ```julia size(multiple_results_allTraits.L) @@ -337,12 +351,6 @@ size(multiple_results_allTraits.L) (7321, 35554) -```julia -length(multiple_results_allTraits.h2_null_list) -``` - - 35554 - To visualize the multiple-trait scan results, we can use the plotting function `plot_eQTL` from `BigRiverQTLPlots.jl` to generate the eQTL plot. In the following example, we only plot the LOD scores that are above 5.0 by calling the function and specifying in the optional argument `threshold = 5.0`: diff --git a/src/BulkLMM.jl b/src/BulkLMM.jl index 1052c5e..ab7d3b5 100644 --- a/src/BulkLMM.jl +++ b/src/BulkLMM.jl @@ -6,6 +6,8 @@ module BulkLMM using Random, Distributions include("./util.jl"); + export p2lod, lod2p, lod2log10p + include("./kinship.jl"); export calcKinship @@ -36,10 +38,10 @@ module BulkLMM include("./bulkscan_helpers.jl"); include("./bulkscan.jl"); - export bulkscan_null, bulkscan_null_grid, bulkscan_alt_grid + export bulkscan, bulkscan_null, bulkscan_null_grid, bulkscan_alt_grid include("./transform_helpers.jl"); - # export transform_rotation + export transform_rotation include("./analysis_helpers/single_trait_analysis.jl"); export LODthresholds, get_thresholds, getLL, plotLL diff --git a/src/analysis_helpers/single_trait_analysis.jl b/src/analysis_helpers/single_trait_analysis.jl index c9515fb..95cc5d6 100644 --- a/src/analysis_helpers/single_trait_analysis.jl +++ b/src/analysis_helpers/single_trait_analysis.jl @@ -28,7 +28,9 @@ end ## Outputs: the logliks (null, alt mean model) under the given h2 function getLL(y0::Array{Float64, 2}, X0::Array{Float64, 2}, lambda0::Array{Float64, 1}, num_of_covar::Int64, - markerID::Int64, h2::Float64; prior::Array{Float64, 1} = [0.0, 0.0]) + markerID::Int64, h2::Float64; + prior::Array{Float64, 1} = [0.0, 0.0], + reml::Bool = false) n = size(y0, 1); w = makeweights(h2, lambda0); @@ -43,13 +45,14 @@ function getLL(y0::Array{Float64, 2}, X0::Array{Float64, 2}, lambda0::Array{Floa X_design[:, 1:num_of_covar] = X0_covar; X_design[:, num_of_covar+1] = X0[:, markerID+num_of_covar]; - return (ll_null = wls(y0, X0_covar, w, prior).ell, ll_markerID = wls(y0, X_design, w, prior).ell) + return (ll_null = wls(y0, X0_covar, w, prior; reml = reml).ell, + ll_markerID = wls(y0, X_design, w, prior; reml = reml).ell) end -function profileLL(y::Array{Float64, 2}, G::Array{Float64, 2}, covar::Array{Float64, 2}, +function profile_LL(y::Array{Float64, 2}, G::Array{Float64, 2}, covar::Array{Float64, 2}, K::Array{Float64, 2}, h2_grid::Array{Float64, 1}, markerID::Int64; - prior::Array{Float64, 1} = [0.0, 0.0]) + prior::Array{Float64, 1} = [0.0, 0.0], reml::Bool = false) ## Initiate the vector to store the profile likelihood values evaluated under each given parameter value ell_null = zeros(length(h2_grid)); # loglikelihood under null @@ -62,7 +65,7 @@ function profileLL(y::Array{Float64, 2}, G::Array{Float64, 2}, covar::Array{Floa ## Loop through the supplied h2 values, evaluate the profile loglik under each h2 for k in 1:length(h2_grid) curr_h2 = h2_grid[k]; - output = getLL(y0, X0, lambda0, num_of_covar, markerID, curr_h2; prior = prior); + output = getLL(y0, X0, lambda0, num_of_covar, markerID, curr_h2; prior = prior, reml = reml); ell_null[k] = output.ll_null; ell_alt[k] = output.ll_markerID; end diff --git a/src/bulkscan.jl b/src/bulkscan.jl index e3fb192..d41457b 100644 --- a/src/bulkscan.jl +++ b/src/bulkscan.jl @@ -2,6 +2,164 @@ # Genome scan functions for multiple traits: # allow modeling additional covariates, two genotype groups ########################################################### +""" +bulkscan(Y, G, K; optional inputs) +bulkscan(Y, G, Z, K; optional inputs) - if modeling additional covariates Z + +Perform genome scan for multiple univariate traits and a set of genome markers + +# Required Inputs +- `Y::Array{Float64, 2}`: Matrix of multiple (m) traits; each column is a trait (dimension: N*m) +- `G::Array{Float64, 2}`: Matrix of genotype probabilities at p tested markers (dimension: N*p) +- `K::Array{Float64, 2}`: Genetic relatedness matrix of the N subjects (dimension:N*N) + +# Optional Inputs + +## Essential Inputs: +- `addIntercept::Bool`: Option to add an intercept column to the design matrix (default: true) +- `reml::Bool`: Option of the scheme for estimating variance components (by REML or ML; default: false) +- `method::String`: Keyword argument indicating which multi-trait scan method will be used; currently supported + options: "null-grid" (fastest, grid-search approximated Null-LMM), "null-exact" (Null-LMM), and "alt-grid" + (grid-search approximated Exact-LMM) +- `output_pvals::Bool`: Option to additionally report the LRT p-values (default: false) + +## Modeling Additional Covariates: +- `Z::AbstractArray{Float64, 2}`: Matrix of additional non-genetic covariates (should be independent to tested + markers) + +## Different Optional Inputs by the Method Chosen: + For the multiple-trait scans, the user may choose to apply one of the three methods, depending on + the need for gaining more precision or waiting for shorter time. The allowed inputs differ by the choice: + +### Grid-search approximation method: "null-grid" (default method) and "alt-grid" +- `h2_grid::Array{Float64, 1}`: a grid of h2-values in [0, 1) where the optimization of profile likelihood + is performed; the finer the grid the better precision but longer wait-time. (default: 0.0:0.10:0.90, 10 values) + +### Null-LMM through exact optimization (the Brent's method), multi-threaded: "null-exact" +- `nb::Int64`: The number of sub-groups of the total number of traits; each group is processed independently and + the processes are parallelized (default: the number of threads of the current Julia session) +- `nt_blas::Int64`: The number of threads BLAS library will be using (default: 1) + +## Permutation Testing: + Currently permutation testing is only supported for single-trait scans. + +## Structure of Weighted Residual Variances: +- `weights::Array{Float64, 1}`: Optional weights for modeling unequal, weighted structure of the residual variances + of the trait (default: Missing, i.e. equal residual variances) + +## Numerical Techniques - for stabilizing the heritability optimization step +- `optim_interval::Int64`: The number of sub-divided regions of the interval [0, 1) of heritability to perform each + numerical optimization scheme (the Brent's method) on (default: 1, i.e. the entire interval) +- `prior_variance::Float64`: Scale parameter of the prior Scaled Inv-Chisq distributed residual variances (default: 0) +- `prior_sample_size::Float64`: Degree of freedom parameter of the prior Scaled Inv-Chisq distributed residual + variances (default: 0) +- `decomp_scheme::String`: Keyword indicating the decomposition scheme for the kinship matrix; either by "eigen" + or "svd" decomposition (default: "eigen") + +# Returned Values: + +The output of the single-trait scan function is an object. Depending on the user inputs and options, the fields of + the output object will differ. For example, for the returned output named as `MT_out` as "multiple traits + outputs": + +## Null-LMM ("null-grid", "null-exact"): +- `MT_out.h2_null_list::Array{Float64, 1}`: a list of h2_null estimated for each trait +- `MT_out.L::Array{Float64, 2}`: 2-dimensional array (dimension: p*m) consisting of the LOD scores for all input traits; each column + contains the LOD scores for one trait + +## Exact-LMM ("alt-grid"): +- `MT_out.h2_panel::Array{Float64, 2}`: 2-dimensional array (dimension: p*m) h2 estimated for each marker and + each trait; each column contains the h2 estimated for each marker for one trait +- `MT_out.L::Array{Float64, 2}`: 2-dimensional array (dimension: p*m) consisting of the LOD scores for all input traits; each column + contains the LOD scores for one trait + +## If the option for reporting p-values is on, the p-values results will be returned as: +- `MT_out.log10Pvals_mat::Array{Float64, 2}`: 2-dimensional array (dimension: p*m) consisting of the -log10(p-values) +for each test (of association between each trait and each marker). + +""" +function bulkscan(Y::Array{Float64, 2}, G::Array{Float64, 2}, K::Array{Float64, 2}; + method::String = "null-grid", h2_grid::Array{Float64, 1} = collect(0.0:0.1:0.9), + nb::Int64 = Threads.nthreads(), + nt_blas::Int64 = 1, + weights::Union{Missing, Array{Float64, 1}} = missing, + prior_variance::Float64 = 1.0, prior_sample_size::Float64 = 0.0, + reml::Bool = false, optim_interval::Int64 = 1, + # option for kinship decomposition scheme: + decomp_scheme::String = "eigen", + # option for returning p-values results: + output_pvals::Bool = false, chisq_df::Int64 = 1 + ) + + n = size(Y, 1); + + # when no covariates are added, make the intercept as the only covariate + intercept = ones(n, 1); + + return bulkscan(Y, G, intercept, K; + method = method, + h2_grid = h2_grid, + nb = nb, nt_blas = nt_blas, + # key step: avoid adding the intercept twice + addIntercept = false, + weights = weights, + prior_variance = prior_variance, prior_sample_size = prior_sample_size, + reml = reml, optim_interval = optim_interval, + decomp_scheme = decomp_scheme, output_pvals = output_pvals, chisq_df = chisq_df) + + +end + +function bulkscan(Y::Array{Float64, 2}, G::Array{Float64, 2}, Covar::Array{Float64, 2}, K::Array{Float64, 2}; + method::String = "null-grid", h2_grid::Array{Float64, 1} = collect(0.0:0.1:0.9), + nb::Int64 = Threads.nthreads(), + nt_blas::Int64 = 1, + addIntercept::Bool = true, + weights::Union{Missing, Array{Float64, 1}} = missing, + prior_variance::Float64 = 1.0, prior_sample_size::Float64 = 0.0, + reml::Bool = false, optim_interval::Int64 = 1, + # option for kinship decomposition scheme: + decomp_scheme::String = "eigen", + # option for returning p-values results: + output_pvals::Bool = false, chisq_df::Int64 = 1) + + if method == "null-exact" + bulkscan_results = bulkscan_null(Y, G, Covar, K; + nb = nb, nt_blas = nt_blas, + addIntercept = addIntercept, + weights = weights, + prior_variance = prior_variance, prior_sample_size = prior_sample_size, + reml = reml, optim_interval = optim_interval, + decomp_scheme = decomp_scheme); + end + + if method == "null-grid" + bulkscan_results = bulkscan_null_grid(Y, G, Covar, K, h2_grid; + weights = weights, + addIntercept = addIntercept, + prior_variance = prior_variance, prior_sample_size = prior_sample_size, + reml = reml, + decomp_scheme = decomp_scheme); + end + + if method == "alt-grid" + bulkscan_results = bulkscan_alt_grid(Y, G, Covar, K, h2_grid; + weights = weights, + addIntercept = addIntercept, + prior_variance = prior_variance, prior_sample_size = prior_sample_size, + reml = reml, + decomp_scheme = decomp_scheme); + end + + if output_pvals + log10Pvals_mat = lod2log10p.(bulkscan_results.L, chisq_df); + temp_tuple = (log10Pvals_mat = log10Pvals_mat, Chisq_df = chisq_df); + return merge(bulkscan_results, temp_tuple) + else + return bulkscan_results + end + +end ########################################################### ## (1) Chunk methods: @@ -32,7 +190,8 @@ function bulkscan_null(Y::Array{Float64, 2}, G::Array{Float64, 2}, K::Array{Floa nt_blas::Int64 = 1, weights::Union{Missing, Array{Float64, 1}} = missing, prior_variance::Float64 = 1.0, prior_sample_size::Float64 = 0.0, - reml::Bool = false, optim_interval::Int64 = 1) + reml::Bool = false, optim_interval::Int64 = 1, + decomp_scheme::String = "eigen") n = size(Y, 1); @@ -45,7 +204,8 @@ function bulkscan_null(Y::Array{Float64, 2}, G::Array{Float64, 2}, K::Array{Floa addIntercept = false, weights = weights, prior_variance = prior_variance, prior_sample_size = prior_sample_size, - reml = reml, optim_interval = optim_interval); + reml = reml, optim_interval = optim_interval, + decomp_scheme = decomp_scheme); end ### Modeling covariates version @@ -55,7 +215,8 @@ function bulkscan_null(Y::Array{Float64, 2}, G::Array{Float64, 2}, addIntercept::Bool = true, weights::Union{Missing, Array{Float64, 1}} = missing, prior_variance::Float64 = 1.0, prior_sample_size::Float64 = 0.0, - reml::Bool = false, optim_interval::Int64 = 1) + reml::Bool = false, optim_interval::Int64 = 1, + decomp_scheme::String = "eigen") m = size(Y, 2); @@ -91,7 +252,8 @@ function bulkscan_null(Y::Array{Float64, 2}, G::Array{Float64, 2}, BLAS.set_num_threads(nt_blas); # rotate data - (Y0, X0, lambda0) = transform_rotation(Y_st, [Covar_st G_st], K_st; addIntercept = addIntercept); + (Y0, X0, lambda0) = transform_rotation(Y_st, [Covar_st G_st], K_st; + addIntercept = addIntercept, decomp_scheme = decomp_scheme); X0_intercept = X0[:, 1:num_of_covar]; @@ -160,7 +322,8 @@ function bulkscan_null_grid(Y::Array{Float64, 2}, G::Array{Float64, 2}, K::Array{Float64, 2}, grid_list::Array{Float64, 1}; weights::Union{Missing, Array{Float64, 1}} = missing, prior_variance::Float64 = 1.0, prior_sample_size::Float64 = 0.0, - reml::Bool = false) + reml::Bool = false, + decomp_scheme::String = "eigen") n = size(Y, 1); @@ -170,7 +333,8 @@ function bulkscan_null_grid(Y::Array{Float64, 2}, G::Array{Float64, 2}, weights = weights, addIntercept = false, prior_variance = prior_variance, prior_sample_size = prior_sample_size, - reml = reml); + reml = reml, + decomp_scheme = decomp_scheme); end function bulkscan_null_grid(Y::Array{Float64, 2}, G::Array{Float64, 2}, Covar::Array{Float64, 2}, @@ -178,7 +342,8 @@ function bulkscan_null_grid(Y::Array{Float64, 2}, G::Array{Float64, 2}, Covar::A weights::Union{Missing, Array{Float64, 1}} = missing, addIntercept::Bool = true, prior_variance::Float64 = 1.0, prior_sample_size::Float64 = 0.0, - reml::Bool = false) + reml::Bool = false, + decomp_scheme::String = "eigen") m = size(Y, 2); p = size(G, 2); @@ -207,7 +372,8 @@ function bulkscan_null_grid(Y::Array{Float64, 2}, G::Array{Float64, 2}, Covar::A results_by_bin = gridscan_by_bin(Y_st, G_st, Covar_st, K_st, grid_list; addIntercept = addIntercept, prior_variance = prior_variance, prior_sample_size = prior_sample_size, - reml = reml); + reml = reml, + decomp_scheme = decomp_scheme); LOD_grid = reorder_results(results_by_bin.idxs_by_bin, results_by_bin.LODs_by_bin, m, p); @@ -263,7 +429,8 @@ function bulkscan_alt_grid(Y::Array{Float64, 2}, G::Array{Float64, 2}, K::Array{ hsq_list::Array{Float64, 1}; reml::Bool = false, prior_variance::Float64 = 1.0, prior_sample_size::Float64 = 0.0, - weights::Union{Missing, Array{Float64, 1}} = missing) + weights::Union{Missing, Array{Float64, 1}} = missing, + decomp_scheme::String = "eigen") n = size(Y, 1); intercept = ones(n, 1); @@ -271,7 +438,7 @@ function bulkscan_alt_grid(Y::Array{Float64, 2}, G::Array{Float64, 2}, K::Array{ return bulkscan_alt_grid(Y, G, intercept, K, hsq_list; reml = reml, prior_variance = prior_variance, prior_sample_size = prior_sample_size, - weights = weights, addIntercept = false); + weights = weights, addIntercept = false, decomp_scheme = decomp_scheme); end @@ -280,7 +447,8 @@ function bulkscan_alt_grid(Y::Array{Float64, 2}, G::Array{Float64, 2}, reml::Bool = false, prior_variance::Float64 = 1.0, prior_sample_size::Float64 = 0.0, weights::Union{Missing, Array{Float64, 1}} = missing, - addIntercept::Bool = true) + addIntercept::Bool = true, + decomp_scheme::String = "eigen") p = size(G, 2); @@ -307,7 +475,8 @@ function bulkscan_alt_grid(Y::Array{Float64, 2}, G::Array{Float64, 2}, K_st = K; end - (Y0, X0, lambda0) = transform_rotation(Y_st, [Covar_st G_st], K_st; addIntercept = addIntercept); + (Y0, X0, lambda0) = transform_rotation(Y_st, [Covar_st G_st], K_st; + addIntercept = addIntercept, decomp_scheme = decomp_scheme); if addIntercept == true num_of_covar = size(Covar, 2)+1; diff --git a/src/bulkscan_helpers.jl b/src/bulkscan_helpers.jl index 0c9b434..e2ce953 100644 --- a/src/bulkscan_helpers.jl +++ b/src/bulkscan_helpers.jl @@ -241,7 +241,8 @@ function gridscan_by_bin(pheno::Array{Float64, 2}, geno::Array{Float64, 2}, grid::Array{Float64, 1}; addIntercept::Bool = true, prior_variance = 1.0, prior_sample_size = 0.0, - reml::Bool = false + reml::Bool = false, + decomp_scheme::String = "eigen" ) m = size(pheno, 2); @@ -249,7 +250,8 @@ function gridscan_by_bin(pheno::Array{Float64, 2}, geno::Array{Float64, 2}, # Y_std = colStandardize(pheno); Y_std = pheno; - (Y0, X0, lambda0) = transform_rotation(Y_std, [covar geno], kinship; addIntercept = addIntercept); + (Y0, X0, lambda0) = transform_rotation(Y_std, [covar geno], kinship; + addIntercept = addIntercept, decomp_scheme = decomp_scheme); prior = [prior_variance, prior_sample_size]; diff --git a/src/scan.jl b/src/scan.jl index 57cc323..4ca0db4 100644 --- a/src/scan.jl +++ b/src/scan.jl @@ -1,41 +1,153 @@ ########################################################### # Genome scan functions for a single trait plus permutation testing: -# allow modeling additional covariates, two genotype groups +# allow modeling of additional non-genetic covariates, two genotype groups ########################################################### """ - scan(y, g, K, reml, method) + scan(y, G, K; optional inputs) + scan(y, G, Z, K; optional inputs) - if modeling additional covariates Z + +Perform genome scan for univariate trait and a set of genome markers + +# Required Inputs + +- `y::Array{Float64, 2} or Array{Float64, 1}`: Single univariate quantitative trait of N measurements (dimension: N*1) +- `G::Array{Float64, 2}`: Matrix of genotype probabilities at p tested markers (dimension: N*p) +- `K::Array{Float64, 2}`: Genetic relatedness matrix of the N subjects (dimension:N*N) + +# Optional Inputs + +## Essential Inputs: +- `addIntercept::Bool`: Option to add an intercept column to the design matrix (default: true) +- `reml::Bool`: Option of the scheme for estimating variance components (by REML or ML; default: false) +- `assumption::String`: Keyword argument indicating whether to estimate the variance components independently + for each marker ("alt") or to estimate once for the null model and use for testing all markers ("null) + (default: "null") +- `output_pvals::Bool`: Option to additionally report the LRT p-values (default: false) + +## Modeling Additional Covariates: +- `Z::AbstractArray{Float64, 2}`: Matrix of additional non-genetic covariates (should be independent to tested + markers) + +## Permutation Testing: +- `permutation_test::Bool`: Option to perform permutation testing on the studied single trait (default: false) +- `nperms::Int64`: The number of permutations required, an integer (default: 1024) +- `rndseed::Int64`: An integer random seed set for performing random shuffling of the original trait + (default: 0) + +## Structure of Weighted Residual Variances: +- `weights::Array{Float64, 1}`: Optional weights for modeling unequal, weighted structure of the residual variances + of the trait (default: Missing, i.e. equal residual variances) + +## Numerical Techniques - for stabilizing the heritability optimization step +- `optim_interval::Int64`: The number of sub-divided regions of the interval [0, 1) of heritability to perform each + numerical optimization scheme (the Brent's method) on (default: 1, i.e. the entire interval) +- `prior_variance::Float64`: Scale parameter of the prior Scaled Inv-Chisq distributed residual variances (default: 0) +- `prior_sample_size::Float64`: Degree of freedom parameter of the prior Scaled Inv-Chisq distributed residual + variances (default: 0) + +## Examining Profile Likelihood - as a function of the heritability estimates +- `ProfileLL::Bool`: Option to return values of the profile likelihood function under different h2 values + (default: false) +- `markerID::Int64`: The ID of the marker of interest +- `h2_grid::Array{Float64, 1}`: Different values of h2 for calculating the corresponding profile likelihood values + (default: an empty array) + +## Other Inputs: +- `method::String`: Keyword indicating the matrix factorization scheme for model evaluation; either by "qr" or + "cholesky" decomposition (default: "qr") +- `decomp_scheme::String`: Keyword indicating the decomposition scheme for the kinship matrix; either by "eigen" + or "svd" decomposition (default: "eigen") + +# Returned Values: + +The output of the single-trait scan function is an object. Depending on the user inputs and options, the fields of + the output object will differ. For example, for the returned output named as `out`: + +## Null-LMM: by the "Null" approximation of the h2 value and applied to testing all markers: +- `out.sigma2_e::Float64`: Estimated residual unexplained variances from the null model +- `out.h2_null::Float64`: Estimated heritability (h2) from the null model +- `out.lod::Array{Float64, 1}`: 1-dimensional array consisting of the LOD scores + +## Exact-LMM: by re-estimating the h2 and sigma2_e independently while testing each marker: +- `out.sigma2_e::Float64`: Estimated residual unexplained variances from the null model +- `out.h2_null::Float64`: Estimated heritability (h2) from the null model +- `out.h2_each_marker::Array{Float64, 1}`: 1-dimensional array of the estimated heritability for each marker model +- `out.lod::Array{Float64, 1}`: 1-dimensional array consisting of the LOD scores + +## Null-LMM and when permutation testing is required: +- `out.sigma2_e::Float64`: Estimated residual unexplained variances from the null model +- `out.h2_null::Float64`: Estimated heritability (h2) from the null model +- `out.lod::Array{Float64, 1}`: 1-dimensional array consisting of the LOD scores +- `out.L_perms::Array{Float64, 2}`: 2-dimensional array of the LOD scores from permutation testing; each column + is a vector of length p of p LOD scores for each permuted copy. + +## If the option for reporting p-values is on, the p-values results will be returned as: +- `out.log10pvals::Array{Float64, 1}`: 1-dimensional array consisting of the -log10(p-values) +- `out.log10Pvals_perms::Array{Float64, 2}`: 2-dimensional array consisting of the -log10(p-values) for each test +(for testing the association between each marker and each permuted trait). + +## Additionally, if the user wants to examine the profile likelihood values under a given set of h2-values: +- `out.ll_list_null::Array{Float64, 1}`: gives the values under the null model under each h2-value +- `out.ll_list_alt::Array{Float64, 1}`: gives the values under the user-specified marker model under each h2-value -Performs genome scan for univariate trait and each of the gene markers, one marker at a time (one-df test) - -# Arguments - -- y = 1d array of floats consisting of the N observations for a certain quantitative trait (dimension: N*1) -- g = 2d array of floats consisting of all p gene markers (dimension: N*p) -- K = 2d array of floats consisting of the genetic relatedness of the N observations (dimension:N*N) -- prior_a = a float of prior distribution parameter -- prior_b = a float of prior distribution parameter - -# Keyword arguments +""" +function scan(y::Array{Float64, 1}, g::Array{Float64, 2}, K::Array{Float64, 2}; + # weighted environmental variances: + weights::Union{Missing, Array{Float64, 1}} = missing, + # regularization options: + prior_variance::Float64 = 0.0, prior_sample_size::Float64 = 0.0, addIntercept::Bool = true, + # vc estimation options: + reml::Bool = false, assumption::String = "null", method::String = "qr", optim_interval::Int64 = 1, + # permutation testing options: + permutation_test::Bool = false, nperms::Int64 = 1024, rndseed::Int64 = 0, + # option for inspecting h2 estimation process: + profileLL::Bool = false, markerID::Int = 0, h2_grid::Array{Float64, 1} = Array{Float64, 1}(undef, 1), + # option for kinship decomposition scheme: + decomp_scheme::String = "eigen", + # option for returning p-values results: + output_pvals::Bool = false, chisq_df::Int64 = 1 + ) -- addIntercept = Boolean flag indicating if an intercept column needs to be added to the design matrix; Default is to add an intercept -- reml = Bool flag indicating if VCs are estimated by REML likelihood; Default is ML -- assumption = String indicating whether the variance component parameters are the same across all markers (null) or not (alt); Default is `null` -- method = String indicating the matrix factorization method to use; Default is QR. + return scan(reshape(y, :, 1), g, K; + weights = weights, + addIntercept = addIntercept, + prior_variance = prior_variance, prior_sample_size = prior_sample_size, + reml = reml, assumption = assumption, method = method, optim_interval = optim_interval, + permutation_test = permutation_test, nperms = nperms, rndseed = rndseed, + profileLL = profileLL, markerID = markerID, h2_grid = h2_grid, + decomp_scheme = decomp_scheme, output_pvals = output_pvals, chisq_df = chisq_df) -# Value +end -A list of output values are returned: -- out00.sigma2 = Float; estimated marginal variance due to random errors (by null lmm) -- out00.h2 = Float; estimated heritability (by null lmm) -- lod = 1d array of floats consisting of the lod scores of this trait and all markers (dimension: p*1) +function scan(y::Array{Float64, 1}, g::Array{Float64, 2}, covar::Array{Float64, 2}, K::Array{Float64, 2}; + # weighted environmental variances: + weights::Union{Missing, Array{Float64, 1}} = missing, + # regularization options: + prior_variance::Float64 = 0.0, prior_sample_size::Float64 = 0.0, addIntercept::Bool = true, + # vc estimation options: + reml::Bool = false, assumption::String = "null", method::String = "qr", optim_interval::Int64 = 1, + # permutation testing options: + permutation_test::Bool = false, nperms::Int64 = 1024, rndseed::Int64 = 0, + # option for inspecting h2 estimation process: + profileLL::Bool = false, markerID::Int = 0, h2_grid::Array{Float64, 1} = Array{Float64, 1}(undef, 1), + # option for kinship decomposition scheme: + decomp_scheme::String = "eigen", + # option for returning p-values results: + output_pvals::Bool = false, chisq_df::Int64 = 1 + ) -# Some notes + return scan(reshape(y, :, 1), g, covar, K; + weights = weights, + addIntercept = addIntercept, + prior_variance = prior_variance, prior_sample_size = prior_sample_size, + reml = reml, assumption = assumption, method = method, optim_interval = optim_interval, + permutation_test = permutation_test, nperms = nperms, rndseed = rndseed, + profileLL = profileLL, markerID = markerID, h2_grid = h2_grid, + decomp_scheme = decomp_scheme, output_pvals = output_pvals, chisq_df = chisq_df) - This function calls either `scan_null` or `scan_alt` depending on the input passed as `method`. - Output data structure might need some revisions. +end -""" -function scan(y::Array{Float64,2}, g::Array{Float64,2}, K::Array{Float64,2}; +function scan(y::Array{Float64, 2}, g::Array{Float64, 2}, K::Array{Float64, 2}; # weighted environmental variances: weights::Union{Missing, Array{Float64, 1}} = missing, # regularization options: @@ -46,7 +158,10 @@ function scan(y::Array{Float64,2}, g::Array{Float64,2}, K::Array{Float64,2}; permutation_test::Bool = false, nperms::Int64 = 1024, rndseed::Int64 = 0, # option for inspecting h2 estimation process: profileLL::Bool = false, markerID::Int = 0, h2_grid::Array{Float64, 1} = Array{Float64, 1}(undef, 1), - x_lims::Array{Float64, 1} = [0.0, 1.0], y_lims::Array{Float64, 1} = [-100.0, 100.0] + # option for kinship decomposition scheme: + decomp_scheme::String = "eigen", + # option for returning p-values results: + output_pvals::Bool = false, chisq_df::Int64 = 1 ) if addIntercept == false @@ -61,7 +176,7 @@ function scan(y::Array{Float64,2}, g::Array{Float64,2}, K::Array{Float64,2}; reml = reml, assumption = assumption, method = method, optim_interval = optim_interval, permutation_test = permutation_test, nperms = nperms, rndseed = rndseed, profileLL = profileLL, markerID = markerID, h2_grid = h2_grid, - x_lims = x_lims, y_lims = y_lims) + decomp_scheme = decomp_scheme, output_pvals = output_pvals, chisq_df = chisq_df) end function scan(y::Array{Float64,2}, g::Array{Float64,2}, covar::Array{Float64, 2}, K::Array{Float64,2}; @@ -75,7 +190,10 @@ function scan(y::Array{Float64,2}, g::Array{Float64,2}, covar::Array{Float64, 2} permutation_test::Bool = false, nperms::Int64 = 1024, rndseed::Int64 = 0, # option for inspecting h2 estimation process: profileLL::Bool = false, markerID::Int = 0, h2_grid::Array{Float64, 1} = Array{Float64, 1}(undef, 1), - x_lims::Array{Float64, 1} = [0.0, 1.0], y_lims::Array{Float64, 1} = [-100.0, 100.0] + # option for kinship decomposition scheme: + decomp_scheme::String = "eigen", + # option for returning p-values results: + output_pvals::Bool = false, chisq_df::Int64 = 1 ) n = size(y, 1); @@ -100,7 +218,7 @@ function scan(y::Array{Float64,2}, g::Array{Float64,2}, covar::Array{Float64, 2} reml = reml, assumption = assumption, method = method, optim_interval = optim_interval, permutation_test = permutation_test, nperms = nperms, rndseed = rndseed, profileLL = profileLL, markerID = markerID, h2_grid = h2_grid, - x_lims = x_lims, y_lims = y_lims) + decomp_scheme = decomp_scheme, output_pvals = output_pvals, chisq_df = chisq_df) else y_st = y; g_st = g; @@ -112,17 +230,20 @@ function scan(y::Array{Float64,2}, g::Array{Float64,2}, covar::Array{Float64, 2} if permutation_test == true results = scan_perms_lite(y_st, g_st, covar_st, K_st; prior_variance = prior_variance, prior_sample_size = prior_sample_size, addIntercept = addIntercept, reml = reml, method = method, optim_interval = optim_interval, - nperms = nperms, rndseed = rndseed); + nperms = nperms, rndseed = rndseed, + decomp_scheme = decomp_scheme, output_pvals = output_pvals); else results = scan_null(y_st, g_st, covar_st, K_st, [prior_variance, prior_sample_size], addIntercept; - reml = reml, method = method, optim_interval = optim_interval); + reml = reml, method = method, optim_interval = optim_interval, + decomp_scheme = decomp_scheme, output_pvals = output_pvals, chisq_df = chisq_df); end elseif assumption == "alt" if permutation_test == true error("Permutation test option currently is not supported for the alternative assumption."); else results = scan_alt(y_st, g_st, covar_st, K_st, [prior_variance, prior_sample_size], addIntercept; - reml = reml, method = method, optim_interval = optim_interval) + reml = reml, method = method, optim_interval = optim_interval, + decomp_scheme = decomp_scheme, output_pvals = output_pvals, chisq_df = chisq_df) end else @@ -139,8 +260,8 @@ function scan(y::Array{Float64,2}, g::Array{Float64,2}, covar::Array{Float64, 2} display(p) =# - results_profileLL = profileLL(y_st, g_st, covar_st, K_st, h2_grid, markerID; - prior = [prior_variance, prior_sample_size]); + results_profileLL = profile_LL(y_st, g_st, covar_st, K_st, h2_grid, markerID; + prior = [prior_variance, prior_sample_size], reml = reml); return (results, results_profileLL); else @@ -188,7 +309,10 @@ A list of output values are returned: """ function scan_null(y::Array{Float64, 2}, g::Array{Float64, 2}, covar::Array{Float64, 2}, K::Array{Float64, 2}, prior::Array{Float64, 1}, addIntercept::Bool; - reml::Bool = false, method::String = "qr", optim_interval::Int64 = 1) + reml::Bool = false, method::String = "qr", optim_interval::Int64 = 1, + decomp_scheme::String = "eigen", + # option for returning p-values results: + output_pvals::Bool = false, chisq_df::Int64 = 1) # number of markers (n, p) = size(g) @@ -196,7 +320,8 @@ function scan_null(y::Array{Float64, 2}, g::Array{Float64, 2}, covar::Array{Floa num_of_covar = addIntercept ? (size(covar, 2)+1) : size(covar, 2); # rotate data - (y0, X0, lambda0) = transform_rotation(y, [covar g], K; addIntercept = addIntercept) + (y0, X0, lambda0) = transform_rotation(y, [covar g], K; + addIntercept = addIntercept, decomp_scheme = decomp_scheme) X0_covar = X0[:, 1:num_of_covar]; if size(X0_covar, 2) == 1 @@ -225,7 +350,12 @@ function scan_null(y::Array{Float64, 2}, g::Array{Float64, 2}, covar::Array{Floa # lod[i] = lrt/(2*log(10)) end - return (sigma2_e = out00.sigma2, h2_null = out00.h2, lod = lod) + if output_pvals + log10pvals = lod2log10p.(lod, chisq_df); + return (sigma2_e = out00.sigma2, h2_null = out00.h2, lod = lod, log10pvals = log10pvals) + else + return (sigma2_e = out00.sigma2, h2_null = out00.h2, lod = lod) + end end @@ -266,7 +396,10 @@ A list of output values are returned: """ function scan_alt(y::Array{Float64, 2}, g::Array{Float64, 2}, covar::Array{Float64, 2}, K::Array{Float64, 2}, prior::Array{Float64, 1}, addIntercept::Bool; - reml::Bool = false, method::String = "qr", optim_interval::Int64 = 1) + reml::Bool = false, method::String = "qr", optim_interval::Int64 = 1, + decomp_scheme::String = "eigen", + # option for returning p-values results: + output_pvals::Bool = false, chisq_df::Int64 = 1) # number of markers (n, p) = size(g) @@ -274,7 +407,9 @@ function scan_alt(y::Array{Float64, 2}, g::Array{Float64, 2}, covar::Array{Float num_of_covar = addIntercept ? (size(covar, 2)+1) : size(covar, 2); # rotate data - (y0, X0, lambda0) = transform_rotation(y, [covar g], K; addIntercept = addIntercept) + (y0, X0, lambda0) = transform_rotation(y, [covar g], K; + addIntercept = addIntercept, + decomp_scheme = decomp_scheme) X0_covar = X0[:, 1:num_of_covar]; if size(X0_covar, 2) == 1 @@ -307,7 +442,12 @@ function scan_alt(y::Array{Float64, 2}, g::Array{Float64, 2}, covar::Array{Float pve_list[i] = out11.h2; end - return (sigma2_e = out00.sigma2, h2_null = out00.h2, h2_each_marker = pve_list, lod = lod); + if output_pvals + log10pvals = lod2log10p.(lod, chisq_df); + return (sigma2_e = out00.sigma2, h2_null = out00.h2, h2_each_marker = pve_list, lod = lod, log10pvals = log10pvals); + else + return (sigma2_e = out00.sigma2, h2_null = out00.h2, h2_each_marker = pve_list, lod = lod); + end end @@ -346,7 +486,10 @@ function scan_perms_lite(y::Array{Float64,2}, g::Array{Float64,2}, covar::Array{ prior_variance::Float64 = 1.0, prior_sample_size::Float64 = 0.0, addIntercept::Bool = true, method::String = "qr", optim_interval::Int64 = 1, nperms::Int64 = 1024, rndseed::Int64 = 0, - reml::Bool = false) + reml::Bool = false, + decomp_scheme::String = "eigen", + # option for returning p-values results: + output_pvals::Bool = false, chisq_df::Int64 = 1) # check the number of traits as this function only works for permutation testing of univariate trait @@ -367,10 +510,16 @@ function scan_perms_lite(y::Array{Float64,2}, g::Array{Float64,2}, covar::Array{ ## Note: estimate once the variance components from the null model and use for all marker scans # fit lmm - (y0, X0, lambda0) = transform_rotation(y, [covar g], K; addIntercept = addIntercept); # rotation of data + (y0, X0, lambda0) = transform_rotation(y, [covar g], K; + addIntercept = addIntercept, + decomp_scheme = decomp_scheme); # rotation of data + + # if the intercept is added, the number of covariates to be regressed out will be one more (the intercept) + n_covars = addIntercept ? (size(covar, 2)+1) : (size(covar, 2)); + (r0, X00, sigma2_e, h2_null) = transform_reweight(y0, X0, lambda0; - n_covars = size(covar, 2), + n_covars = n_covars, prior_a = prior_variance, prior_b = prior_sample_size, reml = reml, method = method, optim_interval = optim_interval); # reweighting and taking residuals @@ -396,7 +545,14 @@ function scan_perms_lite(y::Array{Float64,2}, g::Array{Float64,2}, covar::Array{ lod = L[:, 1]; # lod scores for the original trait; L_perms = L[:, 2:end]; # lod scores for the permuted copies of the original, excluding the lod scores for the original trait - return (sigma2_e = sigma2_e, h2_null = h2_null, lod = lod, L_perms = L_perms) + if output_pvals + log10pvals = lod2log10p.(lod, chisq_df); + log10Pvals_perms = lod2log10p.(L_perms, chisq_df); + return (sigma2_e = sigma2_e, h2_null = h2_null, lod = lod, log10pvals = pvals, + L_perms = L_perms, log10Pvals_perms = log10Pvals_perms) + else + return (sigma2_e = sigma2_e, h2_null = h2_null, lod = lod, L_perms = L_perms) + end end diff --git a/src/transform_helpers.jl b/src/transform_helpers.jl index 907b3b2..731ab5a 100644 --- a/src/transform_helpers.jl +++ b/src/transform_helpers.jl @@ -1,4 +1,6 @@ -function transform_rotation(y::Array{Float64, 2}, g::Array{Float64, 2}, K::Array{Float64, 2}; addIntercept::Bool = true) +function transform_rotation(y::Array{Float64, 2}, g::Array{Float64, 2}, K::Array{Float64, 2}; + addIntercept::Bool = true, + decomp_scheme::String = "eigen") # n - the sample sizes n = size(y, 1) @@ -16,20 +18,39 @@ function transform_rotation(y::Array{Float64, 2}, g::Array{Float64, 2}, K::Array X = g; # Safe; will not make in-place changes to X. end - ## Eigen-decomposition: - EF = eigen(K); - Ut = EF.vectors'; + if decomp_scheme == "eigen" + ## Eigen-decomposition: + EF = eigen(K); + Ut = EF.vectors'; - # return an error if there are any negative eigenvalues - if any(EF.values .< -1e-7) - # throw(error("Negative eigenvalues exist. The kinship matrix supplied may not be SPD.")); - @warn "Negative eigenvalues exist. The kinship matrix supplied may not be SPD." - end + # return an error if there are any negative eigenvalues + if any(EF.values .< -1e-7) + # throw(error("Negative eigenvalues exist. The kinship matrix supplied may not be SPD.")); + @warn "Negative eigenvalues exist. The kinship matrix supplied may not be SPD." + end + + # rotate data so errors are uncorrelated + + return Ut*y, Ut*X, EF.values - # rotate data so errors are uncorrelated + elseif decomp_scheme == "svd" + ## SV-decomposition: + SF = svd(K); + Ut = SF.Vt; - return Ut*y, Ut*X, EF.values + # return an error if there are any negative eigenvalues + if any(SF.S .< -1e-7) + # throw(error("Negative eigenvalues exist. The kinship matrix supplied may not be SPD.")); + @warn "Negative eigenvalues exist. The kinship matrix supplied may not be SPD." + end + + # rotate data so errors are uncorrelated + return Ut*y, Ut*X, SF.S + else + throw(error("Please choose either `eigen` or `svd` for decomposition of the kinship matrix.")) + end + end # Takes the rotated data, evaluates the VC estimators (only based on the intercept model) for weights calculation, and finally re-weights the input data. diff --git a/src/util.jl b/src/util.jl index 5cd0f7a..76a5e16 100644 --- a/src/util.jl +++ b/src/util.jl @@ -178,32 +178,29 @@ function shuffleVector(rng::AbstractRNG, x::Vector{Float64}, nshuffle::Int64; return xx end -## Compare two arrays and return the number of elements with the same indices of each array and are match -function compareValues(x_true::Array{Float64,1}, x::Array{Float64,1}, tolerance::Float64, threshold::Float64) - if size(x_true) != size(x) - throw(error("Dimention Mismatch! Must compare two arrays of same length!")) - end - - passes = falses(size(x_true)) - t_passes = falses(0) - for i in 1:size(x_true)[1] - e = abs(x[i]-x_true[i]) - if e <= tolerance - passes[i] = true - end - - if x[i] >= threshold - if e <= tolerance - push!(t_passes, true) - else - push!(t_passes,false) - end - end +function p2lod(pval::Float64, df::Int64) + + lrs = invlogcdf(Chisq(df), log(1-pval)) + lod = lrs/(2*log(10)) + + return lod - end - # pass_rate = sum(passes) / size(x_true)[1] - # pass_rate = sum(t_passes) / size(t_passes)[1] +end - return (sum(passes), size(t_passes)[1], sum(t_passes)) +function lod2p(lod::Float64, df::Int64) + + lrs = lod*2*log(10); + pval = ccdf(Chisq(df), lrs) + + return pval + +end +function lod2log10p(lod::Float64, df::Int64) + + lrs = lod*2*log(10); + logpval = logccdf(Chisq(df), lrs) + + return -logpval/log(10); + end diff --git a/test/Manifest.toml b/test/Manifest.toml index 36980fc..b951ceb 100644 --- a/test/Manifest.toml +++ b/test/Manifest.toml @@ -15,10 +15,10 @@ uuid = "56f22d72-fd6d-98f1-02f0-08ddc0907c33" uuid = "2a0f44e3-6c83-55bd-87e4-b1978d98bd5f" [[deps.CSV]] -deps = ["CodecZlib", "Dates", "FilePathsBase", "InlineStrings", "Mmap", "Parsers", "PooledArrays", "SentinelArrays", "Tables", "Unicode", "WeakRefStrings"] -git-tree-sha1 = "c5fd7cd27ac4aed0acf4b73948f0110ff2a854b2" +deps = ["CodecZlib", "Dates", "FilePathsBase", "InlineStrings", "Mmap", "Parsers", "PooledArrays", "PrecompileTools", "SentinelArrays", "Tables", "Unicode", "WeakRefStrings", "WorkerUtilities"] +git-tree-sha1 = "44dbf560808d49041989b8a96cae4cffbeb7966a" uuid = "336ed68f-0bac-5ca0-87d4-7b16caf5d00b" -version = "0.10.7" +version = "0.10.11" [[deps.Calculus]] deps = ["LinearAlgebra"] @@ -28,27 +28,27 @@ version = "0.5.1" [[deps.ChainRulesCore]] deps = ["Compat", "LinearAlgebra", "SparseArrays"] -git-tree-sha1 = "e7ff6cadf743c098e08fca25c91103ee4303c9bb" +git-tree-sha1 = "e30f2f4e20f7f186dc36529910beaedc60cfa644" uuid = "d360d2e6-b24c-11e9-a2a3-2a2ae2dbcce4" -version = "1.15.6" +version = "1.16.0" [[deps.ChangesOfVariables]] -deps = ["ChainRulesCore", "LinearAlgebra", "Test"] -git-tree-sha1 = "38f7a08f19d8810338d4f5085211c7dfa5d5bdd8" +deps = ["InverseFunctions", "LinearAlgebra", "Test"] +git-tree-sha1 = "2fba81a302a7be671aefe194f0525ef231104e7f" uuid = "9e997f8a-9a97-42d5-a9f1-ce6bfc15e2c0" -version = "0.1.4" +version = "0.1.8" [[deps.CodecZlib]] deps = ["TranscodingStreams", "Zlib_jll"] -git-tree-sha1 = "ded953804d019afa9a3f98981d99b33e3db7b6da" +git-tree-sha1 = "02aa26a4cf76381be7f66e020a3eddeb27b0a092" uuid = "944b1d66-785c-5afd-91f1-9de20f533193" -version = "0.7.0" +version = "0.7.2" [[deps.Compat]] deps = ["Dates", "LinearAlgebra", "UUIDs"] -git-tree-sha1 = "00a2cccc7f098ff3b66806862d275ca3db9e6e5a" +git-tree-sha1 = "4e88377ae7ebeaf29a047aa1ee40826e0b708a5d" uuid = "34da2185-b29b-5c13-b0c7-acf172513d20" -version = "4.5.0" +version = "4.7.0" [[deps.CompilerSupportLibraries_jll]] deps = ["Artifacts", "Libdl"] @@ -61,21 +61,21 @@ uuid = "a8cc5b0e-0ffa-5ad4-8c14-923d3ee1735f" version = "4.1.1" [[deps.DataAPI]] -git-tree-sha1 = "e08915633fcb3ea83bf9d6126292e5bc5c739922" +git-tree-sha1 = "8da84edb865b0b5b0100c0666a9bc9a0b71c553c" uuid = "9a962f9c-6df0-11e9-0e5d-c546b8b5ee8a" -version = "1.13.0" +version = "1.15.0" [[deps.DataFrames]] -deps = ["Compat", "DataAPI", "Future", "InvertedIndices", "IteratorInterfaceExtensions", "LinearAlgebra", "Markdown", "Missings", "PooledArrays", "PrettyTables", "Printf", "REPL", "Random", "Reexport", "SnoopPrecompile", "SortingAlgorithms", "Statistics", "TableTraits", "Tables", "Unicode"] -git-tree-sha1 = "d4f69885afa5e6149d0cab3818491565cf41446d" +deps = ["Compat", "DataAPI", "DataStructures", "Future", "InlineStrings", "InvertedIndices", "IteratorInterfaceExtensions", "LinearAlgebra", "Markdown", "Missings", "PooledArrays", "PrecompileTools", "PrettyTables", "Printf", "REPL", "Random", "Reexport", "SentinelArrays", "SortingAlgorithms", "Statistics", "TableTraits", "Tables", "Unicode"] +git-tree-sha1 = "04c738083f29f86e62c8afc341f0967d8717bdb8" uuid = "a93c6f00-e57d-5684-b7b6-d8193f3e46c0" -version = "1.4.4" +version = "1.6.1" [[deps.DataStructures]] deps = ["Compat", "InteractiveUtils", "OrderedCollections"] -git-tree-sha1 = "d1fff3a548102f48987a52a2e0d114fa97d730f0" +git-tree-sha1 = "cf25ccb972fec4e4817764d01c82386ae94f77b4" uuid = "864edb3b-99cc-5e75-8d2d-829cb0a9cfe8" -version = "0.18.13" +version = "0.18.14" [[deps.DataValueInterfaces]] git-tree-sha1 = "bfc1187b79289637fa0ef6d4436ebdfe6905cbd6" @@ -97,16 +97,16 @@ uuid = "b429d917-457f-4dbc-8f4c-0cc954292b1d" version = "0.4.0" [[deps.Distributions]] -deps = ["ChainRulesCore", "DensityInterface", "FillArrays", "LinearAlgebra", "PDMats", "Printf", "QuadGK", "Random", "SparseArrays", "SpecialFunctions", "Statistics", "StatsBase", "StatsFuns", "Test"] -git-tree-sha1 = "a7756d098cbabec6b3ac44f369f74915e8cfd70a" +deps = ["ChainRulesCore", "DensityInterface", "FillArrays", "LinearAlgebra", "PDMats", "Printf", "QuadGK", "Random", "SparseArrays", "SpecialFunctions", "Statistics", "StatsAPI", "StatsBase", "StatsFuns", "Test"] +git-tree-sha1 = "e76a3281de2719d7c81ed62c6ea7057380c87b1d" uuid = "31c24e10-a181-5473-b8eb-7969acd0382f" -version = "0.25.79" +version = "0.25.98" [[deps.DocStringExtensions]] deps = ["LibGit2"] -git-tree-sha1 = "c36550cb29cbe373e95b3f40486b9a4148f89ffd" +git-tree-sha1 = "2fb1e02f2b635d0845df5d7c167fec4dd739b00d" uuid = "ffbed154-4ef7-542d-bbb7-c09d3a79fcae" -version = "0.9.2" +version = "0.9.3" [[deps.Downloads]] deps = ["ArgTools", "FileWatching", "LibCURL", "NetworkOptions"] @@ -130,9 +130,9 @@ uuid = "7b1f6079-737a-58dc-b8bc-7a2ca5c1b5ee" [[deps.FillArrays]] deps = ["LinearAlgebra", "Random", "SparseArrays", "Statistics"] -git-tree-sha1 = "802bfc139833d2ba893dd9e62ba1767c88d708ae" +git-tree-sha1 = "f0af9b12329a637e8fba7d6543f915fff6ba0090" uuid = "1a297f60-69ca-5386-bcde-b61e274b549b" -version = "0.13.5" +version = "1.4.2" [[deps.Formatting]] deps = ["Printf"] @@ -151,16 +151,16 @@ uuid = "3f79f04f-7cac-48b4-bde1-3ad54d8f74fa" version = "0.2.1" [[deps.HypergeometricFunctions]] -deps = ["DualNumbers", "LinearAlgebra", "OpenLibm_jll", "SpecialFunctions", "Test"] -git-tree-sha1 = "709d864e3ed6e3545230601f94e11ebc65994641" +deps = ["DualNumbers", "LinearAlgebra", "OpenLibm_jll", "SpecialFunctions"] +git-tree-sha1 = "83e95aaab9dc184a6dcd9c4c52aa0dc26cd14a1d" uuid = "34004b35-14d8-5ef3-9330-4cdb6864b03a" -version = "0.3.11" +version = "0.3.21" [[deps.InlineStrings]] deps = ["Parsers"] -git-tree-sha1 = "0cf92ec945125946352f3d46c96976ab972bde6f" +git-tree-sha1 = "9cc2baf75c6d09f9da536ddf58eb2f29dedaf461" uuid = "842dd82b-1e85-43dc-bf29-5d0ee9dffc48" -version = "1.3.2" +version = "1.4.0" [[deps.InteractiveUtils]] deps = ["Markdown"] @@ -168,19 +168,19 @@ uuid = "b77e0a4c-d291-57a0-90e8-8db25a27a240" [[deps.InverseFunctions]] deps = ["Test"] -git-tree-sha1 = "49510dfcb407e572524ba94aeae2fced1f3feb0f" +git-tree-sha1 = "eabe3125edba5c9c10b60a160b1779a000dc8b29" uuid = "3587e190-3f89-42d0-90ee-14403ec27112" -version = "0.1.8" +version = "0.1.11" [[deps.InvertedIndices]] -git-tree-sha1 = "82aec7a3dd64f4d9584659dc0b62ef7db2ef3e19" +git-tree-sha1 = "0dc7b50b8d436461be01300fd8cd45aa0274b038" uuid = "41ab1584-1d38-5bbf-9106-f11c6c58b48f" -version = "1.2.0" +version = "1.3.0" [[deps.IrrationalConstants]] -git-tree-sha1 = "7fd44fd4ff43fc60815f8e764c0f352b83c49151" +git-tree-sha1 = "630b497eafcc20001bba38a4651b327dcfc491d2" uuid = "92d709cd-6900-40b7-9082-c6be49f344b6" -version = "0.1.1" +version = "0.2.2" [[deps.IteratorInterfaceExtensions]] git-tree-sha1 = "a3f24677c21f5bbe9d2a714f95dcd58337fb2856" @@ -226,9 +226,9 @@ uuid = "37e2e46d-f89d-539d-b4ee-838fcccc9c8e" [[deps.LogExpFunctions]] deps = ["ChainRulesCore", "ChangesOfVariables", "DocStringExtensions", "InverseFunctions", "IrrationalConstants", "LinearAlgebra"] -git-tree-sha1 = "946607f84feb96220f480e0422d3484c49c00239" +git-tree-sha1 = "c3ce8e7420b3a6e071e0fe4745f5d4300e37b13f" uuid = "2ab3a3ac-af41-5b50-aa03-7779005ae688" -version = "0.3.19" +version = "0.3.24" [[deps.Logging]] uuid = "56ddb016-857b-54e1-b83d-db4d58db5568" @@ -244,9 +244,9 @@ version = "2.28.0+0" [[deps.Missings]] deps = ["DataAPI"] -git-tree-sha1 = "bf210ce90b6c9eed32d25dbcae1ebc565df2687f" +git-tree-sha1 = "f66bdc5de519e8f8ae43bdc598782d35a25b1272" uuid = "e1d29d7a-bbdc-5cf2-9ac0-f12de2c33e28" -version = "1.0.2" +version = "1.1.0" [[deps.Mmap]] uuid = "a63ad114-7e13-5084-954f-fe012c677804" @@ -257,9 +257,9 @@ version = "2022.2.1" [[deps.NaNMath]] deps = ["OpenLibm_jll"] -git-tree-sha1 = "a7c3d1da1189a1c2fe843a3bfa04d18d20eb3211" +git-tree-sha1 = "0877504529a3e5c3343c6f8b4c0381e57e4387e4" uuid = "77ba4419-2d1f-58cd-9bb1-8ffee604a2e3" -version = "1.0.1" +version = "1.0.2" [[deps.NetworkOptions]] uuid = "ca575930-c2e3-43a9-ace4-1e988b2c1908" @@ -282,21 +282,21 @@ uuid = "efe28fd5-8261-553b-a9e1-b2916fc3738e" version = "0.5.5+0" [[deps.OrderedCollections]] -git-tree-sha1 = "85f8e6578bf1f9ee0d11e7bb1b1456435479d47c" +git-tree-sha1 = "2e73fe17cac3c62ad1aebe70d44c963c3cfdc3e3" uuid = "bac558e1-5e72-5ebc-8fee-abe8a469f55d" -version = "1.4.1" +version = "1.6.2" [[deps.PDMats]] deps = ["LinearAlgebra", "SparseArrays", "SuiteSparse"] -git-tree-sha1 = "cf494dca75a69712a72b80bc48f59dcf3dea63ec" +git-tree-sha1 = "67eae2738d63117a196f497d7db789821bce61d1" uuid = "90014a1f-27ba-587c-ab20-58faa44d9150" -version = "0.11.16" +version = "0.11.17" [[deps.Parsers]] -deps = ["Dates", "SnoopPrecompile"] -git-tree-sha1 = "b64719e8b4504983c7fca6cc9db3ebc8acc2a4d6" +deps = ["Dates", "PrecompileTools", "UUIDs"] +git-tree-sha1 = "4b2e829ee66d4218e0cef22c0a64ee37cf258c29" uuid = "69de0a69-1ddd-5017-9359-2bf0b02dc9f0" -version = "2.5.1" +version = "2.7.1" [[deps.Pkg]] deps = ["Artifacts", "Dates", "Downloads", "LibGit2", "Libdl", "Logging", "Markdown", "Printf", "REPL", "Random", "SHA", "Serialization", "TOML", "Tar", "UUIDs", "p7zip_jll"] @@ -309,17 +309,23 @@ git-tree-sha1 = "a6062fe4063cdafe78f4a0a81cfffb89721b30e7" uuid = "2dfb63ee-cc39-5dd5-95bd-886bf059d720" version = "1.4.2" +[[deps.PrecompileTools]] +deps = ["Preferences"] +git-tree-sha1 = "9673d39decc5feece56ef3940e5dafba15ba0f81" +uuid = "aea7be01-6a6a-4083-8856-8a6e6704d82a" +version = "1.1.2" + [[deps.Preferences]] deps = ["TOML"] -git-tree-sha1 = "47e5f437cc0e7ef2ce8406ce1e7e24d44915f88d" +git-tree-sha1 = "7eb1686b4f04b82f96ed7a4ea5890a4f0c7a09f1" uuid = "21216c6a-2e73-6563-6e65-726566657250" -version = "1.3.0" +version = "1.4.0" [[deps.PrettyTables]] deps = ["Crayons", "Formatting", "LaTeXStrings", "Markdown", "Reexport", "StringManipulation", "Tables"] -git-tree-sha1 = "96f6db03ab535bdb901300f88335257b0018689d" +git-tree-sha1 = "542b1bd03329c1d235110f96f1bb0eeffc48a87d" uuid = "08abe8d2-0d0c-5749-adfa-8a2ac140af0d" -version = "2.2.2" +version = "2.2.6" [[deps.Printf]] deps = ["Unicode"] @@ -327,9 +333,9 @@ uuid = "de0858da-6303-5e67-8744-51eddeeeb8d7" [[deps.QuadGK]] deps = ["DataStructures", "LinearAlgebra"] -git-tree-sha1 = "97aa253e65b784fd13e83774cadc95b38011d734" +git-tree-sha1 = "6ec7ac8412e83d57e313393220879ede1740f9ee" uuid = "1fd47b50-473d-5c70-9696-f719f8f3bcdc" -version = "2.6.0" +version = "2.8.2" [[deps.REPL]] deps = ["InteractiveUtils", "Markdown", "Sockets", "Unicode"] @@ -346,15 +352,15 @@ version = "1.2.2" [[deps.Rmath]] deps = ["Random", "Rmath_jll"] -git-tree-sha1 = "bf3188feca147ce108c76ad82c2792c57abe7b1f" +git-tree-sha1 = "f65dcb5fa46aee0cf9ed6274ccbd597adc49aa7b" uuid = "79098fc4-a85e-5d69-aa6a-4863f24498fa" -version = "0.7.0" +version = "0.7.1" [[deps.Rmath_jll]] deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg"] -git-tree-sha1 = "68db32dff12bb6127bac73c209881191bf0efbb7" +git-tree-sha1 = "6ed52fdd3382cf21947b15e8870ac0ddbff736da" uuid = "f50d1b31-88e8-58de-be2c-1cc44531875f" -version = "0.3.0+0" +version = "0.4.0+0" [[deps.SHA]] uuid = "ea8e919c-243c-51af-8825-aaa63cd721ce" @@ -362,26 +368,21 @@ version = "0.7.0" [[deps.SentinelArrays]] deps = ["Dates", "Random"] -git-tree-sha1 = "efd23b378ea5f2db53a55ae53d3133de4e080aa9" +git-tree-sha1 = "04bdff0b09c65ff3e06a05e3eb7b120223da3d39" uuid = "91c51154-3ec4-41a3-a24f-3f23e20d615c" -version = "1.3.16" +version = "1.4.0" [[deps.Serialization]] uuid = "9e88b42a-f829-5b0c-bbe9-9e923198166b" -[[deps.SnoopPrecompile]] -git-tree-sha1 = "f604441450a3c0569830946e5b33b78c928e1a85" -uuid = "66db9d55-30c0-4569-8b51-7e840670fc0c" -version = "1.0.1" - [[deps.Sockets]] uuid = "6462fe0b-24de-5631-8697-dd941f90decc" [[deps.SortingAlgorithms]] deps = ["DataStructures"] -git-tree-sha1 = "a4ada03f999bd01b3a25dcaa30b2d929fe537e00" +git-tree-sha1 = "c60ec5c62180f27efea3ba2908480f8055e17cee" uuid = "a2af1166-a08f-5f64-846c-94a0d3cef48c" -version = "1.1.0" +version = "1.1.1" [[deps.SparseArrays]] deps = ["LinearAlgebra", "Random"] @@ -389,9 +390,9 @@ uuid = "2f01184e-e22b-5df5-ae63-d93ebab69eaf" [[deps.SpecialFunctions]] deps = ["ChainRulesCore", "IrrationalConstants", "LogExpFunctions", "OpenLibm_jll", "OpenSpecFun_jll"] -git-tree-sha1 = "d75bda01f8c31ebb72df80a46c88b25d1c79c56d" +git-tree-sha1 = "7beb031cf8145577fbccacd94b8a8f4ce78428d3" uuid = "276daf66-3868-5448-9aa4-cd146d93841b" -version = "2.1.7" +version = "2.3.0" [[deps.Statistics]] deps = ["LinearAlgebra", "SparseArrays"] @@ -399,21 +400,21 @@ uuid = "10745b16-79ce-11e8-11f9-7d13ad32a3b2" [[deps.StatsAPI]] deps = ["LinearAlgebra"] -git-tree-sha1 = "f9af7f195fb13589dd2e2d57fdb401717d2eb1f6" +git-tree-sha1 = "45a7769a04a3cf80da1c1c7c60caf932e6f4c9f7" uuid = "82ae8749-77ed-4fe6-ae5f-f523153014b0" -version = "1.5.0" +version = "1.6.0" [[deps.StatsBase]] deps = ["DataAPI", "DataStructures", "LinearAlgebra", "LogExpFunctions", "Missings", "Printf", "Random", "SortingAlgorithms", "SparseArrays", "Statistics", "StatsAPI"] -git-tree-sha1 = "d1bf48bfcc554a3761a133fe3a9bb01488e06916" +git-tree-sha1 = "75ebe04c5bed70b91614d684259b661c9e6274a4" uuid = "2913bbd2-ae8a-5f71-8c99-4fb6c76f3a91" -version = "0.33.21" +version = "0.34.0" [[deps.StatsFuns]] deps = ["ChainRulesCore", "HypergeometricFunctions", "InverseFunctions", "IrrationalConstants", "LogExpFunctions", "Reexport", "Rmath", "SpecialFunctions"] -git-tree-sha1 = "89a3bfe98f5400f4ff58bb5cd1a9e46f95d08352" +git-tree-sha1 = "f625d686d5a88bcd2b15cd81f18f98186fdc0c9a" uuid = "4c63d2b9-4356-54db-8cca-17b64c39e42c" -version = "1.1.0" +version = "1.3.0" [[deps.StringManipulation]] git-tree-sha1 = "46da2434b41f41ac3594ee9816ce5541c6096123" @@ -437,9 +438,9 @@ version = "1.0.1" [[deps.Tables]] deps = ["DataAPI", "DataValueInterfaces", "IteratorInterfaceExtensions", "LinearAlgebra", "OrderedCollections", "TableTraits", "Test"] -git-tree-sha1 = "c79322d36826aa2f4fd8ecfa96ddb47b174ac78d" +git-tree-sha1 = "1544b926975372da01227b382066ab70e574a3ec" uuid = "bd369af6-aec1-5ad0-b16a-f7cc5008161c" -version = "1.10.0" +version = "1.10.1" [[deps.Tar]] deps = ["ArgTools", "SHA"] @@ -452,9 +453,9 @@ uuid = "8dfed614-e22c-5e08-85e1-65c5234f0b40" [[deps.TranscodingStreams]] deps = ["Random", "Test"] -git-tree-sha1 = "e4bdc63f5c6d62e80eb1c0043fcc0360d5950ff7" +git-tree-sha1 = "9a6ae7ed916312b41236fcef7e0af564ef934769" uuid = "3bb67fe8-82b1-5028-8e26-92a6c54297fa" -version = "0.9.10" +version = "0.9.13" [[deps.UUIDs]] deps = ["Random", "SHA"] @@ -469,6 +470,11 @@ git-tree-sha1 = "b1be2855ed9ed8eac54e5caff2afcdb442d52c23" uuid = "ea10d353-3f73-51f8-a26c-33c1cb351aa5" version = "1.4.2" +[[deps.WorkerUtilities]] +git-tree-sha1 = "cd1659ba0d57b71a464a29e64dbc67cfe83d54e7" +uuid = "76eceee3-57b5-4d4a-8e66-0e911cebbf60" +version = "1.6.1" + [[deps.Zlib_jll]] deps = ["Libdl"] uuid = "83775a58-1f1d-513f-b197-d71354ab007a" diff --git a/test/bulkscan_test.jl b/test/bulkscan_test.jl index 088b689..2265c5c 100644 --- a/test/bulkscan_test.jl +++ b/test/bulkscan_test.jl @@ -136,6 +136,46 @@ test_bulkscan_alt_grid = quote end; +test_bulkscan_general = quote + + stand_pheno = BulkLMM.colStandardize(pheno[:, 705:1112]); + stand_geno = BulkLMM.colStandardize(geno); + + grid_list = vcat(collect(0.0:0.05:0.95)); + + # null-grid + test_bulkscan = BulkLMM.bulkscan(stand_pheno, stand_geno, kinship; + method = "null-grid", + h2_grid = grid_list, + prior_variance = 1.0, prior_sample_size = 0.1); + + test_bulkscan_null_grid = BulkLMM.bulkscan_null_grid(stand_pheno, stand_geno, kinship, grid_list; + prior_variance = 1.0, prior_sample_size = 0.1); + + @test sum((test_bulkscan.L .- test_bulkscan_null_grid.L).^2) <= 1e-7; + + # null-exact + test_bulkscan = BulkLMM.bulkscan(stand_pheno, stand_geno, kinship; + method = "null-exact", + nb = 4, + prior_variance = 1.0, prior_sample_size = 0.1); + + @test sum((test_bulkscan.L[:, 1] .- test_bulkscan_null.L[:, 1]).^2) <= 1e-7; + @test sum((test_bulkscan.L[:, end] .- test_bulkscan_null.L[:, end]).^2) <= 1e-7; + + BLAS.set_num_threads(4); + # alt-grid + test_bulkscan = BulkLMM.bulkscan(stand_pheno, stand_geno, kinship; + method = "alt-grid", + h2_grid = grid_list, + prior_variance = 1.0, prior_sample_size = 0.1); + + test_bulkscan_alt_grid = BulkLMM.bulkscan_alt_grid(stand_pheno, stand_geno, kinship, grid_list; + prior_variance = 1.0, prior_sample_size = 0.1); + + @test sum((test_bulkscan.L .- test_bulkscan_alt_grid.L).^2) <= 1e-7; + +end; ########################################################################################################## @@ -149,6 +189,7 @@ println("Bulkscan functions test: ") eval(test_computeR_LMM2); eval(test_bulkscan_null); eval(test_bulkscan_null_grid); - eval(test_bulkscan_alt_grid); + eval(test_bulkscan_alt_grid); + eval(test_bulkscan_general); end diff --git a/test/scan_covar_test.jl b/test/scan_covar_test.jl index ea3de77..47b5770 100644 --- a/test/scan_covar_test.jl +++ b/test/scan_covar_test.jl @@ -8,8 +8,12 @@ ## For short wait-time reason, test by comparing single trait with covariates scan with the ## multiple trait null_grid algorithm function bulkscan_null_grid() test_scan_covar = scan(pheno_y, geno, pseudo_covars, kinship); -test_grid_covar = bulkscan_null_grid(pheno[:, 4:end], geno, pseudo_covars, kinship, +test_scan_covar_svd = scan(pheno_y, geno, pseudo_covars, kinship; decomp_scheme = "svd"); +test_grid_covar = bulkscan_null_grid(hcat(pheno[:, 2000], pheno_y), geno, pseudo_covars, kinship, vcat(collect(0.0:0.05:0.95), test_scan_covar.h2_null)).L; +test_grid_covar_svd = bulkscan_null_grid(hcat(pheno[:, 2000], pheno_y), geno, pseudo_covars, kinship, + vcat(collect(0.0:0.05:0.95), test_scan_covar.h2_null); + decomp_scheme = "svd").L; tol = 1e-8; @@ -21,6 +25,31 @@ catch e end println("Scan with covariates functions test: ", -@test mean(abs.(test_scan_covar.lod .- test_grid_covar[:, (pheno_id-3)])) <= tol +@test mean(abs.(test_scan_covar.lod .- test_grid_covar[:, 2])) <= tol ) +println("Scan with covariates functions test (SVD): ", +@test mean(abs.(test_scan_covar.lod .- test_scan_covar_svd.lod)) <= tol +) + +println("Scan with covariates functions test (SVD2): ", +@test mean(abs.(test_scan_covar.lod .- test_grid_covar_svd[:, 2])) <= tol +) + +test_scan_covar_pvals = scan(pheno_y, geno, pseudo_covars, kinship; output_pvals = true); +println("Scan with covariates functions test (p-vals output): ", +@test mean(abs.(lod2log10p.(test_scan_covar.lod, 1) .- test_scan_covar_pvals.log10pvals)) <= tol +) + +test_bulkscan_covar_Pvals = bulkscan(hcat(pheno[:, 2000], pheno_y), geno, pseudo_covars, kinship; + h2_grid = vcat(collect(0.0:0.05:0.95), + test_scan_covar.h2_null), + output_pvals = true).log10Pvals_mat; +println("Bulkscan with covariates functions test (p-vals output): ", +@test mean(abs.(test_bulkscan_covar_Pvals[:, 2] .- test_scan_covar_pvals.log10pvals)) <= tol +) + +test_scan_covar_vec = scan(pheno[:, pheno_id], geno, pseudo_covars, kinship); +println("Scan with covariates functions test (vector trait input): ", +@test mean(abs.(test_scan_covar.lod .- test_scan_covar_vec.lod)) <= tol) + diff --git a/test/util_test.jl b/test/util_test.jl index 9e5c34f..01ac152 100644 --- a/test/util_test.jl +++ b/test/util_test.jl @@ -297,40 +297,6 @@ tests_shuffleVector = quote testHelper(test3_shuffleVector); end; -########################################################################################################## -## TEST: compareValues() -########################################################################################################## - -### Test1: check if dimensions match - -test1_compareValues = quote - b_true = convert(Array{Float64, 1}, zeros(rand(collect(1:5)))); - b_test = convert(Array{Float64, 1}, zeros(length(b_true)+1)); - - try - BulkLMM.compareValues(b_true, b_test, 1e-3, 1e5) - catch e - @test typeof(e) == ErrorException - end -end; - -### Test2: check if the results are as desired - -test2_compareValues = quote - b_true = convert(Array{Float64, 1}, zeros(rand(collect(2:5)))); - b_test = copy(b_true); - - b_test[length(b_true)] = 1.0 - b_test[length(b_true)-1] = 2.0 - - @test BulkLMM.compareValues(b_test, b_true, 0.0, 1e5)[1] == length(b_true) - 2 -end; - -tests_compareValues = quote - testHelper(test1_compareValues); - testHelper(test2_compareValues); -end; - println("Utility functions test: ") @testset "Utility Functions Tests" begin @@ -341,7 +307,6 @@ println("Utility functions test: ") eval(tests_rowDivide); eval(tests_rowMultiply); eval(tests_shuffleVector); - eval(tests_compareValues); end