Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Added Minimal Support for Reliability Scores (aka Crombach's alpha) #701

Merged
merged 35 commits into from
Aug 24, 2021
Merged
Show file tree
Hide file tree
Changes from 4 commits
Commits
Show all changes
35 commits
Select commit Hold shift + click to select a range
056b250
added reliability
Jul 25, 2021
fdec3f2
updated docstring
Jul 25, 2021
0afd1aa
Update src/reliability.jl
storopoli Jul 25, 2021
2220f8d
updated naming to crombach_alpha
Jul 25, 2021
120e1a6
Update src/reliability.jl
storopoli Jul 25, 2021
1ec3213
aligned the comment
Jul 26, 2021
6a91ebf
changed calculations and using AbstractFloat
Jul 26, 2021
a4cb7cf
changed for loop to avoid allocations
Jul 26, 2021
3363974
added better performant implementation of crombach_alpha function
Jul 26, 2021
35b1f13
Update src/reliability.jl
storopoli Jul 26, 2021
3c84291
Update src/reliability.jl
storopoli Jul 26, 2021
0a134a1
Update src/reliability.jl
storopoli Jul 26, 2021
02f17f1
Update src/reliability.jl
storopoli Jul 26, 2021
a0dd59b
Update src/reliability.jl
storopoli Jul 26, 2021
07bf0aa
updated docstring for 4 digits
Jul 26, 2021
ff8da3e
corrected tests
Jul 26, 2021
3a0e9a7
expand tests to 1col 2cols Rational and BigFloat
Jul 26, 2021
9131d6b
Update test/reliability.jl
storopoli Jul 26, 2021
d376b62
Update src/reliability.jl
storopoli Jul 26, 2021
f3e9ca0
Update test/reliability.jl
storopoli Jul 26, 2021
6557421
Update test/reliability.jl
storopoli Jul 26, 2021
65a9c63
Update test/reliability.jl
storopoli Jul 26, 2021
45b1150
Update test/reliability.jl
storopoli Jul 26, 2021
d750e54
Update src/reliability.jl
storopoli Jul 26, 2021
440879b
added show io tests
Jul 26, 2021
fc4f4f8
remove Diagonal unused function
Jul 26, 2021
ccfae49
Apply suggestions from code review
storopoli Aug 21, 2021
2f63ba2
changed to crombachalpha and more tests
Aug 21, 2021
a920c41
Apply suggestions from code review
storopoli Aug 21, 2021
6be402b
Added docs,changed Crombach to Cronbach and new struct
Aug 21, 2021
d0afc19
renamed struct in StatsBase.jl
Aug 21, 2021
af0f068
Apply suggestions from code review
storopoli Aug 22, 2021
ebb2f68
replaced realiability to Cronbach's alpha and expanded tests
Aug 22, 2021
0a5e830
Apply suggestions from code review
nalimilan Aug 24, 2021
ce3368a
Fix formula
nalimilan Aug 24, 2021
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 6 additions & 1 deletion src/StatsBase.jl
Original file line number Diff line number Diff line change
Expand Up @@ -212,7 +212,11 @@ export
standardize,
AbstractDataTransform, # the type to represent a abstract data transformation
ZScoreTransform, # the type to represent a z-score data transformation
UnitRangeTransform # the type to represent a 0-1 data transformation
UnitRangeTransform, # the type to represent a 0-1 data transformation

# Reliability
nalimilan marked this conversation as resolved.
Show resolved Hide resolved
Reliability, # the type to represent reliability scores
crombach_alpha # function to compute Crombach's alpha
storopoli marked this conversation as resolved.
Show resolved Hide resolved

# source files

Expand All @@ -232,6 +236,7 @@ include("partialcor.jl")
include("empirical.jl")
include("hist.jl")
include("pairwise.jl")
include("reliability.jl")
include("misc.jl")

include("sampling.jl")
Expand Down
72 changes: 72 additions & 0 deletions src/reliability.jl
Original file line number Diff line number Diff line change
@@ -0,0 +1,72 @@
using LinearAlgebra: Diagonal

struct Reliability
alpha::Float64
dropped::Vector{Pair{Int, Float64}}
end

function Base.show(io::IO, x::Reliability)
storopoli marked this conversation as resolved.
Show resolved Hide resolved
println(io, "Reliability for all items: $(round(x.alpha; digits=4))")
storopoli marked this conversation as resolved.
Show resolved Hide resolved
println(io, "")
storopoli marked this conversation as resolved.
Show resolved Hide resolved
println(io, "Reliability if an item is dropped:")
for i ∈ x.dropped
println(io, "item $(i.first): $(round(i.second; digits=4))")
storopoli marked this conversation as resolved.
Show resolved Hide resolved
end
end

function _crombach_alpha(covmatrix::AbstractMatrix{T}) where T <: Real
k = size(covmatrix, 2)
storopoli marked this conversation as resolved.
Show resolved Hide resolved
σ = sum(covmatrix)
σ_ij = sum(covmatrix - Diagonal(covmatrix)) / (k * (k - 1))
ρ = k^2 * σ_ij / σ
storopoli marked this conversation as resolved.
Show resolved Hide resolved
return ρ
end

"""
crombach_alpha(covmatrix::AbstractMatrix{T}) where T <: Real
storopoli marked this conversation as resolved.
Show resolved Hide resolved

Calculate Crombach's alpha (1951) from a covariance matrix `covmatrix` according to
the Wikipedia formula (https://en.wikipedia.org/wiki/Cronbach%27s_alpha):
storopoli marked this conversation as resolved.
Show resolved Hide resolved

```math
\\rho = \\frac{k^2 \\bar{sigma}_{ij}}{\\sigma^2_X}
```

where k is the number of items, i.e. columns; σᵢⱼ denote the average of
the inter-item covariances; and σ²ₓ consists of item variances and inter-item
covariances.

Returns a `Reliability` object that holds:

* `alpha`: the reliability score for all items, i.e. columns, in `comatrix`; and
storopoli marked this conversation as resolved.
Show resolved Hide resolved
* `dropped`: A `Vector{Pair{item, score}}` reliability scores if a specific item, i.e. column, is dropped from `comatrix`.
storopoli marked this conversation as resolved.
Show resolved Hide resolved

# Example
```jldoctest
julia> using StatsBase

julia> cov_X = [10 6 6 6;
6 11 6 6;
6 6 12 6;
6 6 6 13];

julia> crombach_alpha(cov_X)
Reliability for all items: 0.814
storopoli marked this conversation as resolved.
Show resolved Hide resolved

Reliability if an item is dropped:
item 1: 0.75
item 2: 0.761
item 3: 0.771
item 4: 0.783
```
"""
function crombach_alpha(covmatrix::AbstractMatrix{T}) where T <: Real
storopoli marked this conversation as resolved.
Show resolved Hide resolved
alpha = _crombach_alpha(covmatrix)
k = size(covmatrix, 2)
dropped = Vector{Pair{Int64, Float64}}(undef, k)
storopoli marked this conversation as resolved.
Show resolved Hide resolved
@simd for i ∈ 1:k
storopoli marked this conversation as resolved.
Show resolved Hide resolved
reduced_covmatrix = covmatrix[1:end .!= i, 1:end .!= i]
storopoli marked this conversation as resolved.
Show resolved Hide resolved
@inbounds dropped[i] = Pair{Int64, Float64}(i, _crombach_alpha(reduced_covmatrix))
end
return Reliability(alpha, dropped)
storopoli marked this conversation as resolved.
Show resolved Hide resolved
storopoli marked this conversation as resolved.
Show resolved Hide resolved
end
15 changes: 15 additions & 0 deletions test/reliability.jl
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
using StatsBase
using LinearAlgebra, Random, Test

@testset "StatsBase.Reliability" begin
cov_X = [10 6 6 6;
6 11 6 6;
6 6 12 6;
6 6 6 13]
@test StatsBase._crombach_alpha(cov_X) ≈ 0.8135593220338984
reliabity_x = crombach_alpha(cov_X)
@test reliabity_x.dropped[1].second ≈ 0.75
@test reliabity_x.dropped[2].second ≈ 0.7605633802816901
@test reliabity_x.dropped[3].second ≈ 0.7714285714285715
@test reliabity_x.dropped[4].second ≈ 0.782608695652174
end # @testset "StatsBase.Reliability"
storopoli marked this conversation as resolved.
Show resolved Hide resolved