You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Description:
The current implementation of the Lindemann index calculation algorithm in the lindemann package is primarily executed in a serial manner, which limits its performance and scalability, especially for large datasets typically processed in high-performance computing (HPC) environments. The goal is to optimize the existing algorithm by introducing parallelization using Numba's just-in-time (JIT) compilation and parallel processing capabilities (numba prange).
Proposed Solution:
Enhance the performance of the Lindemann index calculation by implementing a parallel version of the algorithm using Numba. The key steps involved in this enhancement include:
Implementation of Parallel Variance Calculation:
Using Welford's algorithm to compute the mean and variance in a numerically stable way.
Using Numba to calculate the variance.
Chunk-wise Processing:
Splitting the frames into multiple chunks to enable parallel processing (prange)
Calculating the mean and variance for each chunk independently.
Combining Results:
Aggregating the results from all chunks to compute the final Lindemann index.
Using reduction to efficiently combine the results.
Code Implementation:
The following functions were developed in parallel_trj.py to achieve the parallelization:
parallel_variance: Computes the variance in parallel using Welford's algorithm.
calculate_chunk: Processes a chunk of frames to compute mean and variance distances.
calculate: Divides the data into chunks and uses parallel processing to compute the Lindemann index.
Benefits:
Significant reduction in computation time for large datasets.
Efficient utilization of multi-core processors.
Enhanced scalability and performance, making it suitable for HPC environments.
Next Steps:
Parallelization of the -ot flag.
The text was updated successfully, but these errors were encountered:
Description:
The current implementation of the Lindemann index calculation algorithm in the lindemann package is primarily executed in a serial manner, which limits its performance and scalability, especially for large datasets typically processed in high-performance computing (HPC) environments. The goal is to optimize the existing algorithm by introducing parallelization using Numba's just-in-time (JIT) compilation and parallel processing capabilities (numba prange).
Proposed Solution:
Enhance the performance of the Lindemann index calculation by implementing a parallel version of the algorithm using Numba. The key steps involved in this enhancement include:
Implementation of Parallel Variance Calculation:
Chunk-wise Processing:
Combining Results:
Code Implementation:
The following functions were developed in parallel_trj.py to achieve the parallelization:
parallel_variance
: Computes the variance in parallel using Welford's algorithm.calculate_chunk
: Processes a chunk of frames to compute mean and variance distances.calculate
: Divides the data into chunks and uses parallel processing to compute the Lindemann index.Benefits:
Next Steps:
The text was updated successfully, but these errors were encountered: