Skip to content

v1.0.0-rc.3 Limit concurrent file writes

Pre-release
Pre-release
Compare
Choose a tag to compare
@kpjensen kpjensen released this 24 Sep 08:39

Attempt to reduce memory usage by limiting the number of files which can be written to concurrently. The limit is greater of 15 or CPUs*3

Lower the initial upload file part size from 96MiB --> 16MiB. Each subsequent part n increases in size 16 * 1.1^nMiB up to a maximum part size of 700MiB.

Raise the HttpClient pool used for DNAnexus api calls to a minimum of 15 and up to CPUs*3.