Skip to content

Commit

Permalink
Use unblocked files, not components, to determine bucket sizes
Browse files Browse the repository at this point in the history
Summary:
In D8758296 I changed the max_bucket_size constant to mean the number of files
in a bucket rather than the number of components. However, I did not correctly
update the logic to calculate the actual bucket size.

The logic error became clear in D13972924 where I refactored the code to use
the name `ready_components` instead of `jobs`. We are comparing files and
components, when we should be comparing files and files.

Reviewed By: nmote

Differential Revision: D13993319

fbshipit-source-id: fdc7d4d7075b6e06fb60618457248cc7dae61479
  • Loading branch information
samwgoldman authored and facebook-github-bot committed Feb 15, 2019
1 parent 44006e9 commit 12dd2bb
Showing 1 changed file with 3 additions and 3 deletions.
6 changes: 3 additions & 3 deletions src/services/inference/merge_stream.ml
Original file line number Diff line number Diff line change
Expand Up @@ -82,11 +82,11 @@ let max_bucket_size = 500
let bucket_size stream =
(* NB: num_workers can be zero *)
let max_bucket_size =
if stream.ready_components < stream.num_workers * max_bucket_size
then 1 + (stream.ready_components / stream.num_workers)
if stream.ready_files < stream.num_workers * max_bucket_size
then 1 + (stream.ready_files / stream.num_workers)
else max_bucket_size
in
min max_bucket_size stream.ready_components
min max_bucket_size stream.ready_files

let is_done stream =
stream.blocked_components = 0
Expand Down

0 comments on commit 12dd2bb

Please sign in to comment.