Consider options to make clr-debuginfo generation faster

[gtkramer]

There are one or two find command that become expensive as debuginfo build up over time. If we know the packages from the previous build, we might be able to identify which debuginfo packages have changed and generate debuginfo only for these in a staging directory, and then merge it into the final directory.

[phmccarty]

The other part to this issue is to implement a debuginfo lifecycle policy so that debuginfo does not "build up over time" quite like it does at the moment. Enacting such a policy would lessen the impact of running the find(1) commands.

[gtkramer]

Additionally, as a part of the merging, our syncs would no longer attempt to sync the entire directory, but rather, only place the new files into the existing directories. This would avoid statting every file and folder in the very large directory.

[phmccarty]

Right, there are things we can do to optimize debuginfo generation, and we have work to do to speed up the debuginfo syncs as well. I'm not sure how this will work for aws s3 sync, but wherever rsync is used, we can certainly better tailor those commands for the debuginfo content.

[fenrus75]

you can delete it all once in a while we now have like five years so I'd say clean itnow and worry again in five years

…

On Tue, Jan 7, 2020, 17:54 Patrick McCarty ***@***.***> wrote: Right, there are things we can do to optimize debuginfo *generation*, and we have work to do to speed up the debuginfo *syncs* as well. I'm not how this will work for aws s3 sync, but wherever rsync is used, we can certainly better tailor those commands for the debuginfo content. — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#92?email_source=notifications&email_token=AAJ54FPLXHSS3BE4A6UZ2LLQ4UW45A5CNFSM4J4PXTQ2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEIK5KKQ#issuecomment-571856170>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAJ54FKAH7X46TSUOYVDDULQ4UW45ANCNFSM4J4PXTQQ> .