Skip to content

Commit

Permalink
FilePusher: Improved performance when there are a great many nr of fi…
Browse files Browse the repository at this point in the history
…les pre-existing that can be pushed

- This introduces a new setting MaxNrOfFilesToQueueForPushViaScan which will limit the nr of pre-existing files that will be queued at any given time. Default max is 1000
  • Loading branch information
LiberatorUSA committed Mar 2, 2024
1 parent ae42fa6 commit 6f4d1a4
Show file tree
Hide file tree
Showing 2 changed files with 110 additions and 57 deletions.
7 changes: 4 additions & 3 deletions tools/FilePusher/include/FilePusher.h
Original file line number Diff line number Diff line change
Expand Up @@ -185,6 +185,7 @@ class FilePushDestinationSettings
CORE::CString fileCompressionTempDir;
TStringSet fileTypesToCompress;
UInt32 maxNrOfFilesToPushInParallel;
UInt32 maxNrOfFilesToQueueForPushViaScan;
CORE::CString name;
CORE::CString metricsPrefix;
};
Expand Down Expand Up @@ -335,11 +336,11 @@ class FilePushDestination : public CORE::CObservingNotifier
const CORE::CEvent& eventId ,
CORE::CICloneable* eventData );

void
QueueNewFileForPushingAfterUnmodifiedRestPeriod( const CORE::CString& newFilePath );
bool
QueueNewFileForPushingAfterUnmodifiedRestPeriod( const CORE::CString& newFilePath, bool oppertunistic );

bool
QueueAllPreExistingFilesForDir( const CORE::CString& dir );
QueueAllPreExistingFilesForDir( const CORE::CString& dir, bool oppertunistic );

bool
DoesFilenameMatchPushAllFilesPattern( const CORE::CString& filename ) const;
Expand Down
Loading

0 comments on commit 6f4d1a4

Please sign in to comment.