Skip to content

Commit

Permalink
expire, makehistory: How to pre-size the history
Browse files Browse the repository at this point in the history
Better explain the use of the -s flag (as already done for makedbz
documentation).
  • Loading branch information
Julien-Elie committed Nov 2, 2023
1 parent 0c6d73c commit d3ff24d
Show file tree
Hide file tree
Showing 2 changed files with 26 additions and 8 deletions.
10 changes: 6 additions & 4 deletions doc/pod/expire.pod
Original file line number Diff line number Diff line change
Expand Up @@ -126,10 +126,12 @@ also the B<-n> flag.

=item B<-s> I<size>

Optimize the new history database for approximately I<size> pairs
(lines in F<history>). Accurately specifying the size will create
a more efficient database. (The size should be the estimated eventual size
of the file, typically the size of the old file.)
B<expire> determines the optimal size of the new F<history> file from the size
of the old one. In case you want to force a specific size, use this flag to
optimize the new history database for approximately I<size> key-value pairs
(i.e. lines in F<history>). Accurately specifying the size will create a
more efficient database. (The size should be the estimated eventual number of
articles, typically the size of the old F<history> file, in lines.)

=item B<-t>

Expand Down
24 changes: 20 additions & 4 deletions doc/pod/makehistory.pod
Original file line number Diff line number Diff line change
Expand Up @@ -181,10 +181,18 @@ ignored. This option only makes sense in combination with B<-O>.

=item B<-s> I<size>

Size the history database for approximately I<size> pairs. Accurately
specifying the size is an optimization that will create a more
efficient database. (The size should be the estimated eventual size
of the F<history> file, typically the size of the old file, in lines.)
Size the history database for approximately I<size> key-value pairs
(i.e. lines in F<history>). Accurately specifying the size is an optimization
that will create a more efficient database. (The size should be the estimated
eventual number of articles, typically the size of the old F<history> file,
in lines.)

By default, B<makehistory> will create a database optimized for handling about
6,000,000 articles (or 500,000 if the slower tagged hash format is used).
This size does not limit the number of articles the news server can store
in its F<history> file. It will just get slower when that optimal size is
exceeded, until the next run of B<news.daily> which will appropriately resize
it.

=item B<-T> I<tmpdir>

Expand Down Expand Up @@ -238,6 +246,14 @@ For a simpler example:
will scan the spool, removing broken articles and generating history and
overview entries for articles missing from history.

To pre-size the F<history> file for 100,000,000 articles, and generate
overview data at the same time, you may directly use the following command:

makehistory -O -s 100000000

You then do not need running B<makedbz> as the F<history> file has already
been generated and optimized for the expected number of articles.

To just rebuild overview:

makehistory -O -x -F
Expand Down

0 comments on commit d3ff24d

Please sign in to comment.