Plots Per Day

[aaroneden]

I am LOVING ChiaDog and appreciate what you've created. It would be great to include a metric in the daily summary which is the total number of new plots created today and a summary of how long they spent in each stage of plotting. This would help me look for performance changes and tune my plotters and ensure they are working well consistently.

My suggestion would be to pull from the logs the number of plots created (not the number of plots on the drive) as my plotters are all set up to push their files to a NAS harvester.

Best,
Aaron

[martomi]

Have you noticed that in the daily summary we already display the number of new plots detected by the harvester over the past 24 hours? Anything more than that would be outside the scope of this tool - there are a few projects focusing specifically on plotting and I suspect they can provide more detailed metrics.

[aaroneden]

Yes, I did see it, but it appears to be counting the number of plot files on the disk rather than using the logs to count new plots created.  Is that the case? Best, Aaron

…

On May 18, 2021, 12:59 PM -0700, Martin Mihaylov ***@***.***>, wrote: Have you noticed that in the daily summary we already display the number of new plots detected by the harvester over the past 24 hours? — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or unsubscribe.

[pieterhelsen]

Normally, it shows total number of plots as well as new plots.
See below screenshot; first one shows total plots (no new ones during 24h), the second one shows total + new.

[aaroneden]

I understand that, but I’m not communicating my need clearly.  Let me try again: Most of my harvesters are also plotters.  If the machine is both a harvester and a plotter then it’s doing 3 things all day: 1) Creating new plots 2) Harvesting any plots it has 3) Moving plots to my larger servers I’m trying to understand whether Chiadog is calculating “New plots” as number of new files created in the last 24 hours from the disk (which would change) or from the logs (which would not change). If ChiaDog is calculating that metric from the disk then it would be inaccurate for me.  If ChiaDog is calculating that metric from the logs then it would be accurate. Does that help clarify? Best, Aaron

…

On May 19, 2021, 6:01 AM -0700, pieterhelsen ***@***.***>, wrote: Normally, it shows total number of plots as well as new plots. See below screenshot; first one shows total plots (no new ones during 24h), the second one shows total + new. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or unsubscribe.

[pieterhelsen]

Chiadog reads the logfile and checks for increases or decreases in harvester plot count.

chiadog/src/chia_log/parsers/harvester_activity_parser.py

Line 34 in 67328d7

r"([0-9:.]*) harvester (?:src|chia).harvester.harvester(?:\s?): INFO\s*([0-9]+) plots were "

However, if you are simultaneously creating and moving plots off-server, chiadog will have no way of keeping track of the 'actual' created plots. Come daily stat time, it just takes the last reported number of (total) plots, compares that with the plot number from 24 hours ago and determines the 'new' plots based off of that information.

Consider this example:

chiadog reads farming event: 20 plots
a plot is created (total is 21) -> there is no 'event' in the chia log, so chiadog does not notice
a plot is moved (total is back to 20) -> again, there is no event in the chia log
chiadog reads farming event: 20 plots -> no increase and missed plot creation

Obviously this is an extreme example, given the ~18sec timeframe in which these events need to occur, but there is no foolproof way to guarantee correct logging of plot increase just from the Chia logs. The only way to do this reliably is to run plotman analyze, but this is outside of the scope of Chiadog I think.

[aaroneden]

Ahh nice! Then ChiaDog is already doing the core part of what I requested (plot counts).

…

On May 19, 2021, 8:21 AM -0700, pieterhelsen ***@***.***>, wrote: Chiadog reads the logfile and checks for increases or decreases in harvester plot count. https://github.com/martomi/chiadog/blob/67328d7960608179f8a4d9858a70955de29932f9/src/chia_log/parsers/harvester_activity_parser.py#L34 However, if you are simultaneously creating and moving plots off-server, chiadog will have no way of keeping track of the 'actual' created plots. Come daily stat time, it just takes the last reported number of (total) plots, compares that with the plot number from 24 hours ago and determines the 'new' plots based off of that information. Consider this example: chiadog reads farming event: 20 plots a plot is created (total is 21) -> there is no 'event' in the chia log, so chiadog does not notice a plot is moved (total is back to 20) -> again, there is no event in the chia log chiadog reads farming event: 20 plots -> no increase and missed plot creation Obviously this is an extreme example, given the ~18sec timeframe in which these events need to occur, but there is no foolproof way to guarantee correct logging of plot increase just from the Chia logs. The only way to do this reliably is to run plotman analyze, but this is outside of the scope of Chiadog I think. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or unsubscribe.