[FR] Online migration for stale files

[Knud3]

Go-Carbon have fantastic new feature of online migration #438, but it wont affect on stale files. It is problematic in Grafana panels which now include metrics from old and new storage schemas.

It would be beneficial to have some way to trigger migration to all files in new schema.

Maybe cache a copy of storage-schemas.conf and storage-aggregation.conf and when go-carbon process starts it checks diffs to running ones.

Originally posted in #518 (comment)
I also did investigation where problem might be with metrics. It seems that metrics which have had writes is migrated to 1m:14d,1h:1y as you said, but Grafana panels also include metrics which haven't had writes (even out of selected time range), so it only shows 5m:1d,1h:1y.

[deniszh]

Hi @Knud3

Could you please elaborate what's stale files in your descriptions? FIles which stop getting updates?
I'm not aware that current code not migrating them...

[Knud3]

Yes. Files which are not getting updates no more.

Only files which are going through persister get updated.

online migrations are triggered when there are updates on the metric/whisper files. If there are no write to the whisper files, then migrations are not triggered.
Originally posted by @bom-d-van in #518 (comment)

you can also manually migrate the schema of those stale whisper files with whisper-resize.py.
Originally posted by @bom-d-van in #518 (comment)

[deniszh]

Hi @Knud3

Oh, I didn't know that. That could be fixed, indeed