Refresh Zarr cache files after ingestor is finished

[danangmassandy]

The API will cache files from zarr into a directory in /tmp. The cache directory is different for each zarr file (currently, we have CBAM and Salient zarr files). The most useful files to cache are the files for coordinates data, like the date, lat, and lon files because everytime xarray opens a dataset, it needs those files.

Currently, the cache is set to expire after 1 hour. I'm thinking to increase the expiry time because the files are not updated often. CBAM zarr file is not updated once the ingestor is finished processing 10-years NETCDF files. The Salient zarr file is only updated once a week.

The issue in this approach is that the ingestor is run on the worker container, but the cache directory is inside the django container(s). Once the data ingestor is finished updating the zarr file, we should notify the django to refresh the zarr file cache.

[danangmassandy]

This is low priority because we have set exprity time = 1H for the cache files.