You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
I'd prefer using cloud juicefs for my local clusters as well for ease of maintenance. But the perf is not so great due to high network latency (10+x compared with local) to cloud servers. The problem is especially acute for large amount of cluster local (for both producers and consumers) intermediate data, where cluster local cache would help a lot. The cluster data perf could be addressed partially by minio tiering. It'd be great if you can support a cluster local (redis or generic) metadata cache that is incrementally sync'ed (e.g. every 1-10s) to the cloud metadata store. IMO, this would greatly improve the usability of cloud juicefs for on-prem/edge clusters. With a lease based consistent cache implementation, we can also have a pretty performant global juicefs as well.
reacted with thumbs up emoji reacted with thumbs down emoji reacted with laugh emoji reacted with hooray emoji reacted with confused emoji reacted with heart emoji reacted with rocket emoji reacted with eyes emoji
-
I'd prefer using cloud juicefs for my local clusters as well for ease of maintenance. But the perf is not so great due to high network latency (10+x compared with local) to cloud servers. The problem is especially acute for large amount of cluster local (for both producers and consumers) intermediate data, where cluster local cache would help a lot. The cluster data perf could be addressed partially by minio tiering. It'd be great if you can support a cluster local (redis or generic) metadata cache that is incrementally sync'ed (e.g. every 1-10s) to the cloud metadata store. IMO, this would greatly improve the usability of cloud juicefs for on-prem/edge clusters. With a lease based consistent cache implementation, we can also have a pretty performant global juicefs as well.
Wonder if you any plans to address the issue.
Beta Was this translation helpful? Give feedback.
All reactions