You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
In #89 I had to introduce a new kwarg to Active -> storage_type to have correct open/retrieve operations depending on the type of storage (POSIX vs S3 now) - storage_type should/could be picked up from file name as @davidhassell suggested offline - but we can change that later (am not 100% sure we should, though, since future possible storage types may not necessarily be guessed from file name) - this issue is to raise our awareness as we go along.
The text was updated successfully, but these errors were encountered:
You might want to reconsider how the S3 access is configured - is global config in config.py the most appropriate approach? I used it as a fast way to get things working.
Would it be better to pass in the URLs & credentials etc. to the Active object's constructor? Or possibly abstract them into a Storage object that provides a reduce_chunk interface and "knows" how to access the object in storage.
that's a very good point and question! I am thinking we should probably discuss what other types of storage we want to support, and then see what's the best way to have them configured at runtime - I was too thinking about config.py when you opened the PR a few weeks ago and was leaning towards a yaml config file (still sort of an externally loaded configuration) but a config object might be a better approach. For now it does the trick as interim solution 👍
In #89 I had to introduce a new kwarg to
Active
->storage_type
to have correct open/retrieve operations depending on the type of storage (POSIX vs S3 now) -storage_type
should/could be picked up from file name as @davidhassell suggested offline - but we can change that later (am not 100% sure we should, though, since future possible storage types may not necessarily be guessed from file name) - this issue is to raise our awareness as we go along.The text was updated successfully, but these errors were encountered: