v2.0.0-beta.3

Overview

• License management
• Release dataset with versions
• Enable and disable user account through Keycloak
• Jupyterhub integration
• Interface for creating and editing matching criteria for triggering extractors
• Interface for editing metadata definitions
• My dataset tab listing all the datasets created by the user
• Drag and drop upload multiple files
• Footer with links to documentation, source code, and contact information
• Documentation through MKDocs
• Allow public datasets and files to be searchable
• List all the extractors with the ability to enable/disable the extractors
• Filter listeners based on their support for file or dataset
• Helm chart updated to support custom existing secret
• Allow public datasets and files to be searchable
• List all the extractors with the ability to enable/disable the extractors
• Filter listeners based on their support for file or dataset
• Helm chart updated to support custom existing secret

In Depth Changes

• Release/v2.0.0 beta2 by @longshuicy in #928
• 961 add footer for frontend pages by @longshuicy in #998
• adding default value for admin in backend by @ddey2 in #1000
• Keycloak enabled change by @bingzhang in #973
• implemented standard list of licenses by @ddey2 in #963
• filters by file or dataset for listeners by @tcnichol in #970
• fix typo in roles endpoint; disable sharing with myself by @longshuicy in #1002
• Material for Mkdocs docs by @lmarini in #962
• 996 detail information update on by @longshuicy in #1001
• 953 postcss configuration by @longshuicy in #966
• Delete dataset will also delete custom license by @longshuicy in #1008
• fix nested folder bug by @longshuicy in #1024
• Fix registration by @longshuicy in #1006
• removing alignment css by @ddey2 in #1030
• disabling next untill user selects a license by @ddey2 in #1014
• breadcrumb in the public components point to the wrong frontend routes by @longshuicy in #1045
• Docs to set up Clowder in VM by @Vismayak in #1005
• List license and its url on public dataset by @longshuicy in #1046
• 1020 modify extractor description UI by @Vismayak in #1048
• adding license to populate fake data by @tcnichol in #1051
• Search needs to be case insensitive by @longshuicy in #1049
• Enable/disable listeners by @ddey2 in #1022
• 1053 pagination doesnt update on deleting files in a dataset by @longshuicy in #1060
• Fixing logic to post metadata on file upload by @ddey2 in #1054
• 866 readonly user by @tcnichol in #974
• 836 public search option by @tcnichol in #964
• read only user can be made admin by @tcnichol in #1079
• fix precommits by @longshuicy in #1083
• add my datasets tab by @tcnichol in #965
• Implement edit metadata definition by @ddey2 in #977
• 1061 filesystem UI component by @Vismayak in #1064
• 1066 index folders by @longshuicy in #1090
• 1086 file UI component flag current files in datasets by @Vismayak in #1103
• Implementation of feeds page by @ddey2 in #1056
• Added parameters field back to EventListenerDatasetJobMessage. by @lmarini in #1107
• fix elasticsearch delete GH action by @longshuicy in #1109
• Removing version column from File entry view by @ddey2 in #1117
• 1127 missing validator in some rjsf forms by @longshuicy in #1128
• Adding column for listener name to show on Analysis history by @ddey2 in #1118
• Removed snackbar message on clearing metadata form by @ddey2 in #1113
• Download should be incremented as soon as you click on downloads by @ddey2 in #1120
• 1098 superamin messes up dataset pagination by @tcnichol in #1104
• 918 ability to freeze datasets or version by @longshuicy in #941
• users unable to search files by @tcnichol in #1076
• 1069 show message when no metadata is present by @tcnichol in #1112
• added check for rendering step1 by @ddey2 in #1116
• 1082 group is not updated after delete user by @tcnichol in #1111
• when revoke admin also revoke admin_mode by @longshuicy in #1135
• add generic superadmin flag to endpoints by @max-zilla in #1038
• hide extracted results for now by @tcnichol in #1138
• removing left dividers by @tcnichol in #1133
• Integration of Jupyter through docker-compose by @Vismayak in #1088
• Use the update flag when updating fields of existing Elasticsearch entry by @longshuicy in #1136
• 1092 enhanced visualization layout by @longshuicy in #1137
• Remove install package from Pipfile since it doesn't exist anymore … by @lmarini in #1150
• setting active to it's original value for existing extractor by @ddey2 in #1153
• Fix extractor submission related bugs by @longshuicy in #1154
• 1147 clowder 2 helm chart clean up by @longshuicy in #1148
• Docs updates by @lmarini in #1101
• fix tests that are failing by @tcnichol in #1167
• Remove helm chart env by @longshuicy in #1165
• set workers based on recommended number of cores by @longshuicy in #1170
• Annotate image widget by @Vismayak in #1129
• 1168 add contents to docs by @longshuicy in #1188
• error message when sharing dataset with user by @tcnichol in #1140
• Private extractor support by @max-zilla in #991
• Fix/beta 3 extractions list by @lmarini in #1199
• Fix/list dataset extractors list by @lmarini in #1200
• When adding metadata, sometimes two instances of it would show up aft… by @lmarini in #1201
• Rabbitmq host update to release name by @longshuicy in #1203
• Fix refresh token issue when using password flow by @GalMunGral in #1205
• Fix edit file metadata bug by making sure by @lmarini in #1207
• Fix/authenticated datasets access by @lmarini in #1210
• Add endpoint for updating user info by @GalMunGral in #1206
• Default to None if "owner" doesn't exists in msg by @lmarini in #1211
• Use healthcheck to make sure elasticsearch has by @lmarini in #1212
• Only show Create Feed button if in admin mode. by @lmarini in #1213
• v2.0.0-beta3 by @longshuicy in #1151

New Contributors

• @bingzhang made their first contribution in #973
• @Vismayak made their first contribution in https://git...

v2.0.0-beta.2

What's Changed

• Change hostname to edu by @longshuicy in #813
• Have Message when there are no metadata definitions by @tcnichol in #766
• 804 move searchbox to top for metadata definition and group page by @longshuicy in #815
• 787 enable uploading multiple files at once by @tcnichol in #802
• Basic implementation of admin in user collection in mongodb by @ddey2 in #809
• message on new metadata entry by @tcnichol in #764
• Adding check on client side for empty searchterm by @ddey2 in #821
• getting rid of mongo init as beanie takes care of it by @ddey2 in #825
• 464 datasets visible to logged in users with AUTHENTICATED status by @tcnichol in #466
• 769 leveraging the heartbeat info to show extractor status by @longshuicy in #822
• beta.1 release by @lmarini in #810
• Adding admin dependency in authorization of dataset,files, metadata,search by @ddey2 in #819
• Docs cleanup by @lmarini in #829
• Trying to fix readthedocs build with the addtion of .readthedocs.yaml. by @lmarini in #847
• 828 filter and sort extractors by livelihood by @longshuicy in #838
• Register local files by @max-zilla in #816
• 824 display more information on the extractor popup by @longshuicy in #841
• 842 UI component for assign admin by @longshuicy in #845
• allow message and heartbeat image and tag to be overwrite by @longshuicy in #858
• Refactor create superadmin mode by @longshuicy in #851
• need image tag for heartbeat and listeners by @tcnichol in #853
• adding timeout loop for heartbeat and messages by @tcnichol in #860
• hide files pagination on other tabs by @tcnichol in #863
• 779 dataset and file tabs cleanup by @tcnichol in #850
• build the containers by @ddey2 in #889
• 844 toggle on and off of the extractor livelihood by @longshuicy in #869
• 892 latest main branch is not building the frontend image correct by @longshuicy in #893
• 861 bug when preferred username is not the email by @tcnichol in #862
• add admin badge to avatar by @longshuicy in #880
• implement if metadata is required for file and dataset by @ddey2 in #879
• 485 public datasets visible when not logged in by @tcnichol in #832
• upgrade keycloak to v20 by @tcnichol in #897
• add metadata button is back by @tcnichol in #888
• Add external Keycloak functionality by @alemsh in #883
• Hostname localhost by @longshuicy in #908
• extractors removed from public, front end error fixed. by @tcnichol in #909
• 872 refactoring pagination by @longshuicy in #894
• 911 public dataset and file page still query private metadata endpoints by @tcnichol in #913
• set up helm release by @longshuicy in #912
• 904 allow folder renaming by @longshuicy in #915
• Pre commit autoformat by @lmarini in #846
• change to make keycloak point to external url by @longshuicy in #942
• 929 message on empty page if there is no datasets by @tcnichol in #930
• fix version blocking the datasets by @longshuicy in #950
• enable superadmin does not update privileges by @tcnichol in #951
• Fixed dataset parameters by @lmarini in #957
• fix extractor bugs and deployment script bug by @longshuicy in #958
• Drag and Drop One or More Files by @tcnichol in #935
• clear more breakpoints and move role to the Details by @longshuicy in #968
• remove analysis tab from authenticated file by @tcnichol in #972
• style of the sharing table; link to the group/{group_id} page by @longshuicy in #967
• tab can only use display none to hide by @longshuicy in #976
• 981 clowder2 register link on the topbar doesnt work by @longshuicy in #988
• Hide register for now by @longshuicy in #990

New Contributors

• @alemsh made their first contribution in #883

Full Changelog: v2.0.0-beta.1...v2.0.0-beta.2

v2.0.0-beta.1

What's Changed

• 170 add extractor to gui by @tcnichol in #184
• 236 style extractor page to match the mockup by @longshuicy in #242
• Fix/docker compose by @lmarini in #247
• Added dynamic context field w/ Autocomplete by @arunapa in #244
• This will fix the issue with keycloak not connecting to postgres by @tcnichol in #257
• Extractor registration fixes by @max-zilla in #250
• 241-error-on-delete-metadata fixed. by @tcnichol in #245
• fix to deleting metadata in elasticsearch by @ddey2 in #249
• added new fields to metadata by @ddey2 in #239
• deleting metadata on file/dataset deletion by @ddey2 in #240
• 252 use rjsf to dynamically generate extractor parameter forms by @longshuicy in #256
• Dockerize heartbeat by @tcnichol in #76
• 284 rabbitmq client configs should come from settings by @lmarini in #286
• Removed unused Extract button on Dataset page. by @lmarini in #288
• Initial roles and dependencies by @lmarini in #258
• 281 css style change when using rjsf by @longshuicy in #291
• Update Pipfile.lock by @max-zilla in #308
• Add extractor job ids by @max-zilla in #282
• 290 replace contents in metadata with content by @tcnichol in #292
• Metdata version updated with metadata update by @ddey2 in #280
• Add mime type feed support by @max-zilla in #305
• 276 automatically trigger extractor whenever a file is updated by @tcnichol in #279
• Fix to file deletion by @ddey2 in #310
• catch invalid date time by @longshuicy in #320
• update file content model by @longshuicy in #322
• document keycloak setting by @longshuicy in #315
• fix smaller screen text overlapping issue by @longshuicy in #316
• Updating byte size on updating file by @ddey2 in #325
• Initial framework for groups by @ddey2 in #324
• 314 resubmit extractors with parameters on update file by @tcnichol in #323
• 272 show prior extractor executions on filedataset pages by @longshuicy in #327
• Kubenetes add missing environment variables by @longshuicy in #297
• 311 add spinner and disable the upload button when upload files by @arunapa in #331
• download older file versions & metadata by @max-zilla in #326
• Adding download button on UI to download correct verison file by @ddey2 in #334
• fix member typing by @longshuicy in #336
• fixing error of blank page on submit file to extractor by @tcnichol in #338
• Test for v1 extractor triggering by @max-zilla in #329
• Added new component for file actions by @arunapa in #352
• UI for displaying logs on extractors by @arunapa in #317
• Properly interpret extractor statuses by @max-zilla in #356
• Context matches v1 type. by @tcnichol in #328
• Mongo views by @longshuicy in #353
• Replaced EmbeddedSearch with custom search box component to fix session refresh issue by @arunapa in #365
• Add File authorization framework by @max-zilla in #357
• only get current version's metadata by @max-zilla in #363
• Updated File Is Wrong Size by @tcnichol in #361
• Added new Page Not Found component, updated route to use new component by @arunapa in #371
• 348 metadata value not visible on UI by @tcnichol in #362
• changing context to list in metadata defs by @tcnichol in #370
• Added group_id to authorization by @ddey2 in #354
• add metadata auth framework. by @tcnichol in #359
• adding restart unless stopped for this container by @tcnichol in #341
• Implement FileAuthorization dependency by @max-zilla in #385
• matching response type with actual dependency returns by @longshuicy in #390
• add MetadataAuthorization dependency by @tcnichol in #387
• Implement DatasetAuthorization dependency, add to endpoints by @tcnichol in #386
• Group management by @max-zilla in #388
• WIP: 351 add authorization to UI by @longshuicy in #389
• Group management UI by @max-zilla in #393
• Fixes 500 error when going back from group by @tcnichol in #404
• 401 group management UI continue by @longshuicy in #405
• Added hyperlink to jobid, triggers log summary popup when clicked by @arunapa in #395
• User api keys by @max-zilla in #402
• 374 add groups to datasets via gui by @tcnichol in #398
• Add api key to get_current_user, catch missing auths by @max-zilla in #413
• 377 UI display group and user info right next to each of the resource by @tcnichol in #392
• Sharing bugfixes by @max-zilla in #415
• 379 GitHub action to run codegen automatically when there is backend change by @longshuicy in #396
• 406 group management UI improvements by @longshuicy in #416
• fix the search by @longshuicy in #421
• job created timestamp wrong - fixed to local time by @tcnichol in #425
• 403 implement role change on shared tab by @tcnichol in #427
• implement expires parameter for user keys by @max-zilla in #426
• Moved Groups to sidebar, added breadcrumbs for group main page and individual group pages by @arunapa in #429
• Metadata on new file version not updated by @arunapa in #431
• Minor UI updates by @lmarini in #436
• typo of aio-pika by @longshuicy in #460
• 442 UI to acquire api key by @longshuicy in #450
• 441 gui for create new group by @longshuicy in #445
• Fixed the props for search component by @arunapa in #444
• Autocomplete on sharing pop-ups by @max-zilla in #459
• import container by @longshuicy in #462
• 446 gui to manage api keys by @longshuicy in #465
• Sharing tab cleanup by @max-zilla in #430
• Fix reload bug by @max-zilla in #451
• 461 fix library version in pipfile by @longshuicy in #468
• 443 frontend need to display more verbose backend error by @arunapa in #458
• 469 create profile page by @tcnichol in https://github.co...

v2.0.0-alpha.2 | 2022-11-28

Notable Changes

• Use Keycloak for user management, JWT tokens and federated identity.
• Initial implementation of file versioning and nested folders within datasets.
• Initial implementation of metadata entries and metadata definitions.
• Initial implementation of search using Elasticsearch.
• Improvements to overall UI and UX. Standardized components and widgets used.
• Helm charts for Kubernetes deployment.

All Changes

• Frontend refreshing mechanism by @longshuicy in #2
• Registration with keycloak #3 by @longshuicy in #4
• Metadata updates by @max-zilla in #6
• Root directory cleanup by @lmarini in #14
• replace delete with delete_many by @tcnichol in #17
• Download dataset by @tcnichol in #9
• delete metadata when file/dataset deleted by @max-zilla in #15
• Set keycloak login theme by @longshuicy in #5
• modify keycloak social providers style by @longshuicy in #28
• update file version number by @longshuicy in #25
• 22 folderdataset name not showing up in breadcrum after immediately creating file by @longshuicy in #29
• Metadata creation & deletion tests by @max-zilla in #31
• Match delete metadata in the frontend #19 by @longshuicy in #20
• Simple rabbitmq setup. by @lmarini in #37
• Remove unused widgets by @lmarini in #36
• fix delete error by @longshuicy in #40
• 47 keycloak needs volume to store user data by @tcnichol in #48
• 24 store file type size in mongodb by @tcnichol in #27
• 30 add new metadata on existing dataset and file by @longshuicy in #45
• 12 create dataset from zip by @tcnichol in #16
• Build frontend for both linux/amd64 and linux/arm64 by @lmarini in #44
• fix patch endpoint of dataset by @longshuicy in #70
• 41 random jwt signature verification failed error by @longshuicy in #46
• Upgraded node to 16.15.1 LTS. Had to modify several dependencies and … by @lmarini in #33
• setting default limits to 10 for get datasets and get dataset files by @tcnichol in #72
• ro crate and bagit by @tcnichol in #56
• url to link to the correct repo by @longshuicy in #78
• fix content type by @longshuicy in #83
• ensure m1 / arm64 image is pushed to Dockerhub by @KastanDay in #91
• 82 delete files does not always work by @longshuicy in #85
• 93 add favicon by @longshuicy in #94
• 54 add widget type field and break the current widget into smaller components by @longshuicy in #60
• Elasticsearch for v2 by @ddey2 in #73
• Metadata4more resource types by @MBcode in #80
• 57 file does not upload to folder by @longshuicy in #81
• Fix/pytest keycloak timeout by @lmarini in #98
• Fixed file upload. Provide default folder_id = None. by @lmarini in #103
• Register extractor submit file by @tcnichol in #66
• 84 frontend implement delete folders by @longshuicy in #92
• Directories cleanup by @lmarini in #106
• Fix/populate fake data by @lmarini in #107
• Explore page cleanup by @lmarini in #108
• Added indexing for dataset and added documentation #89 by @ddey2 in #109
• Drawer menu by @lmarini in #129
• Added endpoint to search both file and dataset. by @ddey2 in #139
• Updated elasticsearch indices on updating files and datasets by @ddey2 in #140
• Fix/GitHub action pipenv by @lmarini in #159
• 155 elasticsearch dependency injection instead of creating a client everytime by @ddey2 in #157
• WIP: 95 conclude metadata design documentation by @longshuicy in #99
• 166 fix npm dependency issue by @longshuicy in #167
• Scripts to help with development. by @lmarini in #161
• 131 custom widget and appearance for reactivesearch by @longshuicy in #134
• Extractors -> listeners, trigger listeners on saved feeds automatically by @max-zilla in #114
• 127 add parameters to extractor submit add submit dataset by @tcnichol in #128
• 132 reactive search routes when token expire logout by @longshuicy in #164
• Implemented Metadata Definition UI by @arunapa in #150
• include metadata JSON file in output zip by @max-zilla in #178
• Generic error catching in DB by @max-zilla in #152
• fix keycloak version to use 19.0 since it's the most compatibale majo… by @longshuicy in #186
• Track pycharm run configuration in git repository. by @lmarini in #179
• 122 minify keycloak realm json so kubernetes clusters have easier way to load it by @longshuicy in #123
• 175 search box syntax by @longshuicy in #181
• fix frontend build by @longshuicy in #187
• make first name and last name required by @longshuicy in #190
• Fixed missing button by @arunapa in #196
• fix bug when upload file by @longshuicy in #193
• 142 remove url encoding when creating dataset and files by @longshuicy in #189
• Dataset page cleanup by @lmarini in #194
• helmchart to deploy clowder2 in kubernetes by @robkooper in #119
• 23 dashboard and breadcrumb need unique key prop by @longshuicy in #202
• Fixing trailing whitespace issue, renamed list checkbox by @arunapa in #215
• 206 modify time facet not working properly by @longshuicy in #213
• Initial implementation of metadata indexing by @ddey2 in #173
• Fixed list creation issues by @arunapa in #220
• UI tweaks by @lmarini in #216
• Delete bad JWT token after attempting a refresh to avoid 403 on OPTIO… by @lmarini in #219
• Fixed page reload issue and date time picker render by @arunapa in #218
• 202 file breadcrumb folders not in breadcrumb for files fixed by @tcnichol in #214
• Release/alpha 2 by @lmarini in #223

New Contributors

• @max-zilla made their first contribution in #6
• @lmarini made their first contribution in #14
• @tcnichol made their first contribution in #17
• @KastanDay made their first contribution in #91
• @ddey2 made their first contribution in #73
• @MBcode made their first contribution in #80
• @arunapa made their first co...