Avian flu restructure #811

lmoncla · 2024-04-02T19:38:26Z

Description of proposed changes

This pull request separates avian-flu from seasonal-flu and sets them to the be at the same hierarchy.

Original structure: flu/seasonal/ and flu/avian
New structure: seasonal-flu/ and avian-flu/

This was accomplished by editing 3 files:

data/manifest_core.json: seasonal-flu and avian-flu now exist at the same header level. h5nx is set as the default for avian-flu.
redirects.js: I added in redirects for the seasonal flu and avian flu urls. These should be double checked.
static-site/src/components/Cards/pathogenCards.js: added in a pathogen card for Avian Influenza.

Checklist

I would like help with testing out these changes to make sure that all the necessary edits have been made, and that the changes work as intended. Specific things that should be double checked are the redirects, manifest_core.json structure, and that the splash card works as intended.

jameshadfield · 2024-04-02T20:11:53Z

Thanks @lmoncla! I haven't tested this out, but a few thoughts (mainly things to do on our side, most pretty easy).

Once I merge List Resources (prototype UI) #803 (hopefully this week) the format of pathogen cards will change, and I'll need to rejig the quick-links etc.
The redirects would be simpler as /flu/seasonal/:rest -> /flu-seasonal/:rest, as the current redirects don't capture all the various paths we have
- see 3e83194 for a fix
The biggest change is going to be incorporating the redirects into the indexer for two reasons:
- ~~It can't currently use route parameters, so we either have to build this out or list all of the flu paths (there are many!)~~ Solved by Redirect with regex #812
- There's a race condition where if the indexer switches to the new routes then the old routes (/flu/...@...) won't work until the server updates. Maybe the reverse is ok? I'd have to test. There are a few temporary solutions we can use to patch over this (e.g. having the indexer include both /flu/... and /avian-flu/... while we move the server over).
Tests are failing because they are expecting certain /flu/... routes to exist (i.e. not redirect)

jameshadfield · 2024-04-12T05:06:23Z

I've pushed a branch https://github.com/nextstrain/nextstrain.org/tree/flu-restructure which has this working - perhaps we could force-push it into this PR if people are comfortable with that? It consists of 5 commits.

Update the manifest (cherry-picked @lmoncla's 0eaa901)
Update the redirects (cherry-picked @joverlee521's 3e83194)
Update the core routes to send /seasonal-flu/... and /avian-flu/... URLs to Auspice
Update the (new) resource listing UI
Fix the tests which rely on /flu/... datasets / URLs. (This commit is incomplete - tests are still broken)
[not done] update the main splash page cards & the /influenza page cards

If the resource listing index is regenerated from this branch then most things¹ are working well. There's a out-of-sync issue we need to be cognizant of whereby we merge this code but don't promote the site, then when the resource listing updates (it's on a daily schedule) then the /pathogens page will show /seasonal-flu & /avian-flu URLs which will all 404, and any /flu/...@version URL will also 404. We can work through this next week.

¹ Ahh. I've forgotten to update the error-handling pages for flu (static-site/pages/flu/[...flu].jsx) but I think we should get rid of the entire /influenza page which URLs such as /flu/no/dataset/here will load, and instead 404 like we do for a URL like nextstrain.org/mpox/no/dataset/here and this PR is probably the right place to do it. I'll sort this out on Monday.

jameshadfield · 2024-04-16T04:30:36Z

@lmoncla I've updated this branch - if you wish to check this out locally the following commands should work:

git checkout avian-flu-restructure 
git fetch
git reset --hard origin/avian-flu-restructure

@joverlee521 & @victorlin would you mind reviewing the PR? I think the changes on the server side are complete, but we still need to consider the index getting out-of-sync (see #829) as well as changing the pathogen repos to upload the new filenames.

test/requests.test.js

src/routing/core.js

joverlee521

Changes here look reasonable to me 👍

static-site/src/components/Cards/coreCards.js

victorlin

Looked through commits and tested a bit locally. Will test more when the PR is updated with the temporary resource index.

test/auspice_client_requests.json

src/routing/core.js

src/redirects.js

jameshadfield · 2024-04-19T00:30:03Z

Re-pushed with the indexer added as a drop commit. @joverlee521, @victorlin thanks for your reviews so far, very helpful!

Tests are failing (I believe) because the zika root-sequence has been deleted in nextstrain/zika#57. I'll make a separate PR to fix that now & we can rebase onto that. Update: #831

Have changed the PR to a "draft" just to avoid a mistake merge before we drop the final commit.

To match the two salient changes in the previous handful of commits: 1. flu/seasonal/* URLs are now seasonal-flu/* 2. seasonal-flu/* URLs which are invalid (i.e. no dataset) now 404 As part of this I copied `flu_seasonal_*2y.json` on the staging bucket to `seasonal-flu_*2y.json` so we can continue using staging datasets in tests. Includes patches from <#811 (comment)> Co-authored-by: Victor Lin <[email protected]>

jameshadfield · 2024-04-19T01:04:40Z

Merged #831 into master & rebased this PR. Tests should now pass 🤞

test/auspice_client_requests.json

victorlin

Clicked around the review app, looks good. One more small suggestion

data/manifest_core.json

…r the flu/avian and flu/seasonal structure, and into their own groups. avian-flu and seasonal-flu are now separate entities at the same level as ebola, mpox, and dengue

Our core influenza builds are now shifted to different path prefixes (seasonal-flu and avian-flu). "flu/..." URLs will be redirected before they hit this layer of routing, so they don't need to be handled. The URL `/flu` (with or without a version attached) is also redirected (via this commit) so our core build routing layer no longer needs to handle /flu routes.

I searched for 'seasonal' in the repo and updated where I thought was appropriate

for the move from /flu/... urls to /seasonal-flu/... and /avian-flu/...

To use the full paths rather than paths which will be redirected. I changed to full paths to avoid what I felt was an unnecessary redirect cycle. The upside is faster loading pages, although the actual improvement may be tiny. The downside is decoupling the manifest seasonal-flu default from the splash page tile dataset. (Although I see this as another upside, not a downside.)

The original thinking behind this page was motivated by our /sars-cov-2 page, but the influenza page never progressed past a small set of tiles. (The /sars-cov-2 page has similarly fallen out of date, but that's another story.) The (new) /pathogens page contains all the information that was on /influenza plus a whole lot more, so we redirect here. There is one (minor) functional change, URLs /flu/* and /influenza/* which are not backed by a dataset will now go to our generic 404 page rather than to the influenza page with a 404-like banner. This matches the behaviour of all other "coreBuildPaths" routes which aren't backed by a dataset _except for_ `/ncov/*` URLs See also slack discussion: <https://bedfordlab.slack.com/archives/C7SDVPBLZ/p1713122853502539>

To match the two salient changes in the previous handful of commits: 1. flu/seasonal/* URLs are now seasonal-flu/* 2. seasonal-flu/* URLs which are invalid (i.e. no dataset) now 404 As part of this I copied `flu_seasonal_*2y.json` on the staging bucket to `seasonal-flu_*2y.json` so we can continue using staging datasets in tests. Includes patches from <#811 (comment)> Co-authored-by: Victor Lin <[email protected]>

jameshadfield · 2024-04-23T02:19:40Z

Rebased onto latest master & @victorlin's fix incorporated. Tomorrow I plan to:

drop the final commit 256c0b9
merge PR
check build completed & next.nextstrain.org is working
Update index (via newly merged indexer code)
- ( nextstrain.org will be slightly broken at this stage )
promote canary

Any final testing prior to then is of course welcome

lmoncla requested a review from jameshadfield April 2, 2024 19:38

nextstrain-bot temporarily deployed to nextstrain-s-avian-flu--4smxwf April 2, 2024 19:38 Inactive

lmoncla requested a review from joverlee521 April 2, 2024 19:39

victorlin self-requested a review April 2, 2024 20:47

joverlee521 mentioned this pull request Apr 3, 2024

Redirect with regex #812

Merged

1 task

jameshadfield mentioned this pull request Apr 15, 2024

[RFC] Maintain different resource indexes #829

Closed

jameshadfield force-pushed the avian-flu-restructure branch from c8bb121 to 1a2ae73 Compare April 16, 2024 04:25

jameshadfield reviewed Apr 16, 2024

View reviewed changes

test/requests.test.js Outdated Show resolved Hide resolved

jameshadfield reviewed Apr 16, 2024

View reviewed changes

src/routing/core.js Show resolved Hide resolved

joverlee521 reviewed Apr 16, 2024

View reviewed changes

static-site/src/components/Cards/coreCards.js Show resolved Hide resolved

victorlin reviewed Apr 18, 2024

View reviewed changes

test/auspice_client_requests.json Outdated Show resolved Hide resolved

test/auspice_client_requests.json Show resolved Hide resolved

src/routing/core.js Outdated Show resolved Hide resolved

src/redirects.js Show resolved Hide resolved

jameshadfield force-pushed the avian-flu-restructure branch from 1a2ae73 to 9d05894 Compare April 19, 2024 00:14

jameshadfield marked this pull request as draft April 19, 2024 00:15

jameshadfield force-pushed the avian-flu-restructure branch from 9d05894 to 75f89a5 Compare April 19, 2024 01:03

jameshadfield mentioned this pull request Apr 19, 2024

Understand nextstrain.org caching #832

Closed

jameshadfield reviewed Apr 19, 2024

View reviewed changes

test/auspice_client_requests.json Show resolved Hide resolved

victorlin temporarily deployed to nextstrain-s-avian-flu--avz9sw April 22, 2024 17:47 Inactive

victorlin approved these changes Apr 22, 2024

View reviewed changes

data/manifest_core.json Outdated Show resolved Hide resolved

victorlin mentioned this pull request Apr 22, 2024

CI: Transient /charon/getDataset errors when fetching from S3 #836

Open

lhmoncla and others added 5 commits April 23, 2024 13:54

this commit moves the avian-flu and seasonal-flu builds out from unde…

d9aa143

…r the flu/avian and flu/seasonal structure, and into their own groups. avian-flu and seasonal-flu are now separate entities at the same level as ebola, mpox, and dengue

redirects: Add flu redirects

43f35c8

Update doc references

39bbe2a

I searched for 'seasonal' in the repo and updated where I thought was appropriate

Update resource listing data

27861ef

for the move from /flu/... urls to /seasonal-flu/... and /avian-flu/...

jameshadfield and others added 3 commits April 23, 2024 13:54

jameshadfield force-pushed the avian-flu-restructure branch from 75f89a5 to 256c0b9 Compare April 23, 2024 02:17

nextstrain-bot temporarily deployed to nextstrain-s-avian-flu--avz9sw April 23, 2024 02:17 Inactive

joverlee521 mentioned this pull request Apr 23, 2024

profiles/nextstrain-public: Update build/auspice names nextstrain/seasonal-flu#153

Merged

1 task

jameshadfield force-pushed the avian-flu-restructure branch from 256c0b9 to 0948595 Compare April 23, 2024 19:17

nextstrain-bot temporarily deployed to nextstrain-s-avian-flu--avz9sw April 23, 2024 19:18 Inactive

jameshadfield marked this pull request as ready for review April 23, 2024 20:50

jameshadfield merged commit b57ae25 into master Apr 23, 2024
4 checks passed

jameshadfield deleted the avian-flu-restructure branch April 23, 2024 20:50

victorlin mentioned this pull request Apr 26, 2024

Update avian/seasonal flu references nextstrain/public#3

Open

10 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Avian flu restructure #811

Avian flu restructure #811

lmoncla commented Apr 2, 2024

jameshadfield commented Apr 2, 2024 •

edited

Loading

jameshadfield commented Apr 12, 2024 •

edited

Loading

jameshadfield commented Apr 16, 2024

joverlee521 left a comment

victorlin left a comment

jameshadfield commented Apr 19, 2024 •

edited

Loading

jameshadfield commented Apr 19, 2024

victorlin left a comment

jameshadfield commented Apr 23, 2024 •

edited

Loading

Avian flu restructure #811

Avian flu restructure #811

Conversation

lmoncla commented Apr 2, 2024

Description of proposed changes

Checklist

jameshadfield commented Apr 2, 2024 • edited Loading

jameshadfield commented Apr 12, 2024 • edited Loading

jameshadfield commented Apr 16, 2024

joverlee521 left a comment

Choose a reason for hiding this comment

victorlin left a comment

Choose a reason for hiding this comment

jameshadfield commented Apr 19, 2024 • edited Loading

jameshadfield commented Apr 19, 2024

victorlin left a comment

Choose a reason for hiding this comment

jameshadfield commented Apr 23, 2024 • edited Loading

jameshadfield commented Apr 2, 2024 •

edited

Loading

jameshadfield commented Apr 12, 2024 •

edited

Loading

jameshadfield commented Apr 19, 2024 •

edited

Loading

jameshadfield commented Apr 23, 2024 •

edited

Loading