Only use one nginx worker, add a slight wait post-nginx-reload, to pr…

…operly upgrade-test schema changes that aren't backward-compatible on the "get enroll secret" code path (#26008)

For #26003. Will add more context to that ticket.

This is a robustness fix to the upgrade test, which pulls tags/branches
from Docker, so this does _not_ need to be cherry-picked.

To confirm this fix, run the upgrade test from <4.63.0 to >= 4.63.0 and
see it pass rather than fail.

# Checklist for submitter
- [x] Added/updated automated tests
- [x] A detailed QA plan exists on the associated ticket (if it isn't
there, work with the product group's QA engineer to add it)
- [x] Manual QA for all new/changed functionality

test/upgrade/fleet_test.go

@@ -111,6 +111,13 @@ func (f *Fleet) Start() error {
 		return err
 	}
 
+	// drop to one nginx worker process regardless of CPU count to ensure repointing to the correct
+	// Fleet container happens quickly
+	_, err = f.execCompose(env, "exec", "-T", "fleet", "sed", "-i", "s/auto/1/", "/etc/nginx/nginx.conf")
+	if err != nil {
+		return err
+	}
+
 	_, err = f.execCompose(env, "exec", "-T", "fleet", "nginx", "-s", "reload")
 	if err != nil {
 		return err
@@ -361,6 +368,10 @@ func (f *Fleet) Upgrade(toVersion string) error {
 		return err
 	}
 
+	// even with only one worker process, graceful reload of nginx workers doesn't happen instantly,
+	// so we add a wait here to let workers swap so they're pointed at the upgraded Fleet server
+	time.Sleep(250 * time.Millisecond)
+
 	f.Version = toVersion
 
 	return nil