create helper function to fetch data with concurrent requests

[mkholjuraev]

This adds a simple, but useful helper function to utils package. It will help us to fetch a huge amount of data using paginated requests. Additionally, those paginated requests can be done in a certain number of concurrent API calls.

[Hyperkid123]

Do we need the p-all and p-map dependency? In this case, we are dealing with just a simple que.

I am also a bit curious about why you need to limit the number of concurrent requests at all?

[mkholjuraev]

@Hyperkid123 thank you for having a look. I think they will be useful to avoid AKAMAI blocks when there are 10k items to be fetched. As you know if we do not limit concurrent requests, AKAMAI will block after a short time. Let me know if this is not needed. BTW, I writing tests

[Hyperkid123]

I think they will be useful to avoid AKAMAI blocks when there are 10k items to be fetched.

Oh, one more question. Why is it that the UI requires 10k of records? Rendering anything with 10k items (table for example) is never going to end up with good UX unless the view is virtualized.

[mkholjuraev]

Yup, this is one of the major issues on the UI that we face. When for example, a customer selects all existing advisories to be remediated, there is too much data to be fetched on the frontend to make it available for remediation app. Another example is when customers export reports. Maybe, I am exaggerating a bit, but I have seen sometimes this indeed happens. At the same time, I should mention that no table or UI component deals with so much data on the UI

[mkholjuraev]

The ideal solution for the remediation issue would be to do changes in the architectural level, but it is requires a decent amount of effort. Discussions on this have been taken place even now.

[Hyperkid123]

The ideal solution for the remediation issue would be to do changes in the architectural level, but it is requires a decent amount of effort. Discussions on this have been taken place even now.

Yeah, I think the screens should be revisited then. I am not going to block your features, but I'd like it if the problem was properly sorted out at some point. These kinds of "optimizations" should never be required.

If you need any help, let me know.

[bastilian]

@Hyperkid123 One main area that we have been using these hooks or similar solutions are exports, like CSV, JSON and PDF. Another one is when we do batch actions, and we require data that needs to be submitted to an API later on, like a list of all host IDs.

[mkholjuraev]

@Hyperkid123 just wanted to remind you about this.

[Hyperkid123]

@mkholjuraev the CI is still failing.

EDIT: Oh I see. I restarted the job. See if its happy now or we need some adjustment.

[Hyperkid123]

@mkholjuraev you did not push the lock file changes hence the CI failure

npm ERR! code EUSAGE
npm ERR! 
npm ERR! `npm ci` can only install packages when your package.json and package-lock.json or npm-shrinkwrap.json are in sync. Please update your lock file with `npm install` before continuing.
npm ERR! 
npm ERR! Missing: [email protected] from lock file
npm ERR! Missing: [email protected] from lock file

[mkholjuraev]

@Hyperkid123 fixed. PTAL

[gkarat]

LGTM. I think we can merge and release it now

[gkarat]

Oops, there are some conflicts. Can we resolve them? @mkholjuraev otherwise, everything looks good

[mkholjuraev]

@gkarat thanks for having a look. Conflicts are resolved now

[nacho-bot]

                      :soon::shipit::octocat:
     :bug:Shipit Squirrel has this release bugfix surrounded, be ready for a new version:beetle:

[nacho-bot]

     🌱 🌸 🌷 🌻 🌟 New version of package has been released 🌟 🌻 🌷 🌸 🌱

                The release is available on:

        :package:@redhat-cloud-services/frontend-components-utilities/v/4.0.8📦

             :boom:This feature is brought to you by probot🚀