Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

MoveMapGenerator: add proper multithreading support #2827

Open
wants to merge 3 commits into
base: development
Choose a base branch
from

Conversation

schell244
Copy link
Contributor

@schell244 schell244 commented Nov 8, 2024

🍰 Pullrequest

Allow mmap extraction to run properly multi threaded.
No changes have been made to the extraction logic itself, some funtions where just moved into TileWorker to run asynchronous.
Depending on the cpu, mmaps can now be build in <1 hour.

Screenshot 2024-11-08 130912

How2Test

  • start mmap extraction as usual
  • select number of threads to be used
  • resulting mmaps are the same as without those changes

Todo / Checklist

  • None

@0blu 0blu added the CPP A issue / PR which references CPP code label Nov 8, 2024
@mserajnik
Copy link
Contributor

I guess when it’s run from mmap_extract.py it shouldn’t prompt the user but instead just select 1 thread automatically (since mmap_extract.py already spawns multiple workers).

@schell244
Copy link
Contributor Author

@mserajnik can you maybe give it try and confirm that mmap_extract.py now works as expected?

@mserajnik
Copy link
Contributor

@schell244 Currently not at home, but I’ll give it a try later today or tomorrow at the latest!

@mserajnik
Copy link
Contributor

I'm building Docker images off of this PR right now and will test it tomorrow morning.

In the meantime, because it came to my mind: one could make the argument to remove the mmap_extract.py script altogether if MoveMapGenerator now supports multi-threading itself. However, I would keep it for two reasons:

  1. I think the output of mmap_extract.py (that I implemented via Use concurrent.futures and improve user feedback in mmap_extract.py. #2559) is a better overview for the average end user than the elaborate MoveMapGenerator output
  2. Removing the script now would potentially break VMaNGOS-related projects that rely on it (e.g., I use it for my Docker setup; of course, I monitor VMaNGOS development closely and could easily adjust it, but not everyone might do that)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
CPP A issue / PR which references CPP code
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants