Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix EntityWhitelist causing huge performance drop #33322

Open
wants to merge 2 commits into
base: master
Choose a base branch
from

Conversation

MilonPL
Copy link
Contributor

@MilonPL MilonPL commented Nov 14, 2024

About the PR

optimizes EntityWhitelistSystem by caching component registrations

Why / Balance

EntityWhitelist would kill server performance the instant you had an entity with an invalid component name

from a trace:

  • IsValid method: 98,960ms total time
  • Dictionary FindValue: 25,376ms
  • FrozenDictionary GetValueRefOrNullRefCore: 40,373ms

Technical details

  1. component registration caching:

    • new _componentCache dict to store component availability and registration
    • cache initialized during system startup with all available registrations
  2. improved registration handling:

    • added EnsureRegistrations method to convert strings to registrations once and cache results
    • removed redundant StringsToRegs method
    • registration lists are now built once and reused
  3. optimized validation flow:

    • added fast path for empty lists
    • better structured RequireAll logic across all validation checks
You can test it yourself with this, just put it in Content.Shared:
using Content.Shared.Item;
using Content.Shared.Whitelist;
using Robust.Shared.Timing;

public sealed class WhitelistLagTestSystem : EntitySystem
{
    [Dependency] private readonly EntityWhitelistSystem _whitelist = default!;
    [Dependency] private readonly IGameTiming _timing = default!;

    private EntityWhitelist? _problematicWhitelist;
    private const int EntityCount = 25;
    private bool _initialized;
    private TimeSpan _startTime;

    public override void Initialize()
    {
        base.Initialize();
        _startTime = _timing.CurTime;

        // Create whitelist but don't spawn entities yet
        _problematicWhitelist = new EntityWhitelist
        {
            Components = new[] { "MisspelledComponent", "AnotherTypo" },
            RequireAll = true,
        };
    }

    private void InitializeTest()
    {
        Log.Info("Starting whitelist lag test...");

        // Spawn test entities
        for (var i = 0; i < EntityCount; i++)
        {
            var ent = Spawn("MobHuman");
            AddComp<ItemComponent>(ent);
        }

        _initialized = true;
        Log.Info($"Whitelist lag test initialized with {EntityCount} entities");
    }

    public override void Update(float frameTime)
    {
        base.Update(frameTime);

        if (!_initialized && (_timing.CurTime - _startTime).TotalSeconds >= 15)
        {
            InitializeTest();
            return;
        }

        if (!_initialized)
            return;

        var query = EntityQueryEnumerator<ItemComponent>();
        while (query.MoveNext(out var uid, out _))
        {
            if (_problematicWhitelist != null)
                _whitelist.IsValid(_problematicWhitelist, uid);
        }
    }
}

Media

holy shit
image

image

Requirements

Breaking changes

Changelog

no fun

@github-actions github-actions bot added size/L Denotes a PR that changes 1000-4999 lines. S: Untriaged Status: Indicates an item has not been triaged and doesn't have appropriate labels. labels Nov 14, 2024
@TheShuEd TheShuEd added P1: High Priority: Higher priority than other items, but isn't an emergency. T: Performance Type: Performance impacting changes or bugs D1: High Difficulty: Extensive codebase knowledge required. S: Needs Review Status: Requires additional reviews before being fully accepted A: Core Tech Area: Underlying core tech for the game and the Github repository. and removed S: Untriaged Status: Indicates an item has not been triaged and doesn't have appropriate labels. labels Nov 14, 2024
@SaphireLattice
Copy link
Contributor

Wait wait wait

It checked the registrations every single time something was trying to validate?

I’m guessing the huge performance hit is it causing shitton of errors and thus having to do terminal IO which is notoriously awful when spamming stuff…? Er, unless it doesn’t in fact spam? Not quite in position to check the behavior right now.

Oh and the constant lookups don’t help probably

@MilonPL
Copy link
Contributor Author

MilonPL commented Nov 15, 2024

It checked the registrations every single time something was trying to validate?

Yuuuup..

Comment on lines +21 to +26

foreach (var registration in _factory.GetAllRegistrations())
{
_componentCache[registration.Name] = (ComponentAvailability.Available, registration);
}
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Isn't a frozendictionary going to be faster than this

dvir001 added a commit to dvir001/frontier-station-14 that referenced this pull request Nov 29, 2024
whatston3 added a commit to new-frontiers-14/frontier-station-14 that referenced this pull request Nov 29, 2024
* Back to baseline

* space-wizards/space-station-14#33322

* PartOne

* Restore recycler yml

* Pirate bounty touchups, humanprofileeditor

* mail component fix, ID field name

* IdDataField attributes

* Cargo market whitelist

---------

Co-authored-by: Whatstone <[email protected]>
@VasilisThePikachu VasilisThePikachu added the Intent: Hotfix PR Intent: Something that should be applied via hotfix procedure. label Dec 16, 2024
Copy link
Contributor

This pull request has conflicts, please resolve those before we can evaluate the pull request.

@github-actions github-actions bot added the S: Merge Conflict Status: Needs to resolve merge conflicts before it can be accepted label Dec 17, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
A: Core Tech Area: Underlying core tech for the game and the Github repository. D1: High Difficulty: Extensive codebase knowledge required. Intent: Hotfix PR Intent: Something that should be applied via hotfix procedure. P1: High Priority: Higher priority than other items, but isn't an emergency. S: Merge Conflict Status: Needs to resolve merge conflicts before it can be accepted S: Needs Review Status: Requires additional reviews before being fully accepted size/L Denotes a PR that changes 1000-4999 lines. T: Performance Type: Performance impacting changes or bugs
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants