provider problem in multi threaded condition

[FLemon]

Describe the bug

when using dry-system in multi threaded platform, i.e. puma.
when lazy loading provider
there's race condition in the provider start block

in our production environment, we are getting key not found error when calling Container['path.to.key]`

[129] - Worker 0 (PID: 139) booted in 2.14s, phase: 0
Dry::Core::Container::KeyError: key not found: "decorators.decorated_entities.users.with_pending_changes"
Did you mean?  "decorators.builders.users.with_pending_changes"
        /usr/local/bundle/gems/dry-core-1.0.1/lib/dry/core/container/resolver.rb:32:in `block in call'
        /usr/local/bundle/gems/dry-core-1.0.1/lib/dry/core/container/resolver.rb:28:in `fetch'
        /usr/local/bundle/gems/dry-core-1.0.1/lib/dry/core/container/resolver.rb:28:in `call'
        /usr/local/bundle/gems/dry-core-1.0.1/lib/dry/core/container/mixin.rb:132:in `resolve'
        /usr/local/bundle/gems/dry-system-1.0.1/lib/dry/system/container.rb:493:in `resolve'
        /usr/src/app/system/container.rb:151:in `resolve'
        /usr/local/bundle/gems/dry-core-1.0.1/lib/dry/core/container/mixin.rb:145:in `[]'
        /usr/src/app/system/container.rb:137:in `[]'
        /usr/local/bundle/gems/dry-auto_inject-1.0.1/lib/dry/auto_inject/strategies/kwargs.rb:16:in `block (3 levels) in define_new'
        /usr/local/bundle/gems/dry-auto_inject-1.0.1/lib/dry/auto_inject/strategies/kwargs.rb:15:in `each'
        /usr/local/bundle/gems/dry-auto_inject-1.0.1/lib/dry/auto_inject/strategies/kwargs.rb:15:in `block (2 levels) in define_new'
        /usr/local/bundle/gems/dry-system-1.0.1/lib/dry/system/loader.rb:54:in `call'
        /usr/local/bundle/gems/dry-system-1.0.1/lib/dry/system/component.rb:64:in `instance'
        /usr/local/bundle/gems/dry-system-1.0.1/lib/dry/system/container.rb:639:in `block in load_local_component'
        /usr/local/bundle/gems/dry-core-1.0.1/lib/dry/core/container/item/memoizable.rb:35:in `block in call'
        /usr/local/bundle/gems/dry-core-1.0.1/lib/dry/core/container/item/memoizable.rb:34:in `synchronize'
        /usr/local/bundle/gems/dry-core-1.0.1/lib/dry/core/container/item/memoizable.rb:34:in `call'
        /usr/local/bundle/gems/dry-core-1.0.1/lib/dry/core/container/resolver.rb:36:in `call'
        /usr/local/bundle/gems/dry-core-1.0.1/lib/dry/core/container/mixin.rb:132:in `resolve'
        /usr/local/bundle/gems/dry-system-1.0.1/lib/dry/system/container.rb:493:in `resolve'

we tried to replicate the exact issue we are having by putting a new spec in this repo(as shown below), but all we can see is warning of circular require(as shown in the stacktrace below)

the warning kinda illustrated the problem.

/usr/local/bundle/gems/zeitwerk-2.6.15/lib/zeitwerk/kernel.rb:34: warning: /usr/local/bundle/gems/zeitwerk-2.6.15/lib/zeitwerk/kernel.rb:34: warning: loading in progre
ss, circular require considered harmful - /tmp/d20240603-1904-aagks2/lib/animals/cat.rb
        from /usr/src/app/spec/integration/container/providers/resolving_root_key_spec.rb:56:in  `block (5 levels) in <top (required)>'
        from /usr/local/bundle/gems/dry-core-1.0.1/lib/dry/core/container/mixin.rb:145:in  `[]'
        from /usr/src/app/lib/dry/system/container.rb:493:in  `resolve'
        from /usr/local/bundle/gems/dry-core-1.0.1/lib/dry/core/container/mixin.rb:132:in  `resolve'
        from /usr/local/bundle/gems/dry-core-1.0.1/lib/dry/core/container/resolver.rb:36:in  `call'
        from /usr/local/bundle/gems/dry-core-1.0.1/lib/dry/core/container/item/callable.rb:16:in  `call'
        from /usr/src/app/lib/dry/system/container.rb:639:in  `block in load_local_component'
        from /usr/src/app/lib/dry/system/component.rb:64:in  `instance'
        from /usr/src/app/lib/dry/system/loader.rb:47:in  `call'
        from /usr/src/app/lib/dry/system/loader.rb:33:in  `require!'
        from /usr/local/bundle/gems/zeitwerk-2.6.15/lib/zeitwerk/kernel.rb:34:in  `require'
        from /usr/local/bundle/gems/zeitwerk-2.6.15/lib/zeitwerk/kernel.rb:34:in  `require'

To Reproduce

spec put in spec/integration/providers/resolving_root_key_spec.rb

  context "lazy loading" do
    it "is thread safe" do
      threads = []
      2.times do
        threads << Thread.new do
          Test::Container['animals.cat']
        end
      end 
      expect { threads.each(&:join) }.not_to raise_error
    end

Expected behavior

Expected running(or triggering) of the provider steps are thread safe

My environment

• Affects my production application: YES
• Ruby version: 3.0.6
• OS: Linux/MacOS

[flash-gordon]

Can you please explain why you load a provider lazily? I think I haven't stumbled upon this problem because I load my app eagerly.

[FLemon]

Can you please explain why you load a provider lazily? I think I haven't stumbled upon this problem because I load my app eagerly.

Hi @flash-gordon, we now finalize the container in production, but still want to lazy load in development and other non production environment for performance gain

[flash-gordon]

I think you should be safe with your approach, I also use puma in dev mode without eager loading but I haven't encountered the issue.

[FLemon]

we didn't encounter the issue until recently whenever there's intensive high traffic in a small time period, and it is happening more during the moment of rolling update when puma receives heavy request hit upon fresh start/restarted worker

[timriley]

FWIW, I think this would be good to actually make threadsafe, and would certainly welcome PRs for this :)

[solnic]

Seems like lazy-registration is not thread-safe, yeah? Resolving is supposed to be thread-safe because dry-container is. It should be relatively easy to fix it in the AutoRegistrar.

[flash-gordon]

At the same time, an app is kind of expected to be fully loaded in production. Currently, we don't have a standard way to tell a dry-system container which "environment" it operates in. If we had it, it'd be possible to load the app in production eagerly by default with an option to opt out.