Light errors across chunk borders with Sodium 1.20.1

[Scuffleshadow]

The dynamic lighting has difficulty crossing chunk borders with sodium installed. It leaves an ugly shadow, and takes about a second for it to realize it is meant to be lighting the area. In further testing, I discovered that disabling the "Alyways Defer Chunk Updates" option in sodium fixed it, but turning that setting off reduced my fps by 85%. (Unplayable) Fancy setting was on (for lambdylights).

Steps to reproduce the behavior:

1. Enter the game with just sodium, and lambdylights (and fabric api and modmenu ofc).
2. Enter any world, and walk across a chunk border while its dark, and you are holding a light.

Expected behavior
Light would simply act as normal, and not have that weird visual bug, where its only lighting up the chunk you are in.

Screenshots
https://imgur.com/a/ymBM4r3

Desktop (please complete the following information):

• OS: Linux
• Minecraft: 1.20.1 (fabric)
• Fabric Loader version: 0.15.11 AND 0.16.0 (Both have the bug, not tested other versions)
• Mods: fabric-api 0.92.2+1.20.1, lambdynamiclights 2.3.2+1.20.1, modmenu 7.2.2, sodium mc1.20.1-0.5.11