Enchantments aren't applied properly (1.21+)

[LilyKensa]

When using an unenchanted tool while having an enchanted tool of another type, the unenchanted tool will be regarded as enchanted by the client, resulting in an error in the calculated mining time

This leads to a desync with the server, and welp, no block can be mined now

Here's a video exemplification

[Ryhon0]

Werid bug, might be related to another bug I experience.
When you start mining a block, which you don't have a valid tool for and then start mining a block with a valid tool without efficiency in 1st slot and same tool but with efficiency applied in the next, it will not apply the mining speed bonus. Switching to the enchanted tool fixes it until you try to mine a block with no valid tool.
getEquipmentChanges should update the mining speed bonus correctly but it doesn't. Setting the synced hand stack doesn't seem to do anything