Berserk not working after weapon swap

I came across this issue playing a hollow palm character. As I'm pretty starved for links, I tried to set up a berserk on a weapon swap, the idea being to get to max rage, swap weapons, activate berserk and then swap back to get the unemcumbered bonus. However immediately on switch back, the berserk buff stops working even though the rage continues to drain as if it were active. Also once the rage drains to zero, the berserk skill cannot be activated anymore unless you exit and re-enter the map.