Lich Minions + Sigil staff swap BUGs SPIRIT, minions do not respawn

Hi,
I've a lich minion build and a staff that plants the sigil, also with some dedicated points in the passive tree.
After 2 o 3 uses of the sigil+swap spirit reservation is correctly calculated but less and less minions appear, sometimes switching back takes them back, sometimes I'm left with 1 or 2 minions.

I have:
rattling scepter lvl 16 with 2 minions
spectre with 1 o 2 terracotta soldiers
storm mage skeleton 1 for damage: 8
storm mage skeleton 2 for debuff: 1
clerics: 1 or 2

When it bugs out I'm basically unable to proceed because it keeps randomly 1 mage/cleric or 2, with 2 clerics within boss fight it's over since I don't cast anything and rely solely on minions.

I had the chance to fight for long (30 minutes) a boss and seems that by removing and re-putting the skeleton gem on it resets the bug