Spectre's gone twice! - Phrecia

Hi, after summoning my first 3 spectres, I wanted to upgrade the quality of my helm (which wears the spectres gems) knowing that my spectres will just fall dead on the ground and I'll be able to revive them.
Well, only one of the 3 remained on the ground, the rest disappeared.
I tried it once again to make sure and same thing happened.

If spectres were cheap I'd get it but I can't spend at least divine orb each time i have to unequip my helmet for whatever reason.

Is this fixable? It's very annoying.