Rogue Exile + Golden Charm

Have ran into this twice now, I'm not sure the specific Rogue Exiles (one was a female ranged, bow maybe? and she was wisp possessed. The other was male ranged artillerist?) and neither caused my golden charm to proc when I killed them. So far rares and bosses have caused it to proc, just those 2 haven't.