Skill gem gains xp only when used

I find it strange I cannot find any thread related to this idea..
So basically a new idea where there are a lot of new numbers to implement. But this however will automatically balance certain skills like summoning (since you don't summon all the time, the leveling will go very slow) and it adds a more depth to the game. For example: if you are a high level and you find a skill you want but it's a low level skill. You'll need to make a serious effort to be able to use the skill instead of just putting it in your gear and wait until it has become powerful enough to use.
Of course you have the issue of aura's. But aura's like Anger can gain xp when you hit someone, and aura's like the one that gives you an energy shield gains xp when the energy shield gets hit. The energy shield xp system is in fact a great balance (if you are a ranger or witch you make sure you don't get hit often, so the 'safer' players won't have too strong energy shield and a maurader gains more energy shield which he needs more than the rest)!

This kind of system is in fact the only reason I played Oblivion much.

I do like the system, it's just that I think it'd make the gem leveling feature more of a hassle than it needs to be.

The main issue that rises from this is that the skill that the player uses the most becomes powerful, while the skill that they use the second most becomes not quite as powerful. At that point, why would they use the less powerful skill? In order to keep each of their skills up to a similar level, they'd have to use them in nearly equal amounts even if they didn't want or need to use their other skills as often.

your idea would promote bots leveling skills like in WoW...