Weapon swapping without another weapon

I'm not sure if this has an explanation or i missed it somewhere.. Or this is all possible and i wasn't able to figure it out.. But i feel like you should be able to weapon swap without needing a second weapon.. to get the benefits of skill changes. or if you're using a bow/quiver or dual wielding, to swap only one of the weapons/quiver.