Interacting with objects and items on the ground during combat

Not sure if that constitues a bug, but interacting with items on the ground or other objects during combat is very difficult with controller. I understand that there might be some preventive measures not to interrupt combat, but picking up something valuable or interacting with important objects is nearly impossible during combat.

It is very apparent during the first ascendancy hourglass trial, the one where you have to run along the path and quickly interact with hourglasses. It is very difficult to interact with these hourglasses during combat and it basically forced me to avoid this room during my trial attempts.

It's due to the auto-targeting taking focus away and then the interact (X) button casts whatever skill is bound there on that button.

The best advice I can give is remove all skills from X so that it always acts as interact, even when there are enemies nearby. Takes some getting used to but it is a huge improvement. And yeah, those Death Crystals in the Trials are near impossible to do with a controller without this change.