rollercoaster performance (no gpu utilization?)

When using Dx12 (after checking some more it also happens in vulkan) framerate goes up and down every couple seconds even when standing still in hideout (even with passive tree opened). Changing settings doesn't impact that behaviour (even with everything turned to lowest it can go). Game would run perfectly smooth with 144fps (7 MS) and a couple seconds later it would slow down to 70fps or lower (15 MS) and feel really choppy, then 144->70->144->70.... F1 graph shows only GPU bar changing. When checking GPU usage it shows 80% usage but 600MHz clock speeds (during the time when its 144fps GPU clocks are at 2300MHz).

On a side note Dx11 doesn't even load. Game just freezes until I click cancel.


Specs:
GPU: Radeon RX 6950 XT
CPU: i5 13600k
RAM: 32GB DDR5 6000MHz
game installed on M.2 SSD


--update--
This persisted for multiple game restarts until I turned all the graphic settings down to the lowest they could go on vulkan, turned triple buffering on and off, turned everything to basically the highest and now it's smooth both on vulkan and Dx12. I don't understand this game (or my PC) anymore.