"VK : Queue: Submit: ErrorDeviceLost" and Performance issues.

Vulkan currently causes the "VK : Queue: Submit: ErrorDeviceLost" when Multithreading is also on, without it the performance is obviously far worse, with DX12 it is more stable but I want to use the real time capabilities it has to render, however while with vulkan I notcied players load incredibly slow, in particular their assets do not load, obviously in Rogue Harbour where a lot of players exist at once it becomes messy in both APIs, but due to how vulkan behaves it is less impactful to performance with the aforementioned side effect.
This is more of a bug report I guess, as my PC is capable for the modern standards, I can play other similar modern titles well enough to not post of any issues, hopefully poe 2 wont have these issues, but with how things currently are, it is rather worrysome that after a few leagues of not playing consistently I have come to notice a massive performance reduction in the game.
I also do not find using global illumination to be something that should be necessar, side note here but the performance cost can be big and the older copies of the towns and assets which do not need those technologies look objectively better without them, i.e. lionwatch act 1 vs act 6 (if you disable GI you will notice in act 1 it is not needed while in act 6 it is and the GPU usage is significantly higher).
Just small details that hopefully will help out and contribute. Obviously vulkan handles real time asset loading a bit smoother but it is very bugged.
Cheers