Crash on loading screen (Vulkan and Dx 11)

Game constantly crashes or freezes on loading screens every 1-2 hours. Event viewer shows 3 things:
- Kernelbase.dll
- just PathOfExile.exe
- Kernelbase.dll + nvidia driver (nvlddmkm) stopped responding and sucesfully recovered

The problem is Im sure the driver is not crashing because:
- Im mining on second GPU connected to the PC and the application is very sensitive to every driver crash or too high OC causing it
- OBS also doesn't crash but should if the driver really stopped working (crashed)
- I've tried to lower the OC anyway and problem still exist

Im sure I have up to date drivers. I've tried to switch to Dx11 and it's also crashing. I've also tried to reinstall the game. I've never had this kinda problem and I think I noticed it after 3.16.1 patch and it became more frequently since yesterday. 3.16.1b didn't fix the problem, checked it

Here you can see come screens from event viewer. Im sorry it's in polish:
https://gyazo.com/8bfb6599ca7a8ce3266de9df6cf7a9a9
https://gyazo.com/5bccc596e2a9f2d5071bd2d1007c9062
https://gyazo.com/29056d7dffe4407c726fb1c871f500ff
https://gyazo.com/3b0dde9597cb443b10e679798ad4ed28

That's what I found in the logs at the specific time of crashes:
- 2021/11/23 23:47:04 17777234 2a5 [DEBUG Client 11348] SetLocationForced: Attempted to force an invalid location/position, type = Metadata/MiscellaneousObjects/Doodad
- 2021/11/23 22:28:16 13048859 31cb [CRIT Client 6876] [VULKAN] vk::DeviceLostError::DeviceLostError: vk::Device::waitForFences
- 2021/11/24 00:39:51 20944031 8f [INFO Client 11192] Doodad hash: 4049280138
The next line after them is startup (***** LOG FILE OPENING *****)

--------Update--------
I've been playing for 6 hours today and no crashes. I didn't change anything in windows/bios/OC etc. Im wondering if those crashes were related to the server and the client was freaking out giving even a false nvidia crash error to the event viewer. Strange...

I got the same problem now

game crash almost everytime on loading page