Continuous FPS drops

Game was running pretty well until I got to maps. When I started maps, even just basic tier 1's and 2's, the FPS immediately took a hit. I dealt with it for a while, but now that I'm at T16's, the game is starting to get very hard to play with my FPS dropping to 10's-20's whenever I get a semi-big pack. It even drops a bunch of FPS when I'm just fighting a boss or a unique on their own. It's like the game is suddenly more demanding?

At a loss for what I could do to make the game more playable for me. Tried a lot of the universal fixes I've seen people post like disabling sound in the config, lowering or even increasing settings, drivers, deleting shaders, changing channels, admin, Nvidia control panel, all that jazz.

My specs are a GTX 1070, i7-6700k (not OC'd), 32gb ram, 165hz monitor if that matters.

Any ideas or help would be much appreciated, thanks!