GGPK Defragmenter

"
PatrickJr wrote:
The script works fine for me on the Steam Version, is there a reason for us to not use it on the Steam version?

Steam checks hashes of the files for all the game you have installed. They need to be the same as the ones in Steam repository. Since GGPK Defragmenter changes the file, Steam no longer recognizes it as a valid one and re-downloads it.

Every time GGG publishes a patch through Steam, they need to apply it to the core version that remembers all the previous releases so all users get the same patching done. If GGG would clean the file and publish it on Steam, all Steam-based users would get a gigantic download, like they were downloading the game for the first time (since the file is like 99% of the game). It wouldn't matter for people with fast and unlimited internet, but for everybody else it would be a pain in the behind.

Since GGG on it's own client uses a different patching method, you can defragment the file and still get all the patches done + save some space and possibly lower load times. I've just made a switch form Steam to standalone client because 1. I can defragment the file in the future; 2. When Steam is down PoE servers are still online and I can play the game :D
Is there anyway to change the location where it saves the files while defragmenting? because i don't have enough disk space on C where it keeps temp files and my game is installed on another disk that has enough space
File reduction: 4228.07 MB
Thanks!

My GGPK file went from 19GB to 15GB.

However, I lost 10GB of Disk Space and the defrag program shut down on its own without me pressing any buttons to terminate it. I'm not sure where to clean up the loose files on my computer so I can get the space back on my SSD? Where does this program store files temporarily?
Defragmenting G:\games\Grinding Gear Games\Path of Exile\Content.ggpk
This will likely take a while...
Elapsed time: 5.14 minute(s)
File reduction: 3296.1 MB(s)
Successfully defragmented the GGPK.

More than I expected, thank you for making this nifty tool.
"
Jonathan wrote:
We actually have a similar tool for this internally, but our one is an in-place defragment rather than input to output.

Old pack files can get quite fragmented after a lot of game patches. This is especially noticed internally where our developers are patching multiple times per day.

I should mention that the the patcher does reuse free space for deleted files, it's just that there are a lot of oddly sized small chunks left over after filling up available free chunks with new data.

This is exacerbated by the patcher trying to do atomic operations so that it can be closed at any time without causing corruption or loss of already downloaded assets. This means that if we add a file to a directory, for example, we have to make a new directory entry with the file added, commit that to disk, then swap the parent directory entry pointer over to the new one, then commit that to disk.

As files are added or removed it can involve directory entries moving around a lot throughout the pack file, adding to the fragmentation.

There are some improvements I'd like to do one day to prevent quite so much fragmentation, especially of the metadata, but in the mean time, feel free to use this tool.

EDIT: By the way, that code is very clean. Great job!


Why are all the game resources integrated into one GGPK file, is there any benefit to doing this? Does this make loading faster or slower?
Just removed about 3GBs after legion update oO
"
jygxgxx wrote:


Why are all the game resources integrated into one GGPK file, is there any benefit to doing this? Does this make loading faster or slower?


That file is formatted to have its own internal file system. You can improve the internal logistics quite a bit if you implement this correctly - using specialised formats can speed up query times tenfold.
Isn't deleting ggpk and downloading fresh one a better solution? Because the only downside I see is bandwidth cost for people with limited one...


Thanks for the tool, lets hope steam patches faster now

Report Forum Post

Report Account:

Report Type

Additional Info