Reasons Why Desync Happens (Thoroughly Explained /w Cited Commentary On Fixes & Focuses Info)

Perhaps this can shine some light on why Desync happens for those who don't know:

‘Sync Errors’ occur in multi-player games because they are attempting to run an identical simulation on multiple machines. That means that a centralized server is not running the simulation, but instead with enforced latency and a queue of all user inputs, multiple clients are running an identical simulation all at the same time. A lot of games do this because it removes the overhead of server-side simulation and upkeep, and it also makes implementing instant-replay pretty easy.

Ever notice how you can't get a terminal (crashes the game entirely) sync error in games that have dedicated servers? You can just reconnect and get right back in the action.

The downside is that if either of the clients' simulation no longer matches up with the others, the simulation is undefined and cannot continue.

"So which version of reality is the right one?"

Here's a few reasons that a game can go out of sync:

- Random number generation that is not properly deterministic
- Floating point representation and math that is not properly standardized
- An error in the transport of input information between the clients
- ANY case where doing the same thing twice can produce two different results (critical hits?)

More than not, you'll notice there is virtually nothing you (players) can do about this. It is the job of the game developers to ensure sync errors do not occur. Unfortunately, this is a hard problem to solve! Upgrading your internet and computer can help, but in the end, if the software were written well enough, Sync Errors would never happen even with sub-par hardware.

However, we can list the things which can make those Desync happen more or less often:

Your computer can't keep up with the global simulation: It can be because your computer is too slow, or because your ping to the Server is high. In any case, you can't exchange data fast enough with the Server to keep up with the current state.
Your connection bandwidth is insufficient: Some games exchange far too much data to keep the clients in sync. If you can't send or receive enough to keep up with all the data necessary, you will Desync.
Your connection's quality is poor, and you lose a lot of Packets: If you play on WiFi with a bad range, you are likely to lose Packets. You will lose information the Server sent, or the Server won't get your changes. If the game doesn't handle this correctly, it can lead to Desync. Note that lost Packets can happen even if you have a great connection. It can just happen anywhere between you and the Server.

"

ScrotieMcB wrote:

What GGG Really Should Be Focusing On With Desync:

1. One-Way UDP for temporary things, Two-Way TCP for Commands and permanent changes[/b]
Right now, Path of Exile using TCP for everything.
Read this: UDP vs. TCP.
That Author goes a little too far, since TCP would still be smart for certain things (player mouse clicks and keyboard presses, inventory management), and in today's world we should assume all routers are already using TCP in some form at all times (cells sharing WiFi)... but the enthusiasm is good.

2. True Packet Information Entropy (both brevity and compression)
Right now I doubt GGG has taken efforts to make Packets compact, and is instead focusing on making them readable by humans, which greatly increases bandwidth consumption.

This isn't exactly non-standard for the industry; for example, the standard HTTP header* that comes attached to web pages you view looks something like this (which would be shrunk about 15% with lossless compression):

Example HTTP header

GET /tutorials/other/top-20-mysql-best-practices/ HTTP/1.1
Host: net.tutsplus.com
User-Agent: Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US; rv:1.9.1.5) Gecko/20091102 Firefox/3.5.5 (.NET CLR 3.5.30729)
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: en-us,en;q=0.5
Accept-Encoding: gzip,deflate
Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7
Keep-Alive: 300
Connection: keep-alive
Cookie: PHPSESSID=r2t5uvjq435r4q7ib3vtdjq120
Pragma: no-cache
Cache-Control: no-cache

However, if we convert it to this, it's 32% smaller, through brevity alone, loses zero meaning, and further compressible to 39% smaller:

Brief HTTP header

GT /tutorials/other/top-20-mysql-best-practices/ H/1.1
Ho: net.tutsplus.com
UA: Mo5.0 (Win; U; WinNT 6.1; en-US; rv:1.9.1.5) Gko/20091102 FF/3.5.5 (.NET CLR 3.5.30729)
At: t/html,a/xhtml+xml,a/xml;q=0.9,*/*;q=0.8
AL: en-us,en;q=0.5
AE: gz,def
AC: ISO-8859-1,utf-8;q=0.7,*;q=0.7
KA: 300
Cn: k-alv
Ck: PHPSESSID=r2t5uvjq435r4q7ib3vtdjq120
Pr: no-c
CC: no

Read this: Entropy, Compression, and Information Content.
Reducing the size of Packets using these methods would help with bandwidth issues (both for GGG and the players who have them) and also help ease the Packet flow issues between TCP and UDP that where mentioned by the author in the TCP vs. UDP article.

* Although I've run Packet Capture on PoE before (to determine that PoE is 100% TCP), I haven't decrypted the Packet Contents, so I'm using this as a blind example. It's very hard to tell what GGG is doing on this front.

3. Smart Sync Rates:
Players in town need Sync Information a lot less often than players in the middle of combat with Rhoas. The amount of anticipated combat stress on a character should guide how often synchronization Packets are sent out, allowing for more sync attempts in heated situations at the expense of situations where they are not needed, allowing better performance with the same total server bandwidth.

"

Dao wrote:

What would you consider ‘UDP-worthy’?

You mention Mouse and Keyboard is not, as I assume, in your opinion their order and continuity are too important. What part of the data transferred between the Client and Server is not important enough to be ordered, and for that matter can be considered temporary?

The Server cannot even entrust the map to the Client (the reason being maphack,) but would you be satisfied if the map would just appear on your Client in random sequence -- one part missing in the middle and another appearing?

Off the top of my head, I cannot think of a single thing that is less (or more for that matter) important than Keyboard and Mouse, and not important enough to be out of order. There are no temporary things, only one big continuity. In my opinion, in places where desync matters most, continuity matters most as well. Then again, I am not a game designer.

"

ScrotieMcB wrote:

First things first: All UDP Packets would have a simple timestamp on them (Server Time, since Server is only UDP Sender). All Client Calculations would pretty much start with "IF ThisTimestamp > LastTimestamp" -- so getting an out-of-order UDP Packet wouldn't revert any game state. It would simply be discarded.

Since we're not anticipating desyncs longer than a minute (timeout before that), a short that incremented each millisecond, looping back to zero after max, would do the trick.

The importance of the UDP Protocol here is that we do not care about order here -- we just care about the newest information, and the older information can be completely ignored, hence, no need for ordered delivery with the oldest Packet always arriving first. (It also helps that lost Packets won't be retransmitted since they'd be moot by the time they got there anyway, and a waste of bandwidth.)

That said, TCP should be used for things like these:

• Player Commands: User uses SkillA at position X , Y
• Zone Changes: Entering Entrances / Exits, Portals, Waypoints, etc.; Player is dead, Character League Transfers; Login / Logout, Character Selection
• Inventory: Unidentified Unique Glorious Plate from ground to User Inventory: Items X , Y , Z available at Vendor; transfer Kaom's Heart from Inventory to Stash
• Character Maximum / Permanent Stats: Max Life, Max Mana, Player's Current Level, XP to next Level, Passive Point Allocations, Remaining Refund Points; Name, Class
• Party "Permanent" Stats: Name, Class, Level of Party Members; Party Member Death; Players leaving the Party
• Simple Idle / Timeout Detection

Many of these Server-to-Player TCP Communications would require Client TCP Acknowledgement; you don't want to start streaming UDP Position Data while the Client doesn't know you've changed zones yet, or the Client doesn't know it has a new Party Member.

Pretty much everything else -- everything that would fluctuate often during combat -- would be UDP:

• Current Position: of Player, Monster, Projectiles, Items on ground (only goes TCP once picked up); Mini-Map Info
• Character State: Life, ES, Mana, XP
• Party Info: % Life, % ES, Current Position (including Mini-Map)
• Monster / Minion Data: Monster Type, Facing Direction, Alive / Corpse, White / Magic / Rare / Unique, Affixes
• Animation Data: Identifier, Position, Current progress, End Time
• Buff Data: Curses, Auras, Charges, etc.; Level of Effect, Time Remaining

None of these would require TCP Acknowledgement because newer UDP Information is anticipated to be en route anyway.

I probably didn't cover all of them because a bunch of details / minutiae is involved, but the core concept is that if it doesn't change much, use TCP, and if it changes very often, use UDP.

Note that some are kind of arguable. For example, you could argue sending the Monster Type, White / Magic / Rare / Unique and Affixes Data TCP since it doesn't change much, but this might delay, but then you'd need an ACK to start on the UDP, which might slow things down, so I opted to just resend it many times. (This is why compactness is important).

"

Dao wrote:

I highly doubt GGG made a Protocol similar to http, and I highly doubt that in the Multiplayer Gaming Industry it is standard or common. Then again, I am not a Game Designer.

"

ScrotieMcB wrote:

In my experience, the emphasis is on debugging clarity, which leads to very human-readable data. It's rare to see a strong emphasis on compactness, and instead to see a focus on understanding how the client misinterpreted a received communication.

"

Dao wrote:

Any Data Transfer is in fact a Sync attempt, and if the player and mobs are not moving, it is not a combat situation, which means no Sync attempts necessary.

"

ScrotieMcB wrote:

However, players are moving in town -- even when the player playing the game isn't. Obviously we don't need an update on that, nor on the player's progress from Point A to Point B after a command to move to Point B, as much in town as we need it in non-town areas.

We can send less Packets and rely more on the Client to predict player positioning.

Nor do we need it as much when a player moves from Point A to Point B in a non-town area when not in any danger whatsoever. Which means you need some kind of monster proximity/quantity test to measure danger.

"

SilasOfBorg wrote:

HeavyMetalGear, I realize you mean well. But please don't make statements like this:

"If the software were written well enough, Sync Errors would never happen even with sub-par hardware.

"

HeavyMetalGear wrote:

Why, because it holds some truth? I've played countless other FREE and NOT FREE online games that either have way more players playing, beg for players to have more computer performance, etc. and I have never experienced desync as bad I do in PoE in my 14 years of on-and-off online gaming, regardless if using a Wireless connection or not.

That's the reason why GGG is hesitating on fixing this issue because it would require a huge overhaul somewhere in the code/writing of the software because somewhere along the lines there's an error, and then an error behind that error, and another error in the code behind that error . . . The list just goes on.

I never said it was easy to do/fix. I said had they been more careful in the writing of the software we wouldn't have the desync issue as bad we do now. That's not implying that we would be 100% free of desync; I am saying it would be a lot less.

Now they have this massive issue on their hands and it's a conundrum within an aneurysm to fix! The devs brains lock up when trying to deal with it.

GET /tutorials/other/top-20-mysql-best-practices/ HTTP/1.1
Host: net.tutsplus.com
User-Agent: Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US; rv:1.9.1.5) Gecko/20091102 Firefox/3.5.5 (.NET CLR 3.5.30729)
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: en-us,en;q=0.5
Accept-Encoding: gzip,deflate
Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7
Keep-Alive: 300
Connection: keep-alive
Cookie: PHPSESSID=r2t5uvjq435r4q7ib3vtdjq120
Pragma: no-cache
Cache-Control: no-cache

GT /tutorials/other/top-20-mysql-best-practices/ H/1.1
Ho: net.tutsplus.com
UA: Mo5.0 (Win; U; WinNT 6.1; en-US; rv:1.9.1.5) Gko/20091102 FF/3.5.5 (.NET CLR 3.5.30729)
At: t/html,a/xhtml+xml,a/xml;q=0.9,*/*;q=0.8
AL: en-us,en;q=0.5
AE: gz,def
AC: ISO-8859-1,utf-8;q=0.7,*;q=0.7
KA: 300
Cn: k-alv
Ck: PHPSESSID=r2t5uvjq435r4q7ib3vtdjq120
Pr: no-c
CC: no