Technical solution to eliminate desync in single-player sessions

"
...a dead giveaway that you're out of your depth in presuming to speak authoritatively about its engineering. Seriously, do you have any relevant industry experience at all with networked game simulator development?


I love it, I make one post about solving desync, and suddenly every single PoE player is a senior network engineer.



"
In a turn-based RTS game, all user input across all simulators is synchronized in lock-step with each discrete turn cycle.


A 'turn-based' RTS? Haha.

Also, there are numerous RTSs/MOBAs that use a fully deterministic system. Gas Powered Games are big on this sort of thing, and ive gotten a lot of my inspiration from them for my post.



"
With a game like PoE hosted on asynchronous networked simulators, user input occurs asynchronously and is propagated across simulators subject to unpredictable network delays.


A 'game like PoE' huh? Did it ever occur to you that 'genre' has nothing to do with how the game is programmed?



"
Due to the asynchronous nature of the simulation, it is not possible to deterministically reproduce a gaming session solely based on recorded user input and random number seeds.


Sorry, you are wrong. It is possible, and I wrote code to show how to accomplish it.
Last edited by qwave#5074 on Nov 22, 2013, 2:00:41 PM
Also, I vote that we change the topic to strictly address using this proposal to solve the movement/mob pathing related to desync. We should stop discussing combat algorithms, because the community does not appear to be in favor of it. However, I believe there is a lot of potential to use this system strictly with player position and mob movement.
Last edited by qwave#5074 on Nov 22, 2013, 2:18:12 PM
Again, I think stuns and freezes will mess up the synchronization, however it might be an improvement. And yes, Scrotie proposed exactly this some time ago in this thread.
"
Again, I think stuns and freezes will mess up the synchronization, however it might be an improvement.


Maybe, though id have to give it some more thought. If the client can't simulate a hit, then stuns/freezes could potentially desync it. Quite a conundrum.
Last edited by qwave#5074 on Nov 22, 2013, 2:27:50 PM
"
LogoOnPoE wrote:
"
RogueMage wrote:
Spoiler
"
LogoOnPoE wrote:
Regarding "very few rare games" - every game that allows you to view accurate replays, and that's most of RTS games these days, rely on fp determinism.

The fact that you (and HellGauss in another case) would blithely reference the deterministic behavior of turn-based RTS games as if it were relevant to the architecture of an ARPG like PoE is a dead giveaway that you're out of your depth in presuming to speak authoritatively about its engineering. Seriously, do you have any relevant industry experience at all with networked game simulator development? I'd expect even the lowest entry-level QA tech to understand the following basic distinctions:

In a turn-based RTS game, all user input across all simulators is synchronized in lock-step with each discrete turn cycle. Nothing is generated asynchronously, each step of the game is quantized into a single simultaneous turn by the server and reproduced identically on all clients. All network latency is subsumed within each turn cycle, eliminating any possibly of client desynchronization by shackling all simulators to worst-case network delays. As a result, a game session may be preserved and reproduced perfectly, simply by recording the sequence of user input and random number seeds used to generate the session.

With a game like PoE hosted on asynchronous networked simulators, user input occurs asynchronously and is propagated across simulators subject to unpredictable network delays. Each client simulator runs open-loop without supervision by the server, until its reported behavior diverges unacceptably out of sync with the servers'. The client's game state is then corrected to resynchronize it with the server whenever necessary. Due to the asynchronous nature of the simulation, it is not possible to deterministically reproduce a gaming session solely based on recorded user input and random number seeds.

Armchair speculations about the feasibility of retro-fitting a commercial ARPG like PoE into a strictly deterministic simulation are naive. An engineer who has not attempted this type of project before is signing up for a harsh, hands-on education in real-time failure analysis.
Spoiler


You still don't understand the proposed solution after my explanation some pages back. Did you read it? If you did, I give up trying. I guess you wont get it.

Your proposal was examined and dismissed back on pages 53-55 independently by Rhys as well as me. Nothing you or your cohorts have speculated about since have resolved those objections, and neither Rhys nor I have seen fit to discuss it any further. Your weak attempts at mockery in the post above are too half-hearted to bother with, I think current readers would find your earlier performances more spirited:

http://www.pathofexile.com/forum/view-thread/626664/page/53
I would still want that, you all improve on this ideas, if possible. Some of you have good knowledge and interesting ideas even though most of them are not executable, but maybe @GGG team gets their idea upon yours.
GGG thank you for all the great things you are doing. You have combined every element of all other great Rpg's and joined them together as one Diamond, that will shine Forever.

This is coming straight from the heart <3
This is a brilliant proposal!

MAKE IT HAPPEN GGG

Best post i read in years!

qwave for president!

To all those brains who think "..but it's hackable then" ROFL who needs to hack when he actually can PLAY. Wow what a crazy competition on the ladder, i really need that hack and some coke and some... to get my demigod ohhhrly
"
Rhys wrote:

"
adghar wrote:
I've often wondered why GGG doesn't just hire a few more codewarriors with lots of experience in the industry and/or with client/server synchronization and just hammer out some better code.

It's very hard to find experienced game programmers in NZ. And importing people from overseas is hard because we can't match the salaries of most US/EU companies. We kinda have to rely on finding people who wanted to move to NZ anyway... and even then, we aren't the only company here super-keen to hire these people.

Which EU countries and companies did you check? What is avereage salary in NZ for a senior programmer?
There are huge differences in salary in EU. Germany ~2,5k€ month and lets say Czech republic ~1,8k€ Slovak republic ~1,5k€. Those numbers are not 100% correct but at least you can get an image. You will pay 1k more to german developer which may or may not do the better job.
IGN : Mettiu
Last edited by Mettiu#4055 on Nov 22, 2013, 3:51:06 PM
"
Mettiu wrote:
You will pay 1k more to german developer which may or may not do the better job.

It's often a matter of consistency and reliability. I've worked with some absolutely phenomenal and brilliant guys from the former Eastern Bloc. I've also worked with a lot of self taught duds from the same area who were only brilliant in their own minds.

The key difference between programmers I've worked with from the States and those from places like China, Brazil, Eastern Europe, and India (actually have yet to work with anybody good from India, they've all been horrible) is that the average guy from the States is better. There have been geniuses and morons at opposite ends of the spectrum in all the groups. But the middle of the road guys have been more capable and reliable when from major industrialized Western nations.

That matters when you're talking about importing programmers to a foreign country as you're often shelling out a good bit of cash in paperwork, move related compensation, and incentives. Not to mention the cost of getting them ramped up to speed and integrated into your development process. You're always going to be rolling the dice on any employee but weighting that die to give you a better chance at a favorable outcome can sometimes make sense even if the per hour cost is higher.

Language issues can also come into play and you pay a premium to avoid them. Average programmer salary for a given area is a lot less than average programmer who speaks fluent English salary for a given area.
@RogueMage
The thread is about single player game. This does not means that multiplayer is not an issue, but it could handled in another thread. Furthermore it seems to me that the proposed method really fit into GGPO requirements for multiplayer game, so it is a road that it is worth investigating (however i agree that it is not a simple walk).

Consider also that before each software engineerization there is a lot of work with 'pencil and paper' to define the main behaviour of the software, and this is the aim of this thread. Nobody is denying that a lot of problems have to be engineerized and solved and that this would require work.

Wall of text...
Spoiler

Let's analyze how a RTS-ARPG game should work using this architecture, from a single player prospective. As long as the timestamp is discrete, the Real-Time-Simulator is actually a turn-based game. Who are the players? Me and the Server. However the aim of the server is not to win, but only to ensure a challenging and fair gameplay experience.

As in any deterministic game, players do moves. My moves ar input commands. The server is required to do some moves in order to not let me play 'alone', so that i must somewhat adapt my moves to something which is still unknows: in this view, the idea of sending only one initial rng-seed (+ loot info) and let the client go 'alone' is unfeasible, because i could easily repeat the entire simulation until i get a good result.

What is happening with the current implementation? Besides, of course, that is not deterministic, the server 'moves' are monster positions, combat results, and a lot of data. These are being send only once in a while. Furthermore each of these server moves is an unpredictable result of my moves, and the fact that i should send commands to the server and wait for his response, leads to a lot of inconsistency between what is on my pc and what is on the server.

What a good thing would be if server moves were independent on my behaviour and consist on only few bytes.... Is this possible with the proposed architecture? Yes.

In the proposed framework the only 'moves' of the server is loot, craft, (this leads to some desync, but usually loot does not bite) and....a brandly new rng seed each 4 seconds. This seed is independent on player behaviour. The advantage is that server can send this seed a few seconds before it is needed, because the server has not to evaluate player's moves to choose the new seed. Monster moves and combat result are defined in the game itself and in the rng, and does not need to be communicated.

What should happen in practice?

When i enter an istance, the server send me all the required initial data. Then at server-time 00:00:00 it send me 2 rng seed: i will use the first seed for rolls between 00:00:00 and 00:00:04, while the second seed at :04 up to :08. To scramble things, the rng-rools depend also on my behaviour, as i described in a previous post.

For some unknown reason (lag) i receive these seeds when the server time is :01. But it doesn't matter: for me it is time :00 and i can play up to time :08. While playing i send my commands to the server [edit: i timestamp using my time]. For another unknown reason, these commands have 2 secodns of delay (total=3 seconds). Let's say that the server can accept commands timestamped up to 5 seconds before its time (of course, it cannot accept timestamp in future). My gameplay is still safe and well defined.

When does the client send me the third seed, for playing in the :08 - :12 range? It can send me this seed at server-time :04. Even with 1 second of lag, i would not notice it because i have a 'seed buffer' up to :08.

I think that the logic works.

The only real problem of this architecture in multiplayer is that if i cast a command, this command may be later 'invalidated' by other player behaviour. In such a case the 'official' version of the instance should be as if that command has not been sent, and the client must perform an update of the game status (resync). However this is a problem that with current architecture is greater: we may have invalidation up to 5 seconds, since this is the frequency of full updates if i do not \oos. Using a p2p network, a deterministic framework would ensure full updates with a delay equal to the maximum player-player lag, and without an explicit request to the server, which has still only the role of a validator and rngseed-provider.
Roma timezone (Italy)
Last edited by HellGauss#6525 on Nov 22, 2013, 4:47:15 PM

Report Forum Post

Report Account:

Report Type

Additional Info