|
We're investigating what caused it!
Last edited by Chris#0000 on Oct 31, 2012, 5:46:27 AM
|
Posted byChrison Sep 10, 2011, 12:37:15 AMGrinding Gear Games
|
|
It was the underpants gnomes.
|
Posted byardikus#2772on Sep 10, 2011, 12:38:49 AM
|
|
great response time guys. it's awesome and appreciated!!
|
Posted bytake#3634on Sep 10, 2011, 12:39:01 AMAlpha Member
|
|
This problem was caused by a segmentation fault in the master server along with a failure in the systems designed to automatically restart some parts of the realm in response to the problem.
While the website daemons reconnected, the instance spawners got into a state in which they were not authenticated with the restarted realm and also failed to restart.
In addition, our monitoring system didn't have any email events set up for the "Incorrect number of instance spawners" event.
Here are the three changes we will be making in response to this incident:
1) Investigate and fix the cause of the initial crash.
2) Fix the logic that restarts an instance spawner if it is not correctly authenticated with the current master.
3) Make sure we send emails on more of the monitoring events.
While we can never promise that the realm will not crash during beta, the realm shouldn't ever be down for a sustained period. We apologise for this incident and hopefully these steps will mean that it will not happen again!
Path of Exile II - Game Director
|
Posted byJonathanon Sep 10, 2011, 12:51:01 AMGrinding Gear Games
|
|
Thanks for the prompt response and explanation guys.
Your communication has been awesome.
Crashes are to be expected in early beta.
|
Posted byCovertghost#3458on Sep 10, 2011, 1:41:00 AM
|
"
Covertghost wrote:
Thanks for the prompt response and explanation guys.
Your communication has been awesome.
Crashes are to be expected in early beta.
agreed with above.
luckily it was only down for an hour and chris was around to reboot it :)
|
Posted byHaggard32132#7886on Sep 10, 2011, 2:25:07 AM
|
"
Jonathan wrote:
This problem was caused by a segmentation fault in the master server along with a failure in the systems designed to automatically restart some parts of the realm in response to the problem.
While the website daemons reconnected, the instance spawners got into a state in which they were not authenticated with the restarted realm and also failed to restart.
In addition, our monitoring system didn't have any email events set up for the "Incorrect number of instance spawners" event.
Here are the three changes we will be making in response to this incident:
1) Investigate and fix the cause of the initial crash.
2) Fix the logic that restarts an instance spawner if it is not correctly authenticated with the current master.
3) Make sure we send emails on more of the monitoring events.
While we can never promise that the realm will not crash during beta, the realm shouldn't ever be down for a sustained period. We apologise for this incident and hopefully these steps will mean that it will not happen again!
Disregard witches, aquire currency.
|
Posted bydust7#2748on Sep 10, 2011, 5:48:42 AMAlpha Member
|
|
I like the detailed response. Cheers guys
IGN: Cadenza - Account Name
SheHulkPoRN - Critical Staff Sweep
ExquisiteVaginalPoker - 1h Axe/Shield Cyclone (Bringer of Rain)
LollipopCamelTOE - 2H Axe Cyclone
(Closed Beta since August 4th, 2011)
|
Posted byCadenza#3170on Sep 10, 2011, 6:25:34 AM
|
"
Jonathan wrote:
This problem was caused by a segmentation fault in the master server along with a failure in the systems designed to automatically restart some parts of the realm in response to the problem.
While the website daemons reconnected, the instance spawners got into a state in which they were not authenticated with the restarted realm and also failed to restart.
In addition, our monitoring system didn't have any email events set up for the "Incorrect number of instance spawners" event.
Here are the three changes we will be making in response to this incident:
1) Investigate and fix the cause of the initial crash.
2) Fix the logic that restarts an instance spawner if it is not correctly authenticated with the current master.
3) Make sure we send emails on more of the monitoring events.
While we can never promise that the realm will not crash during beta, but the realm shouldn't ever be down for a sustained period. We apologise for this incident and hopefully these steps will mean that it will not happen again!
My God... This post has single handedly won me over. I will never leave PoE or GGG.
Communication.... amazing.
|
Posted byIZombie#0630on Sep 10, 2011, 9:02:37 AMAlpha Member
|
|
i agree +1
|
Posted bywraith676#6086on Sep 10, 2011, 10:11:29 AM
|
