Development Diary: How we got a local LAN party to test our Action RPG

Last weekend, we ran a special Path of Exile pre-alpha test event at the Lanholm LAN in West Auckland, New Zealand. 26 players played for 90 minutes, competing for an exclusive shirt and cash prize. The players had fun, and we learned a lot about running this type of event.

Pick an appropriate venue

Lanholm is a series of LANs that has been going on for almost a decade. Some of our team members met there many years ago, and it's an important part of our local gaming community. We heard there was a LAN planned for last weekend so we decided that it'd be a great opportunity to get some feedback on Path of Exile.

We decided not to publicise the event on our game forums beforehand. It was important to us that we got a wide sample of gamers and hardware, rather than a group of dedicated action RPG fans. If we didn’t know in advance that the event would have enough people, we’d definitely have had to advertise it a bit.

Plan the competition to be fun and intense

Some people at the event were friends who are on our internal alpha testing team. They’re familiar with the game, so we told them in advance that they were excluded from prizes. They were disappointed, but understood that we wanted to keep the competition fair for new players so that more people were interested and we would get better feedback.

We found that the “ladder rush” format worked really well for a 90 minute competition. The people at the top of the ladder were very close, and when it came to the end of the competition, they were fighting neck and neck to be placed first. When the time ran out, the winning player was only 5000 experience ahead (roughly 20 monsters).

We made sure to balance the amount of content available with the time limit that we had decided on. We were given a 90 minute slot in the LAN’s tournament schedule, so we enabled gameplay up to the end of the Brutus fight in the prison. This meant that the fastest players were just running out of new areas as the competition came to an end.

It would have been good to get more than a few hours of testing done on this build before taking it to the LAN. Although there were no major problems, half a dozen polish issues could have been corrected if we had noticed them beforehand. Next time we’ll try to finalise the build earlier than the day that we have to use it. In hindsight it would have been sensible to turn off some of the weird debug commands. Someone pressed F9, which puts the client into movie mode (causing it to run very slowly while taking high resolution screenshots to make movies from). They reported this problem as “Hey man, your game is really laggy”.

Don’t hack up a special build for the event - use it as an excuse to test your deployment process

One of our main goals for this event was to test our client and server deployment process. We wanted to use all of our existing build pipeline to package the special version for this LAN. It would run from a local server rather than one on the internet, and it’d only work at this one event. We did not want to assemble a “frankenbuild”, as that method had burned us in the past. The deployment worked perfectly, and was a great test of our build system.

We actually deployed our servers to a virtual machine that we took to the LAN. We were interested to know how well it would perform under emulated hardware (turns out there was minimal perceptible overhead), and we didn’t want to rip out any of the server machines from the build farm at our office.

We should have made it easier to enable people’s accounts at the event. Traditionally, our servers allow people to sign up for accounts and then we can add game access on a case by case basis (for example, adding people to the official beta test that starts early next year). We decided to retain this model for the LAN event so that we could control who gets access and also only allow players to log in when we were ready. Overall it was quite a lot of effort to tick the box on each player’s account as they created it, so it would have been easier to just allow unlimited access at the venue.

We made sure that the game client that we gave out was only usable at this one LAN. It’s built so that it can’t be used to connect to our alpha servers, and has to be replaced if we run another event. This was done on purpose because we expected that people would leave the venue without deleting the game client from their machines.

Distribute the game client before the competition starts

45 minutes before we started the competition, we began to distribute the game client to players. We were intending to stagger this so that there weren’t 20+ people copying it from the same server, but it turns out that gigabit Ethernet is more than fast enough.

We let players play on the server for a while before we started the competition. This let them get slightly familiar with the game, and allowed us time to solve any problems people had with their accounts. When the competition itself started, we took down the servers, made an announcement, and then wiped the characters/items databases so that players could start fresh.

Get usability and gameplay feedback from the players

Although it feels really important to help players and explain everything to them, it’s often useful to stand back and watch them struggle with complex features before you explain them. Some things that we thought were obvious were missed by players (an item that says “Right click this item then left click another one to identify it” had people asking “How do I use this?”. Also, some people would walk right through the town area and not talk to the clearly marked NPCs, failing to get some important skill gems). We noted down many areas that could be improved with better explanations or visual cues.

Gather as much data as possible with an instrumented game client

In retrospect, we should have instrumented the game client substantially more. Although we paid attention to which machines were performing poorly (and asked what graphics cards those players had), it would have been great to log the average frame rate against the hardware specs of each machine.

We were interested in seeing what proportion of character classes people would pick at the event. There were two available, and we were pleased to see an almost equal split of character chosen.

Try to have fun yourself

Thankfully, everything went fine for us and people had a lot of fun playing. We received a lot of good feedback and many of the players expressed interest in bribing us for beta access. I imagine it would have been stressful if things were going wrong, but it was a great afternoon hanging out with other gamers and showing off the progress we’ve made on our project so far. Hopefully our beta testers feel the same way early next year!