The follow is a report on the state of the technolgy used during ThunderLAN. We have a number of servers and other tech involved, and as we conduct each event, it’s a chance to learn and improve. I don’t remember the last time we had an event and not made improvements. Reporting the status and the improvements to the website may prove interesting to some. So here we go.
One of the first problems we ever faced tech wise, was internet access. I can safely say that getting internet access is a problem of the past. We treat the internet as just access, enough to log into services needed for games, and check websites, but not as a download medium. Moving forward, we will be introducing load balancing, internal DNS, and possibly running games that need to run over the internet or connecting with remote players.
Currently, without load balancing, a single computer has the potential to saturate the internet connection, which in itself is not great, but also dashes any hope of using the internet for anything more then what we currently do. Internal DNS allows us to easily offer services, like the internal website by using a simple website name rather than an IP address. We have been using the OLSH systems for this task, but I would like to see ThunderLAN able to run under its own steam, in case changes are made that don’t suit our needs.
In regards to switches, in this last year, we have bought new ones that are laid out on the tables, and have worked out very well. We also recently swapped our core switch for 2 enterprise class layer 3 switches, which are performing strongly. These switches should serve us very well for the foreseeable future. The new core switches can perform tasks that used to require physical servers to perform. Speaking of servers.
Servers are probably the thing we have most iterated on over the years. We started by trying to be very ambitious with what we offered, setting up complete server manager solutions, with configuring it up for each game server we thought we would need to run. While the idea was a noble one, with the intent of making starting and configuring servers as easy as pressing start on a webpage, so that anyone could run the servers for that night; actually setting up and maintaining such a system was chewing up way to much time. We were also setting things up on servers that had come over from the Chubb LANs, and I think even the eagle training LANs before that. We then inherited some desktop computers from OLSH, which meant we had to reset up the servers again on new hardware.
In the last 2 years, we have redone the servers again, but implementing lessons learnt over the years. Currently we have 2 servers, a game server and a file server. The game server, as the name would suggest, runs the game servers. Instead of trying to run a game control software to manage the servers, we moved to plain scripts or per game config tools, for starting the servers. We also build documentation for config and admin commands for the servers, making it easy to find. The file server holds the files, serving them up as both windows file shares, and via FTP. The file server also hosts the internal website, which is a relatively new addition. The website not only provides links to the downloads, and makes finding them as easy as typing ThunderLAN.local into a browser, but also provides support info and tools for the games, making getting the games up and running, quicker and easier. This website will continue to grow and evolve.
One of the biggest changes is moving the servers to be virtual machine based. Instead of having to rebuild the serves each time we change hardware, they now exist as virtual machines, and are deployed on the physical server of choice. This abstraction of the physical and logical layers of the servers, makes moving forward much easier, removing the need for rebuilds, and allows for backups and iteration to be easier.
Moving forward, we will be looking to get better physical servers. Once obtained, we can simply move the virtual machines onto these new hosts, keeping all the setup of the game servers intact. Our main interests are in getting 10gb+ network adapters, making deploying games magnitudes faster, and moving to proper dedicated server hardware in general.
Each ThunderLAN, as we play games, we build up the servers for each game. Before an event a couple days will be spent updating servers, doing basic checks to see if they show up and can be connected to etc. The last event we had CounterStrike Global Offensive, Renegade X, and Call of Duty 2.
Unfortunately we had issues with both CS:GO and Renegade X. For CS:GO we use a dedicated config tool recommended by valve, both the game and the tool got updates, that caused conflicts that only showed up once actually playing. The tool wasn’t applying the round count, and setting it to 0 causing it to loop the game. We applied the setting directly and that got the game running for the night. It also wasn’t applying the correct map group, causing it to load maps that don’t support buying weapons. Both issues have now been fixed. Renegade X also got a big patch just before the event, which unfortunately caused the game to be unplayable for many people. We’ve reported a lot of the issues and they will be worked on in the next 3 months before the next event. COD2 was perfect, the only issue with the game was it taking a little while to start up, which is an old, common problem for the game.
The lesson learned from this last event is that simply checking if the server is connectable is not enough. Following ThunderLANs will have a dry run session of the games that are organised for the night, testing both the game in gameplay and multiple clients.
Hopefully this report gives you some insight into the ThunderLAN operation, and is an interesting read. I hope to have more info on changes and improvements made after each ThunderLAN posted on the website.