Archive

Archive for the ‘The West’ Category

The 1.20 release

July 4th, 2009 19 comments

1.20 has been released on the German world 1 – quite successfully I would say as there haven’t been so much problems, although quite a few bugs of different severity haven’t been found during the beta testing and we had / have to fix a lot of bugs.

I have to say that I’ve been a bit disappointed by the beta testers regarding cheating: I would expect beta players to try to cheat in the game wherever possible. For example: It’s been possible for attackers to set the starting position on the flag points just by modifying the requests that are sent to the server. I only got aware of that when some players were affected by another bug that occurred when they tried to join more than one fort fight. That happened only after the 1.20 release on world 1.

That’s been a quite bad bug that shouldn’t have been released, yet it got through. Over the months of development I just always “moved” the responsibility of checking illegal positions to other code parts that I had not been working on at that moment – i.e. when getting the values from the players and I wrote it into the database, I expected to filter illegal positions on the fortbattle server. When I implemented the fort battle server, I expected that the values in the database would be correct. Due to the different times when I wrote these parts, I never implemented that checking anywhere. Nice, isn’t it? Yet, such glitches are normally found by evil cheaters quite quickly and they can be used to their advantage. Just imagine a group of attackers spawning on the flag just when the battle starts. Wouldn’t be so nice for the defenders…

So I would like to remind our beta testers: Try to break our rules on purpose! Don’t avoid unfair play but enforce it! You won’t get banned for that (unlike as on the public game worlds). Some of you did already do some testing like that, but we need more of that (like making fort member towns ghost towns (sorry for not having fixed that issue yet on the beta server, but we are not community managers and we had quite busy weeks…)).

Anyway, I wanted to write here now about what all went wrong during the release and how it happened.

Thursday, 7:45 am

I turned up at work at that time because it was the plan to make the release in the morning time so that the first forts would be finished around 4 pm / 5 pm. We thus expected that then the first battles would happen at Friday around 4 pm / 5 pm. I started working on several minor bugs, like fixing a glitch in the fort management and a problem with that particular piece of javascript when executed with Internet Explorer browsers.

9:30 am

About at 9:30 am Anthraxx pushed the button to make the release. About 9:31 am Eiswiesel ran into our office and yelled “Build time for Forts is 30 seconds”. That’s how it’s been on the beta worlds. Uops. We immediately fixed that wrong value, but when we did that, 3 forts had already been built by players. 8 other followed because we forgot to change the values in the task queues that had already been inserted into the database. These players had of course a minor advantage over other players since they were the first to have forts. Some players asked to undo that unfair development but that is not a simple request. We just decided that these lucky ones were just lucky winners in the fort build lottery. Their prize was the immediate attack of other players.

And that was how it came that the first fort battle was to be fought at 9:37 am on Friday instead of 4pm as we planned to. But that was not really to our disadvantage however.

10 am

A few minutes later the first bugs reports dropped in. It was my change at that morning that on the one hand stabilized the IE browser but also introduced a bug that made it impossible to invite other towns. But other bugs got visible too – IE 6 users were unable to build forts because the name field of the founded fort did not turn up on their screens. A little bit later other players reported that a county had 4 forts while the one below had 3. Oups. Another oups when players found a cozy little location of another fort:

fort los angelus

Quite a nice location I’d say, but also quite wrong.  Later it also turned out that the county with 4 forts had not 4 but 3 forts and there was one fort missing on the map that is shown in the minimap. The other county with 2 forts in the minimap had however 3 forts and the 3rd one doesn’t show up in the minimap.

We still have no clue what to do with Fort Los Angelus, neither do we know what is causing the minimap to behave that strange.

Thursday afternoon

Ok, so the first battle was to be fought on Friday morning. I checked if the policy port (1028) was reachable and after a few tweaks it worked. I should have checked the other port as well then (1578? Too lazy to look it up now), but I didn’t. Instead I spent some time on implementing battle logs so that we can later replay the battles. Don’t ask for a playback function, I still don’t know if the recorded actions are complete since I lack a playback function myself. I did also fix some other errors, just like my 3 colleagues in the office.

Around the late afternoon a bug turned up that always existed but never became visible: Due to the fort battles, for the first time in west it made sense for players to gather at a single location. When about 150 players had been luring at the first fort that was declared war on, the player list functionality broke due to too many players at one spot. That was also eventually fixed at afternoon.

When I left the office I had an intensive 11 hour day behind me and I had a bad feeling about the 200 players who were about to fight the first fort battle on a public server. Due to the mishap in the morning time, I had to turn up around 8 am on Friday as well (I normally come in around 10 am).

Friday, 9 am

I still had not checked the second port when the first battle was about to be fought. I just expected that if the policy port was running that the other port would be forwarded as well. Well, checking the port was not that simple because I had no application to test that in a valid way anyway. Had no time to write such a tool.

And this is what went wrong when the battle started at 9:37:

We instantly noticed that we couldn’t connect with flash to the game. That was bad. Looking into the server logs we saw that the server was bombarded with policy file requests that got answered promptly. But no connect request came through. After a couple of WTFs all over the place, hastily hacking on console windows and checking connectivity we realized that of the 3 ports that the server opened, 2 were forwarded correctly and that was the policy port – and the maintenance port that shouldn’t be available to the Internet at all. The game port that is used for the flash communication was simply not addressed by anyone.

After we figured that out we were able to fix it in time and about 10 minutes later the connection worked to my great relief. We could view the battle taking place and the final report to be sent to the spectators.

Later on Friday we noticed that players reported that they were playing on the wrong locations when they joined multiple battles. Also we noticed that the final report was not successfully sent to the users (but that was not noticed by the players due to the lack of information that there should be a report shown). Turned out that the reports became larger than 65kb which was the upper limit for reports at that point. I fixed that bug along with the bug of illegal player positions.

Over the day several battles were fought and the battle server stabilized. We made a few minor changes then – and now I am curious how stable the java server is going to be over the weekend. That is quite a stress test that is currently running. But I am quite optimistic that since the server has survived now for quite some time that it will run stable for some time. Yet I am not sure if we fixed all memory leak issues and so I don’t know if the server is not going down within the next 48 hours. I can’t predict anything there yet, but we’ll see.

I hope that the next week is going to be a bit more calm so that we can fix a couple of other problems and do a bit planning for upcoming features. I would expect that after this huge version release that 1.21 is not going to introduce such large features but will only provide minor features that we couldn’t get into 1.20 yet – but we’ll have to see what we will work on for that release.

We will also see next week when we are going to unroll 1.20 on other worlds and also other languages. This could take the whole week maybe, but I am not sure on that. I hope that the players of other worlds are patient – it shouldn’t take so long anymore.

PS: I am getting constantly requests to increase the number of beta players on the beta world. I am aware that a lot of players want to get into the public beta but you have to understand that I can neither change the player limit nor can I make the decision myself. It has however been decided that the player limit is going to be raised in the future. I can’t really tell you more on that because I simply don’t know more about it myself.

Categories: 1.20, Fort battles, The West Tags:

upcoming release of 1.20

July 1st, 2009 13 comments

So, today we made the decision that 1.20 is to be released on the German world 1 during the morning time of Thursday. Other worlds will follow if it turns out to be running stable. Of course, since we have the beta world we are a bit more confident among the devs, but to be honest: There are a lot of places where we fear that something could go wrong.

As for my part I would expect that the fort battle server is going to have troubles in the first fight. When the first battle is being fought, there’ll be a lot of players involved and also a lot of spectators will try to watch it. However there’s a limit of how many players are going to be able to watch it. We’ve spent quite some time to prevent that an overflow of incoming connections are going to shut down the server. Instead it should simply denying new connections if there are too many connections open. We have a limit of 64 spectators for one battle and a limit of 512 connections in total. So in theory two battles could be fought at the same time or 4-5 when only fewer people are involved.

During the week I did also not only work on the server but also a bit on the client here and there. I added some artworks from our artists and also added the feature to the client that you can see who is actively playing in the game and who is not connected. There are also now popup information on the map so that it is easier to identify players on the map. Next to that we fixed bugs bugs and more bugs. Have I mentioned fixing bugs?

I am really curious how the release is going tomorrow, especially when the first fort battle is starting.

Welcome on the devblog

June 26th, 2009 15 comments
Hi,
I am Eike Decker, also as known as Zet. I am working at InnoGames as a developer since January and I have been assigned to The West at the end of februrary. The first project that I’ve been working on for The West was the tutorial mode for the game which was introduced in 1.19. Since then I’ve been working on the fort battles non-stop. I wrote the flash client as well as the HTML frontend for the battles and also the parts on the game server that are related to the game play. If something doesn’t work as expect in these departments I am probably the person to blame.
The fort battles are a big deal different from any other game that has been developed at InnoGames. The most significant difference is probably the quite intensive interaction between a lot of players during the battles which normally last between 5 and 10 minutes. We’ve had battles with up to 50 players running and so far I am quite content with the results.
Since the introduction of the fort battles on the public beta server I became more interested in user opinions and started reading a few threads about the beta and the fortbattles during the break. Because I have never played The West before starting working on the forts, I have a lot of things to learn about the game and the players. While reading about opinions on the public beta I realized how much the new features are awaited but also that some are upset because of the delay and the limited amount of players on the public beta.
I can understand the disappointment among the players about the delay and other problems and although I am not to blame for all of them I want to express that I appologize for this and that I want to explain this or that with this first blog entry.
The development of the fort battles
As mentioned above, the fort battles are a complete new feature that was developed practically from scratch. I don’t want to bore the audience with technical details but all that I can say is that the technical hurds have been quite complicated to take. While it was fairly simple to develop a prototype server and client, it turned out more to be a problem to integrate the game tightly into the game itself which was quite surprising for me. It is extremely difficult to estimate the development time of a feature and thus it was also expected that this new feature would take some serious time to be completed.
The development can be roughly divided into four components which partially reflect the gameplay:
– The HTML frontend that is used for joining the fort battles
– The Flash frontend that is used while the battle is running
– A Java server that communicates with the Flash frontend
– and a PHP backend that integrates the fort battles into The West.
The HTML frontend
The HTML frontend should allow players to set their starting position and targets that are used during the fort battles. It does not use Flash since we wanted to alow players to take part in the fort battles even if they can not use flash. The frontend should provide all important information about the battlefield and the positions.
The Flash frontend
We decided that flash should be used for the battle itself. It should establish a connection with the server and gather / receive information on demand. This was the key reason why we chose flash next to the additional graphic effects that Flash provides.
The Java server
For a couple of reasons we decided to develop a Java server which would be used only for the fort battles and nothing else. The reason for this was that Java is more efficient than PHP and that a crash of the Fortbattle server should not influence the game itself. The server simulates the game and communicates with the flash clients. In theory, if the Java server would crash, the battle would restart at round 1.
The PHP backend
Most of the database related processing is done in the PHP backend. It informs the Java server about new battles and once a battle is finished, a PHP process receives all results and takes all kind of actions on players and forts.
Features vs. Time, Time vs. Bugs
There are tons of features that did not get into the fort battles. There are plenty

Hi,

zetshotI am Eike Decker, also as known as Zet. I am working at InnoGames as a developer since January and I have been assigned to The West at the end of february. The first project that I’ve been working on for The West was the tutorial mode for the game which was introduced in 1.19. Since then I’ve been working on the fort battles non-stop. I wrote the flash client as well as the HTML frontend for the battles and also the parts on the game server that are related to the game play. If something doesn’t work as expect in these departments I am probably the person to blame. For anything else I am less likely to know what’s going on – I have only a limited overview on other features than the fortbattles. Simply because I have not done much else apart from that.

But as far as I can tell, the fort battles are a big deal different from any other part of The West. The most significant difference is probably the quite intensive interaction between a lot of players during the battles which normally last between 5 and 10 minutes. We’ve had battles with up to 50 players running and so far I am quite content with the results.

intro_blog_attackswarm
Attackers are swarming the defenders on the beta server

Since the introduction of the fort battles on the public beta server I became more curious about user opinions (especially on the fort battles) and started reading a few threads about the beta and the fortbattles during the breaks on the public forums. Because I have never played The West before starting working on the forts, I have a lot of things to learn about the game and the players. While reading about opinions about the public beta I realized how much the new features are awaited but also that quite a few are upset because of the delay and the limited amount of players on the public beta.

I can understand the disappointment among the players about the delay and other problems. And although I am not to blame for all of them I want to express that I appologize for this and that I want to explain this or that with this first blog entry.

The development of the fort battles

As mentioned above, the fort battles are a complete new feature that was developed practically from scratch While it was fairly simple to develop a prototype server and client, it turned out more to be a problem to integrate the game tightly into the game itself which was quite surprising for me. It was extremely difficult to estimate the development time of that feature beacuse of this.

The development can be roughly divided into four components which partially reflects the gameplay:

  • The HTML frontend that is used for joining the fort battles
  • The Flash frontend that is used while the battle is running
  • A Java server that communicates with the Flash frontend
  • and a PHP backend that integrates the fort battles into The West.

The HTML frontend

The HTML frontend should allow players to set their starting position and targets that are used during the fort battles. It does not use Flash since we wanted to allow players to take part in the fort battles even if they can not use flash. The frontend should provide all important information about the battlefield and the positions. Characters that are not controlled by connected players will move to their chosen destiny on their own. They are – no question – much less flexible as the characters that are actively played, however they can still provide the tip on the scales that changes the battle outcome.

The Flash frontend

We decided that flash should be used for the battle itself. It should establish a connection with the server and gather / receive information on demand. This was the key reason why we chose flash next to the additional graphic effects that Flash provides. We accepted the fact that a minority of players are going to be unable to play the game that way (for example if they don’t have Flash or if they are behind a firewall that they cannot influence) in favor of these technical advantages. We will try to bypass these problems in the future and I have already ideas how to work that out. But for the moment, the connection itself seems to be stable enough for the release.

The Java server

For a couple of reasons we decided to develop a Java server which would be used only for the fort battles and nothing else. The reason for this was that Java is more efficient than PHP and that a crash of the Fortbattle server should not influence the game itself. The server simulates the game and communicates with the flash clients. Because it is less of a problem to manage different threads in Java and also having more control on the memory, it also doesn’t cost too much performance. We have no facts so far, but we expect that this solution will work much better while using less resources than using PHP.

In theory, if the Java server would crash, the battle would restart at round 1. Of course this would also upset the players that are involved, but that seemed to be the most acceptable consequence compared with other outcomes of other technologies. We hope that this won’t happen and we also planned to restore partial gameplays once the basic features are working.

The PHP backend

Most of the database related processing is done in the PHP backend. It informs the Java server about new battles and once a battle is finished, a PHP process receives all results and takes all kind of actions on players and forts.

Features vs. Time, Time vs. Bugs

There are tons of features that did not get into the fort battles. There are plenty of ideas that we have for future releases and we tried during the design of the system to provide enough room to make these changes when we have the time to do them. We are only working right now on features that are eminent for the gameplay or that are mostly done. Right now we are denting out bugs that we oversaw so far.

At this point I also want to thank all beta testers who help us to find bugs and testing the game. Some hints had been really a great help to prevent that the release would fail. Of course there are no guarantees for the planned release, but I feel much more confident because of our beta testers.

intro_devblog_fort_hourglass

Future blogging

For the future, I am planning to blog constantly on development about The West. If this is not possible because new features are not yet to be mentioned yet, I’ll just come up with rants on technical details about what I already did at InnoGames.