The Marius.net interview

By Mordia
February 27, 2002

Bungie's boyish bundle of bubbly bathos, Mordia, recently had a heart-to-heart talk with Marius and Conner, the driving forces behind the Marius.net Myth server. The resulting interview is an interesting look at the engineering (and reverse-engineering) process, and it also touches upon some of the difficulties they encountered when they took on the task of maintaining the beast they'd created. A must-read.

Tell us a bit about yourselves. What is it that you do in the real world, how did you guys get to know one another, where'd you go to school, and what kind of beer do you prefer?

DAVE CARLILE: I went to school at a community college in Ankeny, Iowa. I accepted a programming job during my last semester, quit school, and never looked back. Conner and I met while working as programmers at Parsons Technology which eventually became a division of Mattel, Inc. He introduced me to Myth during that time, but I was still too impressed by first person shooters to get into it that much. Still, I closely followed his exploits with Clan Plaid, and even collected many of their t-shirts when they had leftovers. The incompetent Mattel management finally got to both of us, so we quit and job hopped together for awhile. We finally found ourselves at MCI about a year and a half ago where we're doing database development.

TODD SNYDER: Currently I am a database programmer for a large company that most people hate - hmm, sounds familiar, heh. I graduated from University of Northern Iowa in 1997 with a BA in Mathematics and Computer Science. I took a 'while' to get my degree, and worked night stock at a grocery store for most of that time as well. This means that my sleep habits are still stuck in 'night stock' mode - meaning I am usually up late to work on crazy things like marius net, then dead tired in the morning and drink a 6 pack of Diet Pepsi before noon just to wake up.

I met Marius about 4 years ago when I went to work for Broderbund/The Learning Company/Mattel/Mattel Interactive (I missed out of the Parsons Technology and Intuit names) I was there for about 3 years and the company was bought twice then the group I was in spun off. All without changing desks. While there we had a massive project to rewrite a mountain of code in an insane time frame. Marius showed he was a coding god and did the brunt of the work on that project, and though Satan - er I mean the CIO of the company thought it was crap - it did what was expected and over a year after leaving it is still being used more or less unchanged.

DAVE CARLILE: Um... There is only one kind of beer, and that is Guinness.

TODD SNYDER: I prefer cold beer. Other than that - I like Budweiser. I don't drink often anymore, but when I drink, I drink too much to drink anything 'fancy'.

Why did you decide to work on Marius.net? Did you originally intend for it to be a widespread alternative (or replacement, as the case may be) for bungie.net?

DAVE CARLILE: I was pretty sick of programming and decided I wasn't going to do it anymore (database development doesn't count as programming ;). I had gone 2 months without coding anything when Conner approached me with the idea of a bungie.net emulator. At first I didn't want to do it, but he twisted my arm until the idea sounded like too much fun to pass up. The compelling drive for me was the challenge of it, and I also knew how much Conner and his friends loved the game.

I think originally the idea was to have a temporary replacement for bungie.net until it returned, but I think we quietly doubted bungie.net would return if we were successful.

TODD SNYDER: I personally have been a TFL fan since it was released. I preordered TFL from Bungie, and have played it ever since. I became a member of Clan Plaid in January of 1998, and for the last 4 years Myth has eaten up most of my free time in one form or another.

When the bungie.net TFL server went down, then didn't come back - it became harder and harder to play TFL. TCP/IP games meant playing with the same people over and over, and trying to organize games was difficult. uDogs should be applauded for keeping the fire burning - they even started a TFL tourney during the down time.

Without any 'official' word on when the server might come back, and not really expecting it to come back knowing that Bungie was busy with Halo, I started wondering how hard it would be to create a server 'emulator'.

The server basically needs to sit there, accept info from the client, process and respond. I have written many apps that more or less do that in one way or another - but they weren't 'games', so I figured it could be done. Marius is the best programmer I know - and has done some amazing things in his 'free' time before - but never could seem to 'finish' a project because he would either get tired of the idea, or it became too big, or he would feature creep it to death.

I approached him with the idea of a bungie net replacement. It seemed to be a project that we could define a goal and work towards. Our goal was 'simple' - create a temporary server that would act more or less like the original bungie.net TFL server for the community to play on. Marius would do most of the programming, while I would tackle testing/research/planning/etc and the eventual web site.

How did you go about emulating bungie.net without the server code? What special tools, skills and secret powers did you use? How much help did Bungie give you? How long did it take to get up and running, and what kind of testing did you do?

DAVE CARLILE: We started with some information that Hans Birkeland (Vodi) had collected about the Myth 2 network protocol, and hoped Myth 1 was the same or very similar. Conner then used a packet sniffer to look at the data being sent back and forth between the Myth 2 client and the server in order to learn more. We initially made a partial Myth 2 server to get the basics down, then spent hundreds of hours figuring out the differences in packet structure in Myth 1. For a few of the more difficult pieces we used a disassembler to take apart the client code, and also a debugger to trace through the code.

If I remember right, from the time we started coding until the initial release was four weeks.

Early on we did all of the testing ourselves, but in the later stages members of Clan Plaid, uDogs, GfH and various other individuals helped stress test things.

TODD SNYDER: Vodi saved us a ton of time with his information. Without him our project probably never would have been started. After we got the 'connecting' working, we started tackling the other things using the existing bungie.net SB server as our model, and tweaking things for TFL as they arose. It would have been MUCH easier to do this if the TFL server was still up. There weren't many differences, but enough to give us headaches. Four weeks from asking for permission to do this (reverse engineering can be a slippery slope - so we asked Mordia if this project was 'OK') until announcing that we were live sounds about right. It wasn't fun, no doubt about it, but when we got that first game *played* on marius net, it was all worth it, at least for me. Testing consisted of rounding up as many #CP# and Friends of Plaid, logging in, jumping from room to room and so on. Because we were not sure if we could pull it off, we didn't want to make any public announcements or promises until we were ready to go live.

Kinda odd a person from Bungie asking how much Bungie helped... heh. OMG! BUNGIE SUXOR! THEY IGNORD US!!! Heh. Actually, you guys helped more than I expected. Both Marius and I have worked on software for years and been involved in projects with massive deadlines, so we understood the situation. Though it would have been helpful to get the TFL server code (tee hee) or had Modem fly out to give us a hand, we never expected it. Modem was able to clarify a few things here or there as he had time and Mordia was able to bug people for us as well. We could have got it done without this, but it did save us some time - mostly by eliminating some guess work.

How'd you do that neat thing where TFL'ers and M2'ers can see each other?

DAVE CARLILE: As it turns out, the Myth 1 and Myth 2 protocols are very similar. The packet flow is exactly the same for both, and only a few packet structures are different, so it was fairly simple to support both protocols and convert between packet structures where necessary. It took just 2 or 3 hours to get the Myth 2 support in there, and a couple more for fixing a few issues.

TODD SNYDER: Also, since we were using bungie.net SB for our model early on, we knew what the differences were, and knew adding SB wouldn't be overly tough. We looked at the plusses and minuses of having them together or separate, and decided together was better.

What kinds of problems and setbacks did you have getting marius.net up and running the way you wanted it... or at least, the way it does now? What kind of problems do you have with the server or players? Is running the server as much fun as you thought it would be?

DAVE CARLILE: There were two very major problems. The first one involved the rooms. We could get the client to display the available rooms, but they were all disabled except for the first one. We tried for a long time to get the others to work but just couldn't do it. We came up with a workaround we called "virtual rooms". I think .open was the first dot command we added. It allowed creating multiple rooms inside of a single physical room. It wasn't the optimum solution, but we figured it was good enough. After getting the virtual rooms working well, Conner spent a night pouring over hex dumps and finally discovered the now famous 0x0028 that belonged in a certain place in a particular packet. When we put that there the rooms magically all became enabled. We decided to leave the virtual rooms in as a feature. Long after we went live we started investigating why all of the rooms had a limit of 40 people. It so happens that converting 40 to hex gives you 0x0028, and I remember thinking "now where have I seen that before".

TODD SNYDER: This one drove us batty. It's one of those places that if the TFL server was running we probably would have found it immediately.

DAVE CARLILE: The other major problem almost ended the project because we spent days trying to figure it out without any luck. Everything was done - you could login, chat, change rooms, and create games. The problem was, all of the games were red, and try as we might we couldn't change that, and there wasn't any way we could work around it like we did with the virtual rooms. We had pretty much given up, but I finally gave it one more go in the debugger. I eventually found where it was expecting something in one of the packets to contain 0x0A, so I started sending that and viola! So basically marius net comes down to two magic numbers: 0x0028, and 0x0A.

TODD SNYDER: This one was heartbreaking. We felt like we could see the finish line and then ran out of gas. We were so desperate to get this working Marius more or less hexed the .exe so that the games wouldn't be red at one point - and we considered releasing the haxored .exe just to get it done, heh. Luckily Marius tracked this one down, the stuff he did to find it was far beyond my MaD SkIlLz. (:

DAVE CARLILE: The next problem came after we went live. We initially allowed anyone to connect since we had no way of figuring out the password encryption, and we hoped people would act responsibly. This worked okay for awhile, but it wasn't long before the abusers came and there was little we could do to control them. So we designed a workaround for dealing with the password encryption and began requiring people to register for an account, and added a number of tools for adminning.

To be honest, while we were developing the server I never put much thought into what running it was going to be like, and I was very surprised at how some people act. Overall people are respectful and obey the rules, but thanks to the infamous Dough Boy, the few who don't can be encouraged to do so using various convincing methods.

TODD SNYDER: Yes, problems with wanks. I expected it, but I didn't expect it to be so bad. Some people suggest that we make the server 'invitation only' - but that would have defeated the purpose that we created it for. So, I told Marius I'd handle the registrations if he coded it in. With TFL it wasn't too bad, and I was more or less processing all requests totally by hand. Over time, I was able to create various things to make it much easier to process logins. Good thing to - from Jan 1st to Feb 20th, we've received 1700 new username requests.

We still have problems with people who don't quite grasp the concept of the 'be nice, play nice' rules - and we have various ways of dealing with them. These problems are the main reason we have no intention of ever having a complete 'rank' system on marius net with the in game icons. The maintenance/abuse/headaches/cheaters and so on just isn't worth it.

Are you willing to share some of your future plans for marius.net?

TODD SNYDER: Lots of things on the list, but since we are doing this in our free time, I'd rather not comment too much and promise something then not deliver it. We are currently collecting TFL game data for game detail and game summary reports, this has been pretty popular and the game data itself is available in XML format for anyone who wants it, all they have to do is ask. We plan on having the same thing for Soulblighter - I've just been too busy with user requests to get it done recently. The Soulblighter game data is being saved, so once I get done I can process the existing game data. Also - right now none of the interface additions in SB 'work' - the find button, buddy tab, etc. However, we do have the 'dot' commands that work in TFL and SB, so the functionality isn't there, but we plan on coding some of these items as time permits.

DAVE CARLILE: None of our plans are concrete, but we have a few things on the list. We might do something to add support for orders and tracking scores for tournaments. I had also considered redesigning the internals to turn it into a generic chat client with pluggable protocols to allow any sort of game or chat oriented client login, but I'm not sure if I'll ever spend the time to do that.

Anything else you want to add?

DAVE CARLILE: I want to say thanks again to everyone who contributed to the development of marius net, as well as the admins and others who are helping us run it. It couldn't be done without you.

TODD SNYDER: Ditto.