Curse

Inde · Sep 03, 2006, 04:12 AM // 04:12

Just sat down tonight and had some long talks with some tech gods. Very enlightening and I do believe their simple explanation could quite possibly explain a number of issues such as why so many people err7, why reconnects are not in the game, and more. Now just for emphasis this is a theory. It should not be taken as fact. So presenting you:

Guild Wars vs. MMOG's - The Tech Side

MMOG's: They have servers, the servers know where you are, and what you are doing at all times. You log off it, knows you log off. You get booted, it knows you didn't log off, so it can wait for you. When you log back in it says, "Oh Inde just logged in where does she belong?" And it puts you there. Also in groups they get a signal that you were booted (since you didn't actively log) and it awaits a signal that you returned. So with persistant worlds you can go in and out of connections more readily since the servers know where you are at all times. Problem: all those servers cost a lot of money.

Guild Wars: All common areas/shared space are hosted by ArenaNet. From the understanding and investigation of several people I've spoken too, once you leave a common place into a mission/instance you are removed from the ArenaNet servers and begin to go into peer-to-peer connections. So you get booted from the instance and reconnect. You hit ArenaNet's server and it says, "Hello Inde, I have NO clue where you were or who you were connected with nor do I care, welcome to the world." It puts you into the last common place you knew. Problem: you lose the persistant connection but it's CHEAP because you are connected to all your peers versus having datacenters full of servers. This also explains why ArenaNet does not have "worlds" such as other MMOG's but "districts". They don't have to have 100's of servers per world such as World of Warcraft or other MMOG's. They only need as many servers that would support just 1 world to run everything they have to deal with for Guild Wars. This is similar to how Battle.net worked I believe for StarCraft.

Now for the issue of reconnects, and why they have not been implemented such as in other MMOG's. They can't do it as other MMOG's, they are having to find a new way. Losing connections in an instance, when you are peer-to-peer in an instance and you suddenly cannot see another member, it simply drops you (The dreaded 007 [how ironic Licensed to Kill]). Why? Because it doesn't know where you are anymore. So why do multiple people get err007 at the same time? They loose a key peer and they're all sitting there trying to connect to that peer and cannot so Boom another dead group.

So who's problem is it? Not really ArenaNet's unless you drop in a shared area. So all these err7's are your problem. You are the responsible party losing a connection whether your own ISP is being inerrupted, or someone else in your party. But should they do something about it? Yes, they need to figure out a way to improve the handling packet losses or member disconnects (such as having redundant connections waiting for someone to drop). From my understanding, all the tracert's and pings in the world to the Guild Wars server mean nothing when you err7 and drop in an instance, it's your friends you need to try doing the tracert's and pings to and finding out who has the unstable connection since it their computers you are connecting to.

Now I don't know all this for sure, I'm certain some will have more tech knowledge then I but this explanation certainly explains a number of things for me.

EDIT: Interesting discussion beginning, please read through the posts below as we dispute and argue the article before directly quoting what I have said above

nirhan shadowmauler · Sep 03, 2006, 04:18 AM // 04:18

hmm. maybe now that someone respected in the community has said it. all of us that arent total idiots can point to this when we try to explain why you get so many err 007s.

Inde · Sep 03, 2006, 04:20 AM // 04:20

I was a very vocal voice that all the err7's were on ArenaNet's side. So don't give me any credit, of course I've never seen or been told an explanation like that above from anyone at ArenaNet or the community so drew my own conculsions. I'm hoping it can help others to see the problems and enlighten them as it did myself

EternalTempest · Sep 03, 2006, 04:32 AM // 04:32

Thank you for this post, this is some great info on how GuildWars runs on the back end.

Phaern Majes · Sep 03, 2006, 04:48 AM // 04:48

Well just a question then. Why is it when I do drop in an instance, I still have all the items, gold, xp, and skill points that I managed to get? If we aren't connected to the Anet servers how does it know I got everything, if I dropped without a proper logging off?

Carinae · Sep 03, 2006, 04:49 AM // 04:49

But I've had the Error=007 without party members while in towns and in parties with just hench.

arcanemacabre · Sep 03, 2006, 04:55 AM // 04:55

All very possible, but as you said, we don't truly know the way Anet has structured their servers or their code. Honestly, I'd say that peer-to-peer for each instance is only one of many ways it could be done. It does make the most sense.

If it is true, it could be easily tested (in-game) by soloing a little. If you have lag problems, then most likely the problem is on your end. Until Anet tells us that's how their architecture is, though, we can't truly know.

Inde · Sep 03, 2006, 05:04 AM // 05:04

All right more findings. I just logged in with some GWG staff and we were consistantly connected to the ArenaNet servers (which of course explains how it can remember your golds, items, etc.) We couldn't find a direct peer-to-peer connection, but it doesn't mean that the connection to the server wasn't just routing us to others or that the data isn't scrubbed or monitored. Most likely even a combination of both, that you are hitting the ArenaNet server and offloading the instance state to a peer.

I just think the connection to many issues we are all talking about; so many with err7's, reconnects that are in every other MMOG, no monthly fee (because peer to peer is so much cheaper), the fact that we have districts versus worlds as in other MMOG's, and more points to a peer-to-peer connection. It makes sense to me, but maybe someone can enlighten me further as to why GW isn't like this. So let the debate and information continue.

seut · Sep 03, 2006, 05:09 AM // 05:09

Quote:

Originally Posted by Inde

...you are removed from the ArenaNet servers and begin to go into peer-to-peer connections....This is similar to how Battle.net worked I believe for StarCraft.

Sorry, but i seriously doubt that's the way GW works. (Unfortunately i can't find Jeff Strain's "we don't trust the client side"-quote.) If i remember Jeff's last year interview correctly, then GW is using a client-server architecture for each instance to make the game 'hack-proof'.
I agree with the assumption, that a reconnect-feature would require storing more data, leading to more traffic, resulting in higher costs. Anet's business model might not cover this.

Demesis · Sep 03, 2006, 05:11 AM // 05:11

What about some of us who aern't tech 'leet'? Because really if there's a problem with my computer or isp or whatever that's causing the lag, I won't know how to fix it.

Inde · Sep 03, 2006, 05:12 AM // 05:12

Seut, read my post right above yours where I'm saying this:

Quote:

We couldn't find a direct peer-to-peer connection, but it doesn't mean that the connection to the server wasn't just routing us to others or that the data isn't scrubbed or monitored. Most likely even a combination of both, that you are hitting the ArenaNet server and offloading the instance state to a peer.

And in an IGN interview I did find this:

Quote:

In the Q&A section this week we discuss the possibility of Guild Wars using peer-to-peer technology. We’ve contacted ArenaNet, and they inform us this is not true. All towns and instances are hosted by their in-house servers. http://xbox360.ign.com/articles/722/722527p1.html

This still does not mean that they don't offload the instance state to a peer, therefore reducing the CPU cycles needed to maintain the instance. Which explains why you are always connecting to the server (to catch hacks or cheats). And I also don't thinks it's coincidence that Mike O'Brien, founder of ArenaNet, was one of the original creators of the Battlenet system.

seut · Sep 03, 2006, 05:25 AM // 05:25

Quote:

Originally Posted by Inde

Seut, read my post right above yours..

Sorry, i'm a little slow in writing english

Jeff Strain explained parts of Guild Wars' architecture in an interview (i think it was before the release of Prophecies, unfortunately i cannot find it).

gr3g · Sep 03, 2006, 05:27 AM // 05:27

As you have already realised that a full p2p setup won't work, I'll make this as brief as I can.

Assuming the design is as you predict, with one (or a few) central world servers and several distributed instance servers, then I am sure there is some note somewhere in the central world servers as to which instance ID you have been sent to. This is necessary because it has to transfer your character record to the instance server, at least, but also because when you are done with the instance that your character information can be percolated back to the central servers.

Therefore, the problem of reconnecting is fixable as follows: when the instance learns that the link to you has been broken, it uses a callback to inform the central server. When you connect back, the central server knows which instance to shunt you back into.

The problem then becomes how to synchronize connections with an instance after a period of disconnection. For all I know, this might be impossible or extremely hard. Depends on how stateful GW's protocol is. Does the instance server keep a record of everything that has happened in the world----every damage, every death, etc.----or is that info stored in the clients? If the latter, then when you reconnect the instance server will have to ask the clients for a "what's up?" to send to you, which will surely mean lag for every one. (Assuming it's possible.)

Or we could just be barking up the wrong tree. We're basically just speculating on GW's network design here.

Inde · Sep 03, 2006, 05:34 AM // 05:34

seut, Jeff Strain, or anyone else at ArenaNet cannot give away their entire server architecture tradesecrets. I found an interview that Mike O'Brien stated it's not peer-to-peer that you indeed connect to the arenanet servers. That the game itself is on the ArenaNet servers. I don't think the arguement is that the game itself is not on the servers. I guess their justification and definition of what exactly is peer-to-peer could be a debate in itself. But I'm almost convinced that the state of the instance is most likely maintained off guild wars servers.

Here's the podcast with Mike O'Brien but he never clearly answers the peer-to-peer or the "how do you keep the costs so low" question:

http://www.pcgamerpodcast.com/index.php?paged=2

arcanemacabre · Sep 03, 2006, 06:03 AM // 06:03

The more I think about it, the more I realize it's probably not P2P at all. The key thing that disputes this, I think, is rubberbanding. The only reason rubberbanding happens (as far as I know), is because the server says your character is at point A, and the client says your character is at Point B, and correction ensues.

Obviously this can only happen if there was constant communication between the client and server. When lag steps in, a number of things can happen on the server-side without the client knowing about it, at least until the lag stops, or you are d/c. This can happen when grouped and when soloing.

P2P is the most cost-efficent way to go, this is true, but I think in this case Anet has opted for hack-proof over cost. Frankly, I love them for it.

Inde · Sep 03, 2006, 06:13 AM // 06:13

But arcanemacabre, that doesn't dispute the peer-to-peer or what we are discussing with the instance state being on someone's computer. If I'm trying to ping someone who holds the instance state and they show me as being 5 feet away from where I'm at on my computer the rubberband affect could still be valid. It wouldn't matter if it was the server or another person's computer, rubberbanding could still happen.

arcanemacabre · Sep 03, 2006, 06:24 AM // 06:24

Quote:

Originally Posted by Inde

But arcanemacabre, that doesn't dispute the peer-to-peer or what we are discussing with the instance state being on someone's computer. If I'm trying to ping someone who holds the instance state and they show me as being 5 feet away from where I'm at on my computer the rubberband affect could still be valid. It wouldn't matter if it was the server or another person's computer, rubberbanding could still happen.

I'm saying it does this even when soloing... which means it's not pinging off anyone else. Unless you mean P2P with everyone else who's playing, not just in your own instance?

LagunaCid · Sep 03, 2006, 06:27 AM // 06:27

And what would decided whose PC would be the 'peer'? My internet sucks, for example, I get disconnected from it numerous times when playing. And I have never dragged other players.

gr3g · Sep 03, 2006, 06:33 AM // 06:33

I just did a bit of traffic analysis, and I receive about 25x the amount of data I send to the GW server in a single RA round. This makes me think that the state is not being stored on my end.

Here's why I think no significant game state will be stored on clients: because many people use asynchronous connections. My ADSL rx speed is a little less than twice my tx speed. If I were to be serving data to 7-23 other people, my connection would be swamped. This definitely doesn't happen to me.

What is possible is that my own state is stored both locally and at the server, and that the rubberbanding that arcanemacabre mentions is the result of my local state getting ahead of the server's state because of delayed synchronization.

Inde · Sep 03, 2006, 06:36 AM // 06:36

LagunaCid, I of course don't know their entire protocol so no one can answer that. As I said, there are certain server architecture secrets that will never be given out by ArenaNet. While you have never dragged other players, there are other reports from players stating that when they have err7 others have automatically in their party been dropped as well. So this is just a discussion more than anything we could pin down without a response from a tech person at arenanet.

And arcanemacabre this is just theories here. But I do think, as I explained above, it does give insight and perhaps an answer into the subjects of why there are so many with err7's, reconnects that are in every other MMOG, no monthly fees, the fact that we have districts versus worlds as in other MMOG's, etc.