The End of Culling in Guild Wars 2, a.k.a. Choose Your Own Lag


Guild Wars 2 (GW2) launched with a system called “culling”, explained by Habib Loew of Arena Net (AN) as follows:

In order to cut down on the network resources that were used by the game and to reduce the client-side system requirements, we implemented a culling system, which imposed a limit on the number of characters that could ever be reported to your client. This meant that only the characters closest to you would actually get reported.

While this achieved our goal of limiting bandwidth and client-side resource utilization, it had the unfortunate side effect of causing large battles in WvW to be confusing, as there were sometimes many enemy players that were completely invisible. Additionally, there were side effects of culling which could result in stealth characters getting up to two seconds of additional invisibility when coming out of stealth.

While culling created some negative experiences and was a sore point with the community, it also helped to prevent the kind of client-side lag due to mass combat that I’ve experienced in every other MMORPG, including WAR, Aion, and SWTOR. In those games in mass combat, the frame rate dropped significantly and the game client sometimes froze for periods of time as it struggled to render all of the characters and their actions.

ArenaNet’s Solution to Culling

In the article linked above, Habib explained that culling is being removed, and that AN is providing 3 settings that enable you to tune the experience / performance related to characters in WvW:

  • How characters are rendered: high resolution, low resolution “fallbacks” until the high-res loads, or nameplates only
  • The WvW character limit: how many of the reported characters render with a model and how many are rendered only with nameplates
  • The WvW character quality: how many of the characters rendered with a model use the high resolution models and how many use the lower resolution fallback models

The new system will allow players to fine-tune their WvW mass-combat experience, or as I like to call it: “choose your own lag.” We’ll have the see these changes in action, but on paper AN is providing an innovative and flexible solution for a very difficult technology problem not unique to GW2 or even MMORPGs. For most games, you have control over the graphics settings in general but not as much control as the 3 settings described above for character rendering.

I have a very solid Sager gaming laptop, but I know it’s not beastly and I hate lag, so I’ll probably try out the low resolution “fallbacks” option for how characters are rendered and with a limit of 25-30 characters, and see how well it works. For WvW, the most important thing is knowing the numbers of your opponents, then after that knowing what class/spec each player is. I’d rather have lesser graphics than lag that impacts performance.

Server performance will probably not be improved by these changes. It’s possible that with a higher amount of data for characters being sent from the server to the client that server performance may be negatively impacted.

This brings us to the next point: server performance.

The Removal of Culling Doesn’t Address Server Performance

There is a lot going on in real-time in an MMORPG: each game client talks to the server to relay each character’s actions, the server manages all the mob AI and combat calculations to determine outcomes of ability uses (damage, healing, players states due to CC, etc), and the server communicates back to the game clients what is going on, and each game client renders this information so that you see the visual effects of the combat, updates to HP bars, etc.

It’s important to note that culling does not address server-side load, which is why even with culling in place there are times when zones in WvW have server-side lag. After all, even if other characters are culled, the server still knows that they are there. You can tell when there is server-side lag when you and other players experience a noticeable multi-second delay before an ability activates.

What is causing the current server-side lag? I’m not sure, but ArenaNet made at least one design choice which increases server load: the way conditions were implemented.  Colin Johanson discussed the performance of conditions when explaining why there is a cap of 25:

Colin: Currently no. Interesting statistic for you: every condition in the game costs server bandwidth. ‘Cause we have to track how often the condition is running, what the duration of that condition is and what the stack is. So the more stacks we allow them more expensive it gets because we’re tracking every additional stack on there. And so we could, say, you can have infinite stacks. Number one: that becomes really unbalanced. But number two: it’s actually extremely expensive for us, on a performance basis. That’s one of those weird, kind of back-end server issues that can help make game designer decisions regardless of what you want to do with it.

But there’s more to tracking the condition stacks on a target – consider also that condition damage is dynamic in GW2 – that is, it changes over time. In World of Warcraft (WoW), if you cast a Damage-over-Time (DoT) spell on a target, the damage is fixed for each tick at the time of the attacker’s spellcast – that is, the damage is static and does not depend on the attacker’s “state” at the time of each tick. “State” is a term in computer programming that describes all of the variables for a particular thing, e.g. a character. The WoW model for DoT simplifies the calculations performed by the server. By comparison, when you cast a DoT in GW2, the damage inflicted for each tick of the DoT changes based on the state (in this case how much condition damage) the attacker has at the time of each tick. This is why the ticks from burning or bleeds on a target sometimes change over time. If you consider multiple attacker’s placing stacking bleeds on the same target, each set of attacker bleeds requires its own calculations based on the state of each attacker each time the conditions tick. These calculations add up (no pun intended).

Server-side scaling is a very challenging problem, and as I recently wrote in my first article on The Elder Scrolls Online, no game developer has delivered a system that gracefully scales for a high number of players in a small area.

We’ll have to see how AN’s removal of culling coupled with the new character settings pans out.

What do you think of the AN’s proposed system?

Follow Me

Twitter: @taugrim
YouTube: http://youtube.com/taugrimtv
Facebook: http://facebook.com/taugrim

Revision History

  • 2013/03/16: fixed “bugs” in my writeup per Michael’s Chowns comments, and reordered the sections for clarity
About these ads
Posted in Game Design, Guild Wars 2, PVP, World of Warcraft
13 comments on “The End of Culling in Guild Wars 2, a.k.a. Choose Your Own Lag
  1. Matthew Chowns says:

    You have confused a few different potential performance bottlenecks here. There is processing load on the server, network lag between the server and the clients, and each client’s graphics rendering performance.

    Culling, as you’ve described it here, improved your graphics performance because it was no longer rendering those characters on your screen, and it reduced network lag because the server wasn’t sending you updates for the culled characters, but the server obviously still had to process all the data for those characters, their actions, and effects.

    The server’s processing load is unaffected, so your example of GW2 having more complicated condition stacking isn’t really relevant here – it doesn’t matter how many people can see you when you place that DoT, the server still has to calculate the damage for each tick. You could be alone, DoT’ing yourself, or in the middle of a thousand-player pile up, and the server would have to process the same amount for that effect.

    ArenaNet is disabling culling for gameplay reasons and they realize this will cause extra graphics pressure so they’ve also implemented client side options to manage your graphics performance better. However, it will also create extra network pressure because all the updates for every character in the area will be sent to every client. Habib’s article didn’t mention any changes to reduce network lag, so it would appear to me that the decision to disable culling is actually going to have an overall negative impact on network lag that you won’t be able to mitigate with client settings.

    • taugrim says:

      Matthew Chowns :

      You have confused a few different potential performance bottlenecks here. There is processing load on the server, network lag between the server and the clients, and each client’s graphics rendering performance.

      Culling, as you’ve described it here, improved your graphics performance because it was no longer rendering those characters on your screen, and it reduced network lag because the server wasn’t sending you updates for the culled characters, but the server obviously still had to process all the data for those characters, their actions, and effects.

      You are 100% correct – I was conflating topics. I’ve cleaned up the article so hopeful this is more clear.

      Thanks for the feedback!

      Culling is a client-side optimization but as you said the server has to calculate whatever is there, regardless whether a given game client “sees” the actions of culled characters or not.

      Matthew Chowns :

      The server’s processing load is unaffected, so your example of GW2 having more complicated condition stacking isn’t really relevant here – it doesn’t matter how many people can see you when you place that DoT, the server still has to calculate the damage for each tick. You could be alone, DoT’ing yourself, or in the middle of a thousand-player pile up, and the server would have to process the same amount for that effect.

      Actually, I do think the way condition damage is calculated creates additional overhead for the server.

      In a game where you have DoTs that have fixed damage after being cast, there is no need to factor in the state of the DoT caster to determine damage. The DoT has a fixed duration and tick amounts.

      However, in a dynamic DoT damage context such as GW2, to calculate the condition damage on a target, the server has to check for the attacker’s current condition damage stat at each tick, e.g. how much condition damage they currently have based on boons such as Might. This results in additional calculations and lookups of state.

      Matthew Chowns :

      ArenaNet is disabling culling for gameplay reasons and they realize this will cause extra graphics pressure so they’ve also implemented client side options to manage your graphics performance better.

      Correct.

  2. I am pretty excited about the changes, but I remain slightly skeptical. Tonight I jumped into WvW for the first time in a while (T1 on SoR /shoutout to Blackgate Taugrim!) and I noticed some serious ability delay at times. I am pleased to know that it will not be affected by the culling removal though. I think the big WvW issue will shift from culling to lag itself though at this point; both in fps and ping.

    I will have to see how it pans out. My graphics card is pretty old (gtx 460) but everything else on my system is pretty damn close to up to date, so I usually manage just fine. I think this may stretch it though and I will most likely have to set the culling to low and maybe even nameplates for extreme numbers. It will probably be a better experience for me overall though.

    It will be interesting to see how this all pans out in relation to the WvW progression system, leaderboards for WvW, and culling removal. All these things combined will make far more players interested in WvW so more BGs will be filled more often. This may contribute to more lag as well, but that may die down later on.

    The biggest issue that I forgot to mention is that of the dreaded que. I know it won’t be going away anytime soon, especially after the patch during prime time, but they need to tweak it to make it better. Some simple ideas that may help would be to allow you to switch characters while in the que, show your number in line or an eta, and possibly even put you in another borderland that opens up temporarily. A priority system for people who are in groups where 4 people are in and one isn’t or maybe even something related to guilds would also help.

    We will have to see. It is most certainly an exciting time to be a WvW player though and I absolutely cannot wait to see all this stuff in action!

    • taugrim says:

      Michael (@Hagg3r) :

      I am pretty excited about the changes, but I remain slightly skeptical. Tonight I jumped into WvW for the first time in a while (T1 on SoR /shoutout to Blackgate Taugrim!) and I noticed some serious ability delay at times. I am pleased to know that it will not be affected by the culling removal though. I think the big WvW issue will shift from culling to lag itself though at this point; both in fps and ping.

      The ability delay is server-side lag, and as I wrote in my article it exists today. The culling system, or removal of it, probably won’t improve the server lag situation.

      Michael (@Hagg3r) :

      I will have to see how it pans out. My graphics card is pretty old (gtx 460) but everything else on my system is pretty damn close to up to date, so I usually manage just fine. I think this may stretch it though and I will most likely have to set the culling to low and maybe even nameplates for extreme numbers. It will probably be a better experience for me overall though.

      I have a GTX 560m, so mine is a more recent version but also the laptop version, so our performance will likely be comparable, at least according to this site:

      http://www.notebookcheck.net/NVIDIA-GeForce-GTX-560M.48313.0.html

      If needed I’ll do the nameplates setting too.

      Michael (@Hagg3r) :

      It will be interesting to see how this all pans out in relation to the WvW progression system, leaderboards for WvW, and culling removal. All these things combined will make far more players interested in WvW so more BGs will be filled more often. This may contribute to more lag as well, but that may die down later on.

      I’m really looking forward to the leaderboards, but I’m not really interested in the progression system, which as I understand it is tied to characters, which IMO discourages alts.

      Michael (@Hagg3r) :

      The biggest issue that I forgot to mention is that of the dreaded que. I know it won’t be going away anytime soon, especially after the patch during prime time, but they need to tweak it to make it better. Some simple ideas that may help would be to allow you to switch characters while in the que, show your number in line or an eta, and possibly even put you in another borderland that opens up temporarily. A priority system for people who are in groups where 4 people are in and one isn’t or maybe even something related to guilds would also help.

      The queues are a natural consequence of only a limited number (4) zones to be able to WvW in. I’d rather have queues than have no control over populations in a zone. Even with the queues we get server lag in EB on Blackgate.

      Michael (@Hagg3r) :

      We will have to see. It is most certainly an exciting time to be a WvW player though and I absolutely cannot wait to see all this stuff in action!

      Absolutely :)

  3. derSchranzer says:

    i’m really excited about these changes, even though i never had a big problem with culling. in smaller fights, you could always see your enemies and in big zerg fights, you dont really care how many enemies are there or where they are, you just push through them and spam “1”. of course getting rushed over by an enemy zerg was a problem, when you cant see them attacking or when thiefs are invisible 24/7 but everything was handable (is that even a word? ^^)! i think i will do it similar to your decision to use fallback models and try to fit as many characters on the screen without great FPS drops but just as Matthew Chowns i fear many server-side lags

    • taugrim says:

      derSchranzer :

      i’m really excited about these changes, even though i never had a big problem with culling. in smaller fights, you could always see your enemies and in big zerg fights, you dont really care how many enemies are there or where they are, you just push through them and spam “1″. of course getting rushed over by an enemy zerg was a problem, when you cant see them attacking or when thiefs are invisible 24/7 but everything was handable (is that even a word? ^^)! i think i will do it similar to your decision to use fallback models and try to fit as many characters on the screen without great FPS drops but just as Matthew Chowns i fear many server-side lags

      Yea culling has never really bothered me to the extent it does other players.

      If I see opponents heading my way, I’ll wait a bit to let my game client “catch up” with how many are actually there. Although there are times when I’ve been run over by what turned out to be a zerg and not a handful of enemies.

      That said, I am very curious to see how Stealth works once culling is removed. I was really intrigued by AN’s design for Stealth before the game launched, because I thought the approach was better – Stealth was intended to be a short-term buff. In practicality, it’s not difficult for a GW2 Thief to re-gain stealth and for chunks of time, which gives them an advantage.

  4. Dodgycookirs says:

    Culling only really becomes a problem when you can’t see the shape of the enemy formation. That being said, giving the players the option to sacrifice quality for performance gains is a good idea.

    On a side note, Eve has the time dilation system which slows the game down to retain the sense of smooth performance. It seems to be working decently for CCP except for a few extreme situations.

  5. Euchrid Eucrow says:

    Hey Taugrim! Is Maven still relatively active in Guild Wars 2? I was thinking about applying as I have reached something of a critical mass with my accounts(Blackgate from the beginning, 6 level 80’s, one level 65+, one 60+ and 400 in all crafts). I currently think that I’m self-sufficient enough to be a real contributor to the guild. I don’t want to apply if you guys aren’t into gw2 much anymore or simply aren’t taking members or perhaps are repping a larger guild for the missions and such. Anyways, I’m just trying to put a toe in the water here and see what is going on. Thanks in advance for your reply.

  6. miffy says:

    I was wondering if you still played SWTOR? I loved the combat system back when it launched, it was the first MMO to have combat that felt as good as WoW. I just loved as well how duels could last like 10 mins, it felt like SWG to me in that regard and I waited so long to have combat a bit more slow paced and tactical. You actually get a sense there is a fight going on, you could take risks and get away with them because TTK was so long, it makes you feel like a badass.

    However 1.2 happened and drove me away from the game, they did something to speed up the combat and I hated it after that. Though if I’m honest stuff like the fleet stations were killing the game for me anyways, why would you make hubs to take everyone out of the world? Especially when we have capital cities that are meant to serve their purpose, just weird we had dead capital cities for the sake of boring generic fleet stations.

    Just wondering if any of that is fixed now?

    I tried to like GW2 but I didn’t like the art style from the get go, then I found out there was no world PVP, then I tried the combat and hated it and all the running back and forth in WvW made me hate that. I tried going back to GW2 the other week but it’s dead now and it really shows how the public quest dynamic event things do not work in a theme park MMO.

    If you don’t have open world PVP the game is fail for me, Dayz has managed to become my MMO of choice, even though it isn’t one. Though to be fair the term MMO is so broad these days that most MMOs aren’t one IMO anyways. All the best times were in the world having crazy dynamic fights that could go tits up at any moment. Waiting for the Balloon ride on Tatooine is one that comes to mind, it created such great world PVP and trolling, I bet Bioware probably killed that off though thanx to the whiners. In WoW STV, Tarren Mill, around SM, Tanaris, Crossroads or even mind controlling people into Lava lol.

    2005 was the last year I truly liked an MMO though, seems like ever since they’ve all been the same. Sadly that year Blizzard killed off world PVP thanx to Battlegrounds and being able to enter them without going to them. SOE ruined SWG too, that was an MMO I could just go AFK in because I felt like part of a world, my character had to be in that world even when I wasn’t lol. Now MMOs are just games, not virtual worlds and it makes me sad to see how the genre has gone.

    Every MMO comes and says “we’re not making WoW” and what they do is just make WoW lol.

    • Bonkeye says:

      In response to your last comment, consider Camelot Unchained (if it gets funded) http://www.kickstarter.com/projects/13861848/camelot-unchained . Check the comments section and you’ll find someone who’s posted a link to a wiki with all the info nuggets released so far (a google search will also serve).

    • taugrim says:

      I was wondering if you still played SWTOR?

      No, I unsubbed back in the middle of 2012. I was slammed IRL with a new job and didn’t have time for it, and like you I was turned off by the changes in 1.2.

      Just wondering if any of that is fixed now?

      Honestly I don’t know either. AFAIK, the endgame PVP gear is still a treadmill, which I’m not interested in.

      I tried to like GW2 but I didn’t like the art style from the get go, then I found out there was no world PVP, then I tried the combat and hated it and all the running back and forth in WvW made me hate that.

      Art style is subjective. I like GW2. But I do care about game mechanics much more.

      I actually really enjoy WvW – it’s the most fun World PVP experience I’ve had since WAR. I have been experiencing more server lag since culling was removed, and that’s been an issue I won’t play with.

      If you don’t have open world PVP the game is fail for me, Dayz has managed to become my MMO of choice, even though it isn’t one

      The GAMEBREAKER crew raves about DayZ. I was too busy last year when it was popular to download and try it. Watched some streams, looks highly entertaining.

      Every MMO comes and says “we’re not making WoW” and what they do is just make WoW lol.

      There has been gradual innovation starting back in 2008 with WAR (leveling via PVP and PVE, dynamic events, achievements) and later games, e.g. great warfront map design with RIFT and flexible spec’ing system, the awesome 10-49 bracket in SWTOR warzones, etc.

      GW2 started on a path of high innovation – horizontal scaling – but IMO AN dropped the ball by launching without enough endgame content, leaderboards, dailies, etc – things to keep people engaged – and they sought to stem the drop in active players by introducing vertical progression gear, which basically went against what the had promised prior to launch.

      I do believe the next generation of MMORPGs, e.g. TESO (build your class as you want to), CU (pure RVR focus), WildStar will be innovative.

Comments are closed.

Follow
Taugrim on Twitter Taugrim on Facebook Taugrim on Google Plus Taugrim on YouTube Taugrim on RSS

Enter your email address to receive notifications of new posts by email.

Join 676 other followers

Post Categories
Archived Posts
Copyright © 2009-2014 taugrim.com
All rights reserved
Follow

Get every new post delivered to your Inbox.

Join 676 other followers