3101

I have no inside information but I really doubt that a company so dependent on their IT infrastructure is as clueless or shoulder shrugging as many of you suggest - I honestly think its a matter of it being an order of magnitude more complex than people give it credit for.

3102

My installer started at 12+ GBā€¦and is steadily chunking it downā€¦

3103

Lets be clear, I am not picking on them because of the DDoS attack specifically. Its just a history of seeming incompetence when it comes to managing online, 24/7 global systems. As I said, this isnā€™t hard anymore. Which implies their IT is clueless, politically weak or underfunded.

3104

I see you noted your investment background - I wonder if its because MMOā€™s are vastly different than anything else, in it being as dynamic as it is , and as complex systems as they are? I have no idea, I just find it hard to understand why Blizzard, who operate more or less ONLY on their IT infrastructure, should be clueless, politically weak or underfunded. Again, Iā€™m grabbing this out of thin air of course - I just find it amazing if true!

3105

My installer started at 12+ GBā€¦and is steadily chunking it down.

As in itā€™s reducing in reported size faster than the download? Or is it actually many, many gigabytes?

3106

I think the Seasons actually start on Friday.

Think itā€™s time to have another go, see you in the EU clan!

3107

Season starts Friday, confirmed.

Here are the times:

3108

An MMO service is a much more complex thing than financial software. When basically all of your system can be unit tested before release and different machines can run different versions simultaneously, itā€™s a lot easier to have no downtime and no major bugs. Blizzard does a great job and comparing them to applications far less complex than games is just not reasonable.

3109

I question that MMOs are more complex than 24/7 trading platforms. But lets say they are, are they are a magnitude more complex? 12 hour downtime every week for 10 yearsā€¦ Just saying.

3110

So far the best thing about the patch is the new stash tab.

3111

Compare Blizzard to their peers, then. DOTA2, LoL, Path of Exile, etc. Do they suffer from routine extended downtime whenever thereā€™s a patch rollout? Blizzard nearly always does.

3112

Those arenā€™t peers, though. Peers would be SWTOR, Elder Scrolls Online and the like, and they also have regularly scheduled weekly maintenance hours.

3113

How do you figure?

3114

Oh, I got confused since someone mentioned Blizzard doing this for ten years, and I started thinking about World of Warcraft instead - my bad. Yeah, I guess its more like POE, even though then I think D3 has a larger userbase, but no idea how much that matters.

3115

Considering Battle.net is running Starcraft, D2, Starcraft 2, D3, Hearthstone, WoW, and Warcraft 3, Iā€™d say that yes, they have a higher user-base than PoE.

3116

Yes, they are an order if magnitude more complex - a lot of the testing necessary for a game simply canā€™t be automated and that means more downtime to smoke a build before putting it live. Could they invest in twice as many servers and then able to fully vet a build and simply swap which server was live when it was time to go? Maybe, but is that a cost-effective approach?

I donā€™t know what LoL does but most MMOs have downtime for patches. Does LoL not?

3117

Not that I recall, though I will point out that LOL does not exactly have a steller history of server stability (at least in the past - I havenā€™t played in ages).

Steam (DOTA2 + other stuff) doesnā€™t really have much downtime, though it does happen now and then.

For MMOs, no experience except for GW2, which almost never has downtime (the only instance I recall was when they completely changed how ingame currency was stored) - patch and go (and if youā€™re ingame, you get an hour in unpatched instances before you get kicked and forced to patch).

3118

Thatā€™s why I included LoL as well, who has 7.5 million concurrent users. PoE is assuredly much smaller than Blizzard, although they did hit 113,000 concurrent users recently. Not too bad for an indie studio out of New Zealand, and certainly a non-trivial number.

The point is, this stuff is difficult and challenging, but itā€™s not voodoo, and neither is it 1997 anymore. Yes, the occasional bad outage is going to happen. Sometimes a series of highly unlikely events happen that end up in one nasty cascade of failures. My issue with Blizzard is that long downtimes are the routine instead of the exception.

Again, some routine downtime for weekly maintenance is to be expected, as is a limited outage when a new build is brought into production. The occasional catastrophic event and related outage Iā€™m not surprised by either. I do think that Blizzard performs very poorly in this area, though, and could make significant improvements in this area, especially given the resources they have. But just because (Activision)Blizzard has the resources doesnā€™t mean that their IT and Operations teams do. Iā€™ve seen that far, far too often in the corporate world, where IT is given inadequate resources to effectively meet their responsibilities. If I had to guess Iā€™d say thatā€™s what is going on over there. I highly doubt itā€™s a lack of talent or ability.

EDIT: Sorry for shitting up the thread, this really has nothing to do with Diablo 3 (and especially not the DDoS) so Iā€™ll bite my tongue and shut up now. :) Itā€™s just an area Iā€™ve worked in and around and was somehow compelled to chime in with regards to solomaniā€™s comments and the discussion it provoked.

3119

I seriously doubt it has anything to do with lack of talent or ability, or budget for that matter.

Blizzard the business has decided what is an acceptable SLA for their services and how maintenance downtime fits into that. This is what they then ask IT to deliver, with a commensurate budget. I would bet dollars to donuts the business has simply determined that 6 hour outages every other week has little impact on itā€™s bottom line, so sets the SLA and budget accordingly. History has probably demonstrated to them that this kind of window is ultimately acceptable to its customer base, particularly when sufficient notice periods are given.

Yes, IT can achieve extremely high up-times with modern infrastructure, software and enterprise architectures, but it still does not come cheap. There is still a vast difference in the cost of delivering RPO/RTO/maintenance downtime of zero, vs 60 minutes, and again vs 6 hours. Business weighs up the cost vs the impact and decides what is acceptable within budget constraints.

If you are a financial agency that may stand to lose thousand$ or even million$ a second to downtime, you cough up the dough. If youā€™re Blizzard and you are just going to have some customers that canā€™t play a luxury for an hour or two in an off-peak period, with a very small number of those whinging on forums and an even smaller number again cancelling subs or refusing to buy new product, you can probably decide to spend less and give more back to the shareholders.

3120

I think thatā€™s the most likely scenario Shareleo. I think itā€™s not a good IT strategy though for such a customer centric system. These games have a global user base - there is no off period. I hazard to guess Blizz IT would love to have systems that have 99.9999 uptime and they may even have the expertise to do it but the business doesnā€™t care.