NDA's vs. Developer Journals

Games
1

On Friday we had a friend from Paradox (Hearts of Iron II) visiting us. It’s amazing how similar “indie” developers are.

We showed him around our cramped offices and showed the latest internal stuff we were working on with Galactic Civilizations II. We both talked about long-term, unannounced projects. In other words, very open and transparent in how we do things.

And later on, I was thinking about how small developers are very different from larger ones. The big publishers/developers are much tighter lipped about projects.

The Master of Orion 3 project, for instance, had a closed beta with everyone under NDAs.

With Galactic Civilizations II, not only will we have an open beta but we put out weekly developer journals: http://www.galciv2.com/Journals.aspx

This is to let people see what’s going on each week in development.

For example, this week’s big internal debate was over how the ships should “look”. Simulation like? Or more game like? Or something in between. That resulted in this journal entry:

http://www.galciv2.com/Journals.aspx?AID=66480

Which begs the question, it seems to me that game developers would benefit from being more transparent with their fans and potential customers by letting them in on what’s going on early on.

Qt3 is full of game reviewers. I ask you this - how many games have you reviewed where you ended up not liking them because of something that could have been tweaked/changed relatively easily? Some little thing that overwhelmed the rest of an otherwise decent game?

It just seems to me that the way around that is to get the people who would play your game involved as early as possible. No code available yet? No problem, put up what you’re working on. Show them the issues you have going. Build a raport with the game community.

I know there are other developers here who probably disagree with me. That security and NDAs are important. But it strikes me that the game industry is relatively unique in that because we’re striving to make something with an intangible quality “fun” that we should be trying to bring gamers in as early as possible.

2

Well, I agree, and come to think of it, what is the point of game NDAs? Is having your amazing idea stolen and someone releasing the same game a real problem?

3

Cough, save anywhere

It just seems to me that the way around that is to get the people who would play your game involved as early as possible. No code available yet? No problem, put up what you’re working on. Show them the issues you have going. Build a raport with the game community.

As your projects scale up in size and interest, when does this become unmanageable? How do you filter out the noise from the good responses from the fanboys who will take your product to places where only they will be your audience (like flight sims)?

Listening to your audience is great; I’d rather trust the judgment of the people making a game than see every product be diluted by the competing interests of X gamers. I’d think small numbers of serious testers, and then focus groups for interface and other issues late in the project, would be as good or better than being more open at all stages.

And if you screwed up to the point of no return when you hit the focus groups, you need to seriously reconsider your internal dev and design, and that group of serious testers.

Also, the more open you are, the more open you are to, “Why didn’t you include Feature X? You said last year it’d be there!!!” Feature creep, or dropped features, are never good.

But it strikes me that the game industry is relatively unique in that because we’re striving to make something with an intangible quality “fun” that we should be trying to bring gamers in as early as possible.

In the TV industry, they do this all the time and still end up with “According to Jim.” That probably tested great. And in the movie industry, it leads to contrived happy endings.

Games aren’t that unique. But in general, the public doesn’t really know what it wants until it has it. If you let them in early, they might provide valuable insight. Or they might just confuse the hell out of you.

Anyway, I’m buying your game, Brad. I’m not buying one designed by comittee. If I’ve liked your previous games, I trust that you’ll make the right decisions. I don’t want to think you’re being overruled by some Democratic process headed by people like Tom Chick or something. He’d probably just want you to make GalCiv 2 more like Painkiller.

(edit) As an aside, though I’ve not played one of their games, I understand the interfaces for Paradox games get somewhat… muddled. Might that be due to having too much input from too many hardcore fans, who want feature after feature after feature without any consideration how it fits into the whole, or how it makes your interface a disaster for new players?

Look at what the Blizzards and Ensembles do: lots of internal playtesting, small external testing, focus groups, and occasional public things more for PR than anything. (WoW being an exception, for various reasons.)

4

Obviously, from a gamer’s point of view, transparency is much better than the “I could tell you but then I’d have to kill you” approach. The secretive strategy tends to make the developer look like a conceited primadonna to me. It hurts his image, and also his game’s image. I much prefer the old-school, garage feel of developers who are just making their game because they love it (which is one reason I’m so down on the “mainstream,” big-bucks, Hollywood glitz developers and tend to seek out more unknown and uncelebrated games).

5

It’s a tricky balance. You need enough people at the early stages to tell you if you are on the right track, but not so many people that you can’t separate wheat from chaff.

But rarely has something that could be changed “relatively easily” overwhelmed my opinion of a game. Maybe it pushed me from “must buy” to “wait a bit”, but issues like interface or data management are not things that can be changed relatively easily by gamer input. Ship design graphics? Minor thing that can be addressed fairly quickly in game design terms. But not something that will turn a rant into a rave.

The beta process I was involved in had a NDA, but by the time I got involved the bad decisions had already been made and nothing we could say would fix it. The end product was incomprehensible to most players.

Paradox is a good example of the echo chamber of player input, or at least it was until they wised up. Victoria is a historical micromanager’s wet dream, but even experienced strategy guys like me could tell that it was a couple of dozen menus too many. Even their most recent patches for EU2 were driven by the complaints of the hardcore MP gamers to make it more challenging - for them, challenge equalled fun.

Crusader Kings and Hearts of Iron 2, however, clearly show that interface has to be designed not with the fans in mind, but with the masses. Both fall on the right side of the complex versus complicated divide. The question, of course, is when you listen to those that know your games best and when you don’t. It’s a tough line, but a designer has to know when to hold firm to a design document.

The hardcore cannot be perfectly satisfied, ever. (Check the Rome: Total War forums for proof of this.) So you can’t design with them in mind in all things. They may have an idea what they want in a game, but even if you give it to them, you may not be giving it to them in the Platonic ideal that they had in mind.

I know that none of this is helpful, Brad. A lot of the feedback question comes down to knowing who you can trust and at what stage. Some designers have to learn to trust themselves, and some not to trust themselves so fully.

Troy

6

I don’t want to think you’re being overruled by some Democratic process headed by people like Tom Chick or something.

Dude, if it’s headed by me, it sure as hell isn’t going to be Democratic, or even democratic. Cases in point: Deus Ex, Burnout 3, Metroids: Prime Whatever, Doom 3. I would be an absolute autocrat, telling people what not to like in spite of themselves!

-Tom

7

Is having your amazing idea stolen and someone releasing the same game a real problem?

I certainly don’t think so. We have a saying - Ideas are cheap. Execution is what matters.

Anyway, I’m buying your game, Brad. I’m not buying one designed by comittee. If I’ve liked your previous games, I trust that you’ll make the right decisions. I don’t want to think you’re being overruled by some Democratic process headed by people like Tom Chick or something. He’d probably just want you to make GalCiv 2 more like Painkiller.

If you don’t want to end up on one of Tom’s various Emu farms you’ll take that back, comrade! ;)

But seriously, we’re not talking about games designed by committee. We regularly get the “I’m a customer, how dare you talk to me that way!” on our forums because we are very candid in our responses.

Users give us all kinds of suggestions and feature ideas but we readily will say “No, that would change the scope of the game too much.”

But there are often times where we’re not sure of quite the right direction. The example in the journalshttp://www.galciv2.com/Journals.aspx this week has to do with how ships should look.

Or in GalCiv 1, it was beta testers who came up with starbases and galactic resources, not us. Imagine Galactic Civilizations I if there’d been no starbases or galactic resources. It would have been a much different (and in my opinion poorer) game.

So it’s still very important for the designers to maintain firm control over the game’s progression. But I do think it’s very useful to get feedback from users throughout the process.

The age old “Save anytime” feature, for instance, would not be missed by developers if they had people yelling about it prior to going out.

8

Pretty much what Steve said, but I also think your perception is a bit skewed by being smaller and independent. I’ve seen the development process on much bigger games and they’re driven by a completely different mechanics than ‘What makes the game good’. There’s a whole other set of rules at work.

I remember going to see Master of Orion 3 over a year before it was released. I came away thinking, ‘Holy Christ, what a mess’. I would have loved to sit down with a producer or someone to talk to them about what mistakes I thought they were making, but the fact of the matter is they probably didn’t have either the flexibility or the inclination to change what they were doing.

I’d love to know to know more about the interplay between Atari and Quicksilver over that game’s development, because there were plenty of people involved in that game who are far smarter about the industry than me.

-Tom
9

I agree. But it doesn’t explain why NDAs and secrecy are needed, let alone why that would be a better path.

If there was an Age of Empires 3 Journal site that people could hang out on and see the latest/greatest stuff, I’m sure they’d get lots of useless, ridiciulous suggestions but there would be interesting useful things in there as well. Plus, it helps consolidate the fan base.

The MOO3 episode I think is a good example. They had years on that. I think if they had let fans be involved early on, there would have been more coherence in the design.

Obviously you need a very firm hand on the team itself so that crazy stuff doesn’t get put in.

I remember in the original GalCiv 2 design document I threw in the concept that planets had to have housing built in for population. The population was fed by food and the factories then employed X number of people each. So players would have to do a lot of balancing to keep unemployment low.

Before it even got to the players we shook our heads and thought “What the hell are we doing? That sucks. That would be a micro management nightmare!” So it was tossed out. But beta testesr make suggestions like that all the time because they don’t think them through. As long as the designers are willing to say “No” to ideas and defend their decision it can go a long way.

I am pretty familiar with that particular situation so I can’t really say too much publicly on that except that I don’t think Atari can be blamed very much.

But I think MOO3 could have been “saved” if they’d gotten the fans involved earlier. Then the “released” game wouldn’t have been a 1.0 type game but a 1.1.

When we put out the GalCiv II betas, if it turns out the game isn’t fun – that means fun for us too since we want the game to be fun to play for us as well – then we’ll keep at it until it is. I think fans would be more understanding over delays on that kind of thing than vague “We won’t ship until it’s ready”.

But I understand where you’re coming from. Does the transparent approach scale? That is, a game with 100k players may work with this approach but it falls apart when you have 1M players. That I really can’t say since I’ve not made a game that targeted 1M players.

10

At the risk of stating the obvious – shareholder lawsuits (“You actively disclosed information on future projects without requiring an NDA? And now a competitor has a clone that’s doing BETTER?!”) and, more likely, the mistaken notion that a game has a “secret sauce” that is easy to jack.

But, to take devil’s advocate, while ideas are cheap and execution is hard, GOOD ideas are very, very rare but easy to duplicate, whereas solid execution is very hard to duplicate.

So if you’re sitting on that awesome-0 3000 of an idea and, more importantly, you feel you have competitors that can integrate those ideas quicker than you can (e.g. they have existing tech – look at MMOs and how easily ideas from one are stuck into another), then being somewhat tight lipped is sometimes required.

The other issue is that you want the freedom to change your mind later and if you disclose what you’re doing and change direction later there can be a feeling of bait-and-switch, betrayal, or just “those guys are aimless and lack vision” going on.

Obviously you need a very firm hand on the team itself so that crazy stuff doesn’t get put in.

Right. To me, having fans involved early on provides buzz and, more important, gives you a wider range of people for usability testing, bullshit filtering, and what have you. Developers working on something in a vacuum get very skewed perceptions over time.

11

Wouldn’t this be somewhat related to offering demos before the game’s released? So people could have some small inkling if it’s fun or not? Looks like you and the rest of the industry have different motives.

I think Steve just about said it all. And any way you look at it, player input is a double edged sword, with casual players being just as likely to come up with not so thought out ideas as hardcore players are to come up with overly complex ones. What it comes down to with both is that the developer would be looking at their game as reflected off of those people instead of looking at their own game directly to see if it’s what they want or not. Honestly, all of the simple problems that have soured games for me were ones that should’ve been caught by either QA or by one of the developers playing the game for the 5 minutes it would’ve taken to spot the annoyance. I’m not sure this was the question though.

Yes, we really like being “in” on things. Some people like it because it gives them the illusion of having some control over your product, but some also like it simply because it’s nice to have something to validate their enthusiasm for an unreleased niche title other than a couple of screenshots and a cavalier “When it’s done”, which has come to mean “Keep your enthusiasm on tap for when we need it. Don’t call us, we’ll call you.” Something that shows they’re NOT wasting their time in bothering to care about your game, something that shows you’re not just stringing them along.

12

This happened to me for Oddworld Stranger’s Wrath. Would have made the game 10x better if they had just included an audio cue when Stranger’s health got low and made him a bit tougher.

13

I think part of what’s also going on, Brad, is that transparency can run counter to marketing.

A lot of higher profile games dole out information in exchange for cover stories, exclusive coverage, and so forth. I’ll often go to a developer and be told things like “We can only show you three of the weapons we’re going to have in First Person Shooter X” or “We can’t discuss any of our multiplayer features”, because they’re holding off to spread out their preview coverage.

But if you have the developers being open and interacting with the fans, you might lose that bargaining power. It’s certainly harder to control the flow of information. Personally, I think it’s a little silly, but it’s become the nature of the gaming press beast.

-Tom

14

My experience has been that there’s somewhere between a 300:1 and 1000:1 ratio of hokey fan suggestions on a public board, and actual decent, ones. Unfortunately, most game fans, when given the opportunity to voice their opinion, start designing Bongo’s Dream House rather than anything that another player might also appreciate. :?

Another one that always gets me is, “I don’t think it would be very hard to…” Most people really have no idea of what’s going on under the hood (or a complete understanding of how a game mechanic or feature fits into the big picture of the game), so why does everybody think they do? Usually someone saying this is a clear indicator that the proposed change would be anything but easy to make. :P

15

This is now reminding me of that thread on the 3D Realms forums where George Broussard asked the fans about what, in terms of interactivity, they would like to see in Duke Nukem Forever, and there were multiple requests for allowing Duke to take a shower. I almost wonder if it was a prank.

16

IMHO you guys are overanalyzing all this. Some middling game developers and many suits seem to have the opinion that their ideas are golden, and are simply paranoid about being ripped off. In my experience good designers have more cool ideas then they could ever finish, realize that most good ideas aren’t new anyway, and know that the hard thing is actually bringing the ideas to fruition (5% inspiration, 95% perspiration). Those who are paranoid that their ideas will be ripped seem to either have so few ideas they seem precious, or somehow believe that coming up with a cool concept is most of the work.

The only sound reason I can see for NDAs is that sometimes one’s fans can be complete assholes and unfairly bury a game. Kohan 2 comes to mind.

17

Because I’ve found that preview coverage of games tends to make me less excited when I actually play a game for the first time, I ignore most of it. I don’t mind writing previews because I know people like that, but it’s definitely not for me. All the surprise is gone from 95% of released games these days because way too much information about how a game plays is known before it ships.

I guess that makes it easier to decide if you’ll like that game when it comes out, but usually I don’t need that kind of pre-release warning about whether I’m going to enjoy a game or not. There are just too many spoilers in press coverage that make games seem like a lot less than they are by the time you play them.

–Dave

18

Paradox has recently changed their policy to “we won’t change our games only because people on our forums complain about something.” I think that’s perfectly reasonable, and even common sense, but there was still some whining that they didn’t care about their fans anymore, etc.

As for ideas, you have to be honest with yourself. Most ideas are so-so, and it shouldn’t matter if someone else tries to run with them. It can become a good game, and it can become a bad game, depending on execution. If your idea is “Diablo clone but with 3D graphics,” the idea itself won’t make it a good game.
But I think there are a few ideas that are worth a lot, and some of those are the kind where once you hear them you immediately recognize the full potential. I’m kind of in a vegetative state today due to a flu or cold coming on, so I can’t think of any good examples from the past, but I might come back with one later on.

19

you should see the cmak forum at battlefront.com. the main game designer for combat mission has been having a bunch of discussions on what they plan to do for the next cm game engine. the forum goers often talk to about 13 pages (an unwritten rule is to keep threads to 13 pages max to keep the server from crashing). often a second thread on the same topic is opened to keep the discussion going.

14 points describing what cm2 will be like and also a bit of their design philosophy.

list of his last 50 responses in the forum.

he also does a riff on some hardcore wargame fans(“grogs”) and the problem with some of their ideas:
parable of the car designer and the grog

Grog doesn’t even hesitate. He already knows exactly what he wanted. So, he wasted no time telling the designer his ideas, using a voice which indicated that Grog was not just speaking for himself, but all owners of this model of car. “Well, first thing is first… you HAVE to make that cup holder bigger. The new 20oz bottles just won’t fit in there. This is key. The rearview mirror’s night time position is too difficult to operate so I tend not to use it. Make the flip switch larger and with less resistance. The carpet under the peddles is too thin… I wore through it years ago. If the car wasn’t so good I’d probably have traded it in because of this alone. Make the carpets thicker. The vent air has always smelled a bit funny, so if you can fix that to… great. But don’t waste too much time on that because I’ve found driving with the window open a crack fixes that OK. I’d rather see you address the issues with the exhaust system. No… no… it works fine, but when it rots out after 12 months I would like an indicator light to tell me that I need to get it replaced. And if you can fix all of those things, and still have some time, then I suppose a more comfortable driver’s seat would be kinda neat, but really… I don’t need it since my ass is so callused alread I can stick with the same seat. I don’t want that to distract you from the other things! So, you do this and the car will be perfect!”

The car designer sat there stunned. Yes, all of Grog’s suggestions were practical and, for the most part, good things. But the designer was thinking he would be hearing big ideas from this user, not just minor design issues that are almost certainly going to be addressed in the redesign process anyway even without any input. Like the drink holder… of course the designer is going to make the cup holder suited to the current sized drinks. He might even make it adjustable to fit possibly larger drinks in the future. And instead of having an expensive, and rather pointless, indicator light for the exhaust system… the designer was thinking of making something that would last 5 years instead of 1. So after a few minutes of stunned amazement, the designer thought Grog must have misunderstood what he was looking for.

“Grog, I think you must have misunderstood me. I am talking about an entire ground up redesign of the car you love. I want to make it better in all ways, not just the ways you mentioned (though most of them are good suggestions). Don’t you want better gas milage? Gas prices today are terrible, so wouldn’t that be good for you? And I noticed that the body of your car has substantial rust on it. Wouldn’t you like me to design a body that has some innovative anti-rust features so that 7 years from now it will still look like you drove it out of the showroom yesterday? And how about engine performance… I know for a fact that your car, fully loaded, can be a bit slow to accelerate up a hill. I was thinking of changing things around so this wouldn’t happen. Safety is also something most people put at the top of their lists, but you made no mention of this. Don’t you want to have a greater chance of surviving a crash with the least amount of personal injury to you and your passengers as possible? There are all sorts of things I could do to address these issues. Aren’t you interested in any of these?”

Grog, without hesitation, and with the sound of conviction that can only come from someone who has made up his mind long ago, said “No. All of that stuff is unnecessary. Give me the things that I mentioned and leave all the rest exactly as is. Don’t fix what isn’t broken”.

“Are you absolutely sure? I mean, I can do some of these things and it won’t even add any relative cost or detract from the driving experience. Why not put them in?”

“Cost isn’t the issue, functionality is. And besides those things I mentioned, I want the functionality to be identical to what I have now. Don’t mess it up with fancy stuff I don’t want or need”.

The designer, still in shock but oddly not surprised, figured that this was as far as the conversation could go. Grog gave him some good, though extremely minor, suggestions to keep in mind. And he wrote them down with the intention of making sure they were followed up on. But in terms of everything else… well, he obviously asked the wrong person even though Grog is obviously one of his most dedicated customers. So the designer tanked Grog, paid for his coffee, and departed. Grog did the same and got into his beloved car. He sat in it thinking about the joys of a newer car with the changes he requested, even though he doubted the designer would actually put them in. “He’ll screw it up, I’m sure. They always do”.

The End

:D

20

Is that the reason why we don’t have a decent remake of Total Annihilation?
Sometimes all you really do need is to tweak a few things and bring it up to state with current technology.
See X-Com for spiritual heirs that have suffered from the flawed vision of designers.