Jon Shafer's At The Gates

Sorry, but there is now AI that can beat the best human players at Go, which until recently was thought to be as impossible as beating someone at Civilization. And look how it did it: by playing itself!

AI isn’t hard, it’s just that people are doing it wrong. If a large company like Microsoft would just devote a large team of engineers, a couple hundred million dollars, and several years to have server arms of artificial intelligence play each other in strategy video games, we too could reap the reward of having challenging opponents for Gal Civ or Kingdoms of Amalur. So I’m waiting. Impatiently. With my arms crossed. Tapping my foot with a scowl on my face. A mean scowl. Really mean.

How many people are in the market for Impossible to beat AI? Less than you think.
Challenging, sure, but is Google working on that?

For what it’s worth, the design philosophy of Stars in Shadow is along these lines. As a purely single player 4X, there are quite a few aspects only the human gets to endure/enjoy. I believe this is the correct approach.

Edit: The game needs to be either easy for an AI to play, or those aspects left only to the player.

Well, those millions of dollars, several years, and large server farms only produce AI to play one specific game that will be completely hopeless at other games. I suspect, at today’s rates, to get one that’s good at multiple games would be an order of magnitude more expensive :)

Still, AAA game budgets are already rising. What’s a few hundred million to them?!

The ‘only’ in that post is my favourite part!

On Thursday we will be learning where Deep Mind is currently at with playing Starcraft 2. I’m wondering if they have had it playing on the ladder. Whatever it is they are doing some live thing on Twitch/Youtube:

We’ve been past that point for a few years. What I’d like to see if someone come up with an ai that realistically plays like a class player.

Kasparov thinks so

http://science.sciencemag.org/content/362/6419/1087

I admit that I was pleased to see that AlphaZero had a dynamic, open style like my own. The conventional wisdom was that machines would approach perfection with endless dry maneuvering, usually leading to drawn games. But in my observation, AlphaZero prioritizes piece activity over material, preferring positions that to my eye looked risky and aggressive. Programs usually reflect priorities and prejudices of programmers, but because AlphaZero programs itself, I would say that its style reflects the truth. This superior understanding allowed it to outclass the world’s top traditional program despite calculating far fewer positions per second. It’s the embodiment of the cliché, “work smarter, not harder.”

Note the “to my eye”: beauty and intelligence can cogently argued to be in the eye of the beholder.

We are far from applying AlphaZero to anything - it took 40 days on a one of a kind supercomputer to defeat human Go players. After quite a few more runs longer than that to work out the algorithms and parameters that"worked". Not dependable yet or economic to be applied to strategy games that change over time with patches etc.

Yeah the traditional engines like Komodo, Houdini, and Stockfish have been up there for a bit, but what AlphaZero and the opensource Leela project have done is pretty awesome. Lc0 is the 2nd seed in the current TCEC competition Superfinals:

https://tcec.chessdom.com/live.html

i.e. “my style is the best!”

The question could be whether chess pros play like humans, too. I have zero understanding on the matter, but I enjoy typing!

I think Jon Shafer is being smart in that he is advertising this as an asymmetrical game, upfront.

As I listen to people, it seems to me that their reaction to AI limits has to do with their expectations, their preconceptions. If you go into a game thinking of the antagonists as people sitting around a table competing with you, then AI foolishness is a huge turnoff. And in those circumstances, giving the AI extras is also a turnoff. Lots of games not only simulate a world, they are simultaneously simulating an around-the-table experience with humans, and the latter illusion is broken.

You get a whole lot less of this with, say, an RPG. The world simulation remains, but very few people consider mobs or bosses to be the avatars of human competitors. RPG antagonists still need to provide enough competition to be interesting, but the bar is so much lower when it comes to playing by different rules or avoiding stupid mistakes. When was the last time that someone bellyached, “How did that boss get enough experience points to reach level 12?” or “Stupid monsters just keep sending me little groups of kobolds at a time for me to kill”?

Looking at At the Gates, it appears to me that the game itself sends a mixed message in this regard. Rome is quite clearly playing a different game than you are. But the other tribes? At least on the surface, you have approximately the same relationship to them as you do to rivals in Civ or EU. But if I am understanding Jon correctly, he is telling players not to see it this way. These other tribes could give you trouble, or they could prove to be of benefit, but don’t think of them as your co-players, they are not racing you to reach win conditions.

I suspect that the effectiveness of this with gamers will depend a lot more upon in-game cues than on public statements, but who knows, the vocal critic crowd coincides with the crowd likely to read or hear Jon’s public statements.

But in a general sense, I think he is totally correct. The future of quality strategy games, particularly 4X sorts of games, is going to involve making games where players do not perceive their obstacles to be quasi-human players around the table. (Not that this alone will solve the entire problem. @alekseivolchok is certainly right that an AI that gets advantages which negate normal human strategies is a separate and significant problem. And that remains, regardless of whether one thinks of the AI as a quasi-human or just an inanimate speed bump.)

Another good example: in Colonization, the competitor old world powers settling in the New World are often criticized for their poor AI. But nobody thinks twice about the Native American AI, because they are clearly not supposed to be other players.

This is especially baffling as I believe Colonization was entirely single-player!

I think I may have been misconstrued in some of my criticisms of the game as of about a week and a half ago- the problem isn’t that Rome is playing a different game- it’s that Rome is playing no game at all, and the other tribes barely any.

That’s pretty much on the money @FinnegansFather. People still find awesome (or scary) the bots in ancient games like F.E.A.R., but the technology that made that possible didn’t catch up. The kind of emergent behaviours that resulted were hard to predict for a designer, and as a result, in my opinion, the mainstream veered towards relying on behaviour trees which are easy to understand and debug, but result in behaviours like NPCs walking into walls or being nonplussed by somebody putting a cauldron on their heads.

Interestingly, games like the Sims were so successful because they ditched scripting altogether or for the most part (thinking here of the first one, didn’t play later ones).

So I don’t know what to make out of that gap - looks to me that people want their cake (feel like they are sitting around that table next to something that seems intelligent/interact with seemingly intelligent npc’s that go about their lives) and eat it too (have a decent chance of actually come on top/feel part of a story that makes “sense”). As Mark Twain (?) said: “Truth is Stranger than Fiction because Fiction Has to Make Sense”.

I love Kasparov, but he does really have an ego the size of the Kremlin.

The traditional engines still live inside AlphaZero. The same programming techniques for implementing game tree search algorithms are relevant (or even critical) to the kind of tree search that AlphaZero does.

There are several shooters that use GOAP for their bot programming very successfully. Most of the ones I can think of are MP games that use GOAP to substitute for players. I am less aware of strategy games, but I think I remember Armageddon Empires used it.

GOAP is not that hard to design for or debug if planned properly and the number of agents and scope of the planning is limited, although pretending for it to rule 100% of a complex game AI is indeed madness (too granular). It’s not a neural network blackbox, it’s a state machine with hyperinflated connectivity (I find it easier to visualize than behavior trees). For shooters with a limited tactical scope it works wonders. For games like the ones you are discussing here it could be useful for high level planning, reverting to clearer scripted strategies for low level implementation.

Also, GOAP does planning, but not decision making, which is crucial for strategy games.

Also, GOAP has the problem of not really predicting a player’s (or other agent’s) probable future behavior, which can create silly moments than a more scripted system can be hardcoded to avoid. That’s an interesting problem, how to integrate into the GOAP world state probable changes from outside the agent.

Certainly GOAP doesn’t ship with the CryEngine or Unreal. Last time they checked they were doing behaviour trees or some words to the same effect.

I am actually interested in knowing about other reasonably successful games using Orkin’s architecture. Maybe Monolith still uses it?

That is not accurate description: GOAP is all about tree search like chess engines or AlphaZero. The difference - algorithmically - lies in how the search is oriented. The three families of methods are all forms of approximate dynamic programming, differences between them being in representation, nomenclature and ability to deal with non determinism (the tree has more branches, so to speak).

GOAP planning is sequential decision making.

GOAP/chess playing/AlphaZero operate on a model of the “environment” (other agents, Nature) to predict how the world evolves as influenced by the chosen course of action (which could be nothing). This model is given explicitly as rules (describing game rules, laws of Nature or suitably chosen qualitative abstractions) and assumptions (the picture of the outer world is complete, other agents play optimally, etc.). Obviously, models can be wrong, as our assumptions wil probably be wrong sometime or the parameters of the model changing over time or badly estimated. Still, we can use them to obtain a closed loop strategy, that reevaluates at a certain fixed rate future ramifications following from available courses of action up to a certain distance into the future.

That strategy is called Model Predictive or Reciding Horizon Control, and it is “reasonably” robust.

Some good books to learn about this stuff are

Dynamic Programming and Optimal Control
Dmitri P. Bertsekas
MIT Press (4th edition recommended)

Automated Planning and Acting
Malik Ghallab et al
Cambridge (?) University Press

A bit older but still useful also

Planning Algorithms
Steven LaValle
Cambridge University Press