Jon Shafer's At The Gates


We’ve been past that point for a few years. What I’d like to see if someone come up with an ai that realistically plays like a class player.


Kasparov thinks so

I admit that I was pleased to see that AlphaZero had a dynamic, open style like my own. The conventional wisdom was that machines would approach perfection with endless dry maneuvering, usually leading to drawn games. But in my observation, AlphaZero prioritizes piece activity over material, preferring positions that to my eye looked risky and aggressive. Programs usually reflect priorities and prejudices of programmers, but because AlphaZero programs itself, I would say that its style reflects the truth. This superior understanding allowed it to outclass the world’s top traditional program despite calculating far fewer positions per second. It’s the embodiment of the cliché, “work smarter, not harder.”

Note the “to my eye”: beauty and intelligence can cogently argued to be in the eye of the beholder.

We are far from applying AlphaZero to anything - it took 40 days on a one of a kind supercomputer to defeat human Go players. After quite a few more runs longer than that to work out the algorithms and parameters that"worked". Not dependable yet or economic to be applied to strategy games that change over time with patches etc.


Yeah the traditional engines like Komodo, Houdini, and Stockfish have been up there for a bit, but what AlphaZero and the opensource Leela project have done is pretty awesome. Lc0 is the 2nd seed in the current TCEC competition Superfinals:


i.e. “my style is the best!”


The question could be whether chess pros play like humans, too. I have zero understanding on the matter, but I enjoy typing!


I think Jon Shafer is being smart in that he is advertising this as an asymmetrical game, upfront.

As I listen to people, it seems to me that their reaction to AI limits has to do with their expectations, their preconceptions. If you go into a game thinking of the antagonists as people sitting around a table competing with you, then AI foolishness is a huge turnoff. And in those circumstances, giving the AI extras is also a turnoff. Lots of games not only simulate a world, they are simultaneously simulating an around-the-table experience with humans, and the latter illusion is broken.

You get a whole lot less of this with, say, an RPG. The world simulation remains, but very few people consider mobs or bosses to be the avatars of human competitors. RPG antagonists still need to provide enough competition to be interesting, but the bar is so much lower when it comes to playing by different rules or avoiding stupid mistakes. When was the last time that someone bellyached, “How did that boss get enough experience points to reach level 12?” or “Stupid monsters just keep sending me little groups of kobolds at a time for me to kill”?

Looking at At the Gates, it appears to me that the game itself sends a mixed message in this regard. Rome is quite clearly playing a different game than you are. But the other tribes? At least on the surface, you have approximately the same relationship to them as you do to rivals in Civ or EU. But if I am understanding Jon correctly, he is telling players not to see it this way. These other tribes could give you trouble, or they could prove to be of benefit, but don’t think of them as your co-players, they are not racing you to reach win conditions.

I suspect that the effectiveness of this with gamers will depend a lot more upon in-game cues than on public statements, but who knows, the vocal critic crowd coincides with the crowd likely to read or hear Jon’s public statements.

But in a general sense, I think he is totally correct. The future of quality strategy games, particularly 4X sorts of games, is going to involve making games where players do not perceive their obstacles to be quasi-human players around the table. (Not that this alone will solve the entire problem. @alekseivolchok is certainly right that an AI that gets advantages which negate normal human strategies is a separate and significant problem. And that remains, regardless of whether one thinks of the AI as a quasi-human or just an inanimate speed bump.)


Another good example: in Colonization, the competitor old world powers settling in the New World are often criticized for their poor AI. But nobody thinks twice about the Native American AI, because they are clearly not supposed to be other players.

This is especially baffling as I believe Colonization was entirely single-player!


I think I may have been misconstrued in some of my criticisms of the game as of about a week and a half ago- the problem isn’t that Rome is playing a different game- it’s that Rome is playing no game at all, and the other tribes barely any.


That’s pretty much on the money @FinnegansFather. People still find awesome (or scary) the bots in ancient games like F.E.A.R., but the technology that made that possible didn’t catch up. The kind of emergent behaviours that resulted were hard to predict for a designer, and as a result, in my opinion, the mainstream veered towards relying on behaviour trees which are easy to understand and debug, but result in behaviours like NPCs walking into walls or being nonplussed by somebody putting a cauldron on their heads.

Interestingly, games like the Sims were so successful because they ditched scripting altogether or for the most part (thinking here of the first one, didn’t play later ones).

So I don’t know what to make out of that gap - looks to me that people want their cake (feel like they are sitting around that table next to something that seems intelligent/interact with seemingly intelligent npc’s that go about their lives) and eat it too (have a decent chance of actually come on top/feel part of a story that makes “sense”). As Mark Twain (?) said: “Truth is Stranger than Fiction because Fiction Has to Make Sense”.


I love Kasparov, but he does really have an ego the size of the Kremlin.


The traditional engines still live inside AlphaZero. The same programming techniques for implementing game tree search algorithms are relevant (or even critical) to the kind of tree search that AlphaZero does.


There are several shooters that use GOAP for their bot programming very successfully. Most of the ones I can think of are MP games that use GOAP to substitute for players. I am less aware of strategy games, but I think I remember Armageddon Empires used it.

GOAP is not that hard to design for or debug if planned properly and the number of agents and scope of the planning is limited, although pretending for it to rule 100% of a complex game AI is indeed madness (too granular). It’s not a neural network blackbox, it’s a state machine with hyperinflated connectivity (I find it easier to visualize than behavior trees). For shooters with a limited tactical scope it works wonders. For games like the ones you are discussing here it could be useful for high level planning, reverting to clearer scripted strategies for low level implementation.

Also, GOAP does planning, but not decision making, which is crucial for strategy games.


Also, GOAP has the problem of not really predicting a player’s (or other agent’s) probable future behavior, which can create silly moments than a more scripted system can be hardcoded to avoid. That’s an interesting problem, how to integrate into the GOAP world state probable changes from outside the agent.


Certainly GOAP doesn’t ship with the CryEngine or Unreal. Last time they checked they were doing behaviour trees or some words to the same effect.

I am actually interested in knowing about other reasonably successful games using Orkin’s architecture. Maybe Monolith still uses it?

That is not accurate description: GOAP is all about tree search like chess engines or AlphaZero. The difference - algorithmically - lies in how the search is oriented. The three families of methods are all forms of approximate dynamic programming, differences between them being in representation, nomenclature and ability to deal with non determinism (the tree has more branches, so to speak).

GOAP planning is sequential decision making.

GOAP/chess playing/AlphaZero operate on a model of the “environment” (other agents, Nature) to predict how the world evolves as influenced by the chosen course of action (which could be nothing). This model is given explicitly as rules (describing game rules, laws of Nature or suitably chosen qualitative abstractions) and assumptions (the picture of the outer world is complete, other agents play optimally, etc.). Obviously, models can be wrong, as our assumptions wil probably be wrong sometime or the parameters of the model changing over time or badly estimated. Still, we can use them to obtain a closed loop strategy, that reevaluates at a certain fixed rate future ramifications following from available courses of action up to a certain distance into the future.

That strategy is called Model Predictive or Reciding Horizon Control, and it is “reasonably” robust.

Some good books to learn about this stuff are

Dynamic Programming and Optimal Control
Dmitri P. Bertsekas
MIT Press (4th edition recommended)

Automated Planning and Acting
Malik Ghallab et al
Cambridge (?) University Press

A bit older but still useful also

Planning Algorithms
Steven LaValle
Cambridge University Press


I know Digital Legends uses GOAP for their work (public info since they did a presentation on that). There are others. In general it’s a good tool for MP bots (limited set of actions, easy world state definition, clear goal most of the time…).

I certainly don’t expect any game engine to come with it by default, though. It is harder to implement and do the initial setup for than for a behavior tree before you can start defining actions (world state definition is a non-trivial problem). It’s not a generic tool.

GOAP is a tree search when deciding on the course of action, but on debugging/executing the chains of action in a real time game it is a state machine (you execute one action/state until you get the go-ahead to advance to the next state or reevaluate the search). For designing (deciding actions to insert to facilitate certain behaviors you want) it helps me visualizing it as a complex state machine to which I’m adding states but have no real control over the connections. You can also visualize it as the search creating a state machine for the game to run until it’s no longer viable. But I’m not an expert, so I certainly am looking at it from a weird perspective, it’s just a tool I use, not something I develop.

By definition, GOAP can’t decide on the ultimate goal of the search. In chess or in a shooter that goal is easy to decide previously and inject (win, kill the player, get a board state evaluation over a certain level…) but in a more complex strategy game, or in simulations when the agent does not need to “win” at all, just be part of a world, the decision of which goal/world state to go for in any given moment is less clear and requires of a higher level evaluation algorithm (which granted, could be done with a more abstracted GOAP set with a goal of “winning”, but it’s probably overkill).

That’s very cool about modelling probable external change into the search. I am not aware of any game that takes it to that extent.


Years ago, when Atomic Games was doing wargames, I did some testing for them. Some of it consisted of playing the game (the one I remember most was I think Crusader) over and over, against the AI in its current state, and recording my moves so that the programmers could get a bunch of data about, well, playing the game, decisions, strategies, whatever. Now, I’m a terrible player, so I’m not sure how much this helped, but the idea was kind of cool–see how humans approach it and figure out a way to make the machine’s actions at least approximate the sort of thing a person does.

Of course, if there are horrible defects in the AI for that game, it’s probably because it’s emulating my terrible generalship…


Are you the reasons tanks can’t drive straight in any Close Combat game??? ;)


Fwiw, I have 2139 hrs in AoW3 according to Steam (in reality there a couple of hundred more that never got registered), was a beta tester, and played the preceding games to death, and sometimes dream of a resurgent Azrac Empire.

I also organised and hosted 2 tournaments, and played a shit tonne of multiplayer, and exposed and corrected (not just me to be fair) loads of exploits before the game got released.

And sometimes I still lose at Emperor.

There are 2 “tricks” to winning at Emperor:

*scout aggressively so you know when the enemy is coming, and then attack them before they gang up.

  • Abuse your superiority in tactical combat.

You can also essentially abandon your cities and just loot the AI and move onto the next city.


Review up at the4xplorer (this is the guy that has been streaming it and recently interviewed @Jon_Shafer iirc)



" And how about the fact that the AI is basically non-existent…

Yeah, let’s talk about that.

The AI: I understand that the basic structure of this game is meant to mostly be about surviving the harsh wilderness and learning how to balance progression with simply being able to exist, but the AI for both the other Germanic tribes and the Roman Empire is basically brain-dead.

The other tribes do not expand. They do not compete for the same resources that you do. They simply exist for you to eventually declare war on (because allying them currently is literally impossible) and overwhelm with superior armies."