How Atriox Can Beat You in Halo Wars 2 Without Cheating | AI and Games

How Atriox Can Beat You in Halo Wars 2 Without Cheating | AI and Games

In 2017’s Halo Wars 2, players take command
of the UNSC Spirit of Fire once more as they face off against a new enemy – The Banished:
a ruthless band of mercenaries led by the Brute warlord Atriox. Stranded in deep space
with no means of help and outdated combat technology, Captain James Cutter and his crew
must survive a war of attrition against one of the most ruthless warriors in Covenant
history. Developed by 343 Industries and Creative Assembly
and released for both PC and Xbox One, Halo Wars 2 was designed from the ground up to
present Atriox and his subordinates as exciting new additions to the Halo universe. But for
this to work in a real time strategy game, they need to be intelligent and have a sense
of personality. I’m Tommy Thompson and in this episode of AI and Games we’re going to
explore the AI Commander system that enables for Atriox to not only exhibit strategic intelligence,
but be capable of defeating players on various difficulty levels *without cheating*. I’ve previously discussed on the show that
sometimes developers allow AI characters and other gameplay systems to cheat to better
facilitate the experience. As shown in episode 36, the companion AI in Ghost Recon Wildlands
are often invisible to the enemy, spot targets for the player without seeing them and cheat
sync shots to ensure targets go down as expected. But while companions such as the Ghost Squad
and Elizabeth Comstock exploit knowledge of the world for their own purposes, no genre
adopts this principle more, than real-time-strategy. RTS games notoriously cut corners to improve
the experience: from having build behaviours that don’t require them to acquire the resources
needed (hello Warcraft 3), to enemies that ignore the fog of war and always know where
you are. One reason why some RTS games cheat is that it’s an incredibly difficult problem
to write competent AI for. You need to factor at least two levels of intelligent decisions:
the micro scale where you’re moving individual units, and the macro scale where you’re coordinating
the overall army. One need only look at my episodes on Total War back in 2018 – where
the evolution and scale of the AI systems involved at both micro and macro is simply
staggering. This isn’t an easy problem to solve, but Halo Wars 2 employed a fresh angle. The designers intended for Atriox and his
commanders to exhibit strategic play without resorting to cheating. They should make intelligent
moves on both macro and micro scale, adopt specific play styles and adapt to various
game modes and difficulty levels. But more critically, each AI commander should exhibit
human like behaviours: helping teach players to play the gmae, but also inject personality
into these characters and bring them to life within the Halo universe. Now you might think with the recent success
of Google DeepMind training human-like StarCraft players with AlphaStar that employing deep
learning is the way forward. But this isn’t a cost-effective and pragmatic approach for
a video game. AlphaStar isn’t designed to be an interesting opponent for varying skill
levels, its built to win at the highest levels of competition. Plus, there’s the element
of cost. As cool as it all is, AlphaStars training process cost tens of millions of
dollars to train and that’s simply impractical for even the biggest AAA studios. So while 343 and Creative Assembly didn’t
train a machine learning AI based on human players, they employed a similar principle.
Their AI commanders show human-like traits in combat, because they’re based on strategies
by high-ranking human RTS players, with one specific player a critical part of the process. Balint Marczin, more commonly known in RTS
circles as Nakamura, is a competitive RTS player, streamer and caster and at one time
ranked as one of the top 10 players in the world in Halo Wars for both Xbox and PC. But
in addition, Marczin is a game designer at Creative Assembly, with his work to-date including
both Halo Wars 2 and Total War Warhammer 2. His job was to research the macro and micro
layer strategies human players exhibit within the game and work with the programming team
to build a system that would enable for AI to replicate it in various scenarios. This resulted in five macro strategies that
define a particular style of play being encoded into the AI system: – Rush: AI players that attempt to get a large
army on the field as quickly as possible, destroying and capturing expansion bases as
it moves across the map. – Boom: AI players that quickly capture multiple
bases and prepare for a war of attrition. – Map Control: a more aggressive form of Boom
that focuses on capturing and maintaining capture points across the map. – Fast Tech: where the AI moves quickly to
a minimum of tier 2 units production, in an effort to surprise their enemy with heavier
units. – Turtle: A defensive straegy that captures
bases, stays on tier 1 units for longer, deploys turrets and upgrades expansion bases. Meanwhile on the micro layer there are 10
unique behaviours the AI can execute and it allows for a lot of minute control of the
individual units. This includes common behaviours such as focussing fire on individual units,
holding ground at a specific location or ordering special abilities such as cloaks, shields,
grenade throws and healing. But there are also micro tactics commonly exhibited by human
players such as the pull-back and kite. In pullback, the AI commander will monitor a
squad of units in combat and cycle out those that receive heavy damage from attack range,
while kiting is a tactic that keeps enemy units just beyond attack range, allowing for
either long-range attack or opportunities to quickly deliver damage before backing of
once more. So at any point in time, the AI system in
game will read the current game state and feed it into both a macro strategy planner
and a micro behaviour selector, meaning the system can make intelligent decisions about
a long-term strategy to execute, but also run micro behaviours to help ensure that strategy
is being maintained in the long-run. While the micro layer is focussed purely on movement
commands, the macro layer is built into multiple layers that focus on construction, tech ladders,
leader powers and the management of armies. But one of the real successes of this two-tier
system, is that it allows for both macro and micro layers to be customisable based on the
in-game scenario. In certain campaign missions or in multiplayer skirmish matches, the AI
system has complete control of both layers, allowing it to make its own decisions and
conduct strategy its own way. But there are times that the game designers wanted to have
more control over the system. Campaign missions that need more scripted behaviour, or a reduced
competency at macro behaviour in Blitz firefight, or completing disabling the strategic elements
to become purely reactive or having zero intelligence whatsoever like in some campaign or tutorial
missions. All of these game modes, all facilitated by the same AI system. So with all these systems in play, how do
you create unique personalities or customise the AI to operate at different difficulty
levels? The trick lies in the decision making at macro level, which is broken up into three
sub-systems: the alert manager, the strategic decision maker and the strategy manager. The
alert manager is how the system reacts to local information whilst making strategic
decisions, hence it can interject and influence decisions being made to respond to dynamic
changes in the world, such as creating new units and adding defensive structures such
as turrets. The strategic decision maker is reliant on the strategy manager: a system
whose decisions are influenced by one or more strategy tables. At the beginning of a match, the strategy
manager will read in a strategy table. Each table describes the strategy types that are
being used by this AI character and defines their style or personality for the game. While
this defines which of the macro strategies and micro behaviours it includes, the strategy
table loaded into memory is influenced by the game mode and we’ll talk about that in
a second. The strategy table also includes build orders, tech and leader power unlock
orders and preferences and the type of sub-missions this AI will frequently use – ranging from
attacking enemy structures and defensive tactics to scouting the map and foraging for resources. By defining all of this in a table, the game
becomes more heavily data driven. The programming team have built all of the strategies and
behaviours for the AI to execute as well as a XML driven format for designers to feed
strategy tables into the system. This gives designers the freedom to develop new strategies
– and even unique executions of the same strategy – without having to worry about the programming
side of things. This in-turn crafts the personality and behaviour of individual leaders. This
isn’t just for the Banished characters, but also UNSC leaders that appear in Skirmish
modes such as Professor Anders, Jerome-092 and Captain Cutter himself. Plus returning
to the system overview, the strategic decision maker extends the life of a given strategy
table, allowing the system to dynamically add new behaviours to a chosen strategy in
order to flesh it out and show growth over time. Now the last thing to address is difficulty:
since a given leader will play more or less intelligently based on the difficulty level
selected. To address that, there are two approaches: in campaign there are multiple strategy tables
for a given leader – one for each difficulty level – which gives level designers a lot
more control over each playthrough. While in skirmish, a strategy table is built for
Legendary difficulty and then scaled down based on the selected difficulty. This limits
the number of uber units, the max number of turrets deployed, restrictions to tech and
leader power upgrades, and even limitations on the types of micro behaviours the leader
can execute. While this creates plenty of flexibility for
programmers and designers, this is all an incredibly difficult balancing act to pull
off. At launch, the game had 94 strategy tables built: 28 for campaign as well as 66 for skirmish.
These require the design team to be stay on top of the meta of the game as more DLC and
updates roll out. Halo Wars 2 had a fairly extensive set of DLC release post launch,
including new characters as well as an entirely new campaign. This added another 15 campaign
tables and 111 skirmish tables bringing it up to a total of 220. This has seen a lot
of work on design side to manage these AI personalities after launch, with a lot of
the skirmish AIs on the lower difficulties not just heavily tweaked but actually dumbed
down after launch based on player feedback. This resulted in AI players who upgrades,
leader power usage and re-building of destroyed structures much more limited as well as forcing
them to fan out a little more. Given lower-level AI leaders would frequently turtle to a point
that it was too difficult to beat them for more novice players. Halo Wars 2 strove to be a polished and accessible
entry to the franchise for both seasoned RTS players as well as those new to the genre,
with a cast of characters that evoked their personalities when in combat against the player.
What’s really interesting about this is that the creators of the Total War franchise sought
an entirely new approach to building AI for a strategy game, rather than adapting existing
systems from their own work; enabling programmers and designers to work together to achieve
the best outcome for a variety of situations and players. Thanks for watching this episode of AI and
Games on Halo Wars 2. As we wrap up I would like to give special thanks to Creative Assembly
– the developers of Halo Wars 2 – who provided not only feedback but the in-engine development
footage shown throughout this episode. With thanks in particular to Balint Marczin (Game
Designer), Dr Derek Fagan (AI Gameplay Programmer) – whose Game AI North talk from 2017 is the
basis of the video – and Lucy Boxall (Head of Corporate Communications) who made this
all possible. Thank you all so much for your time, it’s greatly appreciated. Halo Wars 2 was a topic voted for by my crowdfunding
supporters who have a say in what we explore in the series. To have a say in what is made
next, plus support me to actually make these videos, join the AI and Games Patreon like
these lovely people already have by checking the links on screen and in the description.
Have a good one folks, I’ll be back.

61 thoughts on “How Atriox Can Beat You in Halo Wars 2 Without Cheating | AI and Games

  1. It's interestingn to know the mechanics behind this, and awesome to see some developers not 'cheat' their way into letting the AI be better. I'd love to see this sort of thing for the next civilisation game. Playing against Deity difficulty is a chore because they just get so much for free or very little. Somewhat annoying.

  2. A huge thank you to Balint, Derek and Lucy from Creative Assembly who provided their assistance in making this episode as best I can. I hope you enjoy the in-engine footage that the studio provided me. Always fun to see how it works under the hood. If you're a fan of RTS games then stick around. I'll be back next month looking at DeepMind's AlphaStar: how it works and why it's still impractical for use in game development.

  3. This appears to be how many of the fan made ais for aoe2 work. The strategies and timings used by pro players are lifted wholesale and plugged into the ai so it can perfectly simulate a mechanically optimal version of that strategy. The best fan made ais are almost impossible to beat without cheesing.

  4. This has me really curious about Star Wars Galactic Battlegrounds, and whether or not they allowed the ai to cheat at all.
    One of the first things the ai does is deploy its scout and gather resources, but I wonder of this is real or just a ruse.

  5. i was only ever to play the first HaloWars game, back on my 360, but even then it was to 'replace' the Command&Conquer games that i used to play a lot more…sure, there was C&C3, and RedAlert3, but my favorite game was always Yuri'sRevenge…anyway, the point im making with this comment is that RA2/YR had a bit of cheatsy AI in the skirmishes– my usual strategy was to blow up the ConYard and their WarFactory, and set up elite units around their barracks to slowly bleed them out of their money…but not only would they be able to produce a ridiculous number of units for super-cheap, but they could also build units that they technically werent allowed to build, due to tech-tree limits…for example, the Flak Trooper (Soviet Anti-Air unit) requires the Radar Dish in order to build, but a Soviet Base that ONLY had a Barracks up was still cranking out Flak Troopers (and some Conscripts) to the bitter end– eventually, the AI would have to sell their barracks to raise enough money to produce a unit, but since the barracks was the last building left, this is when id win…but the game definitely tried cheating its way into a longer fight…

  6. I thought this was gonna be a video on that old xbox game brute force, but am always happy for more halo and rts content

  7. Atriox is the most annoying player in Halo Wars 2, when it comes to legendary dificulty. He just seems to always know when to start spamming that Glassing Beam, or that EMP thingy. It's sort of unfair how he seems to be able to use those whenever he wants.

  8. Is it just me or is there lack of more technical detail of the game. It's more overhead than other videos I've watched on this channel (think hitman video)

  9. 4/5 of those macro strategies are aggressive early moves…You'd think they might have gone for some more variety.

    I understand that aggressive early moved are/can be effective, but in the overall experience with a gane that makes it limiting

  10. Thanks again for doing what you do Tommy – even though i don't work in games or really don't plan on getting into AI i think it's really interesting to learn how these AI systems work.

  11. Wasn't alpha star not SPECIFICALLY built to emulate players? I thought the idea was to apply human limits to a learning AI, so that its strategical choices were its only advantage over a pro player.

    Alpha star actually has a hard limit on its amount of action and camera movement per second.

  12. I don't believe spending tens of millions of dollars on AI should be out of the question, when you look at what modern AAA companies are spending on marketing. For example red dead redemption's marketing budget was apparently $265 million according to google.

  13. This was extremely fascinating. I wish more 4X/RTS developers would incorporate this into their games. I can't stand when the only way to make an A.I. challenging is for them to cheat. Granted, making an A.I. that can compete with a human player is very difficult I'd imagine.

  14. This was super intresting and broken down in a way someone who sucks at RTS (hi its me) can totally understand. Cheers mate

  15. AoE 2 DE also has done a great job creating and updating their AI! Employing very specific strategies in game. While it still has its flaws it has made really big leaps in its difficulty 🙂

  16. Some time the ia are kinda dumb when they spam (group of 3 ) there power on 1 player when in there other side there is a scarab or condor destroying there base

  17. He worked on the AI of warhammer 2? Dont the AI get free units and money and in battle extra invisible stats in the game?

  18. pfffff, this is BS I remember one time I defeated Atriox, Decimus and Shipmaster all on Legendary on a Skirmish match after they somehow eliminated one of my allies by playing cat and mouse. Nothing beats Anders and her Sentinel spam, Nothing.

  19. Wait each commander's AI in skirmish have different personalities or am I getting that wrong? If correct, is there somewhere I can check them out? Would love to see how sergeant johnson AIs normally act since I usually play with that faction.

  20. This is one of the reasons why I dont get so many people despise 343 Industries even now. Sure, Halo 5 sucked. But, Halo Wars 2 is such a great game. And many people dont like it because "its made by 343". This video was truly fascinating. Thank you for the great learning experience! ^w^

  21. Trying to imagine this "super AI" being used for RPG and Sim games. Topped to work with the pre-patch AI from the ES:Oblivion and game difficulty would probably skyrocket.

  22. I rarely even heard you mention Aatrox throughout this video.. most of it was just strategies AI and humans do in Halo Wars 2. I actually thought you were going to tie it all together at least and explain why Aatrox is so powerful. Thumbs down from me.

  23. Honestly my biggest problem with RTS games is I know that the AI are cheating. I'm not very good at these games and whenever I lose against AI it always feels like they cheated to win.

  24. Eff this shit, I'm not becoming a game developer, I'm not programming AIs, I'm not going to college, I'll just become a monk or something, this is too complex.

  25. Damn, imagine if we had this kinda thing in other RTS or if they start using this in further new ones. Could you imagine games like Supcom with that type of AI? Maybe MOWAS as that game's AI is….kinda simple especially on the macro level.

  26. Huh. I might have to pick the game up in light of this. Cheater AIs are a pet peeve of mine (despite the fact that I know the reasons they’re used) and fighting a program on genuinely equal terms is always an interesting prospect.

  27. It is true that the top level gameplay Deepmind strategy would be cost-inneffective and (maybe) hardware inneffective, but that doesn't mean you can't get a DECENT neural network for a fraction of that. Cut way back on network size, and increase the reward frequency by using tested, proven, human knowledge or even a network trained to analyze statistical behaviors of human players who win.

    As for personality, don't make victory the only reward. Have an AI that cares about its K/D, or ending fights quickly. Or gathering a lot of resources. Or causing bloodshed for you whether it wins or not. All of these things can be combined with neural network training procedures to change how the AI behaves.

  28. Total annhilation ai was pretty fun and used pretty good stragies…. until you fought a human player and then it was always rush rush rush with Arm….. Fuck Arm , Core forever XD

  29. One of the only channels that actually thanks you for watching without begging you for subscribes and likes. You genuinely sound happy that I have watched, and I can tell you that I most certainly am.
    You provide quality content, elevated discussion, and play by play breakdown explanations that make any system easy to understand.
    I cannot overstate how enjoyable this type of commentary is for a non educated computer system outsider as myself. But my amazement is still one of wonder, every time you break down how in depth each system is, a testament to your skill as a reviewer/detective everytime you make it understandable.

    Thank you for all that you bring to the forefront of games, singing the unsung heros into the forefront of gaming.
    Subscriber for life, and a fellow of similar interests!

  30. Ahh yes, the game I currently hold number one in. Attriox is overpowered but nowhere as effectice as Isabelle's Spartan or Deci air. =b

Leave a Reply

Your email address will not be published. Required fields are marked *