Bonjour everyone, it’s the French Paradox speaking! For those who don’t know me, I’ve joined the Stellaris team this December after a year and a half as a programmer on Europa Universalis IV.
Today, we are gonna talk about AI.
A good introduction for those new to the field
Fifty Shades of AI
There are several AI modules in Stellaris. For historical reasons we call them “ministers” as each one is supposed to handle a specific role in an AI empire.
There are 3 broad kinds:
- The AI foreign minister handles diplomacy, federations, galactic community, peace deals and the like
- The AI interior minister is in charge of the economy. He keeps budgets and order constructions, both civil and military.
- The AI military minister is in command of all troops and military fleets, and also responsible for laying out strategic plans when at war.
For each of those ministries there are different “ministers” there are several options that can be selected for every empire in the game. All of those have generic one which behaves more or less like we’d expect a player to and is used for most AI empires. Then we have a bunch of specialized ones for special tags such as space monsters, fallen empires, crisis, marauders and the like.
As almost everything in our games, AI is configurable in script for our modders, although I’m not exactly sure what would happen if you assigned a space monster military AI to the caravaneers
In guise of a welcoming gift when I joined the team, I was tasked with reworking the military one...
The Military AI
To give you a little bit of background, there were several generations of military AIs in Stellaris. The generic one (used by most “classic” empires) was redone by the great @sidestep last year, while the more specialized ones (crisis, space monsters) have kept close to what they were on release. In the midst of the sad and dark swedish winter, I managed to bring some improvements that I’ll showcase today.
First of all, I worked on visualization to help us debug how the AI “thinks”. Funny thing is, it already made it look “better” to audiences even if it didn’t actually change any behaviour. It’s actually something that’s been observed in video games: a good AI tells you what it does, which makes it look smarter. One of my favourite examples of that would be the enemies in FEAR.
So by typing 'debug_ai' in the console and observing an AI empire, you can see what it has in mind:
“I don't even see the code. All I see is blonde, brunette, redhead. Hey uh, you want a drink?”
As a simple analogy, imagine that the AI has a war minister that looks at the big picture and rates every potential target, a general staff who assign fleets to some of those objectives, and then admirals who try to lead those fleets on a tactical level to achieve those objectives.
The skulls on top of each system shows military objectives that the AI is considering (the war minister). Red ones are the ones they selected and committed some fleets to, while green ones are other options they haven’t retained for now. Finally for each individual fleet, in those task forces, you see what they are doing at present.
In our screenshot example, the AI decided that taking Tiralam was the most important objective with a score of 4500, and that they estimated that at least 11.2k fleet power was needed to accomplish this. They committed the Kilik Armada, the Jinki-Ki-Ti Armada and the the Grekki Armada to this. Since it makes little tactical sense to attack in a dispersed formation, the AI issued orders to regroup in Broon’s Singularity before proceeding on the attack (something we improved in this patch).
For convenience, the summary is also visible in the outliner:
As seen from the other side of that war
That change alone allowed us to see where the AI was a bit weak and also made evident a few bugs in the production AI that we promptly fixed. A funny one was that in some cases a fleet would end up assigned to two different fleet groups, nicely simulating two admirals fighting over command of a fleet and issuing contradictory orders every day.
Crisis AI
The next step was to rewrite the various crisis to use the generic AI, so that any effort spent on making better would benefit all. In patch 2.6 the specific AI of the Khan, the Prethoryn, the Unbidden and the Contingency will use the same AI as the “standard” empires, with a few twists to still retain their personality.
Without spoiling every secret, here’s a few ideas:
- The Khan doesn’t really believe in defense and will try to beat the closest systems into submission
- The Prethoryn will swarm in every direction they can
- The Contingency will systematically try to stop the biggest threat to the galaxy, until nothing remains
- The Unbidden will be harder to predict, but there’s reason behind their alien way of acting.
One of the biggest challenges we faced was assigning fleets to objectives. Matching X fleets with Y out of Z objectives is not an easy task. Do we try to accomplish as many objectives as we can at the risk of spreading too thin or accomplishing nothing of value? Should we instead focus on the most valuable target and possibly end up in a big fight that we could have avoided? How often should we reconsider our options?
The current version solves this by putting a fleet power value on every target, then grabbing fleets by order of priority until it either has enough to accomplish the objective, or go over the next one. This approach showed its limits when we plugged the crisis AI into it, as it relies a lot on the size of available fleets (it doesn’t know how to split them, it can only merge them).
Teaching the AI how to split fleets proved quite interesting:
What shall we do with this knowledge?
It took several tries to find a good balance, as the AI tended to split too much (most objectives don’t call for that much fleet power, unless you’re fighting your enemy main fleet). In the end, after trying some complex strategies such as keeping statistics on accomplished objectives and deriving a good target number from that, a simpler approach turned out more efficient: put all the nation’s offensive fleet power into one stack, and then consider splitting in 2,3 or more depending on how confident the AI feels about its military power versus its foes.
Knowing some of you like to mod our AI, here’s some new defines you may want to play with once all that hits the shelves.
Most of those changes will be delivered in the patch coming alongside the Federations release (2.6.0), but not all. As you may imagine, changes to the military AI are quite impactful and we don’t want to release the changes without enough testing, so some of them will be delivered in the first support patch (2.6.1).
What is an economy AI and where do I buy one?
Hey all spacefarers, Zack aka. sidestep here. While The French Paradox has been hard at work on the military side of the AI I’ve been occupied with the AI economy, which is important as we all know that credits make the worlds go round.
So what is an economy AI and how does it work in Stellaris? Well, that could be an entire dev diary in itself and differs from game to game. But in short an economy AI decides when and how the AI should spend its resources as well as how it should work to get more of those resources. In Stellaris specifically the AI handles things such as trade, mining bases, starbase modules, policies, edicts, job favoriting and more. But perhaps most importantly the economy AI handles what is constructed on your planets!
Construction is at the core of the Stellaris economy, buildings and districts fuel your economy and help grow more pops, which in turn means more resources. Previously the economy AI did construction on a planet-by-planet basis and with no long term goal in mind. At regular intervals it would go over all planets it owned and through scripted weights it would find something “good” to build, queue it, and then move on and do the same for the next planet and so on. There exists some fundamental problems with doing construction this way however.
The Problems
Firstly it’s very script intensive and requires lots of script maintenance and balancing. If you change how a building works or basically anything in-game which changes the economic balance then you would end up having to rewrite a lot of script. “Oh, if we change this then we would need more alloys but if I increase the weighting for foundries then that would mean that this other building was not built often enough and then if we change that then…”, you see the issue?
What we ended up with was huge, complex, and very explicit and static scripts for each building with a lot of hard to follow ( and expensive to compute ) if/else statements with different weights that nobody could keep track of in the end. Below is an example AI weight script for a single building.
How does weight 200, 400 and 500 compare to other weights of buildings in other files? I have no idea.
Another problem is that an AI working this way is extremely reactive, and not very proactive. It has low income of something which leads to a script evaluating a certain building very high and it then builds that, that’s fine. But what if you’re doing quite alright? How do you then determine what to build? You would have to add even more script that says something along the line of “if you’re doing okay in all these resources then this should be built, but don’t value it too high because maybe we are in need of some auxiliary resource, hmmmm…”, you can see how this would be hard to script and would easily conflict with other script. It’s also a problem because this would all have to be manually maintained and updated as well, in each script for each building.
Yet another issue is that by doing this on a planet-by-planet basis in script we have no real idea of what has been built on any other planet or for what reason. This means that if we on one planet evaluate that an energy producing district is needed because we are running low on energy and then queue that, we will most likely queue the exact same district on the next planet we check since the situation looks exactly the same to the evaluating script even though when construction is done of the first district we might be fine and the second district would then just add to an unnecessary surplus. This might sound like a small issue but imagine if this happened on 12 different planets, it could swing an AI economy completely in the wrong direction and have massive long term consequences. The AI could even end up having to destroy some of those districts to make room for something else, making the entire effort a wasted one.
There were other smaller issues as well, but these were the main ones I wanted to address:
- Less complex and hard to maintain script
- More proactive AI planning
- No isolated planet-by-planet construction
- Overall better economic AI performance
The Solution aka. The Plan™
The wonderful Stellaris QA had put together a list of economic metrics that they thought the AI should be able to achieve at a certain point in time. They had a set for the early game and one for the late game. I started designing and planning how I would do this rework and came to the conclusion that the AI needed to actually plan long term, and not just react to whatever happened to them economically. This tied nicely into these performance metrics that QA had compiled so I just thought “hey, why not make these actual goals in the game!?”, and so the AI Economic Plan was born. The economic plan is at the core of the new economic AI, from the plan the AI derives exactly what it needs to reach the plan goals and decides how to do it in the long term. Plans are fully scriptable objects that simply set goals that the AI should aim for and when. Plans can either be for the early, middle or late game. They also have potential triggers and AI weights so that you can script exactly who, when and why an AI should pick a certain plan ( plans are only replaced when they are fulfilled or no longer valid ).
Above is the default early economic plan used by regular empires. As you can see the potential trigger ensures that no hiveminds for example use this plan since that would mean that they started producing lots of consumer goods that they don’t need.
Below I will go through each scriptable field of the example plan, so if you’re not interested in scripting your own or more in-depth how the system works you can skip this part
Executing The Plan™
So how does the AI turn these plans into anything actionable? Good question! At regular intervals the AI will look at its current economic situation ( income, deficits, budgets etc. ) and compare it to the goals set in the currently active plan. From this it will derive exactly what it needs to reach those goals and create a build plan from that. The build plan is basically a prioritized list of buildings ( or districts ) that the AI wants to build on different planets.
This build plan can be viewed in-game while hovering an empty or locked building slot with ‘debugtooltip’ enabled and can look something like the above image ( the number is the score that the AI has calculated for that building ). The AI will recalculate this plan at regular intervals as well, so that it does not stick to old plans that are no longer relevant.
The AI constructs this build plan by finding out exactly what it can build, what those buildings actually produce when built ( including resources from jobs ) and then try to mix and match buildings to as effectively as possible fulfill the goals identified from the economic plan. This of course takes into account any building upkeep costs or tech limitations present. A benefit of this system is that you no longer have to explicitly script when the AI should build a certain building, instead it will dynamically figure out what each building produces in code. This also means that it is way quicker to add new buildings that produce different resources since you don’t have to manually script how the AI should handle them. Moreover the build plan also tries to take into account on which planet a building would produce the most and build that building there, making the AI more likely to create specialized resource-production planets like “foundry worlds”. It will also not build anything on planets that already have several free jobs. Since this is now done on an empire-wide basis and not planet-by-planet the AI can keep track of and work towards an overarching goal instead of looking at planets in relative isolation. The AI will still go through all your planets to see if any of them are missing housing, amenities or any other planet-specific resources. These buildings are however still part of the economic plan and so if you REALLY need another resource then one of your planets might have to live with an amenities deficit for a while.
So how does the AI actually prioritize the different buildings in the build plan? Another great question! To answer that we need to look at how the AI scores different buildings and districts. The scoring function takes several parameters into account: income needed to reach plan goals, income needed to reach focus goals, current income and deficits, resources produced, miscellaneous resources ( amenities, crime, housing etc. ) produced and lastly if it contributes to pop growth or assembly. The given building gets a score for each of these parameters that is factored by how much we are currently currently missing in that category and how much this building would help to rectify that, along with a defined multiplier. We then add even more score to it if it produces a resource in which we currently have a deficit or if it brings us closer to our focus goal. NOTE: The system remembers all queued buildings so if we have queued 4 mineral districts and that would fix our current mineral deficit then the 5th mineral district would not get the extra deficit score!
Above you can see the multipliers applied to each of these scoring parameters, these are all defines and can therefore be modded.
This all sounds very complicated and “maths-y” so let’s do a quick example to illustrate. If Blorg currently has a steady income in all resources except for minerals where net income is -1, and is using the default plan as outlined above with a mineral income goal of 200 and a mineral focus goal of 30, then when the algorithm is scoring a mineral district it would…
- See that Blorg is currently missing ~200 net mineral income to reach is goal and would give this a very high base score since the relative amount of income missing is huge
- See that the mining district produces resources needed for a focus goal and multiply the score by a defined factor
- See that Blorg currently has a mineral deficit and yet again multiply the score by a defined factor
The algorithm would then give a score for the build plan to use when prioritizing buildings and a mining district ( or several ) would soon be constructed by the Blorg. This would work similarly no matter which building we’re looking at, the important thing is that it scores it relatively to how much you are missing from your goal, and deducts any planned building resources from said goals. Meaning that if you’re only missing 2 mineral income the score wouldn’t be that high and if you have queued a mineral producing building then the AI would no longer look at those buildings because the ones you have planned account for all you need to reach that goal.
Evaluating The Plan™
So, does this system manage to address all the issues that I set out to? Yes, it does! :D
You no longer need to do complex scripts for each building, instead you can tweak the overall plan to affect how important the AI thinks a resource is. The plans also ensure that the AI is always looking forward, being proactive, instead of just reacting to deficits and certain happenings. The AI no longer looks at planets in isolation but as part of a whole, all tied together by The Plan™.
This all sounds very fancy-like and good, but now for the real question; does the economic AI perform better overall? Well...yeah!
In our tests we have seen much more robust AI economies and a better handling of the economy overall. And I’m really happy with it! Of course this wasn’t all me, I had lots of help from QA and other devs, and am very thankful for that because without them this wouldn’t have been nearly as good. Now of course, with the release of Federations closing in I would love to see what you ( the community ) manage to do with this and how you will manage to mod this into oblivion
Lastly, I of course did not forget all of the hard work you put into your mods when writing this. The old style of writing building scripts and weights still work IF there are no economic plans for the AI to use. This means that if you remove all the plans or if an AI fulfills all the plans that it can have, then it will fall back to the old system of building planet-by-planet and use the old building weights.
Thanks for reading, and good luck amongst the stars!