Just in time for Christmas we released version 2.71 which is almost strictly about AI tweaks based on player feedback and saved games. Now, someone might ask "Why do you need save games to make the AI do [obvious thing]?". The answer is that with so many different ways of playing the game (both in terms of game setup and in-game strategies) it turns out people play the game very differently from one another. It's one of the reasons why you have most people who think the AI is really good and some people who think the AI is a total push-over. It's not that the latter group are cheating or even exploiting, they have simply discovered a strategy that is unbeatable.
But as much as version 2.71 improves the AI, I'd like to give you a sneak preview of 2.8...
Over the years, I've written a lot of different types of AI and I've generally tended to make what can broadly be called instanced adaptive AI. That is, the AI performs a series of tests on itself on the background to see how it is doing and makes a series of adjustments based on how it does on that test. This worked really well in GalCiv II for the most part
When Stardock made Galactic Civilizations III, I wasn't really involved in its design and the team chose to go with "modding" as its focus. That is, let the player have a bunch of XML files to modify how the AI works. The problem I have with that system is that it only really works if you get an AI modder who is performing the adaptation for you.
When I returned to GalCiv III for the Crusade expansion, I began working with this system to make it more adaptive. When I joined, there were 8 key areas that the "moddable AI" wasn't really suited for:
Not all 8 are equally important but all 8 need to be addressed, imo, before GalCiv III can claim the AI crown from GalCiv II: Dark Avatar.
Version 2.5 of the game was the first version to start to tackle this:
2.5: Putting together a fleet based on the state of the galaxy.
2.6: Building up its planets based on the state of the game
2.71: Exploring space that adapts to the size of the map
Those 3 items make a huge difference to the way the game plays. Being able to right-size its fleet size is crucial. You don't want an endless stream of tiny fleets uselessly attacking you and it does no good if the AI waits forever to send out some mega fleet. It was version 2.5 that really addressed this and based on what I've read on forums and Reddit I think was the key AI feature that helped word-of-mouth on the game.
Today's update, 2.71 is pretty crucial for those crazies that play on really large maps. The moddable AI had very fixed ways of exploring the galaxy and as we've increased the map size over time, the AI had very limited abilities to adapt. Contrary to what people who lose the game would have you believe, the AI can't "see" the planets. It has to explore them. It has its own FOW just like you (and that's one reason the AI uses so much memory -- every tile has to hold the FOW state for up to 255 players and there's a ludicrous number of tiles on the largest map size.
At 2.5, the AI satisfied most people. At 2.71, we are definitely in "diminishing" returns. But 2.8 is something I'm really excited about. Here's a sneak preview screenshot:
So this is from a game I was playing last night. That's 3 enemy fleets with with transports. On normal difficulty. Things didn't go well for me after that. Which isn't to say my work is done. While 2.8 addresses the moddable AI fleet governor (yea, there's an XML file you can mess with that says what must be in an AI fleet that no one has ever touched) to have that fleet adapt based on the player's needs, it doesn't deal with the moddable ship design template which affects the quality of those fleets (so you end up with meh enemy ships). HIgher difficulty levels can fix that and frankly, 17 "meh" ships will still wipe out most players.
The other thing about that screenshot is that each of those fleets is traveling at 7 moves per turn. This is due to the AI adapting its research path based on the state of the galaxy.
Next up is having the ORDER in which the AI builds improvements be based on its needs rather than based on modding. At this point, you probably wonder, what happens to the modding? The answer, nothing good. Moddable AIs are a bad idea imo. A good AI should work in a way that is well beyond our understanding. Not because it's "smart" but because it should be very complex.
My job, as the AI guy, is to simply come up with tests the AI uses to measure itself and adapt to it. That's how I've been doing it since the OS/2 days. The more data I have access to, the better tests I can perform and the AI makes adjustments based on that.
I don't do this for you guys, I do it for me. Because an adaptive AI will surprise you. Last night when those 3 fleets showed up, I woke my poor wife up when I yelled "Oh shit!" because I was totally not expecting that. Until the adaptive transport fleet AI was made, the AI had never put together a fleet group like this.
After the adaptive improvement construction is handled, next up is the AI ship building being adaptive. That's going to take some time.
As some of you know, what motivates me is you guys. That's why I harp on the Steam reviews so much. If people don't like the game, I don't really want to work on it. If you haven't already reviewed the game, I encourage you to do so.
There is a small chance that I might get an opt in of my first pass of 2.8 before Christmas. But people are starting to go on vacation so no promises. In the meantime you can hang out with us on Discord.
Thanks for the article, interesting read.
Hmm, let's assume what you are doing is like training a neural network. Is the learning closer to supervised or to reinforcement learning?
Does the learning only occur on your end, i.e. between patches or also in between games which I play on my local pc?
On my end, I only learn what tests to give the AI and then provide conditions on what passing the test would require. How it does that it up to the AI.
That's why I was shocked to see 3 fully loaded invasion fleets.
But the actual learning is only happening within a game. My part is only to devise new tests.
Regarding ship design, I think the AI should just cheat. Like just let them know what weapons their main enemy is using so it can pick defenses easily.
AI is something that can be majorly difficult to nail, but when done right it can be the best part of a game.
Outstanding.
Keep up the great work everyone!
Well, yeah. It's great work by Frogboy and the GC3 team. I'm just worried that he's going to make it so smart I won't be able to beat it anymore
I'm against a cheating AI if it's possible to achieve the same effect by letting the AI use the game mechanics properly.
Appreciate the post Brad and your efforts.
To get to that Dark Avatar level I would also suggest there is a need to look back at the DLC before you get heavily involved at Crusade and ensure the AI is reasonably effective at using them. Specifically, improving AI use of Mercenaries, exploring Precursor Anomalies and being more aggressive with colonisation of Precursor Worlds. I also don't think I've seen the AI use special components from Precursor Treasures, ever.
It's funny, almost 25 years ago I studied Adaptive AI, little did I know at the time of it's role in good games! I went the Engineering path ...
Started a 2.7.1 game last night and saw very effective colony ship use at the started (AI at gifted levels)- enough so that the AI may hit some admin point limits. Be interesting to see how it makes balancing choices to get more admin points as the game goes on.
Great stuff.
About the call for reviews: unfortunately imho most people who take the time to visit the forums prolly alrdy wrote a review. Its the other poeple that would be great to reach, but the same time there is no channel to do so.
This is the kind of stuff I've been waiting for. It gets old playing the game time after time when the only way the AI is any good is because the deck is stacked in it's favor and it still only churns out simple ships and fleets. I did expect more tactical surprises from GCIII. This might be part of the out cry to have the multi-player in GCIII. They were looking for a more unpredictive opponent.
Thanks Stardock for continuing to improve this game.
With 2.7.1 I could see a more rounded strategy was needed.
With 2.8 I wonder if people will be needed.
ForesterSOF iam sure you are alone with your question in reference to 2.8 and that's good so.
Another good measure of success would be ... the complaints of Noobs on Normal difficulty become so loud that Brad has to nerf the AI on Normal difficulty ... like Gal Civ 2.
Iam also and can't await it!
Al this talk about adaptive AI is very interesting.
One thing i am curious about; after having modded new promotions:
How the Ai can understand how and when to use promotions ?
There is nothing in there to help the AI decide what to do !
Can't wait to get my hands on this! The AI always sending one group of ships, then small weak ships in huge streams was never difficult to defeat. Hopefully it's better now!
Yeah me too, but this is the kind of thing that would be hard to code and the AI would be wrong so often that it wouldn't be worth doing.
I would not deem it too complicated to let the AI analyze the weapons and defenses it sees on enemy ships it encouters and act accordingly, just like the player does. But it's up to Frogboy to make the last decision here.
It already does that part. The AI already knows what it wants in an ideal ship. What it doesn't know how to do (which it did in GalCiv II) is actually design the ship. Instead, it goes through those ship templates and sees what the best ship it can build based on the manufacturing capacity of the empire.
This is an area that would need a lot of work to reach the next level because it would require the AI to coodinate between its colonies on a future strike force (i.e. have some planets far in back that can spend 34 turns building a super dreadnought). Humans do this now.
The problem is an economic one: Players can just turn up the difficulty level and get the same effect. Moreover, every time I do implement some major new AI, I find some crazy balance thing (oh, you found a module with 50 attack? Oh hey, there's a special planetary improvement that will increase your max ship speed globally by 8!).
My AI only can get better based on the tests that *I* set up for it. So when I find out there's some major debalancing thing (like palyers with 30 moves per turn thing) that the game itself was never designed for (if the AI was building hundreds of 30 moves per turn ships you'd be waiting hours between turns due to path finding analysis). The pathfinding system was intended to max out at around 8 moves per turn.
But when I fix those kinds of things, a handful of people get mad and give it negative Steam reviews and those are devastating to sales.
Here's a rough approximation of conversions to sales (per page view) based on Steam user score:
Steam user Score / Conversions:
>90: 10
>80: 6
>70: 2
>60: 1
<60: trivial
So fixing the AI or improving balance results in a massive hit in sales.
Right now, ES2 has about 4 times as many people playing it as GalCiv III. It has an 82% positive score compared to GalCiv III's 72%.
If you look at 10 GalCiv III reviews (which we do endless analysis of)
You will get:
-1 Because of bugs/polish
-1 Because they just don't like it
-1 From someone mad about a balance change
More should be done on bugs and polish. I have 1,755 hours into GalCiv III and agree the game should get dinged for this.
You can't do much about the people who thought it would be a different game or just found it not to their liking. It's part of this particular genre.
But the last one, that's the one that hurts. You could argue that the base game shouldn't have had the crazy balance stuff (like the Precursor DLC in particular) but I think you just have to address this stuff. But it comes at a huge cost. So we effectively spend money to lose money when we "fix" this stuff.
This is why I am so adamant about people who haven't reviewed the game on Steam to go review it IF they want to keep seeing these updates. You can't really fully counter every -1 we get from people who are mad that they won't be able to move their ships 40 moves per turn (you should fix the AI to deal with it!!!11!). But it can make the difference between a Mostly Positive and a Mixed.
Thanks for the insight about ship movement and turn times because of path finding. To counter that I see other possibilities other than nerfing max moves per turn (although that will make still other people mad ...):
- Reduce the number of ships by making ships more expensive (perhaps adjusted to map size, so that the cost is higher on larger maps because the games are longer and the player has more time to wait).
- Reduce base speed of ships generally to 1. Make it so that only large ships can carry interstellar drives but those give interstellar movement to the entire fleet such a ship is in. That would mean less single ships flying around but more fleets. Cargo hulls must still be able to carry interstellar drives, survey ships and explorers must then be cargo hulls, so that they together with colonizers and constructors are still usable. That would mean a cargo hull ship would be enough to give a fleet interstellar movement, but one could adjust AI combat behaviour so that the AI goes more aggressively against support ships, so a fleet with only a cargo hull ship to provide interstellar movement is in danger of being crippled when losing that ship.
Apart from that, my personal opinion is that a singleplayer sandbox game does not have to be balanced perfectly. Adjusting to imbalances that are different in every single game is part of the fun for me
Well hallelujah. Unfortunately this admission is way too little, way too late imo. Bugs aren't just a problem with GC3; they're a disaster. It is literally the second-worst commercial video game *from the bug standpoint* that I personally have ever seen, losing out only to the all-time winner in this category: Master of Orion 3.
Yes it's come a long way and yes it's not terrible at the moment. But there are still, right now, half a dozen issues at least that I'm vaguely aware of from reading the forum that prevent me from even considering picking it up again. Here's a simple illustration of the scope of the problem:
Time taken to reach even minimally-acceptable basic playability, again in my personal opinion based on 300+ hours playing and 1000+ hours modding:
Launch: 6+ months (version 1.0 to 1.6 or so)
Crusade: 3+ months (version 2.0 to 2.3)
2.5 Rework: 3+ months (version 2.5 to 2.7)
Yes I'm pissed off, b/c I assumed the devs would do at least a decent job in ALL areas and stupidly wasted a huge amount of time as a result. But that's not the point. I'm always focused on being objective and calling stuff like I see it, and I was still holding out some sliver of hope things would get better.
Unfortunately 2.7 was far from comprehensive bugfix-wise, and with another expansion looming, things look pretty hopeless given the above list. Sigh.
As I said in another post though, I never review anything since I prefer to avoid interacting with other humans as much as possible, so I'm not part of your Steam problem and never will be; it's annoying enough as it is I have to write these forum posts. But before you say "you don't have to write/post anything!", yes I do in this case. After continuing to be upset with the way this product has gone for 2 years straight I repeatedly get so mad I have no choice but to vent. I do apologize for my lack of self-control in that regard.
I've seen this comment made a few times now and each time I've gone through the negative Steam reviews and come to a totally different conclusion. I'm very aware that I have a bias towards making ongoing balance fixes and AI improvements ... so I checked again. Specifically it's extraordinarily rare to find a review that negatively calls out one of your balance fixes or AI improvements.
Now about a third of Steam reviews are almost incoherent, making them difficult thematically to classify or apply constructively, so probably should be ignored. The rest had the following key themes repeated over and over: crashes/bugs and lack of innovation. Less common are pricing, tutorial, or UI issues. Occasionally there is a general comment about the AI (rather than a specific recent change you've made). To be honest, my take was there plenty of good ideas that could be considered in the next expansion (noting I said expansion, not patches).
Crusade has 84% positive reviews but most interestingly EVERY SINGLE ONE of the last 20 (i.e. recent reviews) is positive (at the time of writing this post). It doesn't seem consistent with your comment. My take on why that is ... obviously Crusade is a vastly improved game ... but it's SO much better that it dwarfs any other annoyances or pet peeves (even from the "don't like change because" club). As much as I like the frequent patches, it does seem to be a better strategy, from the point of view of Steam reviews, to bundle up changes for this reason.
Has it been seriously considered to make Crusade the base game sooner rather than later i.e. for all new players (the original could still be on the Beta branch)? Would the higher proportion of positive Steam reviews outweigh the price differential? What if the price was adjusted accordingly as well?
Steam reviews seem to encourage an excessively short term sales focus. What about the impact on existing customers who have already given reviews for future sales? Before Gal Civ III released, I was an "automatic" founder after great experiences with Gal Civ 2 and Sins. After Gal Civ III released I was shocked by how bad the balance and AI was, even though I totally understand today's product lifecycle, and sadly dropped out of the "automatic" founder club for Gal Civ IV and would have happily waited for a mega-sale and plenty of post-release updates. After all the recent effort with Gal Civ III, I'm happy to say I'll definitely be back as a Gal Civ IV founder. And before you say I would buy it anyway, no, there are a whole of bunch of 4X releases that I never bother buying. Scottish heritage and all.
@Lunar: Buzz off. If you didn't think GalCiv III was playable at release, then you aren't the target market for the game.
My issues on polish and bugs amounts to dozens of little things ranging from things like "You have received a message from a MAJOR CIVILIZATION" (okay, WHO?) or having the amount of military production to a shipyard not show up until you build something.
-
@ice
We have looked into this but it is basically impossible due to the way Steam works. You can do bundles but you can't replace GameID-A with GameID-B without everyone who has GameID-A already now having GameID-B and we're not really in a position to hand out hundreds of thousands of free copies to Crusade.
Version 2.5 merged the two quite a bit but the citizen based economy is just so much more compelling I think.
There are many great features available to you once you register, including:
Sign in or Create Account
