Some people are good at making things. Others are good at breaking them. I guess I’m a breaker.
As some long time players know, we’ve been trying to find ways to reduce the memory foot print of Kumquat (the engine behind Elemental). It’s a non-trivial thing to do, especially when long ago when we were specing this we were thinking of the day of 64-bit gaming when the 2 gigabyte memory limit was no longer a concern.
2 Gigs
Now, a quick thing about that. No matter how much memory is on your computer, a PC game will only be able to see 2 gigabytes of it unless the game itself is 64-bit which, to my knowledge, there are no major games that fit that yet. This is something pretty high on our priority list for the future but with most people still running 32-bit Windows, we have to focus on the here and now.
Crazy Games
Anyway, I have a saved game that upon loading would instantly crash the game because it was on a crazy sized maps with crazy sized cities with crazy numbers of units. The challenge to the team was to find a way to make this work.
Most games work around this problem with unit limits. That’s why Sins of a Solar Empire and Starcraft 2 (to name two examples) have very definite numbers of units in the game. War of Magic has no such limit. There’s nothing stopping someone from creating tens of thousands of unique units.
Multithreaded
The Kumquat engine is highly multithreaded. Now, what does that mean? It means that if you have multiple cores, execution of the game will occur on multiple cores. The AI has its own thread. The graphics engine has some threads the main game has a thread, the music has a thread and so on.
It’s the reason why, even on a relatively low end system, you can zoom out and see the world and then zoom in and see individual people living out their lives in one smooth, fluid motion. It’s a pretty impressive technical feat imo. But it comes at a cost – memory. The easiest, fastest way to implement this ability was to give each unit its own instance so that it could be threaded. That doesn’t mean each unit is its own thread, that would create a different set of problems. It’s a bit more complicated than that (ok, it’s a lot more complicated than that). But what it means is that the engine can choose to put different groups of units into their own thread in order to make sure everything remains smooth. But it means doubling up on memory – or tripling or quadrupling and so on.
Now, this isn’t something that was a mystery to us. We knew about this. The problem was, how do we have our cake and eat it. The solution basically boiled down to months of work to write a different system to do this. So for the past few months, that has been what Codecritter and the gang have been trying to do. Now, in v1.19, some people people have reported a performance drop so that’s being addressed presently as well.
But the good news is that, in theory, the next beta of v1.2 (v1.19c? d?) should largely eliminate the memory issue for most people.
Far Reaching effects
Most people don’t connect gameplay limitations to memory. But there’s a definite connection. Because when you’re fighting a memory limit, it greatly limits what you can add to the world. You end up having to bland things out because, frankly, adding more “stuff” becomes very problematic. Most people don’t make these connections (if you read forums they think the designers or developers are just oblivious or didn’t “see” some issue). But that is the challenge of designing a game – any game – on any platform is making guesses as to how much of a design can fit in. That’s another reason why sequels are so tempting. You already know the scope.
The bottom line is that it’s good news for War of Magic and great news for Fallen Enchantress.
Nice, thanks for sharing!
I'm too dumb to understand this stuff. But...uh...less crashing and freezing and junk, right? Thanks!
Great Read! I've found value in most all of the Dev Journals I've read. This one rates towards the top of the list! TY
Thanks for keeping us updated Frogboy!
Eh, the 2GB limit isn't a problem for me, since the blue eyed monster's motherboard wet poof after months of storage, I'm gonna be 1GB'it for a while.
Out of curiosity, to what extent can virtual memory be used to compensate for a lack of RAM?
there is a clothmap only setting already - that should do what you are looking for.
Thanks for the update Frogboy! As an indie game developer myself, I am highly tolerant of such issues. I know you guys are working hard to fix the issue, and I have faith that you will! I have to say that EWoM handles this issue FAR better than Civ5. When my Civ5 games get too big, performance comes to a halt and finally crashes. Your game continues to operate at peak performance, but then eventually crashes. I also find that I don't get the OoM error for a long period even when the game is extensive and nearly finished. Civ5, once it starts crashing, it becomes unplayable. I have two incomplete Civ5 games to show for it. Crash every other turn.... EWoM doesn't do this!! I get to finish!
EWoM wins in my book (plus the gameplay is better ).
Awesome, looks like I found an excuse to play the game again before the next patch. Thanks RF.
Hopefully you get those memory leak gremlins eriadicated- I want the engine to be in great shape for FE.
As for the 64-bit games, can I ask when do you think you'll make that a requirement?
Realistically, what percentage of people playing Stardock games are still using 32-bit OS's? I"m curious.
One really, really crazy idea. You said 2 expansions for Elemental? If FE is well received, why not make the 3rd game an expansion of FE, but make the 3rd game the 64-bit version?
Accidental double post- feel free to delete.
Isn't it possible to use more than 4GB of memory even on 32-bit OSes with PAE-enabled kernel? I know Windows XP has a hard-coded limit, but newer version should be able. I know for certain that Linux can.
There are actually quite a few "major games" that have released with 64bit. Not one that's ever actually needed it though. 45% of computers on steam are running 64 bits OS's, excluding macs. Did anyone at stardock tally the 64bit vs 32 systems during the Beta? If you were to weed out the majority of MomPCs from the steam numbers you could very possibly have a majority in the 64bit category.
The downside with a slim majority is when you make a game for 60% of the computers out there, you can only sell to 60% of the market.
This is why Stardock's been big on reasonable specs for their games- and it works well for them. I think with Elemental: 2nd expansion, that splitting the game into 32/64-bit versions since it will again be separate games, and the number of folks who will be getting it free/reduced, could work.
I don't know where you got that percentage (94.37% chance its made up on the spot, though not necessarily by you), but you do realize if someone at Stardock tallied that and got the same numbers as you what would happen right. Think of it this way, you are part of a team that is producing a PC game, would you cut out 55% of your potential customers right off the bat?
Also how many Stardock produced titles do you think there are on Steam, considering Stardock has their own digital distribution platform (Impulse). Elemental certainly isn't on there last I checked, and it may be in both Valve and Stardock's best interest to keep it that way due to it being the closest thing Steam has to competition. Though this is verging on industry politics so I stand by my earlier points and not theoretical ones.
Thanks for the continued work and informative posts!
It's funny that a "2010 AAA game" would have such a late 90's architecture.
FYI shoe horning concurrency into an older design is not fun or easy.
Your task doesn't just become twice as hard, it becomes exponentially harder.
Stardock doesn't make games for the money... They should be the first to make a 64-bit game... It would be revolutionary... turn the industry on it's head and the rest of us would reap the benefits for years to come (at least until we start complaining that 64-bit games are too slow and we need 128-bit only games).
Thanks for the update, im looking forward to seeing the improvements in 1.2 patch
A question for Stardock, are the system specs for the new expansion going to be similiar to war of magic?
Virtual memory is your hard drive, so basically it's a performance killer. You'll get optimum performance if everything is stored in system RAM.
Weed out the majority of the "MomPCs" and you weed out the majority of your potential customers.
can you explain that middle part again
Stardock does like their games to make money. Brad himself has said the reason he hasn't gone 64-bit is due to the amount of older systems, though that was back in the GC2 days. Yeah, Stardock makes games because they like doing it, but Stardock wouldn't be able to keep making games if they all lost large amounts of money.
I think the time is coming, if it hasn't arrived already for games to transition to 64-bit.
I would just like to add that the Memory/Speed work/tweaks that were applied to 1.19b seems to really show in up Tactical.
The response now, after a units turns ends and the next begins, is way better.Keep after it Dev..
OT: Now if we could get the Cartographers Table back on its legs, that would be ideal.
IF it works and we no longer get the OOM issue then I shall plant a big wet sloppy internet man kiss on the Frog because this issue has been the chief joykiller and main gripe for me to date with this game.
There is a 56% chance that only 32% out of the 100% of those people who dnld 1.19c (and beyond) will be happy. Because being Happy would screw up their current belief system...
(+/- 3%) for balance reasons. LOL
There are many great features available to you once you register, including:
Sign in or Create Account