Finally a REAL developer journal.
In game development, one of the most important polish points is making sure that the game is always responsive to the user. It’s better to take 1 second longer than to stutter or hang for frames.
Unfortunately, it often gets very expensive in engineering hours to meet that objective.
In Elemental, we have two areas that are particularly problematic in that area before you even get into the game:
#1 When you first launch the game.
and
#2 Waiting for the game to start (i.e. the time after you start the game and you actually get to the map).
In both cases, the delay is due to parsing in XML. It’s completely a CPU bound issue.
Elemental is designed to support multi-core but we also support single core processors too. And when you first launch the game, that’s where the guy with the quad core is going to notice the biggest difference versus the guy with a single core CPU.
THIS…
BECOMES…
Which has to be parsed…
…
Anyway…
The “stuff” that has to be loaded before you even see the title screen is in the \english directory. All the other XML gets loaded after. The problem is, there’s a lot of stuff in that english directory.
Thankfully, tile designs don’t have to go in there. But UNITS do and “GoodieHuts” and anything you would expect to encounter while setting up the game.
As of now (1:43am on Wednesday, August 11) there is 3.31 MEGABYTES of XML (that’s just text mind you) that has to be read in and parsed. Heck, there's 65K of hats that have to be parsed (I mean literally, hats as in things to put on your guy’s head).
But by far, it’s the units that are the biggest chunk of it. And we’re adding more creatures every day. So this is something we’ll have to find a way to deal with between now and release otherwise, the first impression will be crummy for people with slower machines.
Either we have to find a way to move the unit parsing into a background thread (not as easy as it sounds) or we need to find a way so that we don’t have so many units being loaded up front.
Now, the other issue, which there really is no way around, is the time it takes after you hit “Start game” and you get to the map. All those wonderful tiles have to be loaded and parsed. There’s 11 megabytes of XML there. It’s the nature of the beast unfortunately and one that I think most would agree is worth it. But you can bet, many of us are looking forward to going through all 770 tiles we have as of tonight and seeing which ones we could get rid of.
In Elemental, we have two areas that are particularly problematic in that area before you even get into the game: #1 When you first launch the game. and #2 Waiting for the game to start (i.e. the time after you start the game and you actually get to the map).
Hey I wouldn't worry about that. 'Game starting delay' is the last opportunity I get to sip my tea before it goes cold with that nasty white froth floating on it nine hours later.
Heh, I doubt it will be as bad as GalCiv 2 when I first got it. I used, (thank god I can say 'used') to run it on a single core system running Vista with half a gig of Ram. It used to take literally 15-20 mins for a game to load, up to half an hour for late game saves.
I'm still running the single core, but Ive more than quadrupled my RAM. Sooo nice.
Besides, if E:WoM can run on "a crappy netbook" I think it will be bearable on my machine.
If I may ask, what is the expectation of load times in a worse case on single core?
5 seconds +- 1000 seconds. Honestly it is impossible to answer this question accurately for a general case.
I should think that a close generalization should be possible, or lacking that, the goal of how long they would like to see it take in a worse case on single core. I do take your meaning though.
Won't this be a concern for modders who add tons of units and tiles? If so, then cutting units and tiles will only be a short term solution that will cause problems later.
Get rid of ALL tiles. I only use the cloth map anyway.
Seriously, the tiles CAN be delayed until later, and you can use the cloth map until they are available.
Please don't remove content that you already have in the game. The bigger the better.
Amen!!! We'll deal with the loading times!!! Just put some info worth reading on the loading screens and I don't think anyone will mind.
Even if I had a dual+ core CPU, I wouldn't want this.
I'd rather all 2, 4, 8, however many of my cores giving their resources to getting me into Elemental not playing out a minigame.
The question is not load time of the vanilla version of the game as a binary would be generated only when the game will be updated but rather load time for "mod & modding intensive players".
Will the binary be "wholly" re-generated every time we launch a new game after changing our mods and/or mods option?
Thanks.
Well, on a quad core i7 you're more likely to be waiting for disk I/O on at least one core to read XML, so the hit would be pretty negligible. But on a single core (the people who will be delayed the most) any CPU time spent handling a minigame is CPU time spent not doing XML processing.
But how do you generate a random map with stuff on it if you don't know what the "stuff" can be?
The tiles aren't just graphical, they contain things that can affect gamplay. It's data that other parts of the game will need. Even the cloth map has to know that I modded in a "Tower of Infinite Pwnage" so you can see that something is there on the map.
If there's some things that aren't needed right away you could try to do lazy initialization, but for something like this it's going to be a lot more complicated then simply caching the parsed XML result as a binary. The second load will be just reading in the binary, which is blazing fast even on a slow PC.
So if you don't mod, you'll only have to worry about the delay after updates. Downloading a mod will make you worry about it more often, and actually writing a mod will give you more frequent delays.
But! The game could cache the core XML files anyway, then always load mod XML files uncached and combine them in memory. That would still be a significant performance boost, and you wouldn't see large delays due to the core game XML being reparsed just because you added a mod. (Can you tell I deal with this stuff a bit at work? )
I told you, man, S-expressions. I mean, you must have already succumbed to Greenspun's 10th what with the whole "data-driven" thing.
But yeah, a cached binary format is probably the quickest, easiest, least-redevelopment-requiring fix.
Frogboy, why isn't this an option?
https://forums.elementalgame.com/390597
I don't know what your setup is but it seems like the obvious thing to do to fix the load time?
I don't have a single core so I don't know how bad it is for them but I myself don't find it bad (dual core). I think what you really need is a progress indicator. Waiting is fine, waiting with having no idea how much longer it is going to take can be very frustrating. Also I find the inconsistancy in loading times very odd. I really can't remember ever having a game have such drastic differences in loading time. The problem is expectations. If it loaded fast once then I expect it to load fast again and if it doesn't I think it is broken. Again just giving some feedback to the user would be helpful. Just add a little message on the loading screen saying, "Parsing XML..." and have a little progress bar. That way we know that from one load to another something is different and can see why. Hopefully something along these lines was already in the plans.
removing content (apart from really unnecessary/redundant stuff, which I assume frogboy is talking about) now does seem kind of pointless when the game is aimed to be moddable and thus should get more content very soon.
that leaves only optimization. as for xml parsing speed, don't blame xml. At work we parsed around 100mb of xml in under half a minute (5 files, each 18mb). and this was using the standard xml deserialization from .net, not some super-parser
so my guess would be its good old windows filesystem.. fat and ntfs are notorious for being ultra slow, in my experience especially for reading lots of small files. so one BIG xml with everything in it possibly would be way faster (just try a 'cat * -R > bigfile.xml' if you have cygwin or a linux box and parse that).
if the difference is as large as I assume, binary serialization would make a huge difference (less files, less data and faster parsing).
MAYBE just zipping the xml files together in one zip would even be faster (despite having to unzip on load) if the real bottleneck is I/O and not parsing
or you could just use profiling to find the real bottlenecks instead of just trying what helps
and loading just a random amount seems a bit.. cheap. I think the game should be capable of loading at least the threefold the current data within a sane amount of time - if not, the game should be optimized, not content just add limits.
This is because XML is a data-exchange format and not really efficient for data storage. It just has way to much data-redundancy and uses way too much CPU to process. Coming up with a better storage format is no harder than writing the XML schema, but I guess it is too late for that now, but please consider taking a royal dump on XML in your next game
One thing I have always thought was needed is a simple progress bar. I do not care if loading/world building takes a while BUT it there is no feedback you start wondering if the game has simply hung.
So loading screen should have some visible method of showing me that its doing something. Then if it takes a minute at least I do not start to think something has gone wrong.
FKN
William
I think the prevailing process indicator system is leaning towards a slide show of art and lore screens while you wait. I had a thought on this and would like to suggest excerpts from the book associated with the game.
This is what's needed I think. Simple, easy, and it lets people know the game hasn't hung.
Please do this instead of on first/every load. Have install and patches create/update your cache, segmented out to reduce refresh time. Potentially even include a pre-built cache to copy over during install in addition to updating cache whenever checksums don't match.
Some games take a ridiculous amount of time to save/load/start that could be avoided with caching / using a faster storage mechanism.
I strongly disagree with your opinion:
the community wants moddable games. the easier modding is, the better the community content gets (as more people will have the skills to create something). proprietary formats where widely used in the past when space (1.44 mb, 750mb, 4.4 gb) where hard limits and no modding was planned. adding modding, the need for easily understandable, standardized formats arises, also to allow users to create their own tools, editors whatever. modding with proprietary formats is possible but very limited as everybody just has to stick to the given tools (which often aren't very user friendly). this essentially removes a large part of the community as prospective modders.
xml is by far the easiest, most standardized data storage to be used by a modding community. the only real cons are speed and size, and only the first is really a concern nowadays. so instead of suggesting to throw everything (all the advantages and the few minor disadvantages) out of the window and start building good old static games, I'd prefer they stick with xml and learn to optimize its speed or use some caching function (which can be used by every new game). This will not diminish the flexibility of the game as your suggestion most probably would.
ora
A Progress Bar that starts at 0(zero) on the Left and shows the actual # of .xml files (how many 3.31Mb that are getting loaded(parsed), on the Right end, would be cool.
Watching the number count up, for us who know what they actually, the Bar doesn't have to say .xml, we could try and guess when the "Hats" were loading. LOL
Some Tips and Lore screen would be good as well. Blank screens are bleh!
+1 For this idea. Show me the cool stuff you are doing and I will wait for it.
Sammual
There are many great features available to you once you register, including:
Sign in or Create Account