It is difficult to explain the issues of Sins have without going into too much "technical detail" which some may not understand. The main problem is that Sins is an older generation 32 bit game. With a two gigabyte memory limit. This is nothing new. All 32 bit games have a 2 gig memory limit. However Sins is NOT like other 32 bit games. Most games have an ability to load their assets "on the fly", or only as needed. Some games can also unload assets that are not needed. This is how they prevent themselves from reaching the 2 gig limit. Some games have been patched to where they can use up to 4 gigs of ram on 64 bit systems.
Sins also can load only what is needed at game start up, but Sins does NOT unload what it does not use. Any assets that are loaded remain loaded throughout the entire game! If you use all three factions (Six in the case of Rebellion), Then Sins will load ALL of its game assets at start up. Entrenchment, and Diplomacy in some games loaded between 1.7 to 1.9 gigs worth of assets at game start up! Graphic settings, The size of the Map, and the amount of Players, or AI opponents also determine how much ram Sins will use in any given session.In 2006 when Sins was first conceived 32 bit Windows XP was the standard OS of the time. Single core CPU's were also still the standard. Though dual core CPU's were starting to become mainstream, and 2 gigs was considered to be "plenty" of memory. So most game developers at the time didn't have to worry about hitting that dreaded 2 gig brick wall. As time went on Computers, and Sins itself evolved into their present state which is now Rebellion, and soon to be Windows 10 running in multiple core 64 bit systems. With 16+ gigs of ram. Things have changed so much over the years. However Sins still remained the 32 bit game as it was designed in 2006. That uses only a single core of your CPU. With a two gigabyte hard coded memory limit. Sins WILL CRASH when the game reaches that 2 gig limit.There is some misunderstanding about the 2 gig crash issue. The dump happens when Sins reaches 2 gigs of Ram Usage. That means the ram that Sins uses by itself. NOT what your entire operating system, and/or other programs you are running combined uses. Only the ram that Sins uses. No matter how powerful your system is. If it is 32, or 64 bit, or if it is Windows XP, Vista, 7, 8, etc. etc. Sins WILL CRASH when it hits its 2 gig ram limit. It is hard coded into the game.Nobody was aware that this would be a major problem back when the game was first designed, because Sins was designed based on the hardware standards of 2006-2007. Before 64 bit OS's, Quad Core cpu's, and 4+ gig ram systems became the standard that they are now. There is nothing that can be done about it. Except to re-write Sins in 64 bit. Which is out of the question, because it is too cost prohibitive for the developers to do this (Ironclad said so themselves). Sins is a great game, and it is a shame that this is happening. With the Entrenchment, and Diplomacy expansions Ironclad had unintentionally pushed the game over its own hard coded limitations with all of the new added content.
Over the years as of the release of Diplomacy 1.3, and Rebellion. Stardock, and Ironclad did indeed fix all of the major game crashing issues of Sins. Using the fix's from This Project as a guide. However, NONE of these fix's ever made it into Entrenchment, or Original Sins! While the Stardock, and Ironclad developers have fixed most of these issues over years. They can not do anything to change the game itself short of reprogramming Sins over from scratch. The developers can do no more, but there is something that WE can do as modders to prevent Sins from hitting that dreaded 2 gig brick wall, and improve the performance of Sins "just a little bit more". I have taken the initiative, and applied to Sins what i have learned from my Homeworld Modding days. I present to you....
The goal of this project is to improve your Vanilla Sins gaming experience, and to eliminate the 2 gig crash issue. No changes will be made to Original Sins, or any of its Expansions in game play at all. No stat changes will be made. Nothing "new" will be added. There are other mods that can do that. Original Sins will remain Original Sins, and the Expansions will remain as they are. What we are doing is taking what game assets that Sins already has, and we are "optimizing" them to improve the performance, stability, and to moderate the ram usage in game. The side benefit will be that some lower end systems can enjoy almost the same gaming experience as the high end users do (Especially Laptop Users).The project initially started out as an idea to reduce the texture sizes, and poly counts of some models. Not just arbitrary reductions. The goal was to maintain the quality of Sins in game. To make it look as if nothing has been changed at all. We started with optimizations to the Planet's, and Skybox's, because they were the "worst offenders" as far as ram usage, and performance was concerned. Unfortunately my modeling skills left much to be desired in Softimage XSI at the time, and there were un-fixable mesh errors on the optimized planet meshes. Despite that the "proof of concept" was established. A lot of ram was saved, and there was a very noticeable increase in performance.Then we studied the Particle files, and Textures. Many of the particle textures in our opinion were way too high in resolution for the purpose that they served. A reduction in texture resolution by HALF showed no visible difference in game at all! There were some exceptions where the particle resolution reduction difference was very noticeable in game, and those textures were reverted back to their original resolutions. Remember the goal was NOT to reduce in game quality. Also all of the particle textures were changed from TGA format to DDS format, because DDS uses a little less memory, and is much more efficient, because DDS format loads directly into video memory. The particles can also utilize the mip mapping features of DDS format. Where TGA format does not. Therefore enhancing in game video performance (increased FPS). As of version 0.07 no TGA format textures load in TSOP. With the exceptions of the Mouse Cursors, and Scenario Pictures. This is due to those textures being controlled by the hard coded exe. When you change the texture format those textures do not show up in game.The developers mentioned something about a Memory Leak. I heard that a memory leak can be caused by a file searching for something over, and over again if it is not there. Example a mesh file calling for a texture that does not exist. Vanilla Sins had MANY particle, and mesh files that had that issue. We removed all of the entries that pointed to non existent textures in the mesh, and particle files. We also removed the entries that have lines like for example "C;\g\main\datasource\textures\effects\aura4.dds", and just used "aura4.dds". This will default the search to Sins, or the Mods "Textures" folder, and speed things up a bit. Also various typo's were corrected in some mesh, and particle files. Some ships had mesh nulls that were Miss-Labeled. For example the Akkan had nulls, and flairs that were miss-labeled. The engine trails, and flairs didnt show up in game. the Marza was missing its bomb nulls. etc. etc. All of this has been corrected as well. If there is a ships mesh file in the mods "Mesh" folder then it had something wrong with it that was fixed. The Soundata, and Galaxydef files had errors too. Regardless if any of these issues caused any memory leaks, or not these issues MUST be fixed! (and they were as of Diplomacy 1.3, but NOT for Original Sins, or Entrenchment).After the Meshes, Particles, Textures, and the User Interface were optimized, and the very obvious issues were fixed there was a tremendous boost in game performance. Sins ram usage was greatly reduced. As of TSOP 0.07 I could run a game of Entrenchment, or Diplomacy ALL DAY, and I NEVER used more than 1.5 gigs of ram! This was using Maximum Settings. Including Bloom enabled. Tested on Huge Random Multi Star maps with 9 AI's. The 2 gig crash problem has been solved! Our primary goal with this project has been achieved! Without having to use any external memory increasing programs! Stardock, and Ironclad took notice, and they used TSOP as a guide in the Diplomacy 1.3 patch, and for developing Rebellion.Now that the Stardock, and Ironclad have fixed the issues of the game as of Diplomacy 1.3, and Rebellion. TSOP has been focusing more on Original Sins, and Entrenchment. Since the Ironclad/Stardock fixes never made it into Original Sins, or Entrenchment. However, That is not stopping us from optimizing Diplomacy, or Rebellion "just a little bit more" You can also use Large Address Aware to give Sins an extra gig to play with, but in vanilla Original Sins, and Entrenchment the Endless Search Loop issues that were never fixed will cause the game to eventually go over that extra gig (Run vanilla Entrenchment, and watch your ram usage keep going up, and up, and up). TSOP is a solution that will stop that issue.
____________________________________________________________________________________________________________________
Downloads
The Sins Optimization Project v1.0
For Original Sins 1.195
For Entrenchment 1.055
For Diplomacy 1.37
Stardock, and Ironclad fixed all of the issues that compelled us to make TSOP. Therefore there is no TSOP for Diplomacy
For Rebellion 1.82
As with Diplomacy, Stardock, and Ironclad fixed all of the issues that TSOP pointed out, and then some in the case of Rebellion. Sins: Rebellion is as optimized as any Sins game is going to get.
NOTE: TSOP 0.07 is still hosted on ModDB, and it still works in the current versions of Trinity (OS, E, and D), but you will be playing as if Sins was patched to an older version of the game. Plus some Icons and Buttons wont be visible.
Changes in TSOP 1.0
TSOP 1.0 updated to the latest versions of Original Sins, and Entrenchment.
The User Interface is completely overhauled to use pure DDS format.
Trade Ships, Strike Craft, Mines, and Constructor Unit Meshes, and Textures are Optimized.
Starbase Textures are optimized mimicking the Stardock Rebellion Optimizations.
Sounds, and Music are optimized.
All TGA format Textures are replaced with DDS format, and TSOP "force" loads them. The only TGA textures that remain are the Mouse Cursers, and Scenario Pictures. This is due to hard code in the EXE.
"Strategic Texture Reductions" for the Particles return using newer methods to create a better looking texture.
(Original Sins, and Entrenchment) Many Mesh Null issues, and various File Errors were corrected, and Unnecessary Text Entry's were removed in both the Mesh, and Particle files. Entry's pointing to Non Existent Textures in both the mesh, and particle files removed. See the various "Fixed Files" texts in the mods Read Me for details.
(Original Sins, and Entrenchment) The Diplomacy 1.3 graphic fix's have been incorporated into the Original Sins, and Entrenchment versions of TSOP (Pipeline Effects, "White Line" error fix, and Corrected Mesh files).
TSOP 1.0 for Trinity will see the return of Optimized Planets using the planet meshes from Rebellion! but with our Optimized Original Sins Textures!
There will be no version of TSOP made for Diplomacy, or Rebellion, because there is no need for it. Only Original Sins, and Entrenchment need TSOP, because the 2 gig issue, and various other issues were never fixed in those two versions of the game.
This is the FINAL version of TSOP that i will make. If the community wants to take over the project they are more than welcome to.
Community input, and contributions are more than welcome, and they are encouraged! This is a Community Project! All can pitch in!
Note: TSOP does not fix the known issue of "Late Game Lag". It does help prolong the lag until further into the game in some cases, but sooner, or later the lag will happen. The late game lag issue is a CPU issue due to the fact that Sins (including Rebellion) is optimized to use only single core processors. Therefore Sins will use only one core in multi core CPU's. To help with the late game lag issue we recommend the following:
Playing on smaller maps.
Using fewer AI opponents.
Disabling the Trade Ship icons (this actually helps out a lot)
Stacking the icons in the empire tree (same as with disabling trade ship icons the less icons rendered the better).
Using a Strike Craft Reduction mod (The many strike craft in the game are a big CPU hog. There are some good strike craft mods that reduce the number, but maintain the balance. Search for them).
You do not need to download all versions of TSOP. You only need to download the version of TSOP for the specific version of Sins you wish to play. The Retail version of Trinity is just all three versions of Sins bundled into one package. Installation for Trinity versions of Sins are no different than if you purchased each version of Sins separately.
_____________________________________________________________________________________________________________________
Other mods are more than welcome to use TSOP. No questions asked! In fact we encourage it! Learn from it! Use it as guide, and an example! Merge it into your projects!
TSOP is intended to be a Stand Alone mod. Run with no other mods enabled. There are issues enabling TSOP with other mods. Enabling with other "Graphic Enhancement" mods is NOT recommended, because the graphic enhancement mod will defeat the entire purpose of TSOP. All of our optimized files will be overridden by that enhancement mods "enhanced" files. Increasing your ram usage instead of decreasing it. Some enhancement mods like Distant Stars are incorporating elements of TSOP into their future updates.
Read the included READ ME text's for installation instructions.
i tried the mod out and didn't notice any change in performance (still lags heavily on a single system large), i suppose its just the limitations of my machine though
Just an heads up of things planned for the 0.06 update.
Bad files listed in MyFist0's "bad files" topics will be fixed.
Work on fixing the planets, and skybox's will continue. Will try to make improved versions. If that fails then we will just remove them from the mod.
Searching for more errors, and "typos" in EVERY file that we can convert to text, and open with wordpad.
My main focus now is what IMO is the biggest waste of memory in the game, and that is the User Interface. The UI uses mostly huge TGA textures averaging about 5 megs a pop. Most of the buttons (Research, Build icons, etc) use 4 separate textures of that size to represent the button being pushed, or disabled. A simple conversion to DDS brings each texture down to 1 meg, or less without altering anything at all. You cant resize the UI textures, because it messes up the grid system. Also there are some DDS texture already in the UI, but they have Mip Maps..Why? You cant zoom the UI in, or out, and the effects settings have no effect on the UI at all. Removing the Mip Maps saves a little over 1 meg for each texture. The main problem i am running into is when i replace some TGA's with DDS the game still loads the TGA from the core files. Once i find a way to overcome this there will be a HUGE savings in ram usage. Even larger than when the particles were optimized. I expect original sins to not use over a gig of ram when i am done with this.
I will keep you posted on progress.
aaa11, We are still fighting the late game lag. If your machine is very old chances are there will only be a slight difference in performance. You still need over a gig of ram to run Original Sins with TSOP at max settings, and at least 1.5 gigs for Entrenchment, and Diplomacy. What are your system specs?
I am going to bring up the subject of running other mods with TSOP again, because it seems like everyone wants to run TSOP with Distant Stars, or Bailknights graphics mod, or some other enhancement mod (or all of the other mods here combined). Then they get upset, and wonder why when enabling multiple mods over TSOP it fails, or they say they see no improvement at all. Sins is one of the few games i have modded that allows you to enable more than one mod at a time. It was never my intention to have this mod run with other mods.
First off an "enhancement mod" is exactly that. It "enhances" something in the game. Usually by doubling the size of the textures, or by creating a better looking textures that are larger in size than the originals they replaced, or by adding more, and larger particle effects. Distant Stars adds multiple planets, and new particles. They are having trouble with the 2 gig issue as well as many of the other enhancement mods here. They recommend running at lower settings if you lag, or crash. With good reason, because their enhancements push Sins over the edge.
If you run DS, or Bailknights mod, or any other enhancement mod over TSOP that has Planets, Skybox's, Ship/Effect meshes, Textures, or Particles with the same file names as TSOP's, then that enhancement mods files will OVERRIDE the files of TSOP. Basically defeating the entire purpose of this project. TSOP will not fix the 2 gig problem, or improve the performance of someone elses enhancement mod. Our Planet, and Skybox meshes, and textures are different than other mods as well, and will show errors if another mod has meshes and/or textures of the same filenames. TSOP is meant to be stand alone, and run by itself with no other mods enabled.
This is why i encouraged other mods to use TSOP as a guide, and example for their future updates. Distant Stars is doing this. I wish all the other mods here would do this. So we wouldnt have the "Why doesnt this work with DS, or some other mod?" post here.
We are trying to set a standard for Sins modding.
I will work on fixing the rest of the mesh points for the models that are not optimized unless you did them. I will go through the structures and and such.
I would like someone experienced in the research grind system to look at the pirate research added in Dip 1.011. The dev should not crash while switching players with no mods enabled.
Also, if there is anyone experience in the SoundEffects.sounddata, I added 4 missing references to the top but I just copied other effects that were close to what I needed. Any testers and change suggestions are quite welcome. Have a look at the top 4.
I agree totally. As Soa2 and TSOP are now one mod.
This is going to take a while.
2ghz processor
ati radeon 3100 graphic card
4 gb ram
laptop
About the same as my laptops specs. Sins runs ok on mine, and slightly better with TSOP enabled. I cant max the settings because it will turn into a slide show even with the project enabled. So i run at medium settings on the laptop.
Myfisto if there is a mesh file in the mod then i already looked it over. and fixed it. However if there are other files that we missed something then point them out.
The only meshes we lowered the poly counts for are the strike craft, trade ships, planets, and skybox's
BTW it is decided that the savings that the planets and skybox's give is not very much, and not worth the current graphics error. So we are reverting it back to vanilla for 0.06. These were the very first things we optimized, and only done as a proof of concept. The way the vanilla planet meshes are made is unique, and very different than what i am used to. Cant figure out for the life of me why a spherically mapped mesh doesnt work well in game. Removing them wont cause any significant increase. You can delete the planet, and skybox mesh/textures in the mod now. It shouldnt hurt anything if you do.
We need to keep the planet meshes in to fix the texture references but not reduced mesh.You did miss these...
I will look over all of the reference files again, and fix/add as needed Can you PM me a complete list of files found with errors so far on your end? The bad files topics is becoming confusing, and i am not sure what is fixed now and what is not.
Yes i will do a compare on TSOP and mine and send you the differences.
if i enable sound mod over this one, it will override the reduced quality sounds right?also, do star meshes/textures count as planets?
furthermore, the game's graphic quality isn't visibly reduced by optimization right?
Yes new sounds will override TSOP if they have the same file names.
yes they do.
We tried to make any visible difference as little as possible. I personally dont notice any difference in quality with the exception of the planets and skybox's which are going to be removed next update.
Good to see you two are getting the coding and UI working out. Thumbs
Now I just have to get some time freed up to get the modeling done....
I'll get you those starbases ASAP.
SivCorp
Myself, and Myfist0 are combining efforts to make a compiled list of EVERY obvious bad, and/or questionable file in Sins. Along with what is wrong with that file. I am currently in work on the particles looking through EVERY one of them over again because i didnt document what was wrong with them when i optimized them the last time. Not making that same mistake again :/ The list of particles with errors, miss-labels, or questionable entrys is LONG, and i am only half done searching through them.
And we wonder why sins is broken :/
I also managed to get ALL of the Sins TGA formatted textures converted to DDS format. This is for another version of TSOP. Which i will call TSOP "Lite". Dont let the name fool you there will be nothing "lite" about it. This is for all the graphic nuts out there that want maximum resolution textures.
TSOP Lite will have all of the mesh, particle, and file optimizations, and fixes that we discovered so far, but the Textures themselves will remain "unmolested". All that was done was a simple conversion from TGA to DDS format. So that there is not a single TGA in game. The memory savings just in the UI textures alone is HUGE.
The problem is the game still loads some TGA's from the core files. Like the curser textures. Does anyone know which file controls the mouse cursers in game? because i cant find it.
Quick Question - I'm trying to download the trinity version of your fantastic mod and I keep getting 404 file not found errors from file share. Is it just me ?
I can confirm that I am also receiving error 404 not found using firefox 3.68
harpo
Ya FileFront is getting very unreliable. In the future we should have multiple mirrors. Here is the Trinity aDrive Mirror
I was going to mirror the rest but an update is coming
EDIT: FileFront is changing quite a bit. See A Tour of FileFront 2.0. I guess it will be buggy for a bit so I will get it all up on aDrive ASAP
We plan on having a website for the optimod. working out details right now.
Almost done redoing particles, and documenting errors. This is just for Original Sins, entrenchment and diplomacy is next, but the count of files is much smaller.
Textures all DDS now still having trouble with some UI TGA's loading from the core files.
The mouse cursors are controlled by the game EXEs. If you do a notepad++ search of the root directory it will find the game exes and the dev.exes so no editing them without a hack.
Figures :/ So much for that idea. I guess the cursers remain tga. Difference is 6kb for a TGA vs 2kb for a DDS. I know it seems like nothing but every little bit does help.
Finished documenting the particle errors
453 particles for original sins alone. 40 of them had errors. Several are not used by the game at all. One had an obvious typo. A few have a strange name for its effects though i dont think that causes an error in itself. Most of them call for a DDS texture where one doesnt exist in the core game files. It does NOT use the TGA texture of the same name. So those particles are missing a few textures in game, and it creates an endless search loop for a file that doesnt exist. The files mis-labeled are in the "smoke" series of textures. The particles call for a DDS, but they only exist as a TGA in the core game files.
Looking over, and correcting meshes again. This time checking ALL of them.
I am checking the meshes again myself and have a doc for each version coming shortly
EDIT: Here ya go
Save as and compare to see the differences.
A few models were replace with Ent. See the diferences from the change logs
I should do random gen maps with the "guass"cannons (mis-spelled on purpose on the info cards)
0.06 is being skipped because the construction ships add on IS the 0.06 update.
The upcoming 0.07 update removes the modded planets and skybox's from TSOP. The savings gained with the modded planets, and skyboxes was minimal at best. It is not worth the errors, and the un-fixable pole pinch problem. This is also for better mod compatibility. Though i still do not recommend that you run other mods with TSOP, BUT... i know you guys are going to run the other mods over TSOP anyway. Despite all the warnings not to do so, and then complain when the other mods files override TSOP's files, and 2 gig crash the game. So we will keep on seeing the relentless "why doesnt DS, or Bailknights mod work with TSOP?" posts.. :/
The UI has been completely overhauled. It will now use ALL DDS textures. With the exception of cursors, and map pictures. The cursors are controlled through the exe, and cant be changed. The map pictures are part of the UI itself, and are an odd size that is not a power of 4. If i re-sized them to a power of 4 then that would create an error in the UI grid (i tried it. It was ugly). The memory savings just by that alone is huge. Original Sins with TSOP used to level off at 1.2-1.3 gigs playing Systems of War with 4 AIs. Now it levels off at 1.2-1.3 gigs playing a Random Huge Multi star map 102 planets with 9 AI's!
All errors listed in MyFisto's "bad file" topics have been fixed.
I think its time for the rest of the mod community to adopt TSOP's standard. We have achieved almost every goal we set for ourselves so far. We have proven our point. TSOP WORKS. Study what we have done, and learn from it.
TSOP's goal has changed to become more than just an "Optimization" mod. Our goal now is to become the sins community bug fix patch. Along with all of the optimizations of TSOP. It is a BUG HUNT now lol. I said it before, and i will say it again. All members of the sins modding community are more than welcome to join us if they know ways to fix, and/or improve Sins. Iron Engine wise. Not Stat wise. There are other balance mods for that purpose. Plus there is another topic for balance issues, and possible fixes. Our focus is on the game engine itself. What makes Sins tick.
If you have not noticed on another topic, Blair Frasier has released the source code for the .XSI to .Mesh files. This will help the community (mainly the code monkeys) understand the mesh files better, and possibly make import/export plugins for other 3d modeling programs like 3dsmax.
I am about 90% done with the 0.07 update for Original Sins. Entrenchment, and Diplomacy shouldnt take very long at all. Figure by this weekend, or next worst case.
Agreed, we at the Distant Stars mod community are hard at work in implementing this mod
Just a quick note from a lurker to say a great big "Thank you" to Major Stress and the other modders who have worked--and are still working hard--on improving Sins of a Solar Empire. Just so you know that your efforts are all greatly appreciated. This mod, along with the Netbook resolution mod, have allowed me to play Sins on my Acer Aspire One (with 1 GB RAM!) virtually as smooth as on my desktop at home. So thanks again and keep up the good work!
I am now beginning to think the late game lag is a CPU issue (or a lack of a decent CPU issue). Not a graphics issue.
In my "Stress" tests of TSOP 0.07 for Original Sins (major pun intended), I find the game has awesome framerate.. Up until the point to where the AI's have maxed their research out, and can build almost everything. By this point in time there are already hundreds of units, and structures on the map. Along with the NPC craft associated with them. Heated battles rage between the AI's that didnt align themselves with each other.
My framerates drop to around 20ish near the end game, and continue dropping. By the time i finish Systems of War it is a slide show. It doesnt matter what i set my graphic settings to. When i set my graphics to all low settings i get the same FPS as i do on max settings. This is why i believe it is a CPU issue.
thoughts? any ideas on what we can do?
There are many great features available to you once you register, including:
Sign in or Create Account