It is difficult to explain the issues of Sins have without going into too much "technical detail" which some may not understand. The main problem is that Sins is an older generation 32 bit game. With a two gigabyte memory limit. This is nothing new. All 32 bit games have a 2 gig memory limit. However Sins is NOT like other 32 bit games. Most games have an ability to load their assets "on the fly", or only as needed. Some games can also unload assets that are not needed. This is how they prevent themselves from reaching the 2 gig limit. Some games have been patched to where they can use up to 4 gigs of ram on 64 bit systems.
Sins also can load only what is needed at game start up, but Sins does NOT unload what it does not use. Any assets that are loaded remain loaded throughout the entire game! If you use all three factions (Six in the case of Rebellion), Then Sins will load ALL of its game assets at start up. Entrenchment, and Diplomacy in some games loaded between 1.7 to 1.9 gigs worth of assets at game start up! Graphic settings, The size of the Map, and the amount of Players, or AI opponents also determine how much ram Sins will use in any given session.In 2006 when Sins was first conceived 32 bit Windows XP was the standard OS of the time. Single core CPU's were also still the standard. Though dual core CPU's were starting to become mainstream, and 2 gigs was considered to be "plenty" of memory. So most game developers at the time didn't have to worry about hitting that dreaded 2 gig brick wall. As time went on Computers, and Sins itself evolved into their present state which is now Rebellion, and soon to be Windows 10 running in multiple core 64 bit systems. With 16+ gigs of ram. Things have changed so much over the years. However Sins still remained the 32 bit game as it was designed in 2006. That uses only a single core of your CPU. With a two gigabyte hard coded memory limit. Sins WILL CRASH when the game reaches that 2 gig limit.There is some misunderstanding about the 2 gig crash issue. The dump happens when Sins reaches 2 gigs of Ram Usage. That means the ram that Sins uses by itself. NOT what your entire operating system, and/or other programs you are running combined uses. Only the ram that Sins uses. No matter how powerful your system is. If it is 32, or 64 bit, or if it is Windows XP, Vista, 7, 8, etc. etc. Sins WILL CRASH when it hits its 2 gig ram limit. It is hard coded into the game.Nobody was aware that this would be a major problem back when the game was first designed, because Sins was designed based on the hardware standards of 2006-2007. Before 64 bit OS's, Quad Core cpu's, and 4+ gig ram systems became the standard that they are now. There is nothing that can be done about it. Except to re-write Sins in 64 bit. Which is out of the question, because it is too cost prohibitive for the developers to do this (Ironclad said so themselves). Sins is a great game, and it is a shame that this is happening. With the Entrenchment, and Diplomacy expansions Ironclad had unintentionally pushed the game over its own hard coded limitations with all of the new added content.
Over the years as of the release of Diplomacy 1.3, and Rebellion. Stardock, and Ironclad did indeed fix all of the major game crashing issues of Sins. Using the fix's from This Project as a guide. However, NONE of these fix's ever made it into Entrenchment, or Original Sins! While the Stardock, and Ironclad developers have fixed most of these issues over years. They can not do anything to change the game itself short of reprogramming Sins over from scratch. The developers can do no more, but there is something that WE can do as modders to prevent Sins from hitting that dreaded 2 gig brick wall, and improve the performance of Sins "just a little bit more". I have taken the initiative, and applied to Sins what i have learned from my Homeworld Modding days. I present to you....
The goal of this project is to improve your Vanilla Sins gaming experience, and to eliminate the 2 gig crash issue. No changes will be made to Original Sins, or any of its Expansions in game play at all. No stat changes will be made. Nothing "new" will be added. There are other mods that can do that. Original Sins will remain Original Sins, and the Expansions will remain as they are. What we are doing is taking what game assets that Sins already has, and we are "optimizing" them to improve the performance, stability, and to moderate the ram usage in game. The side benefit will be that some lower end systems can enjoy almost the same gaming experience as the high end users do (Especially Laptop Users).The project initially started out as an idea to reduce the texture sizes, and poly counts of some models. Not just arbitrary reductions. The goal was to maintain the quality of Sins in game. To make it look as if nothing has been changed at all. We started with optimizations to the Planet's, and Skybox's, because they were the "worst offenders" as far as ram usage, and performance was concerned. Unfortunately my modeling skills left much to be desired in Softimage XSI at the time, and there were un-fixable mesh errors on the optimized planet meshes. Despite that the "proof of concept" was established. A lot of ram was saved, and there was a very noticeable increase in performance.Then we studied the Particle files, and Textures. Many of the particle textures in our opinion were way too high in resolution for the purpose that they served. A reduction in texture resolution by HALF showed no visible difference in game at all! There were some exceptions where the particle resolution reduction difference was very noticeable in game, and those textures were reverted back to their original resolutions. Remember the goal was NOT to reduce in game quality. Also all of the particle textures were changed from TGA format to DDS format, because DDS uses a little less memory, and is much more efficient, because DDS format loads directly into video memory. The particles can also utilize the mip mapping features of DDS format. Where TGA format does not. Therefore enhancing in game video performance (increased FPS). As of version 0.07 no TGA format textures load in TSOP. With the exceptions of the Mouse Cursors, and Scenario Pictures. This is due to those textures being controlled by the hard coded exe. When you change the texture format those textures do not show up in game.The developers mentioned something about a Memory Leak. I heard that a memory leak can be caused by a file searching for something over, and over again if it is not there. Example a mesh file calling for a texture that does not exist. Vanilla Sins had MANY particle, and mesh files that had that issue. We removed all of the entries that pointed to non existent textures in the mesh, and particle files. We also removed the entries that have lines like for example "C;\g\main\datasource\textures\effects\aura4.dds", and just used "aura4.dds". This will default the search to Sins, or the Mods "Textures" folder, and speed things up a bit. Also various typo's were corrected in some mesh, and particle files. Some ships had mesh nulls that were Miss-Labeled. For example the Akkan had nulls, and flairs that were miss-labeled. The engine trails, and flairs didnt show up in game. the Marza was missing its bomb nulls. etc. etc. All of this has been corrected as well. If there is a ships mesh file in the mods "Mesh" folder then it had something wrong with it that was fixed. The Soundata, and Galaxydef files had errors too. Regardless if any of these issues caused any memory leaks, or not these issues MUST be fixed! (and they were as of Diplomacy 1.3, but NOT for Original Sins, or Entrenchment).After the Meshes, Particles, Textures, and the User Interface were optimized, and the very obvious issues were fixed there was a tremendous boost in game performance. Sins ram usage was greatly reduced. As of TSOP 0.07 I could run a game of Entrenchment, or Diplomacy ALL DAY, and I NEVER used more than 1.5 gigs of ram! This was using Maximum Settings. Including Bloom enabled. Tested on Huge Random Multi Star maps with 9 AI's. The 2 gig crash problem has been solved! Our primary goal with this project has been achieved! Without having to use any external memory increasing programs! Stardock, and Ironclad took notice, and they used TSOP as a guide in the Diplomacy 1.3 patch, and for developing Rebellion.Now that the Stardock, and Ironclad have fixed the issues of the game as of Diplomacy 1.3, and Rebellion. TSOP has been focusing more on Original Sins, and Entrenchment. Since the Ironclad/Stardock fixes never made it into Original Sins, or Entrenchment. However, That is not stopping us from optimizing Diplomacy, or Rebellion "just a little bit more" You can also use Large Address Aware to give Sins an extra gig to play with, but in vanilla Original Sins, and Entrenchment the Endless Search Loop issues that were never fixed will cause the game to eventually go over that extra gig (Run vanilla Entrenchment, and watch your ram usage keep going up, and up, and up). TSOP is a solution that will stop that issue.
____________________________________________________________________________________________________________________
Downloads
The Sins Optimization Project v1.0
For Original Sins 1.195
For Entrenchment 1.055
For Diplomacy 1.37
Stardock, and Ironclad fixed all of the issues that compelled us to make TSOP. Therefore there is no TSOP for Diplomacy
For Rebellion 1.82
As with Diplomacy, Stardock, and Ironclad fixed all of the issues that TSOP pointed out, and then some in the case of Rebellion. Sins: Rebellion is as optimized as any Sins game is going to get.
NOTE: TSOP 0.07 is still hosted on ModDB, and it still works in the current versions of Trinity (OS, E, and D), but you will be playing as if Sins was patched to an older version of the game. Plus some Icons and Buttons wont be visible.
Changes in TSOP 1.0
TSOP 1.0 updated to the latest versions of Original Sins, and Entrenchment.
The User Interface is completely overhauled to use pure DDS format.
Trade Ships, Strike Craft, Mines, and Constructor Unit Meshes, and Textures are Optimized.
Starbase Textures are optimized mimicking the Stardock Rebellion Optimizations.
Sounds, and Music are optimized.
All TGA format Textures are replaced with DDS format, and TSOP "force" loads them. The only TGA textures that remain are the Mouse Cursers, and Scenario Pictures. This is due to hard code in the EXE.
"Strategic Texture Reductions" for the Particles return using newer methods to create a better looking texture.
(Original Sins, and Entrenchment) Many Mesh Null issues, and various File Errors were corrected, and Unnecessary Text Entry's were removed in both the Mesh, and Particle files. Entry's pointing to Non Existent Textures in both the mesh, and particle files removed. See the various "Fixed Files" texts in the mods Read Me for details.
(Original Sins, and Entrenchment) The Diplomacy 1.3 graphic fix's have been incorporated into the Original Sins, and Entrenchment versions of TSOP (Pipeline Effects, "White Line" error fix, and Corrected Mesh files).
TSOP 1.0 for Trinity will see the return of Optimized Planets using the planet meshes from Rebellion! but with our Optimized Original Sins Textures!
There will be no version of TSOP made for Diplomacy, or Rebellion, because there is no need for it. Only Original Sins, and Entrenchment need TSOP, because the 2 gig issue, and various other issues were never fixed in those two versions of the game.
This is the FINAL version of TSOP that i will make. If the community wants to take over the project they are more than welcome to.
Community input, and contributions are more than welcome, and they are encouraged! This is a Community Project! All can pitch in!
Note: TSOP does not fix the known issue of "Late Game Lag". It does help prolong the lag until further into the game in some cases, but sooner, or later the lag will happen. The late game lag issue is a CPU issue due to the fact that Sins (including Rebellion) is optimized to use only single core processors. Therefore Sins will use only one core in multi core CPU's. To help with the late game lag issue we recommend the following:
Playing on smaller maps.
Using fewer AI opponents.
Disabling the Trade Ship icons (this actually helps out a lot)
Stacking the icons in the empire tree (same as with disabling trade ship icons the less icons rendered the better).
Using a Strike Craft Reduction mod (The many strike craft in the game are a big CPU hog. There are some good strike craft mods that reduce the number, but maintain the balance. Search for them).
You do not need to download all versions of TSOP. You only need to download the version of TSOP for the specific version of Sins you wish to play. The Retail version of Trinity is just all three versions of Sins bundled into one package. Installation for Trinity versions of Sins are no different than if you purchased each version of Sins separately.
_____________________________________________________________________________________________________________________
Other mods are more than welcome to use TSOP. No questions asked! In fact we encourage it! Learn from it! Use it as guide, and an example! Merge it into your projects!
TSOP is intended to be a Stand Alone mod. Run with no other mods enabled. There are issues enabling TSOP with other mods. Enabling with other "Graphic Enhancement" mods is NOT recommended, because the graphic enhancement mod will defeat the entire purpose of TSOP. All of our optimized files will be overridden by that enhancement mods "enhanced" files. Increasing your ram usage instead of decreasing it. Some enhancement mods like Distant Stars are incorporating elements of TSOP into their future updates.
Read the included READ ME text's for installation instructions.
0.07 should still work with Entrenchment, and Original Sins, because they were not touched with the diplomacy 1.2 update. The only things i changed with TSOP Entrenchment, and Original Sins for 0.08 were the reversion of textures back to full resolution, and added the pipeline fix from diplomacy 1.2 Ill see if i can get 0.08 back up.
How would i intergrate this into a mod.
File by file, i would imagine? I wouldn't want to replace the mesh or textures folders completely.
Im using Zombies races mod for Entrenchment V 1.051 as the base.
At the moment im having no problems with my mod CTD wise and ive got the best part of 6 races in. I've only had 1 CTD in 2 months of modding and that was when i tried to reduce the size of the Shadow battlecrab, so the mothership could have little baby battlecrabs. It didnt work, so i gave up.
The only thing ive noticed, it takes a while to load the game from a previous save.
Would this make a difference because at the moment the Mod is still in TXT format.
Theo
What I would try is to 1st make backups. Then use a backup of your mod and paste the TSOP over top. When you come across duplicate files, DO NOT overwrite, choose to make another copy. This will give you both, the original from your mod and TSOPs copy with a [1] at the end. Then use Notepad++ compare feature (under plugins) to see what differences there are between the 2 copies and determine which to use.
BIN does load faster but I would not convert until the mod is completed for release.
If your main ship is a Capital Ship or a carrier you should be able to have baby crab fighters or look at the TEC carriers spawn missile launcher ability which is actually a frigate. I'm sure if you made a new thread asking how you will get many options from the community.
Good luck with your mod.
And where have you been young man?
Edit-
That seems like a lot of hardwork, would you leave the optimization until the end?
I had to return to my home planet for a while. I am also as old as 1 of your Redwood trees.
The more files you modify before optimizing, the more duplicate files you willl have to compare.
I actually started my mod using tsop as a base and added the 3 races manually and optimized a lot more files as I progressed.
I got a good question to ask...
Now that Diplomacy 1.21 is upon us has anyone had any major graphic lag (not cpu lag from too many units, but graphic only lag) since it was released. I would like to hear from the peeps that have machines that are the minimum requirements.
The answer will determine where we go with TSOP.
I tried to implement TSOP as a patch, and failed. It mainly has to do with the particles, maily the vasari missiles (as in once i patch the missile particles the meshes disappear). I do not know why this is, because the particle files are worded identically. No errors or typos. There are no errors in the mesh, and it works fine when its run in mod format. So patching = failure.
Rebellion is coming, and the iron engine itself will pretty much be the same. However with the diplomacy 1.2 updates we really do not need TSOP anymore, because the patch fixed the 2 gig issue, and a litany of other thing graphic wise (fsaa issue, white lines on meshes issue etc.).
What if anything should TSOP focus on if the game is basically fixed?
While Dip alone might not require TSOP, the modders sure can use it as a base when adding races and ships. I did manage to reduce the planet memory quite a bit so we can add that and a few other things that use up about 50mb of memory. Mostly from unused loaded textures and wasted texture space, some sheets have over 50% empty space.
This will make adding races and/or planets much more stable late games. Don't forget about the huge multi-star maps that people like me prefer as well with the extra races. I still want to squeeze every drop out of this engine I can.
Also another add-on that changes the mines to a very simple model.
Lol, I think most of us on here are mature trees. Well the ones who make any sense anyway!
I really needed this information at the time of starting, so it looks like i've got my work cut out.
The problem for me is that i wouldn't know which file would take preference.
If i'm serious about releasing my mod to the public, do you think it should be for Diplomacy 1.21.
In your honest opinions, do you think i could get this released, i've got a lot of begging to do, so little time. Do you think i'm best just doing it for personnel use. You know, if this is to be done correctly, i need a small team like the other Mods, like SOA2, SOGE and SGI etc, the very least a code-person and there ain't no volunteers.
As the mod stands now, i'm really enjoying the whole process.
IMO, the game plays really well. Who wouldn't want to see a Shadow Battle Crab go against a Wraith Hive ship or the Daedalus against the Defiant? Or the Galactica against a Vorlon Heavy Cruiser........ or 3 Star Trek Capitals being creamed by a Super Star Destroyer.......................................................etc,etc,etc
Sorry that was blatant advertising in this thread. I do apologise.
I actually achieved the answer to the question i set myself.
Yes it is possible.
Can i have some more Karma now?
Can this mod be used in conjunction(stacked) with other mods presently available for dip 1.19 & entrenchment 1.05?
Sorry
Sins 1.19
Entrenchment 1.05
Dip 1.01
within the trinity bundle?
I apologaize, i missed the paragraph above in the introduction where this mod is jumpy with other mods.
I have my answer.
TSOP not being stackable with other mods is the main reason i tried to implement it as a patch. So there wouldn't be any mod issues. Unfortunately it met with failure.
Myfisto is correct. TSOP was designed to be incorporated into others mods, or used as a foundation to build your mods. It is to make you aware that even with the diplomacy 1.2 (and soon 1.3) updates that sins still has limits. Using TSOP as a guide you can give yourself much more room to play with in your mod, and not worry about hitting the 2 gig brick wall. Most of the mesh, and particle fix's from TSOP have been incorporated into the diplomacy 1.2 patch.
For those without diplomacy TSOP is still useful, because the 1.2 updates have no effect on Entrenchment or Original Sins.
Does the squadron automation work with the diplomacy 1.21. I really like it, but i get a mini dump when i try to use it,
No it does not work. I will try to take it off the page today.
Says the vanilla sins file is no longer on the site, why?
Is this mod currently being revised and updated? If it is not I will likely be 'officially' discontinuing my Multiple Stars Graphics Mod support for this mod as the primary user base utilizes the Diplomacy / Trinity 1.2 patch.
If desired however I can continue to take a look and get a stable version of the MSGM mod together for TSOP. Please let me know. Thanks.
To all TSOP people who have helped contribute to this AMAZING Mod, please do not give up hope. The work and progress that you have made has been terrific, and has astounded me personally. Even the Devs have admitted the work you have done is fantastic and if I am not mistaken got them to implement or look at some of your work in the patches they released.
Please, Please continue to optimise sins whilst keeping the same quality, I will always use this mod when creating any of my mods and I very much encourage all the modding community to do the same.
Its a fantastic piece of work and a very big thank you from me!
For Diplomacy there is pretty much nothing more we can do with it. Ironclad fixed the 2 gig issue, and a bunch of other graphics issues with the 1.2 patch. IC utilized elements of TSOP to help fix it. Plus IC recoded stuff on their own end to accomplish this. The only thing IC didnt change with the 1.2 patch are the textures themselves. TSOP uses all DDS textures, but IC is insistent on using TGA format for some stuff. WHY? I dont know.
For Entrenchment, and Original Sins, both the 0.07, and 0.08 versions are still current, because IC never updated Entrenchment, or Original Sins since 1.191, and 1.051. The only difference between TSOP E, and OS 0.07, and 0.08 are that the textures in are full resolution in TSOP E/OS 0.08.
The only thing we can do with Diplomacy 1.31 is convert the textures to DDS. IC pretty much included everything else from TSOP with the patches.
Ill try to fix the downloads later.
Speaking of the TSOP team where did the rest of you guys go? Myfisto? SivCorp?
It has been very busy time for me. The worse the economy gets the cheaper people sell their souls for. Mephistopheles
I am still here. I've had a full reinstall and some core parts replaced on my computer recently, and haven't done much with sins.
I had been doing some work for the DS team with the flagship mod. But thats been about it so far.
But I'm getting back into it. What should we be doing from this point out? Are we just optimizing meshes and textures?
Thanks for the response Major Stress, even if its an optomisation of particles, meshes and textures its still worth a down load, and memory that is saved no matter how small is worth it for any Mod that wants to add a load of new content.
Please keep it up, it is an awsome project I am thrilled to hear that the Devs included many of your fixes!!! If that doesn't prove this project was/is still worthwile I don't know what would.
A loyal fan to TSOP, Ben.
TSOP will still be useful for Diplomacy in the fact that Diplomacy still shares many files with Entrenchment, and Original Sins. Files that did NOT get updated with the 1.2 thru 1.31 patch.
I attempted to make TSOP a full blown patch (replacing core game files with TSOP files), but I failed. There must be something i missed, or screwed up, and its in the Vasari Phase Missiles effect. The phase missile mesh doesnt show in game when i replace the core files with TSOP files. However the trails, and particle effects for the phase missile do. I checked the mesh file, the particle files, and all other files that are connected, and i cant find anything wrong. There is no reason why the mesh shouldnt show as far as i can see. Regardless i deemed the TSOP patch a failure solely because of that error.
Conversion of the textures to DDS format. Plus the files that call for those textures to "force load" dds can still be done. Should be a simple matter of checking over the 1.31 files to see what has changed.
I am not sure if Rebellion will use the same "mod" format for the expansion, or if it will be stand alone. Ether way i hope IC makes the switch to DDS format. It will save a lot of headache.
An area of optimization that is untapped is the Ability/Buff timings especially with passives.
By example, Psychoak found an interesting issue with the Tech planetary shield passive that uses PrioritizeNewBuffs in the BuffPlanetShieldOnSpawner buff. Turn on entity creation counts and watch this spawner leap off the charts...
This results in the buff being reset every invocation when it should be PrioritizeOldBuffs. Further, this buff has a finish condition checking if the research prerequisite for the tech shield generator which doesn't make much sense from a gameplay perspective or performance. Basically, why should losing the ability to build shield generators stop your currently built ones from working.
My recommendation would be to change this to PrioritizeOldBuffs and remove the research pre-requisite in both buffs.
My last comment is on timing. When this buff is fixed it is currently set to enter the game event queue every 1 second. This periodic action is really tight for something that only affects 1 object in the gravity well. I think in certain cases the timing of certain periodic actions could be increased to reduce the number of actions the game needs to process. I've bumped this up to 10 seconds in my own mod, but I could see it being higher.
Looking at the timing of other periodic actions might prove fruitful as well.
Yeah, I've forgotten to change that with quite a few abilities I created and it does drain a lot of system resources. Fortunately planetary shields aren't used that much but I'm surprised that a bug that obvious has survived this long.
Anyways, I have a question for you optimization experts. I had assumed that Sins loaded all textures in their entirety on start up, but someone I talked to had the idea that perhaps it only loads the areas in the pixel boxes as defined by the .brushes files. This would make some files like the buttons_research2 file that looks like an unbelievable waste of space make more sense, and might explain what the onlyLoadOnDemand line in the brushes file does. So I was wondering if anyone had done any testing on the issue that could explain what exactly the engine does?
onlyloadondemand True only works for the wallpaper type images like when the game is loading. All brushes in-game do load the whole image into the RAM. You can see this with the dev.exe. I tested with 1 very small image on a large texture sheet and the dev listed it as one of the worst offenders at 16mb,
I resized the research2's from 2048x2048 to 512x512 which worked fine and made them 1mb each instead of 4mb
1 big chunk could be the planets. I started the stock planets on Zombies thread and made the texture a hell of a lot smaller.
Daily Planet
I have no idea why IC uses a huge texture sheet for the dark side with no lights. FAIL! Also using optimized mesh.
Ready for download soon. Updated the 1st 2 planets with GF icons and fixed the shield links for dwarf and giant.
There are many great features available to you once you register, including:
Sign in or Create Account