It is difficult to explain the issues of Sins have without going into too much "technical detail" which some may not understand. The main problem is that Sins is an older generation 32 bit game. With a two gigabyte memory limit. This is nothing new. All 32 bit games have a 2 gig memory limit. However Sins is NOT like other 32 bit games. Most games have an ability to load their assets "on the fly", or only as needed. Some games can also unload assets that are not needed. This is how they prevent themselves from reaching the 2 gig limit. Some games have been patched to where they can use up to 4 gigs of ram on 64 bit systems.
Sins also can load only what is needed at game start up, but Sins does NOT unload what it does not use. Any assets that are loaded remain loaded throughout the entire game! If you use all three factions (Six in the case of Rebellion), Then Sins will load ALL of its game assets at start up. Entrenchment, and Diplomacy in some games loaded between 1.7 to 1.9 gigs worth of assets at game start up! Graphic settings, The size of the Map, and the amount of Players, or AI opponents also determine how much ram Sins will use in any given session.In 2006 when Sins was first conceived 32 bit Windows XP was the standard OS of the time. Single core CPU's were also still the standard. Though dual core CPU's were starting to become mainstream, and 2 gigs was considered to be "plenty" of memory. So most game developers at the time didn't have to worry about hitting that dreaded 2 gig brick wall. As time went on Computers, and Sins itself evolved into their present state which is now Rebellion, and soon to be Windows 10 running in multiple core 64 bit systems. With 16+ gigs of ram. Things have changed so much over the years. However Sins still remained the 32 bit game as it was designed in 2006. That uses only a single core of your CPU. With a two gigabyte hard coded memory limit. Sins WILL CRASH when the game reaches that 2 gig limit.There is some misunderstanding about the 2 gig crash issue. The dump happens when Sins reaches 2 gigs of Ram Usage. That means the ram that Sins uses by itself. NOT what your entire operating system, and/or other programs you are running combined uses. Only the ram that Sins uses. No matter how powerful your system is. If it is 32, or 64 bit, or if it is Windows XP, Vista, 7, 8, etc. etc. Sins WILL CRASH when it hits its 2 gig ram limit. It is hard coded into the game.Nobody was aware that this would be a major problem back when the game was first designed, because Sins was designed based on the hardware standards of 2006-2007. Before 64 bit OS's, Quad Core cpu's, and 4+ gig ram systems became the standard that they are now. There is nothing that can be done about it. Except to re-write Sins in 64 bit. Which is out of the question, because it is too cost prohibitive for the developers to do this (Ironclad said so themselves). Sins is a great game, and it is a shame that this is happening. With the Entrenchment, and Diplomacy expansions Ironclad had unintentionally pushed the game over its own hard coded limitations with all of the new added content.
Over the years as of the release of Diplomacy 1.3, and Rebellion. Stardock, and Ironclad did indeed fix all of the major game crashing issues of Sins. Using the fix's from This Project as a guide. However, NONE of these fix's ever made it into Entrenchment, or Original Sins! While the Stardock, and Ironclad developers have fixed most of these issues over years. They can not do anything to change the game itself short of reprogramming Sins over from scratch. The developers can do no more, but there is something that WE can do as modders to prevent Sins from hitting that dreaded 2 gig brick wall, and improve the performance of Sins "just a little bit more". I have taken the initiative, and applied to Sins what i have learned from my Homeworld Modding days. I present to you....
The goal of this project is to improve your Vanilla Sins gaming experience, and to eliminate the 2 gig crash issue. No changes will be made to Original Sins, or any of its Expansions in game play at all. No stat changes will be made. Nothing "new" will be added. There are other mods that can do that. Original Sins will remain Original Sins, and the Expansions will remain as they are. What we are doing is taking what game assets that Sins already has, and we are "optimizing" them to improve the performance, stability, and to moderate the ram usage in game. The side benefit will be that some lower end systems can enjoy almost the same gaming experience as the high end users do (Especially Laptop Users).The project initially started out as an idea to reduce the texture sizes, and poly counts of some models. Not just arbitrary reductions. The goal was to maintain the quality of Sins in game. To make it look as if nothing has been changed at all. We started with optimizations to the Planet's, and Skybox's, because they were the "worst offenders" as far as ram usage, and performance was concerned. Unfortunately my modeling skills left much to be desired in Softimage XSI at the time, and there were un-fixable mesh errors on the optimized planet meshes. Despite that the "proof of concept" was established. A lot of ram was saved, and there was a very noticeable increase in performance.Then we studied the Particle files, and Textures. Many of the particle textures in our opinion were way too high in resolution for the purpose that they served. A reduction in texture resolution by HALF showed no visible difference in game at all! There were some exceptions where the particle resolution reduction difference was very noticeable in game, and those textures were reverted back to their original resolutions. Remember the goal was NOT to reduce in game quality. Also all of the particle textures were changed from TGA format to DDS format, because DDS uses a little less memory, and is much more efficient, because DDS format loads directly into video memory. The particles can also utilize the mip mapping features of DDS format. Where TGA format does not. Therefore enhancing in game video performance (increased FPS). As of version 0.07 no TGA format textures load in TSOP. With the exceptions of the Mouse Cursors, and Scenario Pictures. This is due to those textures being controlled by the hard coded exe. When you change the texture format those textures do not show up in game.The developers mentioned something about a Memory Leak. I heard that a memory leak can be caused by a file searching for something over, and over again if it is not there. Example a mesh file calling for a texture that does not exist. Vanilla Sins had MANY particle, and mesh files that had that issue. We removed all of the entries that pointed to non existent textures in the mesh, and particle files. We also removed the entries that have lines like for example "C;\g\main\datasource\textures\effects\aura4.dds", and just used "aura4.dds". This will default the search to Sins, or the Mods "Textures" folder, and speed things up a bit. Also various typo's were corrected in some mesh, and particle files. Some ships had mesh nulls that were Miss-Labeled. For example the Akkan had nulls, and flairs that were miss-labeled. The engine trails, and flairs didnt show up in game. the Marza was missing its bomb nulls. etc. etc. All of this has been corrected as well. If there is a ships mesh file in the mods "Mesh" folder then it had something wrong with it that was fixed. The Soundata, and Galaxydef files had errors too. Regardless if any of these issues caused any memory leaks, or not these issues MUST be fixed! (and they were as of Diplomacy 1.3, but NOT for Original Sins, or Entrenchment).After the Meshes, Particles, Textures, and the User Interface were optimized, and the very obvious issues were fixed there was a tremendous boost in game performance. Sins ram usage was greatly reduced. As of TSOP 0.07 I could run a game of Entrenchment, or Diplomacy ALL DAY, and I NEVER used more than 1.5 gigs of ram! This was using Maximum Settings. Including Bloom enabled. Tested on Huge Random Multi Star maps with 9 AI's. The 2 gig crash problem has been solved! Our primary goal with this project has been achieved! Without having to use any external memory increasing programs! Stardock, and Ironclad took notice, and they used TSOP as a guide in the Diplomacy 1.3 patch, and for developing Rebellion.Now that the Stardock, and Ironclad have fixed the issues of the game as of Diplomacy 1.3, and Rebellion. TSOP has been focusing more on Original Sins, and Entrenchment. Since the Ironclad/Stardock fixes never made it into Original Sins, or Entrenchment. However, That is not stopping us from optimizing Diplomacy, or Rebellion "just a little bit more" You can also use Large Address Aware to give Sins an extra gig to play with, but in vanilla Original Sins, and Entrenchment the Endless Search Loop issues that were never fixed will cause the game to eventually go over that extra gig (Run vanilla Entrenchment, and watch your ram usage keep going up, and up, and up). TSOP is a solution that will stop that issue.
____________________________________________________________________________________________________________________
Downloads
The Sins Optimization Project v1.0
For Original Sins 1.195
For Entrenchment 1.055
For Diplomacy 1.37
Stardock, and Ironclad fixed all of the issues that compelled us to make TSOP. Therefore there is no TSOP for Diplomacy
For Rebellion 1.82
As with Diplomacy, Stardock, and Ironclad fixed all of the issues that TSOP pointed out, and then some in the case of Rebellion. Sins: Rebellion is as optimized as any Sins game is going to get.
NOTE: TSOP 0.07 is still hosted on ModDB, and it still works in the current versions of Trinity (OS, E, and D), but you will be playing as if Sins was patched to an older version of the game. Plus some Icons and Buttons wont be visible.
Changes in TSOP 1.0
TSOP 1.0 updated to the latest versions of Original Sins, and Entrenchment.
The User Interface is completely overhauled to use pure DDS format.
Trade Ships, Strike Craft, Mines, and Constructor Unit Meshes, and Textures are Optimized.
Starbase Textures are optimized mimicking the Stardock Rebellion Optimizations.
Sounds, and Music are optimized.
All TGA format Textures are replaced with DDS format, and TSOP "force" loads them. The only TGA textures that remain are the Mouse Cursers, and Scenario Pictures. This is due to hard code in the EXE.
"Strategic Texture Reductions" for the Particles return using newer methods to create a better looking texture.
(Original Sins, and Entrenchment) Many Mesh Null issues, and various File Errors were corrected, and Unnecessary Text Entry's were removed in both the Mesh, and Particle files. Entry's pointing to Non Existent Textures in both the mesh, and particle files removed. See the various "Fixed Files" texts in the mods Read Me for details.
(Original Sins, and Entrenchment) The Diplomacy 1.3 graphic fix's have been incorporated into the Original Sins, and Entrenchment versions of TSOP (Pipeline Effects, "White Line" error fix, and Corrected Mesh files).
TSOP 1.0 for Trinity will see the return of Optimized Planets using the planet meshes from Rebellion! but with our Optimized Original Sins Textures!
There will be no version of TSOP made for Diplomacy, or Rebellion, because there is no need for it. Only Original Sins, and Entrenchment need TSOP, because the 2 gig issue, and various other issues were never fixed in those two versions of the game.
This is the FINAL version of TSOP that i will make. If the community wants to take over the project they are more than welcome to.
Community input, and contributions are more than welcome, and they are encouraged! This is a Community Project! All can pitch in!
Note: TSOP does not fix the known issue of "Late Game Lag". It does help prolong the lag until further into the game in some cases, but sooner, or later the lag will happen. The late game lag issue is a CPU issue due to the fact that Sins (including Rebellion) is optimized to use only single core processors. Therefore Sins will use only one core in multi core CPU's. To help with the late game lag issue we recommend the following:
Playing on smaller maps.
Using fewer AI opponents.
Disabling the Trade Ship icons (this actually helps out a lot)
Stacking the icons in the empire tree (same as with disabling trade ship icons the less icons rendered the better).
Using a Strike Craft Reduction mod (The many strike craft in the game are a big CPU hog. There are some good strike craft mods that reduce the number, but maintain the balance. Search for them).
You do not need to download all versions of TSOP. You only need to download the version of TSOP for the specific version of Sins you wish to play. The Retail version of Trinity is just all three versions of Sins bundled into one package. Installation for Trinity versions of Sins are no different than if you purchased each version of Sins separately.
_____________________________________________________________________________________________________________________
Other mods are more than welcome to use TSOP. No questions asked! In fact we encourage it! Learn from it! Use it as guide, and an example! Merge it into your projects!
TSOP is intended to be a Stand Alone mod. Run with no other mods enabled. There are issues enabling TSOP with other mods. Enabling with other "Graphic Enhancement" mods is NOT recommended, because the graphic enhancement mod will defeat the entire purpose of TSOP. All of our optimized files will be overridden by that enhancement mods "enhanced" files. Increasing your ram usage instead of decreasing it. Some enhancement mods like Distant Stars are incorporating elements of TSOP into their future updates.
Read the included READ ME text's for installation instructions.
Thanks
Update:
The late game lag issue seems to have eased up a little on my "big box". Being that it was a brand new machine windows 7 was still doing its "windows experience" nonsense, and bogging down the machine. It was not just sins that had a lag issue. Plus it has a raid harddrive so i think that may have something to do with it too. Sins is smoking as it should now on that box. Still get late game lag but not near as bad as before.
Working on the entrenchment version of 0.07. All of the bad file fixes are in place, and just a few entrenchment specific texture changes needed, but i have to go over all of the brush files, and make sure they all point to a dds texture... tedious.
Have not had time to test with no shield meshes yet. If it does improve the situation, and we do go that route we will make it an optional add on, because it does effect the visuals of the game.
The Entrenchment, and Diplomacy versions MUST include ALL of the game info in the mod folder to run correctly. Just like it does for Original Sins. Otherwise sins will just pull files from the core game defeating the purpose of the project. So the Entrenchment, and Diplomacy versions will be the same size download as Original Sins. The goal is to get the entire game to run from the mod folders. At least until we can make this a patch. Then you wont need the mod folders to run TSOP.
Working crazy hours again so no eta on release.
looking good guys.... Still overly busy here, sorry for not getting any complete models to you yet Stress. I am working on it when I get a free moment.
TSOP: Entrenchment 0.07 finished. Did a 6 hour "stress" test on systems of war with 3 unfair AI's. Holds at 1.5 gigs (better than previous builds 1.7 gigs). On my intel i5 box there is late game lag, but not near as bad as before the UI overhaul, and bug fixes. Game is very playable at maximum settings (including bloom enabled) until the very end.
On to the Diplomacy build.
harpo
To the TSOP team.. If you got any optimized meshes ready for import, Now would be a good time to e-mail them to me so i can get them in game before the 0.07 release. I just have a few things to change for the Diplomacy build, and i am done.
The method I a using is untested but like i said I have all of the meshes in XSI now but that 3DS script mirrors every one so I have to manually flip the mesh and all the nulls. Thats what I am on now.
If you make a link in the OP to the http://soase.weebly.com/tsop-downloads.html page I will make sure everything is mirrored and if there is new files (that we test 1st), I will make a new add-on link and redo the main mods.
The TEC fighters Above and Aura points need repair. This is from the original mesh and probably not noticed when optimizing.
The Above point is way to high where on all other SC it almost touches the ship. The Aura point is above the Center Point and the Center point is Left of 0. Very bad.
I am off to school for the morning but can fix the optimized mesh when I get home, out of time.
I have not noticed the above point change when i imported the optimized strike craft. If it did change it must have happened during conversion to mesh. i can copy the coordinates from another strike craft and replace it. Simple fix.
Will update main topic as soon as 0.07 is ready. I need to do the final Diplomacy "stress" test. Then i am done. Minus any last minute optimized mesh importing, or last minute bug fixes that need doing. I dont have time to watch 2 hours worth of uploading right now, but will upload it asap. More than likely by this weekend.
I think you guys will be impressed with 0.07. It is performing better than my expectations. Even on my crappy intel GPU laptop. I think the focusing the textures to all DDS format was the answer. Plus strategic texture, and mesh reductions. Plus fixing all of the bad files we found. Just know that if you have a very low end system (1 gig of ram, or under) that you still wont be able to run the Sins at maximum settings, and your performance increase will only be slight.
Original Sins with TSOP enabled on maximum settings (including bloom enabled) requires 1.3 gigs of ram. Tested on Random Huge multi star 100+ planet maps with 9 hard AI's
Both Entrenchment, and Diplomacy with TSOP enabled require 1.5 gigs of ram to run at maximum settings.
Entrenchment and Diplomacy now use the same amount of ram to run, and both HOLD at 1.5 gigs. Tested on Systems of War with 3 unfair AI's
All games have typical late game lag, but nowhere near as bad as vanilla.
Going over 0.07 changes again.
1) Removed the modified planets, and skybox's, and restored the vanilla setup. This is due to the fact that the unfixable bugs, and errors created by them were not worth what little memory was gained.
2) UI for all 3 TSOP versions completely overhauled. UI now uses all DDS format textures. In fact the entire game now uses DDS format with only a very few exceptions due to hard code. All Brush, Particle, Mesh, and Texture Animation files edited to "force" the strict use of DDS textures.
3) All errors listed in the past "bad file" topics have been fixed.
This is on top of all of the other optimizations we have done since march.
The original mesh was bad. Just missed it when importing, I almost missed it as well on the original. And dont forget it all three. Aura should not be above Center and Center should be three zeros.
Great job on the update, I cant wait to try it.
Ok still running into a snag.. This time it is on Diplomacy. Diplomacy runs steady at 1.5 gigs on maximum settings with TSOP enabled. So ram is NOT the issue. Rest assured you will NEVER 2 gig crash with TSOP 0.07.
I "Stress" Tested with a few 100+ planet multi star random maps with 9 vicious AI's, and get the same 2 issues each time.
1) Typical late game lag still persists. On my big intel i5 box the FPS at game start is at 60 (seems to be limited to 60 because if i nerf settings it stays at 60). By the end game the framerate has dropped to around 19 FPS. Laggy but still playable.
2) Particle effects disappear. I read sometime back that there is a limit to how many particles that can be rendered at once. Apparently Diplomacy pushes that limit, because this does NOT happen with Entrenchment.
In ether case i dont think very much can be done without further research into the problems. We are still working on optimized meshes. Though i think it is more a cpu issue than a graphics issue now.
Other than that 0.07 is ready. Just need the free time to upload it.
found more bad buff points in the Envoys
Frigate_PhaseEnvoy & Frigate_PsiEnvoy Aura and Above are at zeros
Frigate_TechEnvoy also all the same as center buff point but its not at zero and is right of the middle. WTF?
Time to update the "bad file" lists.
Awesome work guys ! Just wanted to say thanks. I've followed what you have done here and have been optimizing my Dark Secret mod and with each change have noticed good results. Thought i would throw some ideas your way, don't know if they will actually help that much but none the less ideas. 3 things I did a long time ago and just feel they are not necessary (1) Removed all planet ambience sound loops from the planet entity files - can you really hear this in outer space? lol (2) shut off the blinking fading stars from the starscape file - does anyone really notice this while playing? (3) Removed planet elevator cars - cool to see but an animation thats not really needed.
Some other thoughts and things I've been looking at:
(1) LensFlare data file is referencing 1 .tga texture (lensflarebar.tga)
(2) Would it help to have all planet meshes use "Planetgas0-da" for the dark side texture? I'm currently doing this in my mod with a few planets I've created and they look fine, just no city lights.
(3) Would it help to cut back on the Dust Clouds. Planets with a puffy, dense and so on dust cloud template are rendering 2 to 6 particles. In a 100+ planet game that would add up fast.
Don't know if any of that made sense or not but thanks again for all the hard work.
you just can turn of showing elevators ^^ ive done this and it helps a lot
turning of showing trade ships also increase a little bit (because there still there its only a little ^^)
0.07 is uploaded. Hopefully we dont have any corrupt file issues this time. If you do follow the instructions on the main topic.
Once Myfist0 works his magic we will have an all in one download link. Also i need the link for the website to put on the main topic.
I did optimize the lenflares for 0.07. It is not just a bar. There are a few other textures used as well.
Planet car textures were also optimized. Along with the Atmospheric glows, Cloud maps, Night side lights, Star coronas, Nebula map objects (magnetic fields, black hole, etc.), and The Starfield background itself. I was pretty methodical when i looked through, and optimized the texture files.
The textures used for dust clouds were optimized as well.
As was said most of these features mentioned can be turned off in the game options.
great work so far guys!, love what your doing!
but i jsut ran into one little snag in the diplomacy 0,07 > the texture for the starbase upgrades is just one of those black fields instead of the upgrade button, you can still click it though, is it just my download that doesnt show it or do others have it aswell ?
edit: clarification: i ment the button when you already have a starbase, and want to go to the menu where you chose what upgrade to build on your starbase.
( thought this might save some confusion )
TSOP Main Page http://soase.weebly.com/tsop.html
Downloads Page http://soase.weebly.com/tsop-downloads.html
Frequently Asked Questions http://soase.weebly.com/tsop-faqs.html
Working on the aDrive Mirrors now and will also do the Trinity 7z on FileFront and aDrive
Anything on the weebly that should be changed or edited just send me a PM
I will test out the SB button ASAP
"edit: clarification: i ment the button when you already have a starbase, and want to go to the menu where you chose what upgrade to build on your starbase."
I know the texture you are talking about. i will look into it. It may have been missed when i worked over the diplomacy textures in which case just copy it from the entrenchment version. I will double check it asap..... so much for being methodical
'just copy it from the entrenchment version'
what exactly? the texture of the button, or the -if i can even figure out what it is- menu thing that says'i have all these buttons on me' ? or just everything remotly spacebase-ish ?
Just wondering, graphics aside, does the games audio effect ram at all?, any room for improvement?
Can the effects,sounds,music files be cleaned up/degraded to lower mem usage?
I did lower the music and sounds from 128kb/sec to 112kb/sec. I did try it at 96kb/sec but we decided that was too much quality loss. The audio is 1 thing that is loaded on-the-fly but the smaller the file, the less ram needed. Alone, it will not make a noticeable difference, it is all the little things together where the difference is clear.
How do I install TSOP?
Can I use this in my mod?
Can I stack TSOP with my favourite mod?
Is Polygon Count important?
Who is involved with TSOP?
Could you please upload the mod to other minor links ? for some reason i can't download anything from firefront
Does it work with other mods like Legend of galactic heroes ?
I just tried a bunch of time to get it on aDrive and will never try that POS site again.
Another person that does not read the OP
First of all, great job on this project guys, I get about a 100% game speed execution improvement on large maps involving multiple AI's on my high end system.
I have a question.
Just to clarify, as of 0.07:
"As of version 0.07 there are no longer any modded planet, and skybox meshes. Unfortunately the problems caused by the unfixable mesh errors from the poly reductions were not worth what little memory was gained from those optimizations. As i said the goal was to make it look like nothing has been changed at all."
So I downloaded 0.07 for OS today and while I was going over the files I noticed:
"Planet_Desert_0.mesh ... Planet_Volcanic_3.mesh" with dates 7/29/2010 and file sizes around 465 KB, also there are:
"SkyboxBackdrop2.MESH ... SkyboxBackdrop7.MESH" with dates generally at 7/31/2010 and file sizes around 219 KB.
So if there are to be no more modded planet and skybox meshes, are these files then intended to restore the original files installed with OS 1.191? If so, it would not appear this is successful, because my original files for the planet meshes are dated 10/16/2008 with file sizes around 124-130 KB and the skybox meshes are dated 2/24/2009 with file sizes of about 63 KB. Furthermore, I thought the purpose was smaller files taking up less memory? So now we are not fixing planet and skybox meshes anymore, but yet we'll replace the original files with larger ones? Could someone please explain?
There are many great features available to you once you register, including:
Sign in or Create Account