It is difficult to explain the issues of Sins have without going into too much "technical detail" which some may not understand. The main problem is that Sins is an older generation 32 bit game. With a two gigabyte memory limit. This is nothing new. All 32 bit games have a 2 gig memory limit. However Sins is NOT like other 32 bit games. Most games have an ability to load their assets "on the fly", or only as needed. Some games can also unload assets that are not needed. This is how they prevent themselves from reaching the 2 gig limit. Some games have been patched to where they can use up to 4 gigs of ram on 64 bit systems.
Sins also can load only what is needed at game start up, but Sins does NOT unload what it does not use. Any assets that are loaded remain loaded throughout the entire game! If you use all three factions (Six in the case of Rebellion), Then Sins will load ALL of its game assets at start up. Entrenchment, and Diplomacy in some games loaded between 1.7 to 1.9 gigs worth of assets at game start up! Graphic settings, The size of the Map, and the amount of Players, or AI opponents also determine how much ram Sins will use in any given session.In 2006 when Sins was first conceived 32 bit Windows XP was the standard OS of the time. Single core CPU's were also still the standard. Though dual core CPU's were starting to become mainstream, and 2 gigs was considered to be "plenty" of memory. So most game developers at the time didn't have to worry about hitting that dreaded 2 gig brick wall. As time went on Computers, and Sins itself evolved into their present state which is now Rebellion, and soon to be Windows 10 running in multiple core 64 bit systems. With 16+ gigs of ram. Things have changed so much over the years. However Sins still remained the 32 bit game as it was designed in 2006. That uses only a single core of your CPU. With a two gigabyte hard coded memory limit. Sins WILL CRASH when the game reaches that 2 gig limit.There is some misunderstanding about the 2 gig crash issue. The dump happens when Sins reaches 2 gigs of Ram Usage. That means the ram that Sins uses by itself. NOT what your entire operating system, and/or other programs you are running combined uses. Only the ram that Sins uses. No matter how powerful your system is. If it is 32, or 64 bit, or if it is Windows XP, Vista, 7, 8, etc. etc. Sins WILL CRASH when it hits its 2 gig ram limit. It is hard coded into the game.Nobody was aware that this would be a major problem back when the game was first designed, because Sins was designed based on the hardware standards of 2006-2007. Before 64 bit OS's, Quad Core cpu's, and 4+ gig ram systems became the standard that they are now. There is nothing that can be done about it. Except to re-write Sins in 64 bit. Which is out of the question, because it is too cost prohibitive for the developers to do this (Ironclad said so themselves). Sins is a great game, and it is a shame that this is happening. With the Entrenchment, and Diplomacy expansions Ironclad had unintentionally pushed the game over its own hard coded limitations with all of the new added content.
Over the years as of the release of Diplomacy 1.3, and Rebellion. Stardock, and Ironclad did indeed fix all of the major game crashing issues of Sins. Using the fix's from This Project as a guide. However, NONE of these fix's ever made it into Entrenchment, or Original Sins! While the Stardock, and Ironclad developers have fixed most of these issues over years. They can not do anything to change the game itself short of reprogramming Sins over from scratch. The developers can do no more, but there is something that WE can do as modders to prevent Sins from hitting that dreaded 2 gig brick wall, and improve the performance of Sins "just a little bit more". I have taken the initiative, and applied to Sins what i have learned from my Homeworld Modding days. I present to you....
The goal of this project is to improve your Vanilla Sins gaming experience, and to eliminate the 2 gig crash issue. No changes will be made to Original Sins, or any of its Expansions in game play at all. No stat changes will be made. Nothing "new" will be added. There are other mods that can do that. Original Sins will remain Original Sins, and the Expansions will remain as they are. What we are doing is taking what game assets that Sins already has, and we are "optimizing" them to improve the performance, stability, and to moderate the ram usage in game. The side benefit will be that some lower end systems can enjoy almost the same gaming experience as the high end users do (Especially Laptop Users).The project initially started out as an idea to reduce the texture sizes, and poly counts of some models. Not just arbitrary reductions. The goal was to maintain the quality of Sins in game. To make it look as if nothing has been changed at all. We started with optimizations to the Planet's, and Skybox's, because they were the "worst offenders" as far as ram usage, and performance was concerned. Unfortunately my modeling skills left much to be desired in Softimage XSI at the time, and there were un-fixable mesh errors on the optimized planet meshes. Despite that the "proof of concept" was established. A lot of ram was saved, and there was a very noticeable increase in performance.Then we studied the Particle files, and Textures. Many of the particle textures in our opinion were way too high in resolution for the purpose that they served. A reduction in texture resolution by HALF showed no visible difference in game at all! There were some exceptions where the particle resolution reduction difference was very noticeable in game, and those textures were reverted back to their original resolutions. Remember the goal was NOT to reduce in game quality. Also all of the particle textures were changed from TGA format to DDS format, because DDS uses a little less memory, and is much more efficient, because DDS format loads directly into video memory. The particles can also utilize the mip mapping features of DDS format. Where TGA format does not. Therefore enhancing in game video performance (increased FPS). As of version 0.07 no TGA format textures load in TSOP. With the exceptions of the Mouse Cursors, and Scenario Pictures. This is due to those textures being controlled by the hard coded exe. When you change the texture format those textures do not show up in game.The developers mentioned something about a Memory Leak. I heard that a memory leak can be caused by a file searching for something over, and over again if it is not there. Example a mesh file calling for a texture that does not exist. Vanilla Sins had MANY particle, and mesh files that had that issue. We removed all of the entries that pointed to non existent textures in the mesh, and particle files. We also removed the entries that have lines like for example "C;\g\main\datasource\textures\effects\aura4.dds", and just used "aura4.dds". This will default the search to Sins, or the Mods "Textures" folder, and speed things up a bit. Also various typo's were corrected in some mesh, and particle files. Some ships had mesh nulls that were Miss-Labeled. For example the Akkan had nulls, and flairs that were miss-labeled. The engine trails, and flairs didnt show up in game. the Marza was missing its bomb nulls. etc. etc. All of this has been corrected as well. If there is a ships mesh file in the mods "Mesh" folder then it had something wrong with it that was fixed. The Soundata, and Galaxydef files had errors too. Regardless if any of these issues caused any memory leaks, or not these issues MUST be fixed! (and they were as of Diplomacy 1.3, but NOT for Original Sins, or Entrenchment).After the Meshes, Particles, Textures, and the User Interface were optimized, and the very obvious issues were fixed there was a tremendous boost in game performance. Sins ram usage was greatly reduced. As of TSOP 0.07 I could run a game of Entrenchment, or Diplomacy ALL DAY, and I NEVER used more than 1.5 gigs of ram! This was using Maximum Settings. Including Bloom enabled. Tested on Huge Random Multi Star maps with 9 AI's. The 2 gig crash problem has been solved! Our primary goal with this project has been achieved! Without having to use any external memory increasing programs! Stardock, and Ironclad took notice, and they used TSOP as a guide in the Diplomacy 1.3 patch, and for developing Rebellion.Now that the Stardock, and Ironclad have fixed the issues of the game as of Diplomacy 1.3, and Rebellion. TSOP has been focusing more on Original Sins, and Entrenchment. Since the Ironclad/Stardock fixes never made it into Original Sins, or Entrenchment. However, That is not stopping us from optimizing Diplomacy, or Rebellion "just a little bit more" You can also use Large Address Aware to give Sins an extra gig to play with, but in vanilla Original Sins, and Entrenchment the Endless Search Loop issues that were never fixed will cause the game to eventually go over that extra gig (Run vanilla Entrenchment, and watch your ram usage keep going up, and up, and up). TSOP is a solution that will stop that issue.
____________________________________________________________________________________________________________________
Downloads
The Sins Optimization Project v1.0
For Original Sins 1.195
For Entrenchment 1.055
For Diplomacy 1.37
Stardock, and Ironclad fixed all of the issues that compelled us to make TSOP. Therefore there is no TSOP for Diplomacy
For Rebellion 1.82
As with Diplomacy, Stardock, and Ironclad fixed all of the issues that TSOP pointed out, and then some in the case of Rebellion. Sins: Rebellion is as optimized as any Sins game is going to get.
NOTE: TSOP 0.07 is still hosted on ModDB, and it still works in the current versions of Trinity (OS, E, and D), but you will be playing as if Sins was patched to an older version of the game. Plus some Icons and Buttons wont be visible.
Changes in TSOP 1.0
TSOP 1.0 updated to the latest versions of Original Sins, and Entrenchment.
The User Interface is completely overhauled to use pure DDS format.
Trade Ships, Strike Craft, Mines, and Constructor Unit Meshes, and Textures are Optimized.
Starbase Textures are optimized mimicking the Stardock Rebellion Optimizations.
Sounds, and Music are optimized.
All TGA format Textures are replaced with DDS format, and TSOP "force" loads them. The only TGA textures that remain are the Mouse Cursers, and Scenario Pictures. This is due to hard code in the EXE.
"Strategic Texture Reductions" for the Particles return using newer methods to create a better looking texture.
(Original Sins, and Entrenchment) Many Mesh Null issues, and various File Errors were corrected, and Unnecessary Text Entry's were removed in both the Mesh, and Particle files. Entry's pointing to Non Existent Textures in both the mesh, and particle files removed. See the various "Fixed Files" texts in the mods Read Me for details.
(Original Sins, and Entrenchment) The Diplomacy 1.3 graphic fix's have been incorporated into the Original Sins, and Entrenchment versions of TSOP (Pipeline Effects, "White Line" error fix, and Corrected Mesh files).
TSOP 1.0 for Trinity will see the return of Optimized Planets using the planet meshes from Rebellion! but with our Optimized Original Sins Textures!
There will be no version of TSOP made for Diplomacy, or Rebellion, because there is no need for it. Only Original Sins, and Entrenchment need TSOP, because the 2 gig issue, and various other issues were never fixed in those two versions of the game.
This is the FINAL version of TSOP that i will make. If the community wants to take over the project they are more than welcome to.
Community input, and contributions are more than welcome, and they are encouraged! This is a Community Project! All can pitch in!
Note: TSOP does not fix the known issue of "Late Game Lag". It does help prolong the lag until further into the game in some cases, but sooner, or later the lag will happen. The late game lag issue is a CPU issue due to the fact that Sins (including Rebellion) is optimized to use only single core processors. Therefore Sins will use only one core in multi core CPU's. To help with the late game lag issue we recommend the following:
Playing on smaller maps.
Using fewer AI opponents.
Disabling the Trade Ship icons (this actually helps out a lot)
Stacking the icons in the empire tree (same as with disabling trade ship icons the less icons rendered the better).
Using a Strike Craft Reduction mod (The many strike craft in the game are a big CPU hog. There are some good strike craft mods that reduce the number, but maintain the balance. Search for them).
You do not need to download all versions of TSOP. You only need to download the version of TSOP for the specific version of Sins you wish to play. The Retail version of Trinity is just all three versions of Sins bundled into one package. Installation for Trinity versions of Sins are no different than if you purchased each version of Sins separately.
_____________________________________________________________________________________________________________________
Other mods are more than welcome to use TSOP. No questions asked! In fact we encourage it! Learn from it! Use it as guide, and an example! Merge it into your projects!
TSOP is intended to be a Stand Alone mod. Run with no other mods enabled. There are issues enabling TSOP with other mods. Enabling with other "Graphic Enhancement" mods is NOT recommended, because the graphic enhancement mod will defeat the entire purpose of TSOP. All of our optimized files will be overridden by that enhancement mods "enhanced" files. Increasing your ram usage instead of decreasing it. Some enhancement mods like Distant Stars are incorporating elements of TSOP into their future updates.
Read the included READ ME text's for installation instructions.
Original Sins meshes do not seem to have the issues of other 3rd party open meshes like those in the SoA 2 mod. So there "should" be no problems with conversion. Unless you create an open area through the optimization process.
As far as where TSOP should go depends on how much IC fixes the memory issues in the next patch, and "IF" they do use TSOP as a guide for the next patch. I say we just do business as usual and stick to our plan.
aye but the chances of a patch these days are...well kind of bleak in the whole scope of things, though it would truly be a welcomed by the whole community, I'm fairly sure the "sins team" has moved onto their next project and shelved us.
....someone not heard the news.
I wouldn't hold my breath .
you're right I just read the news on the home page talk about a shock! now I'm excited
I am slightly optimistic about the new update. I can see them doing to Sins like what was done with X3 Terran Conflict, and adding the ability to access more than 2 gigs of ram despite it being a 32 bit game. Plus the ability to access multiple cores or hyperthreads will really beef up Sins late game performance. This is all just speculation, but it is what could be done to Sins. On top of the Optimizations that we have pioneered here to drastically improve the game, and make room for the supposed new units.
Irregardless that big question of 'what's next' for the Sins universe will likely be answered in some fashion and that is what many gamers are curious about. Is it just a patch, is it Sins 2, is it both? Who knows. *shrugs* We'll find out one way or the other though.
It will NOT be Sins 2. The update will more than likely be a major add on like Entrenchment was to Original Sins. If they do plan on adding a new faction (like the "unknown enemy" that the Vasari are running from) They will have to make lots of room in the existing game, because if they add a faction in Diplomacys current state then Sins will most certainly be guaranteed a 2 gig crash, and we will be back to square one.
square one's scary, even as a thought at all the work you all put in, if that ends up being the case I think I'll happily deny the update
Could their extended 3rd patch be ... *drum roll*
https://forums.sinsofasolarempire.com/177648 ?!?!
In fact, it is not off-topic... the SOAP mod is great for reduce the memory print from sins but there is several other way that sins dev can use for reduce the memory use that SOAP dev cannot use...
by example, implement a "loadondemand" for everything can really be a complement to the SOAP mod... i will be really unhappy that Ironclad/Stardock use only a work similar to these made by the SOAP team... they can make better because they can "mod" the game engine...
The last Stardock/Ironclad post was not releted to a new expension, to a sins II but to the memory problem... something directly related to the SOAP mod...
What will be really great, it is Stardock/Ironclad who collaborate with the SOAP team... mixing their own optimization with these of the mod... the SOAP devs are maybe not real game devs but they have show their capacity to have original idea and have some really positive result...
I'd like to see a "sub menus" button or extra pages to the build ques, I'd make additions/alterations all that more possible while keeping original content or make a lot more room for some of those "Pact available creations" like what Zombie's working on. All around basically wanting them to make it more "open" to the modders and I'm using so many "quotes" I must sound like one of those "air quotes" people lol
There are many ways to reduce a memory print. TSOP uses just a few of those methods. SOAP may use other methods. Ether way can be a "right" way. Since there really is no single "right way" to do this. However, there is a "wrong" way, and i hope Ironclad does not chose the "wrong" way. The wrong way IMO is to chose a quick, and dirty approach.
Like for example a major poly, or texture reduction that is so drastic that the visible results are plain as day, and makes units look like lego's instead of starships.
Or a change of code that while it may make the memory usage more efficient in return makes the game play aspect WORSE instead of better.
There is absolutely nothing wrong with Sins game play from my point of view. What needs to be fixed is the 2 gig problem from the developers level. Which the developers have the ability to change much more than a few models, and textures. Like what we did. They have the ability to re-write the entire game if that is what it takes, and does prove to be cost effective.
I do hope that they release a 64-bit version as Sins II.
As for optimizing what is already there I haven't implemented any of the work here into Requiem simply because Requiem hovers around the 1.5Gb mark and never crashes due to this (and I run max settings on everything).
One bit of optimization that I've been thinking about is how to bring it to the attention of the developers that they have a serious bug in their look up code for Strings. Specifically, Requiem hasn't worked multiplayer for Diplomacy forever and I'm only now fixing it because I spent a good 2 hours fiddling around. In a nutshell: Diplomacy is hardcoded somewhere to look up starbase names as an offset from the beginning of the file. I had to re-arrange string entries to prevent a constant de-sync bug when spawning a starbase for one of my extra races.
Maybe the next job of the Optimization project is to actually get the Dev's engaged and not only fixing bugs but also implementing the good work of this project into their next patch??
No, its not. Sorry. But I'm doing my own optimization and I've run into a major bump - since I had to redo some of the models because of the textures taking up too much memory and the old models look like crap with lowres textures, I recently found out that - convertXSI doesn't like my dotXSI files from Softimage 2011!
I've even tried exporting a simple sphere primitive, same thing. Reading this topic I know that other people have similar issues, anyone found a cure? I found some stuff to try but it would save me a lot of time if someone has a specific solution that works.
And as usual, I answer my own question as soon as I post about it. Seems Mod Tool 7.5 dotXSI is still ok to use. Sorry for the offtopic guys.
Its okay, nice to see you back ManSh00ter
Glad to see some modding Veterans back. Good to see you both Manshooter, and JasonFJ.
I can confirm that Softimage 2011 does NOT work with ConvertXSI. At least for my meshes. Others have had mixed results. Mostly bad from what i have read. Stick with 7.5, and below.
Read back a few pages. We had a good discussion about 7.5, Mod Tool, and ConvertXSI. ConvertXSI does not like models with "open areas", or open meshes (which is what 99.999% of the SoA 2 models are). We need to rethink how we mesh models for Sins. Unfortunately making a "solid" mesh with no intersecting polys, or open areas may add a few extra polys to your models. It also means i have to re-mesh EVERY model for SoA 2. Read back. It gets into some details about this.
Thanks Major Stress, I've been catching up with the very useful info in this thread. I have pretty much the same set of problems as you, which is why I've started redoing every model for the Last Stand (sigh).
I have to do some research before I can post anything conclusive, but regarding open models, I have had pretty much 100% success importing models which have open mesh parts using Softimage 7 (NOT 7.5, turned out 7.5 does not translate vertex normals well).
Also, I have some of the redesigned meshes with some open parts, I'll let you guys know if I manage to place them ingame without error.
And yeah, don't know if this has been mentioned before (haven't seen it in the last eight pages or so), but for those people who have an issue converting textures to DDS format in Windows 7 64bit, the only method I found that works is to use NVidia's Texture Tools 2 which have a 64bit version and are really, really fast (if you got a Nvidia GPU).
P.S. Concerning deleted faces, in my experience that usually happens if you have weak poly joints (vertices shared by more than 4 polygons) such as cylinder tops, especially if those parts of the mesh are stretched (think cone tips) or have quads in them. Retopologizing those parts of the mesh so that the weak joint area is very small, or triangulating the offending section might resolve the issue.
So far it has been a 50/50 chance that an open model will come out corrupt with ConvertXSI using Softimage 7.5. I do not have 7. I do have 6.0 (the same version the sins developers used), but it does not work on Vista, or Windows 7. Nor do i have an old 32 bit machine that can run windows XP to run xsi 6.0.
It is indeed the open areas on the meshes that are causing the issues. Like you said Manshooter not all of the meshes with open areas come out corrupt, but if made with 7.5+ there is a much higher chance that they will come out corrupt. I think 7.5 exports a little differently than xsi 6.0 which is why i have had very little issue using xsi 6.0 to export open meshes. ConvertXSI was designed to work with meshes made from XSI 6.0, and nothing has changed about it since the update when Ironclad updated the shaders (switched the team color, and specular channels). Perhaps when they update Sins they may update the Forge Tools as well with a ConvertXSI that is more compatible with Softimage 7.5+. I wish i had my old box operational to test my theory.
Agreed about the flipping faces. It is generally how i made the mesh that caused the issue. Too little tri's in a certain area or using a quick, and dirty cylinder "cap" when i should have manually added the faces. Wonder if the depth of the cone has anything to do with it. Plus the UV mapping.
So far i feel the "safe" bet is to remesh models into a "solid" mesh, and study how Ironclad UV mapped its models. Sins does not seem to like duplicated, or cloned sections very much that share the same UV coordinates as another section. Example: Warp nacelles that share the same UV's on a texture, or half sections cloned to share the same half texture.
I am currently testing a new UV mapping workflow for my own project - as I have to redo most of the models I did so far I am interested in a most time-efficient workflow possible.
My previous, and current, workflow involves building symmetrical meshes, UV mapping one half to maximize texture space, then texturing that in ZBrush (which REALLY doesn't like any UV overlapping), after which I use the textures on a symmetrized mesh. This results in complete UV overlap for the whole model of course. Can't say I had any texture problems with that, except of course the seams issue (but ZBrush makes that a minor one). There were a number of shading problems though.
My new workflow involves doing a second, completely new UV set for the tangents, this one done for the whole symmetrized mesh. IC docs suggest simply copying the existing set of UV's for the tangents and then adjusting that set to get rid of any seams of lighting issues. Since the layout of the second UV set and texture space are irrelevant, building it from scratch to fit the tangent smoothing requirements perfectly seemed feasible - and it is. This method got rid of all the weird lighting issues I experienced before, including shading seams, rough lighting transitions, mesh pinch issues etc. Smooth as Data's bottom.
Which is why I would suggest to anyone who can to switch to Softimage 2011 - it has a really cool Unfold function which makes UV mapping a ten minute job (and a quality one at that) even for complex models. Maybe newer versions of 3DSMax have the same (after all they're all Autodesk products).
Other than that if your overlapped UV's do not share edges or vertices, you can simply make sure they're not overlapping in the second, tangent UV set. I do not think the engine cares that much for the texture UV's as it does for the tangent set.
Well, i have made for the B5 mod models with numerous piece sharing identical UV coordinates...
Main problem was related to ingame lightning... but it seem that Tobi have no problem with it... maybe because he calculate the tangent in Cinema 4D...
Seam are a other problem... need to be careful during the segmenting... best is to cut in not very visible place or along hard edges...
Anyway, Tobi have become a expert at correcting UV map with a lot of pieces having the same UV coordinates... can be a good idea to contact him about it...
@Thoumsin
Thanks for the flowers, but I'm having some very visible seams on some parts of the models due to the way I'm doing the base textures. (Hyperion for example)
Anyways, I haven't had any lightning problems since going the Cin4D (or whatever program) -> MeshLab -> XSI route. I calculate the normals in MeshLab, and use as a base the cubic projection of XSI for the tangents. after that, some models (e.g. Vorlon crusier) need correction of the tangent uv. I do select the individual faces and adjust projection (orientation/rotation on the uv map) until the lightning is smooth.
As for parts sharing the same texture position on the UV map, this is absolutely no problem at all for sins, one only needs to be careful if one wants to bake Lightning or Ambient Occlusion into the texture. As I use Cinema4D I can use snap to point/edge to perfectly overlay parts on the UV.
Instead of adjusting individual faces, it is better to focus on creating large, continuous UV islands (such as you would have for, for example, organic meshes like faces), which in itself eliminates a lot of shading seams. This requires building a new set of UV's from scratch though, to avoid having to reorient a lot of stuff, especially if your UV map has a lot of small islands.
And it won't help with texture seams though. The best way to avoid texture seams is to use a polypainting app.
There are many great features available to you once you register, including:
Sign in or Create Account