For those that do not know what the Sins Optimization Project is, please click the link below and some descriptions from their original post.
https://forums.sinsofasolarempire.com/378149
The project initially started out as an idea to reduce the texture sizes, and poly counts of some models. Not just arbitrary reductions. The goal was to maintain the quality of Sins in game. To look like nothing has been changed at all.
I studied the Particle files, and textures. Many of the particle textures IMO were also way to high resolution. A reduction by 1/2 showed no visible difference in game in most cases. There were some exceptions where it did effect the appearence in game, and those textures were left alone. Remember the goal was not to reduce in game quality. Also all of the particle textures were changed from TGA format to DDS format, because DDS uses far less memory, and is much more efficient. The particles also utilize the mip mapping features of DDS.
The developers mentioned something about a Memory Leak. I heard that a leak can be caused by a file calling for something over, and over again if it is not there. Example a mesh file calling for a texture that does not exist. Vanilla Sins has MANY particle, and mesh files that do that. I removed all of the redundant entries that pointed to non existent textures in the mesh, and particle files. I also removed the entries that have lines like "C;\gs\main\datasource\textures\effects\aura4.dds", and just put "aura4.dds". This will default the search to the mods texture folder, and speed things up. Also various typo's were corrected in some mesh, and particle files. Some ships have mesh nulls that were miss-spelled. For example the Akkan had typos on its engine nulls, and flairs. The engine trail, and flair didnt show up in game. the Marza was missing its bomb nulls. All of this has been corrected as well. If there is a ship mesh file in the mod then it had something wrong that was fixed. Regardless if this caused a memory leak, or not these errors had to be fixed.
After the Meshes, Particles, Textures, and the User Interface were optimized, and some very obvious bugs were fixed there was a tremendous boost in game performance. Sins ram usage was greatly reduced. As of version 0.07 no game i ran ever went above 1.5 gigs on Diplomacy with maximum settings. Including Bloom enabled. Tested on Huge random multi star maps with 9 AI's. The 2 gig crash problem is solved. Our primary goal with this project is achieved!
What I would like to know is how much of this has been transferred over into the 1.2 Beta patch? The community has, from a Mod standpoint, done a lot of down in the dirt work to make larger games playable reliably online. I would like to hope that this effort has been rewarded by being included in some part in the official patch. Already I have noted the 1.5gig 'observed max usage' comments from the 1.2 Beta and between the TSOP Mod being comparable. Has anyone looked in detail at a comparison between these two excluding game balance changes? Thoughts?
No changes to mesh files in the beta. Its all engine and gameplay.constraints with some pirate entities.
Was never able to test it in game because each time that somebody have promise to convert it to .mesh, it was never done...
Go to http://www.love-from-russia.be/package_planet.zip ... 30 mb to download... inside, you have a .obj who is a simple sphere with a UV who use a home made "almost" cylindrical projection... any usual planet texture with ratio 2/1 will work :
- http://www.mapsharing.org/MS-maps/map-pages-planets-map/images-planets-map/8-planet-jupiter-map.jpg
- http://www.oera.net/How2/PlanetTexs/MoonMap2_2500x1250.jpg
- http://24.190.6.205:333/earth/planet_earth.jpg
- http://www.mmedia.is/~bjj/data/io/io.jpg
... you can manually edit the .mtl for try the cl-01, cl-03, cl-04... cl-02 being the one used by the actual .mtl , earth... and i think that it look pretty good...
Since it is a manual UV, it is not really a spherical or cylindrical projection... it remain some UV surface waste but it is minimum ( a few % )... and the pole pinch problem is minimum too...
If it please you, it is yours... if needed, i have 12 map set ( color, bump, ambient... last being the cloud layer ) for custom planet using these model/UV...
Don't try to unfold the sphere again, it will mess everything... there is no real segmentation info... UV was calculated and added directly to the .obj ... when open with software like wings3d, and click the "windows", "windows UV editor", the UV map is show to be square shapped ( screenshot below ) when in fact, it is 2/1 ratio... in the same screenshoot, on the left, where the green line hit the planet, it is the north pole... only with extrem zoom, some pole pinch distortion is visible... Sins cannot zoom so much, so it is not a problem...
Original sins planet are 1271 vertices, 2320 tri ... these one is 1742 vertices, 3480 tri... the light increase of poly is not a big problem... it remain half of the poly count of the heavy tec cruiser... and since almost the full surface of the texture map can be used, resolution can be half without loose of quality ( 2048x1024 in place of 2048x2048 )... meaning a reduction of 50% of the memory use for planet texture ( 2.7 mb in place of 5.3 mb )... a total of 5.3 mb memory spared by planet type ( cl and da only, cloud are already 2/1 ratio in sins )... there is a minimum of 69 planet texture ( 326 mb ) who can profit of it... who lead to 163 mb ram spared without loose of quality... without speaking of the huge amount of already existing texture with these ratio 2/1 who can be found freely on numerous Nasa, Space, fan site...
I always welcome open discussion as long as it does not distract from the main point. Which is TSOP being merged into the 1.2 patch, and Rebellion.
I get upset when people try to hijack the TSOP topics to push for LAA, or some risky EXE hack which increases the games ram usage. The main point is We should not need LAA, or a Hack to run the game smoothly, or to keep it from dumping. The game should run fine with no need for ANY alterations whatsover. It should run fine straight out of the box. We as a community should not have to fix the game. We should not have to do ANYTHING at all. It is the Developers responsibility to fix this Excelent, but Broken game. I did what i did because of my love for the game, and it was a shame to see what was happening to it. Obviosly if there was no fan base, no market, or if this was a really bad game then we would not even be having this discussion right now, and Sins would be truly dead. Yet here we are
The main purpose of TSOP was to eliminate the 2 gig crash bug that plagued Sins since Entrenchment, and got Worse with Diplomacy. I ran the idea of TSOP accross to the devs last year when the 2 gig issue was at its worst. They said "go ahead and try it". So i did. Understand that i am not a programmer. I am a 2d, and 3d graphic artist. A Modeler. I know nothing about code. Everything in TSOP was done on the Graphics level, and some very obvious data errors that were easy to fix with wordpad. The goal was to make sins run better, use less ram, and to eliminate the 2 gig dump without changing any game play aspect. We succeded.
We all knew that sins had a 2 gig hardcode ram limit, but what people did not understand that it the actual ram that Sins used on your system. Not what Sins plus everything else you were running along with it. So people were saying "I am 3 gigs ram, and sins was running just fine". Yes it would run fine if sins was using 1.5 gigs, and the rest of your system was using the other 1.5 gigs. Sins crashed when Sins itself uses 2 gigs of ram. It is hardcoded to dump when it did.
The textures play a BIG role in how much ram sins uses. Most of the ram Sins uses IS used by the textures. Sins is different from other games, because of how sins loads its files. Other games load, and unload assets "as needed", or "on the fly". Example: Mass Effect. When scenes change you can see the textures still loading up from low to high resolution as the map changes. Sins does NOT do that. Sins loads most if not ALL of its game assets at map startup, and keeps them in memory until ether 1) The game is over (by victory/defeat, or user ending the game), or 2) The game dumps, because it hit the 2 gig hardcode limit.
There are many textures in the game assets that are very large (some up to 4096 resolution), but waste almost 1/2 of the textures space due to the method UV mapping the original developers used (especially for the planets, and skybox's). There will always be "some" waste on textures, but what i saw was exccesive IMO. Most of the particles also use textures that are way too large for thier purpose. There is no need for a particle to require a texture with 512 resolution when the particle mesh is scaled to the size of a pinpoint in game. This is the kind of waste TSOP eliminated. We left the Ships themselves alone, because there was nothing wrong with the textures. I already explained the failure with the Planets/Skybox's earlier. The biggest offenders in the waste department were indeed the Particles.
The obvious bugs we found i wont list them all, but you can look at the TSOP topics to see for yourself:
Typos, and Mis-labels of mesh nulls, and textures in the mesh files. MANY of them.
Examples: Marza not having a bomb, Above, Center, or Aura null, and Akkan's engine, and Nav light points miss-spelled.
Entrys calling for textures that do not exist in the textures folder at all (both the mesh, and particle files) causing an endless search loop...A memory leak. TSOP substituted some of these missing textures with the closest matching alternative texture. Basically fixing the particle. In some other cases the entry was removed.
Entrys calling for folders, and file paths that do not exist on the users computer causing the game to use more cpu ticks, ram whatever to search for the these files (mainly in the Mesh, and Particle files).
Examples: In the Buff_GaussRailgunTravel.particle you will find entrys like this.. "C:\gs\entrenchment\DataSource\Textures\Effects\orbitaldefense.tga" when a simple "orbitaldefense.tga" would do, and default Sins to search in its textures folder. In the mesh files you will find similar entrys. Example: In the Star_Blue.mesh file you will find texture entrys like this.. "C:\Dropfolder\Models\sun\pictures\starblue.dds" when a simple "starblue.dds" would do.
To me all of that amounts to waste. So TSOP eliminated that waste, and managed to free up to 800 megs of ram for Diplomacy.
To me the biggest advantage of DDS textures is the Mip Mapping feature. It serves two purposes...
1) The further you zoom out the lower the resolution of the texture. Sort of like texture level of detailing. Since there is no mesh lodding in Sins. You dont need to see the Kol at 2048 res when you are zoomed out to almost an icon. The particles gain the same mip mapping benefit when switched to DDS.
2) It switches to a lower resolution when you chose it in your game options. Example Highest detail would be 2048, Medium 1024, Low detail 512 etc. Since TSOP switched the particle textures from TGA to DDS they too will benefit from this feature. Increasing the games performance.
For the UI i switched to DDS just to keep the standard. Some UI was already in DDS format. There was a huge memory benefit form converting the rest of the UI. I see the artifacts now. I used the Nvidia photoshop plugin to convert them. Ill see if i can convert a "cleaner" texture.
As of Beta 3 NO models were changed yet in the beta. NO textures were changed yet in the beta. NO particles were changed yet. Only a few files in the GameInfo folder were changed.
EDIT: As of Beta 4 some of the mesh fix's from TSOP are now in the patch.
A good question.. Is the memory usage identical between a Loaded TGA, and a DDS texture?
Sorry Thomsen, but i havent had any free time to try your method, but will do asap.
Sorry for the novel.. I needed to make my point
Yep, thats it. Just need to be aware of what the impacts are when optimizations are being made.
So, haven't checked what they did but here is a rough summary of changes with Beta 4.
58 meshes were changed.
24 particles were changed.
no textures changed.
I think Major Stress's point is easy to understand, even for a computer-illiterate person like me.
If I understand correctly, the OP was asking this:
1. Does the new patch make TSOP superfluous because it incorporates TSOP's good ideas?
or the correllary:
2. Should we and can we run TSOP with the new 1.2 Patch without conflicts?
Please correct me if I am wrong, but the answer to 1. appears to be "no". I would like a clear answer to #2 if possible. Thank you.
1. Yes it is correct. TSOP has not been fully incorporated into Sins yet.
2. No issues found. The one we found on DS was the player.entity files which TSOP does not touch.
Hopefully after the 1.2 patch we will not need TSOP. As it stands now Diplomacy beta 4 doesnt use more than 1.3 gigs on max settings on my big machine, and runs pretty smooth on my crappy laptop. Late game lag is still present, but i am not sure if much can be done about that. If IC decides to use the texture optimizations of TSOP it may help a little, but short of converting Sins to 64 bit the late game lag may always be present. I think its too many units on the map at once causing the lag (fighters, mines, structures etc.) putting a heavy strain on the cpu. IC said they were addressing that issue.
I'm going to try your model out. I found some nice textures too that look good on it. I'll send you the results when I get it working in game.
Well, if you wish know more about the type of projection from where my UV is based, look at http://en.wikipedia.org/wiki/Equirectangular_projection ... it is not a perfect equirectangular projection but it is not so bad since my 3D software ( wings3d ) have not these type of projection... and these projection have one of the simplest formula for x,y...
For texture, you can find hundred of them ( if not thounsand ) at http://www.celestiamotherlode.net/ because they use these type of projection... your have star trek planets, babylon 5 planets, star war planets, and more there... all textures being 2/1 ratio and fitting on my .obj sphere... not only almost all area of the texture map is used but it can be usefull for a lot of mods related to some sc-fi show...
About in game, i have no idea how the dark side is made, directly on texture like at http://24.190.6.205:333/earth/planet_earth.jpg or via real sunlight... no idea about the cloud too ( texture already 2/1 ratio in sins )... no idea if bump map can be used ( for mountain ) or specular ( for bright mountain top with snow and water surface ), etc... there is so little modding information about planet for sins...
Poiny is my post was more related to the Major Stress post about texture map surface being wasted and projection problem at the pole...
Ya, I'll figure it out. I don't really care too much about the particulars. I've wanted to add some new planet textures and will figure it out as I go. I did find some nice variations that look good on your model, so half way there.
I don't know if I am doing something wrong or something but it seems as if the changes that would have been applicated because of the beta changes won't take effect if you are running the SOP mod.
The mod will be able to be enabled while you have patch on but none of the changes from any of the betas will not applied. I'm basing this off the fact that Vasari colonize ship reverts back to its default colonize ability properties which is having a 20 percent construction boost instead of what the beta adds which is an extra constructor.
You are correct. Running TSOP over the beta right now will negate any, and all of the betas changes. We have the entity files in the mod folders in order to "force load" all of the needed files from the TSOP mod directorys (another reason why TSOP is not compatible with stacking other mods on top of it).
Ok I was just checking. I look forward to the next release I assume is coming after the final patch. Got to have alot more playing without lag with it. Loving that. Makes sins more fun and bearable.
As it stands right now with 1.019f TSOP is really no longer necessary. Diplomacy doesnt use over 1.3 gigs of ram. However the textures were not changed, or converted to DDS format. Entrenchment, and Original Sins were not touched, and Diplomacy shares many files with Entrenchment, and Original Sins. So we may still need it. Unless IC incorporates TSOP for those two versions with Rebellion. Which is supposed to be a stand alone game from what i hear.
There are still some late game lag issues. Which i think nothing can be done about on a graphics level. Even with TSOP, and the reduced poly meshes in TSOP the late game lag is still there. So it must be too many units putting too much of a strain on the CPU/s. IC said it was looking into that.
We are waiting for the final patch to see how much of TSOP is actually incorporated. We do intend to make a new build for Diplomacy, but it may be just an optimized graphics version for people with lower end systems. The Entrenchment, and Original Sins versions will still be there for those that dont have Diplomacy.
AFAIK the dark said uses the -da file. You have the cl for dayside and da for night side
This is the first shot of Thoumsin's planet mesh in game. It's a 1024x1024 texture for both the light and dark side and looks pretty darn good. Figured I could import more planets if I kept the texture sizes down for my mod.
Looks good from the pole too...
Impressive. I can't wait to bomb earth to oblivion.
Since sins don't accept bump map for planet, result is not "pretty darn good"... but you can improve it greatly by "backing" the bump map in the color map...
Step by step method ( you need GIMP )
- Open Gimp
- Load the bump map
- Load the cl map
- cl windows being selected : filter > projectie > bump projectie ( "projectie" is Dutch, mean "projection" )
- select "bumpmap onkeren" ( invert bumpmap )
- "diep" on 10 ( "diep" is "deep" in English )
- click "OK", save your new cl map with more realistic relief...
- Make a test in game and enjoy the improvement !!!
Same method can be used on ship who use a grey scale bump map ( it spare a texture and if the bump map is good made, the render quality remain )... in case of the colored normal map, the method don't work...
Will post a render from the few planets i have at a later time... need bakking the bump in the cl first...
Yep, I use Gimp and that approach. Step one is get something working before I worry about refining it. I think the result is pretty darn good considering the minimal effort put in. This is my first planet to wire up so wasn't real concerned about making everything perfect up front as I prefer to iteratively move through things.
I have grey scale bump maps for most the planet textures I have that I will bake into the cl.
I plan to release these as open to anyone to use outside of my mod as well...
Well, it will greatly improve the result...
- render ( not ingame ) using only the basic cl
- render using the modified cl ( bump map baked in the cl with Gimp )
- render using a other modified cl ( bump map, clouds baked in the cl with Gimp )
- render of various planet texture using the model( using texture half the size from SOASE )
No problem with it on my side... the model was made 2 or 3 year ago for the solar sins mod from Danman... but nobody have use it and have since only wasted space on my computer harddrive... if it can be usefull for other, it is perfect for me...
What do you mean? Have you change the scale ration 2/1 to 1/1 or do you have put the light and dark side on a single texture ( 1024x512 for light side + 1024x512 for dark side together on a single 1024x1024 texture ).
If it is the second case, your ingame screenshot are very good when related to the low resolution used... it mean one dds using 1.3mb in place of 2 dds using each 5.3mb... it is 87.73% memory spared... 286 mb who can be spared if all game planet texture/mesh are rebuild... hmmm... TSOP + new planned SOESE patch + new mesh/texture for planet can lead to a game who is near to use less that 1GB !!!
Sorry, no... I mean two 1024x1024 textures which is 2x1.3meg. The Sins base terran texture is 2x5.3meg, so still a significant savings for the the planet texture. It only becomes noticeable when zooming 100% into planet.
If it is spherically UV mapped send the mesh files my way. Ill see if i can get them into the next build of TSOP.
There are many great features available to you once you register, including:
Sign in or Create Account
