Sins of a Solar Empire : Real-Time Strategy. Unrivalled Scale.
© 2003-2013 Ironclad Games Corporation Vancouver, BC. All rights reserved.
© 2006-2013 Stardock Entertainment

[MOD] The Sins Optimization Project version 0.08 (updated 9/5/11)

The Sins 2 gig crash issue has now been DEFEATED!

By on March 15, 2010 8:56:37 PM from Sins of a Solar Empire Forums Sins of a Solar Empire Forums

Major Stress

Join Date 05/2006
+139

It is difficult to explain the issues Sins have without going into too much "tech" which some will not understand. The problem is that Sins is a 32 bit game. With a two gigabyte memory limit. This is nothing new. Since all 32 bit games have a 2 gig memory limit. However Sins is NOT like other 32 bit games. Most games load their assets "on the fly", or only as needed. Some games unload what is not needed. Sins on the other hand loads almost ALL of its assets at game startup. These assets remain loaded throughout the entire game. The size of the map, and the amount of players, or AI's also determine how much ram will be used.

In 2006 when Sins was first conceived 2 gigs was considered to be "plenty" of memory. So most game developers didn't have to worry about hitting that dreaded 2 gig brick wall. As time went on Computers, and Sins itself evolved into their present state which is Diplomacy, and dual/quad core 64 bit systems. Things have changed over the years. Much more content was added to Sins with the two expansions. Most people now use 64 bit systems, and have much more than 2 gigs of ram. However Sins still remains a 32 bit game. With a two gigabyte hard coded memory limit. Sins WILL CRASH when the game reaches that 2 gig limit.

There is some misunderstanding about the 2 gig crash issue. The dump happens when Sins reaches 2 gigs of Ram Usage. That means the ram Sins itself uses. NOT what your entire operating system, and/or other programs you are running combined uses. Only the ram that Sins uses. No matter how powerful your system is. If it is 32, or 64 bit, or if it is Windows XP, Vista, or Windows 7. Sins WILL CRASH when it hits its 2 gig ram limit. It is hard coded into the game.

Nobody was aware that this would be a major problem back when the game was first designed, because Sins was designed based on the hardware standards of 2006-2007. Before 64 bit OS's, Quad Core cpu's, and 4+ gig ram systems became the standard that they are now. There is nothing that can be done about it. Except to re-write Sins in 64 bit. Which is out of the question, because it is too cost prohibitive for the developers. Sins is a great game, and it is a shame that this is happening. Entrenchment, and Diplomacy unintentionally pushed Sins over its own limitations with all of the new added content.

The Sins developers can not do anything to change the game at this time. However there is something that WE can do as modders to prevent Sins from hitting that dreaded 2 gig brick wall. I have taken the initiative, and applied to Sins what i have learned from my Homeworld Modding days. I present to you....

 

aka the "Taste Great, Less Filling" mod.

The goal of this project is to improve your Vanilla Sins game experience, and to eliminate the 2 gig crash issue. No changes will be made to Vanilla Sins gameplay at all. No stat changes will be made. Nothing will be added. There are other mods for that. We are optimizing what assets Sins already has to improve performance, and stability in game. The side benefit will be that some lower end systems can enjoy almost the same experience as the high end users.

The project initially started out as an idea to reduce the texture sizes, and poly counts of some models. Not just arbitrary reductions. The goal was to maintain the quality of Sins in game. To look like nothing has been changed at all. 

As of version 0.07 there are no longer any modded planet, and skybox meshes. Unfortunately the problems caused by the unfixable mesh errors from the poly reductions were not worth what little memory was gained from those optimizations. As i said the goal was to make it look like nothing has been changed at all.

Just with a few optimizations alone the games performance had a noticeable improvement. Though not much ram was saved.

I studied the Particle files, and textures. Many of the particle textures IMO were also way to high resolution. A reduction by 1/2 showed no visible difference in game in most cases.  There were some exceptions where it did effect the appearence in game, and those textures were left alone. Remember the goal was not to reduce in game quality. Also all of the particle textures were changed from TGA format to DDS format, because DDS uses far less memory, and is much more efficient. The particles also utilize the mip mapping features of DDS. Therefore enhancing in game video performance. As of version 0.07 no TGA format textures load in TSOP. With the exceptions of Mouse Cursors, and Scenario Pictures. This is due to those textures being controlled by the hard coded exe.

The developers mentioned something about a Memory Leak. I heard that a leak can be caused by a file calling for something over, and over again if it is not there. Example a mesh file calling for a texture that does not exist. Vanilla Sins has MANY particle, and mesh files that do that. I removed all of the redundant entries that pointed to non existent textures in the mesh, and particle files. I also removed the entries that have lines like "C;\gs\main\datasource\textures\effects\aura4.dds", and just put "aura4.dds". This will default the search to the mods texture folder, and speed things up. Also various typo's were corrected in some mesh, and particle files. Some ships have mesh nulls that were miss-spelled. For example the Akkan had typos on its engine nulls, and flairs. The engine trail, and flair didnt show up in game. the Marza was missing its bomb nulls. All of this has been corrected as well. If there is a ship mesh file in the mod then it had something wrong that was fixed. Regardless if this caused a memory leak, or not these errors had to be fixed.

After the Meshes, Particles, Textures, and the User Interface were optimized, and some very obvious bugs were fixed there was a tremendous boost in game performance. Sins ram usage was greatly reduced. As of version 0.07 no game i ran ever went above 1.5 gigs on Diplomacy with maximum settings. Including Bloom enabled. Tested on Huge random multi star maps with 9 AI's. The 2 gig crash problem is solved. Our primary goal with this project has been achieved! Without having to use some risky "EXE hack"!

Now TSOP is focusing on other ways to optimize the game to reduce late game lag as much as possible.  Also we are searching for more bugs as part of the Community Bug Fix effort.... It's a "Bug Hunt" now

NOTE: As of the latest Diplomacy series of patches v1.2+. Diplomacy no longer needs TSOP. Much of TSOP was incorporated into the 1.2-1.3+ series of patches. So the Diplomacy version of TSOP will no longer be linked here since it is now deemed obsolete.

_____________________________________________________________________________________________________________________

Our Website

http://soase.weebly.com/tsop.html


Some sections are still under construction

______________________________________________________________________________________________________________________

Downloads

The Sins Optimization Project v0.08

The User Interface is completely overhauled to use pure DDS format.

Trade Ships,  Strike Craft, Mines, and Constructor Unit meshes are Optimized.

Sounds, and Music are optimized.

All TGA format Textures are replaced with DDS format, and TSOP "force" loads them. The only TGA textures that remain are the Mouse Cursers, and Scenario Pictures. This is due to hard code in the EXE.

As part of the Community Bug Fix Effort, Many Mesh Null miss-labels were corrected, and Unnecessary Text Entry's were removed in both the Mesh, and Particle files. Entry's pointing to Non Existent Textures in both the mesh, and particle files removed. See the various "Fixed Files" texts in the mods Read Me for details.

The Diplomacy 1.2 graphic fix's have been incorporated into the Entrenchment, and the Original Sins versions of TSOP (Pipeline Effects, "White Line" error fix, and Corrected Mesh files).

As of TSOP 0.08 ALL textures have been reverted back to original Vanilla Sins Resolutions, but are all in DDS format. The Mip Mapping feature of DDS format "should" reduce any early game lag. Late game lag is still a problem, and is a CPU issue from too many units on the map in late game. Not a graphics issue. Graphic optimizations can not fix this.

As of the Diplomacy 1.2 patch, The issues that compelled us to create TSOP are now fixed! TSOP for Diplomacy is no longer required, and is now deemed "obsolete"! Many of the TSOP fix's, and improvements have been incorporated by Ironclad into the Diplomacy 1.2+ series of patches! We will no longer work on a Diplomacy version of TSOP. However, If we find something "game breaking" wrong with the new series of patches. Then we shall reconsider.

The Planets, Skybox's, Asteroids, Structures, Map objects, and various Other Ways to reduce the Late Game Lag issue will be looked into for any future updates.

Community input, and contributions are more than welcome, and they are encouraged! This is a Community Project! All can pitch in!

You do not need to download all versions of TSOP. You only need to download the version of TSOP for which version of Sins you wish to play. The Retail version of Trinity is just all three versions of Sins bundled into one package. Installation for Trinity versions of Sins are no different than if you purchased each version of Sins separately.

TSOP 0.08 For Original Sins v1.191

 

TSOP 0.08 For Entrenchment v1.051

 

TSOP For Diplomacy is no longer required. Since the Diplomacy 1.2 patch TSOP for Diplomacy is now obsolete.


Since TSOP Diplomacy is now obsolete. We will not host a "trinity" version.

However, when Rebellion is released we will look into what is needed (if anything at all), and perhaps make a combined version of TSOP in the future.

 

Add-On's, and Patches

TSOP Optional Lower Texture Resolution Add On

Optional Add On for those with VERY LOW END SYSTEMS. This Add-On "replaces" the full resolution TSOP 0.08 textures with the "strategically reduced resolution" textures from TSOP 0.07. Use this add-on ONLY if you suffer extreme graphic lag in the early game. If you still lag badly with this add on, chances are your system can not handle Sins. Use at your own risk.

Our Downloads page with optional Add-On's for TSOP

Read the Read Me's, and associated documentation for each version of TSOP, and add-on for important installation, and removal instructions. Plus other important information.

Sorry for the separate downloads. Together the project is over 600 megs with the sound, and music files, and most file host wont let you upload that much.  If your download is corrupt try downloading another version. The Mesh, Particles, Textures, Sounds, and Music files are all interchangeable between each version. Replace any corrupt files with another versions files.

_____________________________________________________________________________________________________________________

This project is far from a perfect solution, and it is still rough around the edges. It is improving as time passes.

Other mods are more than welcome to use TSOP. No questions asked! In fact we encourage it! Learn from it! Use it as guide, and an example! Merge it into your projects!

TSOP is intended to be a Stand Alone mod. Run with no other mods enabled. There are issues enabling TSOP with other mods. Enabling with other "Graphic Enhancement" mods is NOT recommended, because the graphic enhancement mod will defeat the entire purpose of TSOP. All of our optimized files will be overridden by that enhancement mods "enhanced" files. Increasing your ram usage instead of decreasing it. Some enhancement mods like Distant Stars are incorporating elements of TSOP into their future updates.

Read the included READ ME text's for installation instructions.

1102 Replies
Search this post
Subscription Options


Reason for Karma (Optional)
Successfully updated karma reason!
March 18, 2010 1:17:16 AM from Sins of a Solar Empire Forums Sins of a Solar Empire Forums

FYI...the recent patch created a new current Diplomacy folder: Mods-Diplomacy v1.011  (this moved locations from the original folder Mods-Diplomacy v1.01 that the mod was originally specified to go in).

I discovered this when the mod disappeaared from my mod enable choices menu after the patch.  I also had to place a copy prior to the patch in the generic Mods folder.  It would not list until I did.  Simply paste the Optimization mod folder from here into Mods-Diplomacy v1.011 and you are good to go (possibly into Mods as well if that doesn't do the job).

Reason for Karma (Optional)
Successfully updated karma reason!
March 18, 2010 2:50:49 AM from Sins of a Solar Empire Forums Sins of a Solar Empire Forums

The mod "should" have no issues with the patch just move it to the 1.011 folder and you should be ok.

For 0.02 ill make sure its 1.011 compatible. Crap Original Sins, and Entrenchment were updated as well. ill have to double check and make sure the particles were not updated. EDIT the update just fix's the desyncs so stats, and gameinfo shouldnt have been altered.

Reason for Karma (Optional)
Successfully updated karma reason!
March 18, 2010 6:54:59 AM from Sins of a Solar Empire Forums Sins of a Solar Empire Forums

I had no issues simply recopying the patches to the new folders.

Reason for Karma (Optional)
Successfully updated karma reason!
March 18, 2010 2:32:55 PM from Sins of a Solar Empire Forums Sins of a Solar Empire Forums

Is there only the puple looking Skybox working?

All the other ones are broken for me, same for the Desert Planet.

Reason for Karma (Optional)
Successfully updated karma reason!
March 18, 2010 4:09:19 PM from Sins of a Solar Empire Forums Sins of a Solar Empire Forums

All of them should be working, maybe the file u downloaded was corrupt

Reason for Karma (Optional)
Successfully updated karma reason!
March 18, 2010 6:19:48 PM from Sins of a Solar Empire Forums Sins of a Solar Empire Forums

The only thing that can be corrupt are the Archives and then they give me some Errors.

When the Files itself are corrupt then all or did he made for every Archive some new ones?

Reason for Karma (Optional)
Successfully updated karma reason!
March 18, 2010 6:46:26 PM from Sins of a Solar Empire Forums Sins of a Solar Empire Forums

ihated2, I would suggest downloading from the alternate link as the filefront copy also corrupted for me, but the alternate is fine

harpo

 

Reason for Karma (Optional)
Successfully updated karma reason!
March 18, 2010 7:54:03 PM from Sins of a Solar Empire Forums Sins of a Solar Empire Forums

What should be the stack order in v1.05?

The optimizer and than the desired mod or should it be last.

Reason for Karma (Optional)
Successfully updated karma reason!
March 18, 2010 8:02:24 PM from Sins of a Solar Empire Forums Sins of a Solar Empire Forums

 Stress Rocks

Reason for Karma (Optional)
Successfully updated karma reason!
March 18, 2010 11:00:06 PM from Sins of a Solar Empire Forums Sins of a Solar Empire Forums

I got a corrupted file from Fileshack the first time so used the altenate link on the other site and it was fine.

I am running a single Athlon XP 3000 cpu, 3gig of single channel dram, 512 graphics card and IDE hard drives. (This is an old Slot-A 462 AMD setup). 

The Optimization mod is already a HUGE improvement.  No more jerky motions in massive combats, no slowed, jerky scrolls over mass fleets and mostly clean shutdowns--not the long time out and  dump when a large, long game is ended.  No more stuttering audio when lots of effects and movement are happening.  I can't say how much is attributed to the patch but the Mod definitely improved things as I had installed it just prior.

I'd highly recommend the mod/patch combo.  Works wonders on older comps.  Can't wait to see the future slimming.

I am pretty good at audio editing (flash designer, radio, sound board and video production background).  Anything to be gained by slimming those files?  Would it help?  I haven't peeked at them in an editor yet.  happy to render some changes if you like.

Reason for Karma (Optional)
Successfully updated karma reason!
March 19, 2010 6:41:15 AM from Sins of a Solar Empire Forums Sins of a Solar Empire Forums

Quoting devancio,
What should be the stack order in v1.05?

The optimizer and than the desired mod or should it be last.

 

Thats where i loaded the Mod. =/

I will try it again ...

 

@SIN-Imperium: I dont think that the Audiofiles are a Problem - and i use the custom Sounds in Distant Stars too. Maybe there is a Problem for PCs without a Soundcard and a older / weaker CPU.

 

Edit: Err its down ...

Reason for Karma (Optional)
Successfully updated karma reason!
March 19, 2010 7:33:24 AM from Sins of a Solar Empire Forums Sins of a Solar Empire Forums

Quoting IHaTeD2,

@SIN-Imperium: I dont think that the Audiofiles are a Problem

I don't know how they load but I would think it would be pretty minimal too.

Reason for Karma (Optional)
Successfully updated karma reason!
March 19, 2010 9:57:31 AM from Sins of a Solar Empire Forums Sins of a Solar Empire Forums

I think the jerky audio is because that is one of the few things sins loads only as needed. Along with some particles. Now that i am working on them i can see some load immedietly and others load up only when necessary.

Whew sins has a LOT of particles. I am half done with thwm. Also got to the stars themselves. That was easy.

Sins has a bunch of missing textures for some effects too. Ill make a list of what i found missing that files are calling for later.

Reason for Karma (Optional)
Successfully updated karma reason!
March 19, 2010 10:48:02 AM from Sins of a Solar Empire Forums Sins of a Solar Empire Forums

I was wundering about the sounds and music. 

I am currently replacing all my music and sound files and beyond that events have to be in mono and music can be in stereo I was not sure of the quality. Now there very high (fairly large) quality and my system plays all them all fine but I would also like to know the best way for all systems please. 

Preview the theme to my new upcoming mod "Sins of Humanity(Ogg Vorbis sound file, length 57s, 387kbps). BSG's OST "Call to Arms" mixed with "Sins of Humanity" speech from Appleseed.

 

Reason for Karma (Optional)
Successfully updated karma reason!
March 19, 2010 10:57:16 AM from Sins of a Solar Empire Forums Sins of a Solar Empire Forums

I think the sporadic stuttering audio is becasue when the cpu is fully occupied, audio gets a lower priority than graphics and the like--so it chops when your processor is taxed.  The audio files aren't the big issue but I do optimize files in flash movies and it saves quite a bit of work for the viewer on large projects. 

I think we have ogg vorbis files here which are lossless, hq audio.  I haven't looked at the parameters and all but some games specify file types and compressions.  Have to experiment.

For audio to help a really minimal system, the best thing would be to just change most of them to simople tones to indicate the action.  that might actually help a really slow comp but not as much fun.

Reason for Karma (Optional)
Successfully updated karma reason!
March 19, 2010 11:12:26 AM from Sins of a Solar Empire Forums Sins of a Solar Empire Forums

Can u please update the Alternate Link?

Rapidshare only allows 10 Downloads for Free uploadet Files ...

Reason for Karma (Optional)
Successfully updated karma reason!
March 19, 2010 11:35:04 AM from Sins of a Solar Empire Forums Sins of a Solar Empire Forums

This is a fantastic mod concept, and I look forward to future optimizations.  I have a couple friends I play with whose systems start chugging hard mid-late game, so any relief at all is quite welcome.

Reason for Karma (Optional)
Successfully updated karma reason!
March 19, 2010 11:40:36 AM from Sins of a Solar Empire Forums Sins of a Solar Empire Forums

For audio to help a really minimal system, the best thing would be to just change most of them to simople tones to indicate the action. that might actually help a really slow comp but not as much fun.

Well I will never do work for a minimal system, more like the average gaming system.

Reason for Karma (Optional)
Successfully updated karma reason!
March 19, 2010 12:57:40 PM from Sins of a Solar Empire Forums Sins of a Solar Empire Forums

The Key is to optimize the Performance with a less in Quality (or maybe an improvement). Things like that are happend in The Elder Scrolls many times just because the Game isnt realy optimized.

Reason for Karma (Optional)
Successfully updated karma reason!
March 19, 2010 4:12:43 PM from Sins of a Solar Empire Forums Sins of a Solar Empire Forums

Greetings All,

 

Stress, you have done what I have been thinking about for a long time.

I have been doing some optimization of my own.  For example with the Asteroids.  Does anyone care if every asteroid they colonize is different looking?  I made them all the same mesh, saving 5-6 meshes and textures in the process.

I am also working on optimizing the particle effects, especially engines.  They are not that much different from each other, so I am trying to see if multiple ships can use the same particle, to optimize them.

Little things like this make a big difference.

 

Anyways, I am here to help if you need any.  We can work together on this if you want.  Just say the word.  Feel free to email me at: danman3712@gmail.com

DANMAN

Reason for Karma (Optional)
Successfully updated karma reason!
March 19, 2010 8:36:11 PM from Sins of a Solar Empire Forums Sins of a Solar Empire Forums

I am not doing this just for "minimal" systems... All systems will benefit from sins not crashing when it hits 2 gigs. High end systems will pretty much not see any change in the game at all. Except for the pole pinch on the planets (which i plan on fixing). The point to all of this is to create a "buffer zone" so that sins doesnt reach that 2 gig brick wall. Which Sins is hardcoded to dump when it reaches that point.

The side benefit is that low end systems will see a slight performance increase. Not a great increase but maybe a few more FPS. I am trying to keep the quality of original sins as intact as possible.

Danman i already got all of the engine trails optimized. Both fighter, and ship. However  I have not got to the Asteroids yet. I would like to try to keep them as they are, but maybe shave a few polys off, and nerf the non colonizable asteroids textures a bit. No need for 1024 textures for a decoration, or piece of debris.

With the particle textures optimized the ram usage dropped in original sins by a good 100 megs. This is with them half done. I havent got to the weapon particles yet, but i got the Capital Buffs, Capital Ability's, Damage Effects, Exhaust (engine trails), Explosions, Flairs (nav lights, and blinkers). I still have a long list of particles to optimize, but the process goes quick so it shouldnt take long.

Many particle textures were left unchanged, and just converted from TGA to DDS. With the particle file edited to point to that dds texture. Some particle files called for a dds, or tga textures that didnt exist, because they were in the opposite format. Must be typo's in the particle file They exist now, because all particle textures are now dds. It makes a huge difference when a missing texture finally shows up

There are a couple textures missing from the game itself that the particles call for. One is the "evasivemanuevers .dds" The other is the "weaponjamming.dds". Those 2 effect meshes call for these textures but they dont exist. If there is another particle that replaced these files then they shouldnt be in the game, because the game loads all this stuff wether it is used or not.

Reason for Karma (Optional)
Successfully updated karma reason!
March 19, 2010 10:47:56 PM from Sins of a Solar Empire Forums Sins of a Solar Empire Forums

Quoting Major Stress,
High end systems will pretty much not see any change in the game at all. Except for the pole pinch on the planets (which I plan on fixing). The point to all of this is to create a "buffer zone" so that sins doesnt reach that 2 gig brick wall. Which Sins is hardcoded to dump when it reaches that point.

I had started and run the game in debug for some time and noticed a list of texture/mesh errors that came with the vanilla files of Sins and Entrenchment (not sure about Dioplomacy).  I'm wondering f those are the things you had found.

I also am playtesting a custom map and got a shot of polar pinch showing on one of the backgrounds (skyboxes?).  Here is the link.  it isn't readily visible anywhere else--just on this particular spot which is usually under a planet object.

http://junkers.worldstrider.info/PolarPinch.jpg

Reason for Karma (Optional)
Successfully updated karma reason!
March 19, 2010 11:45:32 PM from Sins of a Solar Empire Forums Sins of a Solar Empire Forums

Good find. I didnt see that myself until you pointed it out, and yes that is polar pinch. Combined with bad tangents on the mesh. Ill see if i can get rid of that in the next update... no promises though.

All Ability's and buffs are now optimized.. Ram usage dropping like a rock

I am also reverting the planet meshes back to their original poly counts (using the gas giant mesh which is uv-mapped spherically). Hopefully that will help ease up on the pole pinch.

Reason for Karma (Optional)
Successfully updated karma reason!
March 20, 2010 12:16:09 AM from Sins of a Solar Empire Forums Sins of a Solar Empire Forums

I can live with it but a fix is sweet too.  Can you let us know when an updated version is actually available?

Reason for Karma (Optional)
Successfully updated karma reason!
March 20, 2010 1:02:12 AM from Sins of a Solar Empire Forums Sins of a Solar Empire Forums

I should be done with the particles tonight barring any major setback.

Reason for Karma (Optional)
Successfully updated karma reason!
Stardock Forums v1.0.0.0    #108434  walnut1   Server Load Time: 00:00:00.0000734   Page Render Time: