Major Stress Major Stress

[MOD] The Sins Optimization Project version 1.0 (updated 1/25/15)

[MOD] The Sins Optimization Project version 1.0 (updated 1/25/15)

Because all good things must come to an end.

It is difficult to explain the issues of Sins have without going into too much "technical detail" which some may not understand. The main problem is that Sins is an older generation 32 bit game. With a two gigabyte memory limit. This is nothing new. All 32 bit games have a 2 gig memory limit. However Sins is NOT like other 32 bit games. Most games have an ability to load their assets "on the fly", or only as needed. Some games can also unload assets that are not needed. This is how they prevent themselves from reaching the 2 gig limit. Some games have been patched to where they can use up to 4 gigs of ram on 64 bit systems.

Sins also can load only what is needed at game start up, but Sins does NOT unload what it does not use. Any assets that are loaded remain loaded throughout the entire game! If you use all three factions (Six in the case of Rebellion), Then Sins will load ALL of its game assets at start up. Entrenchment, and Diplomacy in some games loaded between 1.7 to 1.9 gigs worth of assets at game start up! Graphic settings, The size of the Map, and the amount of Players, or AI opponents also determine how much ram Sins will use in any given session.

In 2006 when Sins was first conceived 32 bit Windows XP was the standard OS of the time. Single core CPU's were also still the standard. Though dual core CPU's were starting to become mainstream, and 2 gigs was considered to be "plenty" of memory. So most game developers at the time didn't have to worry about hitting that dreaded 2 gig brick wall. As time went on Computers, and Sins itself evolved into their present state which is now Rebellion, and soon to be Windows 10 running in multiple core 64 bit systems. With 16+ gigs of ram. Things have changed so much over the years. However Sins still remained the 32 bit game as it was designed in 2006. That uses only a single core of your CPU. With a two gigabyte hard coded memory limit. Sins WILL CRASH when the game reaches that 2 gig limit.

There is some misunderstanding about the 2 gig crash issue. The dump happens when Sins reaches 2 gigs of Ram Usage. That means the ram that Sins uses by itself. NOT what your entire operating system, and/or other programs you are running combined uses. Only the ram that Sins uses. No matter how powerful your system is. If it is 32, or 64 bit, or if it is Windows XP, Vista, 7, 8, etc. etc. Sins WILL CRASH when it hits its 2 gig ram limit. It is hard coded into the game.

Nobody was aware that this would be a major problem back when the game was first designed, because Sins was designed based on the hardware standards of 2006-2007. Before 64 bit OS's, Quad Core cpu's, and 4+ gig ram systems became the standard that they are now. There is nothing that can be done about it. Except to re-write Sins in 64 bit. Which is out of the question, because it is too cost prohibitive for the developers to do this (Ironclad said so themselves). Sins is a great game, and it is a shame that this is happening. With the Entrenchment, and Diplomacy expansions Ironclad had unintentionally pushed the game over its own hard coded limitations with all of the new added content.

Over the years as of the release of Diplomacy 1.3, and Rebellion. Stardock, and Ironclad did indeed fix all of the major game crashing issues of Sins. Using the fix's from This Project as a guide. However, NONE of these fix's ever made it into Entrenchment, or Original Sins! While the Stardock, and Ironclad developers have fixed most of these issues over years. They can not do anything to change the game itself short of reprogramming Sins over from scratch. The developers can do no more, but there is something that WE can do as modders to prevent Sins from hitting that dreaded 2 gig brick wall, and improve the performance of Sins "just a little bit more". I have taken the initiative, and applied to Sins what i have learned from my Homeworld Modding days. I present to you....

 

The goal of this project is to improve your Vanilla Sins gaming experience, and to eliminate the 2 gig crash issue. No changes will be made to Original Sins, or any of its Expansions in game play at all. No stat changes will be made. Nothing "new" will be added. There are other mods that can do that. Original Sins will remain Original Sins, and the Expansions will remain as they are. What we are doing is taking what game assets that Sins already has, and we are "optimizing" them to improve the performance, stability, and to moderate the ram usage in game. The side benefit will be that some lower end systems can enjoy almost the same gaming experience as the high end users do (Especially Laptop Users).

The project initially started out as an idea to reduce the texture sizes, and poly counts of some models. Not just arbitrary reductions. The goal was to maintain the quality of Sins in game. To make it look as if nothing has been changed at all.

We started with optimizations to the Planet's, and Skybox's, because they were the "worst offenders" as far as ram usage, and performance was concerned. Unfortunately my modeling skills left much to be desired in Softimage XSI at the time, and there were un-fixable mesh errors on the optimized planet meshes. Despite that the "proof of concept" was established. A lot of ram was saved, and there was a very noticeable increase in performance.

Then we studied the Particle files, and Textures. Many of the particle textures in our opinion were way too high in resolution for the purpose that they served. A reduction in texture resolution by HALF showed no visible difference in game at all! There were some exceptions where the particle resolution reduction difference was very noticeable in game, and those textures were reverted back to their original resolutions. Remember the goal was NOT to reduce in game quality. Also all of the particle textures were changed from TGA format to DDS format, because DDS uses a little less memory, and is much more efficient, because DDS format loads directly into video memory. The particles can also utilize the mip mapping features of DDS format. Where TGA format does not. Therefore enhancing in game video performance (increased FPS). As of version 0.07 no TGA format textures load in TSOP. With the exceptions of the Mouse Cursors, and Scenario Pictures. This is due to those textures being controlled by the hard coded exe. When you change the texture format those textures do not show up in game.

The developers mentioned something about a Memory Leak. I heard that a memory leak can be caused by a file searching for something over, and over again if it is not there. Example a mesh file calling for a texture that does not exist. Vanilla Sins had MANY particle, and mesh files that had that issue. We removed all of the entries that pointed to non existent textures in the mesh, and particle files. We also removed the entries that have lines like for example "C;\g\main\datasource\textures\effects\aura4.dds", and just used "aura4.dds". This will default the search to Sins, or the Mods "Textures" folder, and speed things up a bit. Also various typo's were corrected in some mesh, and particle files. Some ships had mesh nulls that were Miss-Labeled. For example the Akkan had nulls, and flairs that were miss-labeled. The engine trails, and flairs didnt show up in game. the Marza was missing its bomb nulls. etc. etc. All of this has been corrected as well. If there is a ships mesh file in the mods "Mesh" folder then it had something wrong with it that was fixed. The Soundata, and Galaxydef files had errors too. Regardless if any of these issues caused any memory leaks, or not these issues MUST be fixed! (and they were as of Diplomacy 1.3, but NOT for Original Sins, or Entrenchment).

After the Meshes, Particles, Textures, and the User Interface were optimized, and the very obvious issues were fixed there was a tremendous boost in game performance. Sins ram usage was greatly reduced. As of TSOP 0.07 I could run a game of Entrenchment, or Diplomacy ALL DAY, and I NEVER used more than 1.5 gigs of ram! This was using Maximum Settings. Including Bloom enabled. Tested on Huge Random Multi Star maps with 9 AI's. The 2 gig crash problem has been solved! Our primary goal with this project has been achieved! Without having to use any external memory increasing programs! Stardock, and Ironclad took notice, and they used TSOP as a guide in the Diplomacy 1.3 patch, and for developing Rebellion.

Now that the Stardock, and Ironclad have fixed the issues of the game as of Diplomacy 1.3, and Rebellion. TSOP has been focusing more on Original Sins, and Entrenchment. Since the Ironclad/Stardock fixes never made it into Original Sins, or Entrenchment. However, That is not stopping us from optimizing Diplomacy, or Rebellion "just a little bit more" ;)

You can also use Large Address Aware to give Sins an extra gig to play with, but in vanilla Original Sins, and Entrenchment the Endless Search Loop issues that were never fixed will cause the game to eventually go over that extra gig (Run vanilla Entrenchment, and watch your ram usage keep going up, and up, and up). TSOP is a solution that will stop that issue.

____________________________________________________________________________________________________________________

Downloads

The Sins Optimization Project v1.0

For Original Sins 1.195

Sins Optimization Project 1.0 for Original Sins


For Entrenchment 1.055

Sins Optimization Project 1.0 for Entrenchment


For Diplomacy 1.37

Stardock, and Ironclad fixed all of the issues that compelled us to make TSOP. Therefore there is no TSOP for Diplomacy


For Rebellion 1.82

As with Diplomacy, Stardock, and Ironclad fixed all of the issues that TSOP pointed out, and then some in the case of Rebellion. Sins: Rebellion is as optimized as any Sins game is going to get.



The Sins Optimization Project for Rebellion

The Sins Optimization Project for Rebellion

NOTE: TSOP 0.07 is still hosted on ModDB, and it still works in the current versions of Trinity (OS, E, and D), but you will be playing as if Sins was patched to an older version of the game. Plus some Icons and Buttons wont be visible.


Changes in TSOP 1.0

TSOP 1.0 updated to the latest versions of Original Sins, and Entrenchment.

The User Interface is completely overhauled to use pure DDS format.

Trade Ships,  Strike Craft, Mines, and Constructor Unit Meshes, and Textures are Optimized.

 Starbase Textures  are optimized mimicking the Stardock Rebellion Optimizations.

Sounds, and Music are optimized.

All TGA format Textures are replaced with DDS format, and TSOP "force" loads them. The only TGA textures that remain are the Mouse Cursers, and Scenario Pictures. This is due to hard code in the EXE.

 "Strategic Texture Reductions" for the Particles return using newer methods to create a better looking texture.

(Original Sins, and Entrenchment)  Many Mesh Null issues, and various File Errors were corrected, and Unnecessary Text Entry's were removed in both the Mesh, and Particle files. Entry's pointing to Non Existent Textures in both the mesh, and particle files removed. See the various "Fixed Files" texts in the mods Read Me for details.

(Original Sins, and Entrenchment)  The Diplomacy 1.3  graphic fix's have been incorporated into the Original Sins, and Entrenchment versions of TSOP (Pipeline Effects, "White Line" error fix, and Corrected Mesh files).

 TSOP 1.0 for Trinity will see the return of Optimized Planets using the planet meshes from Rebellion! but with our Optimized Original Sins Textures!

There will be no version of TSOP made for Diplomacy, or Rebellion, because there is no need for it. Only Original Sins, and Entrenchment need TSOP, because the 2 gig issue, and various other issues were never fixed in those two versions of the game.

This is the FINAL version of TSOP that i will make. If the community wants to take over the project they are more than welcome to.

 

Community input, and contributions are more than welcome, and they are encouraged! This is a Community Project! All can pitch in!

Note: TSOP does not fix the known issue of "Late Game Lag". It does help prolong the lag until further into the game in some cases, but sooner, or later the lag will happen. The late game lag issue is a CPU issue due to the fact that Sins (including Rebellion) is optimized to use only single core processors. Therefore Sins will use only one core in multi core CPU's. To help with the late game lag issue we recommend the following:

Playing on smaller maps.

Using fewer AI opponents.

Disabling the Trade Ship icons (this actually helps out a lot)

Stacking the icons in the empire tree (same as with disabling trade ship icons the less icons rendered the better).

Using a Strike Craft Reduction mod (The many strike craft in the game are a big CPU hog. There are some good strike craft mods that reduce the number, but maintain the balance. Search for them).

You do not need to download all versions of TSOP. You only need to download the version of TSOP for the specific version of Sins you wish to play. The Retail version of Trinity is just all three versions of Sins bundled into one package. Installation for Trinity versions of Sins are no different than if you purchased each version of Sins separately.

_____________________________________________________________________________________________________________________

Other mods are more than welcome to use TSOP. No questions asked! In fact we encourage it! Learn from it! Use it as guide, and an example! Merge it into your projects!

TSOP is intended to be a Stand Alone mod. Run with no other mods enabled. There are issues enabling TSOP with other mods. Enabling with other "Graphic Enhancement" mods is NOT recommended, because the graphic enhancement mod will defeat the entire purpose of TSOP. All of our optimized files will be overridden by that enhancement mods "enhanced" files. Increasing your ram usage instead of decreasing it. Some enhancement mods like Distant Stars are incorporating elements of TSOP into their future updates.

Read the included READ ME text's for installation instructions.

2,238,262 views 1,134 replies +10 Loading…
Reply #276 Top

Hello just a quick simple question, I downloaded the Optimisation Mod, and intend to use it. But I have one question before I do, did you edit the String File in any way for use with this MOD? I have checked and the count number is still the same, its just that I have edited my String File a lot, and do not wish to loose the work I have put in to editing it. If you have edited the String File, then I will have to merge mine with yours, however if you did nothing to it, I can simply use mine.

Many thanks for the effort and hard work put into this MOD, I know how RL can prevent you working on such things, sometimes for months at a time. So BIG thank you from myself and everyone in the Modding community.

Kind regards, Ben

Reply #277 Top

The only part of the string i edited was the game title to show The Optimization Project instead of Sins of a Solar Empire. Nothing else was changed.

Reply #278 Top

Thank you Major Stress for your response, and that is fantastic news. I have already implemented your work into my MOD with zero errors found by the Developer program :)

 

Thank you again.

 

Ben

Reply #279 Top

There is a persistant crc fault in battle08.ogg in the Diplomacy version when using 7zip to extract redownloaded twice same problem.

Reply #280 Top

we know

look pg 11 - i share for now

entrenchment also missing 3 folders

 

Reply #281 Top

Hey all!

Sorry, I've been on vacation out in the badlands of Wisconsin, getting fat on cheese and sausage.  :drool:

 

But I am back now and glad to see the Major is only partially retired.  I will check into that trade ship issue, it may be the one where I did some creating on it... we shall see.

 

And I am working on the strike craft, I've got some done already... but I was waiting till I got them all done before sending them to ya.  Do you want what is done now Major?

Reply #282 Top

Quoting myfist0, reply 280
we know

look pg 11 - i share for now

entrenchment also missing 3 folders

 
End of myfist0's quote

 

Cheers your a star :)

Reply #283 Top

I looked over the advent trade ship, and it looks like there will not be very much reduction that can be done.  The design of this ship makes for a big problem without reworking the texture altogether.  So I'll just make sure to drop the polys on the advent strikecraft and extra amount....

 

Redone Trade vessle optimized for the advent.

Original - 3,288 Vertices, 2,716 Faces

Optimized - 1,384 Vertices, 2,316 Faces

 

It may only be 400 faces, but every bit counts i guess.

Reply #284 Top

400 x god knows how many ships = holy crap  :grin:

Reply #285 Top

Quoting SivCorp, reply 281
And I am working on the strike craft, I've got some done already... but I was waiting till I got them all done before sending them to ya.  Do you want what is done now Major?
End of SivCorp's quote

:puke: Loads of people are waiting on strike craft. I would say depending on how long before you think they will be done (no rush). If its the summer or more then we would like what you have. If sooner may as well wait for 1 new version. 

Nice to see your back. Hope you had a good vacation.  :grin:

Reply #286 Top

Glad to see you back Siv. I would re-uv map the model you already did, but like i said before i have zero free time. Which forced me into "semi-retirement".

Indeed every little bit helps.

Reply #287 Top

Quoting VirtuVerse, reply 285

Quoting SivCorp, reply 281And I am working on the strike craft, I've got some done already... but I was waiting till I got them all done before sending them to ya.  Do you want what is done now Major?
Loads of people are waiting on strike craft. I would say depending on how long before you think they will be done (no rush). If its the summer or more then we would like what you have. If sooner may as well wait for 1 new version. 

Nice to see your back. Hope you had a good vacation. 
End of VirtuVerse's quote

 

I did have a very well rested vacation, thanks.  And yes, I know many are waiting on strike craft.  I already have the Vassari done.  I will get the rest done very soon, maybe today if no work comes in...

cross your fingers :inlove:

 

And I'll look into re uv mapping the old optimized model... its just been forever the last time i uv mapped.  anyone have a little tutorial i can refresh on for sins? :pout:

 

 

Reply #288 Top

Running updates ----

 

Advent fighter and bomber are now done! 

 

Only Tec remains

Reply #289 Top

Strike craft are complete!

\o/

Here are the numbers for those who are curious...

Advent -

Fighter

original - 632 vertices, 640 faces

optimized - 256 vertices, 472 faces

Bomber

original - 582 vertices, 470 faces

optimized - 213 vertices, 390 faces

Tech -

Fighter

original - 1612 vertices, 1228 faces !!!

optimized - 471 vertices, 882 faces

Bomber

original - 944 vertices, 740 faces

optimized - 299 vertices, 594 faces

Vas -

Fighter

original - 692 vertices, 542 faces

optimized - 221 vertices, 438 faces

Bomber

original - 741 vertices, 618 faces

optimized - 229 vertices, 454 faces

 

I will send these to the Major, and they should DRASTICALLY improve late game performance, especially for the Tech

 

I'm going to work on the builder bots next, and then that should be about it.  Enjoy!

+2 Loading…
Reply #290 Top

Your da bomb!  :grin:

+1 Loading…
Reply #291 Top

Aw it was nothing... it only took me 1.5 hours.  Not too bad.

 

Reply #292 Top

Quoting SivCorp, reply 291
Aw it was nothing... it only took me 1.5 hours.  Not too bad.

 
End of SivCorp's quote

Only took the community what, 6 years. You and major will breathe new life into this game.

Have some karma for yur coffee

ooohh, [TOP] Guns could be a good name for a developer account but I don't think the major can stay away to long. He needs his fix.

Reply #293 Top

 

thank ya much!

 

I don't seem to get much Karma, getting into the double digits will be nice.

 

oh, on another note - I will be working on re UV mapping the advent mesh.  It will take a while though, so if you want to use the -400 mesh for now, you can.  I will get to that pesky advent soon enough....

+2 Loading…
Reply #294 Top

Quoting SivCorp, reply 293
I don't seem to get much Karma, getting into the double digits will be nice.
End of SivCorp's quote

 

That better :thumbsup:

 

We played a 4 player game with this last night and oh my god what a difference, I am one of the ones been suffering with cpu lagging later in the game aswell.

 

Also I been using ntcore to force sins.exe to use 4 gb also seems to help rather alot.

Reply #295 Top

Speaking of the 4Gb patch (http://www.ntcore.com/4gb_patch.php) has anyone else had success in reducing crashes with it?

Unfortunately I never get dumps so nothing to try for me.

P.S. Not to say this project is not totally awesome, I'm just looking for lazy ways to reduce dumps :)

 

Reply #296 Top

I also have never had sins actually crash from the 2gb limit.  Just crazy amounts of lag. 

 

And I know why you want the lazy way out JasonFJ, that great Star Wars mod is pushing the limit, isn't it?  Those models could use some major reductions too ;)

Reply #297 Top

tried v0.04 first time yesterday.  Wow the reduction in lag from midgame on was very noticeable!   Now I wonder how I've put up with the Sins of Solar Empire slideshow before! haha.  Awesome work! 

Is it possible to just copy over the modded files into the sins installed folder ? >.> this way it'll work even without loading mod? >.<

Reply #298 Top

the mod stays loaded after exit, so I dont see the reason to worry about it.  Just leave the mod running and your good as gold. :thumbsup:

Reply #299 Top

You never want to mess with base files.

Reply #300 Top

Quoting SivCorp, reply 287
And I'll look into re uv mapping the old optimized model... its just been forever the last time i uv mapped.  anyone have a little tutorial i can refresh on for sins?
 
End of SivCorp's quote

The SoaSE documentation comes with info about UV mapping and I made an online version for easy access and updating.

Getting Rid of Non-Tris and Non Quads  
Creating the UV Map  
Hooking-up Textures to the Model  
Creating the Tangent Map  
Setting-up Meshpoints  
Cleaning-up Your Model  
Exporting your Finalized Model  
Creating a Second Set of Materials

I also think people would like to know what you are using to edit SoaSE meshes. there was lots of older posts of people wanting to modifie the original models.

I have this so far but the texture sheets do no hook up correctly.