LoTeK_83 LoTeK_83

The Push for Higher Hardcode Limits

The Push for Higher Hardcode Limits

Hello Stardock and Ironclad!

We, the endless Sinners, the players of Sins of Solar Empire, the good ol' gamers, and the modding community as well would like to make our voices hear loudly in regard of a problem that is causing many headaches and stealing away many hours of sleep to some pious souls who daily bravely dare to undertake the task of creating really rockin' mods challenging the gorge of eternal patching.

Let this be told, in good and bad, we will always love ya...but all we wanted to say is:

"Push for more higher hardcode limits!!!"

Thank you,

Sinners! Sign this please.

 

 

46,345 views 78 replies
Reply #51 Top

Quoting ManSh00ter, reply 46
2GM limit is an OS limit, not the processor limit. Windows XP have a little switch you can use to enable them to allocate more than 2GB of RAM to an application (if you have more than 2GB, of course), I use it myself.
End of ManSh00ter's quote

If you're referring to PAE, as far as I know, all it does is make each program believe it has its own little 32-bit mapped section of RAM, so on a 32-bit machine with 4GB of RAM you can use all 4GB in a roundabout way. It's still limited to that 2GB until someone compiles a 64-bit build.

Reply #52 Top

I understood that PAE enables an application to address 1GB extra over the 2GB limit. Windows XP, as far as I know, has a 2GB limit on how much memory can be allocated to a given process - which is evident when you use application such as ZBrush and try to work with, say, 8 or ten million polyon models.

According to this page  certain application can be made "aware" of the extra available RAM. I may be reading it wrong, I've just read it helps with certain memory-intensive apps such as ZBrush.

Reply #53 Top

This discussion about PAE is completely irrelevant, basically if you have DEP (the windows kernel runtime protection) turned on (which unless you turned it off or you have no service packs on XP. you do) you have PAE enabled (right click my computer->properties and look under the ram information, it probably says 'physical address extension') but to Manshooters and carbon016 point the application has to be compiled to support it, otherwise it is just ignored.

Reply #54 Top

Quoting Carbon016, reply 51
If you're referring to PAE, as far as I know, all it does is make each program believe it has its own little 32-bit mapped section of RAM, so on a 32-bit machine with 4GB of RAM you can use all 4GB in a roundabout way. It's still limited to that 2GB until someone compiles a 64-bit build.
End of Carbon016's quote

 

In fact, since the Pentium Pro ( who is a very old ), processor are 36 bit ... PAE allow the use of the full 36 bits range on 32 bits OS... what happen with PAE is that the address range of material begin after the 4 gb limit OS limit... http://msdn.microsoft.com/en-us/library/aa366796(VS.85).aspx

 

For the 2gb limit, a other switch is used... http://msdn.microsoft.com/en-us/library/bb613473(VS.85).aspx ... this lead to 3gb for application... if i good remember, one of the first software to use these option fully was photoshop...

 

By the way, actual processor are not real 64 bits processor... the address range is 48 bits... planned range for future processor is 56 bits...

 

One details... for application be able to use the 3gb application range, the application need to be compiled with the linker flag /LARGEADDRESSAWARE when building the executable... very simple operation for Ironclad/Stardock since they have the source code...

 

About PAE, it is not only related to the processor... Hardware devices should be DAC-capable or 64-bit capable with LME drivers; otherwise, the device will function as "legacy" 32-bit and will be double buffered, with lower relative performance. Not really a problem for today computer but it was a problem when PAE was released log time ago... by example, old graphic card driver was not able to use address over the 4gb limit...

 

By the way, this is not really on topic, it is more a OS issue... Stardock/Ironclad cannot ask user of Sins to begin modify their boot.ini ( XP )  or registerkey ( Vista ) for allow their game to run at the top performance... never forget that the majority of computer's user don't know how work a computer, how work a OS, how to tune it... The standart Joe gamer will not understand what is PAE or 3GB... The standart Joe gamer have already difficulty to understand why they need to upgrade their driver, how to open a zip file, where to copy a mod, how to show they hidden folder, etc... Stardock/Ironclad cannot afford to loose these Joe gamer for please the hardcore gamer/modders... it is these numerous Joe gamer who allow Stardock/Ironclad to earn enough money for survive...

Reply #55 Top

We understand that, but we modders have no such qualms. If you're a mod gamer, you pretty much need to know how to work your computer anyway, rarely do you get mods which self install with a few clicks. Joe Users can keep on playing their vanilla games, all we ask for is that we get some room to flap our creative wings in... having these limits be adjustable from an external file would be great, since both the vanilla settings stay as they are meant to be by Ironclad, and we can modify them to suit our needs.

But at this point, I'd be happy just to see the mesh limit increased. That's number one issue I think.

Reply #56 Top

By the way, actual processor are not real 64 bits processor... the address range is 48 bits... planned range for future processor is 56 bits...
End of quote

sure it is! its just that you only get to use 48 bits for memory addressing.

bits actually refers to the length of the integer registers.

Reply #57 Top

signed, get this down for the life of a great game.

Reply #58 Top

Quoting crashmatusow, reply 56
sure it is! its just that you only get to use 48 bits for memory addressing.

bits actually refers to the length of the integer registers.
End of crashmatusow's quote

If what you say is true, there is no 64 bits Microsoft OS... All Microsoft x64 OS use the LLP64 data model : 16 bits short, 32 bit integer, 32 bit long, 64 long long, 64 pointer....

Linux/Unix OS use the LP64 data model : 16,32,64,64,64 bits ...

Only HAL OS using the ILP64 data model : 16,64,64,64,64 bits ... have a real 64 integer unit...

There is a big difference between what a hardware can and cannot ... and what a OS can and cannot... since game is based on the hardware and OS, only common thing can be used... So, for the time being, forget about 64 bit integer with Windows, Linux, MAC, OpenSolaris, etc ... operating system...

Reply #59 Top

It's a switch you added to the boot.ini file in WinXP using the /3GB switch.  I'm not quite sure where it is located in 32bit Vista off hand though.  If I recall you can force it on by enabling DEP in a certain way or something.  It's been quite a while since I looked into as I switched to 64bit Vista quite a while ago.

There is a catch though.  Enabling 3GB isn't all that helpful unless the application process is also set up to run as a 3GB process.  With SupCom for instance, an enterprising community member created a little utility that mofified the executable to add the 3GB enabled flag in the .exe for those who did tune Windows to use the larger memory space.

I admit though, I am a bit fuzzy on the details of all this now.  As I said, it has been a while since I worried about it.

As for the thread topic... Here, here.  Up the limits please.  I personally haven't hit them yet (or so it seems) but more expandability and moddability is always better.  If it were not for modding I'm sure I would have left Sins ages ago.

-dolynick

Reply #60 Top

yes, these limits need to be increased. The mesh one seems to be the biggest issue, however there are some other important ones also listed above.

Reply #61 Top

Just a Thought with the current limits in place and the planed diplomacy expansion coming they will most likely exceed there own limits!!!  *_* :thumbsup:

Reply #62 Top

Quoting JTAYLORPCS, reply 61
Just a Thought with the current limits in place and the planed diplomacy expansion coming they will most likely exceed there own limits!!! 
End of JTAYLORPCS's quote

Don't think so... but it is maybe the reason why the 3th expension was canceled... no more working room for create a last expension... 

Reply #63 Top

Quoting Thoumsin, reply 62



Quoting JTAYLORPCS,
reply 61
Just a Thought with the current limits in place and the planed diplomacy expansion coming they will most likely exceed there own limits!!! 



Don't think so... but it is maybe the reason why the 3th expension was canceled... no more working room for create a last expension... 
End of Thoumsin's quote

 

Maybe so , Maybe not  but , I guess the real question is can they up the limits?????????

Reply #64 Top

thsis is an idea that probably will not be in sins/entrenchment/diplomacy, but would help with sins2

ie seperate the textures from the meshes ie the texture and size are  controlled from the entity, that way mesh IS ONLY the shape, and can be used by MANY items eg suns, large planets, meduim planets,small planets & moons, and can have several factions that have completely different textures, but the same basic shape.

the main penalty that I see is greater difficulty in alligning the texture to the meshes, but with for example 1 mesh for 4 suns, 6 colonisable planets, the gas giants, it would save at least 10 mesh locations in this example, but would require a re-design of the engine so is unlikely to happen in sins/entrenchment/diplomacy.

harpo

 

Reply #65 Top

It cant possibly be THAT difficult to change a number. If they cant make the limits external in a config file. Then they can at least bump the number from 400 meshes to 800. How hard can that be? The vanilla meshes will still be at the number originally planned. If we push over the mesh limit then oh well. That is our problem. If they do increase the limit then at least we KNOW what that limit is.

I dont give a damn about the textures. There are many ways to get around that limit without losing detail, or screwing up your system. The Homeworld mods (HW1 not HW2) are a shining example of how that can be done.

All the devs have to do is open up visual c++, or basic, or whatever they used to code this mod killing limit in, and change a number... How hard is that?

Reply #66 Top

Quoting harpo99999, reply 64
the main penalty that I see is greater difficulty in alligning the texture to the meshes, 
End of harpo99999's quote

Not really since UV are stored in the mesh, texture align themself... scale up or down in a proportional way is not a problem too...

Reply #67 Top

Quoting Major, reply 65
It cant possibly be THAT difficult to change a number.
End of Major's quote

It can be... if all memory for array is reserved at the begin, doubling it will double the memory use... these array are like empty box... need a bigger box for store more data...

If data array are reserved in a dynamic way, it is not a problem...

Only Stardock/Ironclad have the source code of sins, so it is impossible for us to know if double a "number" can have some influence on the rest...

Maybe having the limit in some ini file is the only real possible way... change them from hardcoded to non hardcoded... so, if modder mess up the game, it is their sole responsability...

Reply #68 Top

Quoting Thoumsin, reply 67

Quoting Major Stress, reply 65It cant possibly be THAT difficult to change a number.
It can be... if all memory for array is reserved at the begin, doubling it will double the memory use... these array are like empty box... need a bigger box for store more data...

If data array are reserved in a dynamic way, it is not a problem...

Only Stardock/Ironclad have the source code of sins, so it is impossible for us to know if double a "number" can have some influence on the rest...

Maybe having the limit in some ini file is the only real possible way... change them from hardcoded to non hardcoded... so, if modder mess up the game, it is their sole responsability...
End of Thoumsin's quote

 

I don't think these guys are declaring fix sized arrays for this kind of data when they load up the game, lol, you can see they use vectors when you crash the game for example. (heh, "Mr C++, give me an array of exactly... Oh - why not eh? ONE HUNDRED MILLION DOLLARS *cough* sorry, I mean bytes").

The real reason you can't just go change numbers willy-nilly is that you don't want to mess with dependency. (or you may encounter face-melting bug horridness, however bad the coding practice it still happens when time needed > time avalible).

 

I doubt there would be any real problems if they put the numbers up to silly levels, but that requires MOAR TESTING I guess.

Reply #69 Top

I can understand that "if' we are talking about texture memory. Textures load in terms of MB's. a 2048x2048 texture can use up to 20 MB's (the average useage it reads in photoshop for me). Multiply that by 3 if you include the -da, and -nm textures. Then by how many ships use that size texture. Meshes themselves only use a fraction of memory. Im talking KB's. Every mesh in the game right now couldnt use more than 100 megs if we loaded every single one of em up. I could be wrong. hey i am a modeler not a programmer. Each game renders stuff differently. I guessed at the low end because sins was designed to run on low end machines.

Reply #70 Top

Brad has often said that Gal Civ will scale to take advantage of the ever increasing capabilities of the PC, I would think/hope that philosophy would influence projects that he is involved with.

Yes, Gal Civ – Sins two different games, two very different engines but the philosophy ought to transcend the engine.

Reply #71 Top

except you're talking about  two different engines/games/AND companies. i doubt brad has even looked at much of the source code for sins except for maybe to trim through for some coding tricks to pick up on or for a proof of concept build.

Reply #72 Top

I don't think rendering should have an issue with the number of stuff loaded into memory. Whatever an engine does, it can only render something which actually exists in the game world. Meshes loaded into memory but not present in the game world should not affect rendering performance at all, unless the engine does some crazy stunt like "behind the scenes" rendering, which would be ridiculous since it serves no purpose.

So yeah, I don't think extra meshes would impact the application footprint that much, especially when compared to how much mem goes for textures.

Reply #73 Top

Doing my bit for the cause.... I'd love to see a configuration file that would allow editting of previously hardcoded limits.

What I'd also like to see if weapon types customized per race. Having different research paths for weapon types is great, but having them called the same across all races is cosmetically displeasing.

Reply #74 Top

Quoting Carbon016, reply 26
A related but annoying issue is some of (I say "some of" because while I've encountered one so far, I'm sure there's more) the hardcode limits on strings. For example, go to the following string in English.str:

StringInfo
    ID "IDS_GAMEMESSAGE_PLAYERCAPITALSHIPLOSTDETAILS"
    Value "Intelligence reports the loss of the level %d %s near %s"

While many other strings can be edited, sometimes removing the variables, this one can't. In fact, if you try to edit this string to the following:

StringInfo
    ID "IDS_GAMEMESSAGE_PLAYERCAPITALSHIPLOSTDETAILS"
    Value "Intelligence reports the loss of %s near %s"

..the game will instantly crash with a dump upon destroying a capital ship. Since these sort of things usually happen with GameInfo, only Beyond Compare saved me from a lot of busywork - once I had used an old copy of my mod to sub the String directory in, it was narrowed down with some difficulty.

While the fact that certain variables are passed into these strings and removing them might be a little troublesome at least makes a little sense (although it's kind of stupid, because people should have the flexibility to edit these sort of things), interestingly there's another quirk. Even if you keep the variable but place it elsewhere in an attempt to obfuscate it:

StringInfo
    ID "IDS_GAMEMESSAGE_PLAYERCAPITALSHIPLOSTDETAILS"
    Value "Intelligence reports the loss of %s near %s (%d)"

The game will crash. This seems like a hard-code problem to me in the way these things are being passed, probably specific to this one string. I'm a pretty rudimentary programmer, but I assume a function is passing these things in order as a integer, then two strings (%d, %s, %s), and so when they're rearranged, the integer is being passed as a string, and (more importantly) the string is being passed as an integer, causing a crash. What's weird though is that this string seems to behave a lot weirder than any others in this respect, because a lot will let you remove %s or %d and function with no problems.

It's difficult to make mods intuitive when you can't stop them from blasting base strings, and it's more difficult when troubleshooting them is an hour of trial and error.

 
End of Carbon016's quote

 

I guess the do a sprintf (cheap, really :grin: ) - it would explain the syntax. It's the easiest way to do it, but once u have a termination problem ( no null termination ) you would read (and "execute") the wrong memory adress and your program crashes. so, better not touch it!

 

And Yes, Up the Limits!!! Especially meshes, textures particels and WEAPONS PER SHIP!

Reply #75 Top

UP THE HARDCODE LIMITS !!!! RESISTANCE IS FUTILE !!!!!!!! 

:borg: