4GB+ flag on Rebellion like Skyrim? » Forum Post by DavidtheDuke

Nope. Would require an engine modification which is still not going to happen for Rebellion.

Maybe for Sins 2.

Successfully updated karma reason!

that's poor...this game needs more RAM so much as it gets incredibly laggy in huge fleet battles

Successfully updated karma reason!

lag is related to

1 CPU speed

2 RAM speed

and a poor third IS

3 amount of ram

if you hate LAG then boost the first two (overclocking the cpu helps for #1, for #2 SPEND a LOT of money of the FASTEST ram you can find for the motherboard)

harpo

Successfully updated karma reason!

Quoting JA_394,

reply 1

Nope. Would require an engine modification which is still not going to happen for Rebellion.

Well, have not yet test it for Rebellion but with sins/ent/dip/tri unmodified .exe, it is possible to use a max of 4gb if you kill the Microsoft OS and move to a other OS who allow you to fully enjoy the power of the game... for the non-believer, look below :

If you have a server board with 2 or more CPU ( need each CPU sharing the same memory ) who have a hardware speculative multithreadding capabilities, it is possible to assign multiple thread to the game... my case is 2 CPU and 2 thread ( with 8 CPU, it can be 8 thread )... the amount of core change nothing, it is a per processor threading and not a per core threading... in case of online play, these hardware speculative multithreading lead always to a direct desync...

Once Kubuntu 12.04 LTS and Rebellion are out of beta stage, i will test Rebellion on Linux and maybe make a topic about it... there is plenty of people with dual boot around and IF Rebellion perform better on Linux, why not inform people about it... move from Impulse to Steam is already a positive move for Linux user since Impulse have never work on Linux but Steam ( with playonlinux ) work perfectly...

As Rebellion itself, on win xp pro x64, the game usualy use 1-1.1 gb ram only... huge random multistar map with 10 player can reach in late game, 1.5gb ram use during huge battle with hundred if not thousand ship... Stardock have make a very good work at optimizing the memory use... not yet know any visual slowdown and i have a slow CPU ( 2.66 ghz ) and old ram ( FB-dimm ddr2 667 )... i think that the 2gb crash have become history with Rebellion ( until mods are released for Rebellion )

Successfully updated karma reason!

I don't think the issue is the memory it self is the problem, it seems there is desyncing/memory leaking somewhere else.

Anything from Trade-ships or Ship Abilities being used can cause these issues.

Successfully updated karma reason!

Well, I have 16GB of RAM and don't use a pagefile. I would be willing to help test out any such flag. It seems pretty basic to me, just put the same flag on the .exe, not saying that I actually understand how to. Is it too technical of a reason why it can't be done for SOSE?

Successfully updated karma reason!

I've used the 4 GB flag with Sins for years...i've been over 2 GB on 10 players games. It works.

Successfully updated karma reason!

I use the 4gb flag, I also play with SHOWERRORS TRUE, so instead of crashing I can continue. Since I fixed my issue I haven't minidumped once.

Successfully updated karma reason!

Where/how do I install/use this? Do I have have to modify the hex or something (I have no idea how to )?

Successfully updated karma reason!

I can email you a simple program that will add the flag...send me a PM.

Successfully updated karma reason!

I don't understand, my sins doesn't crash at all. Of course my sins doesn't even reach 2gb maybe 1gb. That with a huge map with over a thousand ships on my faction alone.

Successfully updated karma reason!

Quoting minidump-hellboy,

reply 7

...i've been over 2 GB on 10 players games. It works.

Well, your picture don't show that it work... you have 202457x kb ... let say that it is 2024579 kb... /1024 = 1977.127 mb ... /1024 = 1.93 gb ...

Show a picture where you are over 2097152 kb and it will be the proof that you have break the 2gb "limit"...

Successfully updated karma reason!

Timmaigh

Join Date 05/2009

+76

Quoting Thoumsin,

reply 12

Quoting minidump-hellboy, reply 7...i've been over 2 GB on 10 players games. It works.

Well, your picture don't show that it work... you have 202457x kb ... let say that it is 2024579 kb... /1024 = 1977.127 mb ... /1024 = 1.93 gb ...

Show a picture where you are over 2097152 kb and it will be the proof that you have break the 2gb "limit"...

Very good. Could be fooled myself.

Successfully updated karma reason!

listen up here genius. It goes over that.

Quoting Thoumsin,

reply 12

Quoting minidump-hellboy, reply 7...i've been over 2 GB on 10 players games. It works.

Well, your picture don't show that it work... you have 202457x kb ... let say that it is 2024579 kb... /1024 = 1977.127 mb ... /1024 = 1.93 gb ...

Show a picture where you are over 2097152 kb and it will be the proof that you have break the 2gb "limit"...

listen up here genius. It goes over that. I've seen it as high as 2.3 GB. You're good at math, but you fail common sense. If I take the flag off, it crashes. SupCom will crash @ 1.98 GB because that's the limitation. Use the flag, I've been up to 2.6 GB in SupCom with about 10,000 units in play.

The version of the 4 GB patch I use, is packaged for SINS you moop. With the Maelstrom Mod. It's an excellent mod. I've spoken with the author of the mod on several occasions. He wouldn't package it if it didn't work.

Successfully updated karma reason!

Quoting minidump-hellboy,

reply 14

You're good at math, but you fail common sense.

Please, explain !!!

A binary kilo was always 1024... sure, some people have manipulate these fact, like when harddisk have stop to use the binary kilo for use the metric one... allowing them to sell harddrive with a lower capacity that the featured one...

Common sense dictate by example that you don't mix land mile ( 1.609 km ) with the nautical mile ( 1.852 km )... same thing with the metric kilo and the binary kilo...

Quoting minidump-hellboy,

reply 14

If I take the flag off, it crashes

On windows OS... without flag, i can use 4gb on a other OS... and with flag, i was able to reach over 11.5gb before a crash on the same other OS...

All these so called limit are operating system limit, no physical one...

It is the same thing that 32 bit vs 64 bit... the funny thing is that since pentium pro from 1995, hardware have be 36 bit, good for 64 gb ram... look at http://en.wikipedia.org/wiki/Physical_Address_Extension#Microsoft_Windows and you will see that some 32 bit windows OS can have plenty of ram... sure the desktop 32 bit version is limited to 4gb ( by license reason ) where there is ( 2gb for application and 2 gb for OS ) but it is not true for real professional version ( server, datacenter, etc )... more funny is that the so called "x64" was in fact 40 bits in the first "x64" processor... actual "x-64" processor are 48 bits... at max, the "x-64" architecture can only be increased to 52 bits in futur generation processor... "x-64" architecture will never be able to reach the 64 bits !!!

A lot of thing about computer are in fact commercial trick for push people to buy something new or to sell less for the same price...

Successfully updated karma reason!

What you're neglecting to mention is that running PAE on a 32 bit OS to get that 36 bit memory addressing had significant performance penalties that don't exist in x64. As for the number of bits, that's because it's cheaper to make hardware with only 40 lines for the memory address instead of 64, particularly since there's no PC out there that can use over 1TB of RAM anyway.

And none of that has anything to do with the developers turning on the compiler option to allow the game to be large address aware, so I'm not sure why you'd even bring it up.

Successfully updated karma reason!

Quoting Tridus,

reply 16

What you're neglecting to mention is that running PAE on a 32 bit OS to get that 36 bit memory addressing had significant performance penalties that don't exist in x64.

First, 86-x64 ( AMD64 or/and intel EMT64 ) is the same thing that PAE... well, in fact, it is PAE who have be extended... take a look at http://support.amd.com/us/Embedded_TechDocs/24593.pdf , page 52 of 566, physical memory section...

Quoting Tridus,

reply 16

As for the number of bits, that's because it's cheaper to make hardware with only 40 lines for the memory address instead of 64, particularly since there's no PC out there that can use over 1TB of RAM anyway.

As for the 40 to 52 lines, it is not related to the hardware but to the concept of 86-x64, the architecture itself... as for computer using 1TB or more, look at the altix 3700 ( http://techpubs.sgi.com/library/tpl/cgi-bin/getdoc.cgi?coll=linux&db=bks&fname=/SGI_EndUser/Altix3000/ch03.html ) who allow 2TB and was discontinued in 2006... more recent is the altix UV who can use 16TB of memory...

Sure, they are not home desktop but indirectly, you use computer like them... internet hosting use "farm" of so computer for web hosting... your electronic banking use similar computer... etc...

Quoting Tridus,

reply 16

And none of that has anything to do with the developers turning on the compiler option to allow the game to be large address aware, so I'm not sure why you'd even bring it up.

True if you have a 64 bit windows OS who will use WOW64 who reserve 4gb by 32 bit application... but for a 32 bit OS, LAA alone is not enough... first PAE for have your full 4gb of ram at your disposal ( address range of material is pushed over the 4gb limit )... second, the 3gb boot.ini switch who change the 2-2 to 3-1 ( 3 gb application - 1gb OS )... finally, exe with LAA enable for the use of the 3gb for application...

by the way, it is not a compiler option but a execution flag...a running time, it simply say to the system that it can use large address...

Sure that around 50% of people are now using a x64 OS, that for them, it will be very easy... but the is always the other half who are using 32 bit OS... in so case, for a working LAA, it is not so simple ( but possible )...

Finally, remember that a business will try to release software who can run on the maximum of system... what is best for the devs wallet? 50% of people who have 64 bit or "50% 32 bit + 50% 64 bit"...

By the time that sins II is released ( in a few years ), a huge majority of people will have more to 64 bit and then i am pretty sure that the game will be a native 64 bit game ( who will not run on 32 bit system )...

Quoting DavidtheDuke,

reply 9

Where/how do I install/use this? Do I have have to modify the hex or something (I have no idea how to )?

first post at http://www.techpowerup.com/forums/showthread.php?t=112556

Successfully updated karma reason!

TobiWahn_Kenobi

Join Date 08/2008

+83

@Thoumsin

What I still don't understand technically (as a dev) is how can sins address more than 4gb with just 32 bit pointers? Are they substituted for 64 bit pointers while executing?

If I set a pointer in a 32 bit application to a specific address space I do only have access to a limited addressability. How does your setup overcome this?

PS.: Wheres the Vorlon Planetkiller?

Successfully updated karma reason!

Quoting JA_394,

reply 1

Nope. Would require an engine modification which is still not going to happen for Rebellion.

Maybe for Sins 2.

...No, it wouldn't. It's merely a patch on the .exe that you can apply, and people have indeed applied to Trinity.

Successfully updated karma reason!

Quoting TobiWahn_Kenobi,

reply 18

What I still don't understand technically (as a dev) is how can sins address more than 4gb with just 32 bit pointers? Are they substituted for 64 bit pointers while executing?

first reading : http://www.geoffchappell.com/notes/windows/license/memory.htm ... from the paragraph up the "physical address extension" title to "PAE is a ugly hack?"... it somehow explain the paging system... a little like a harddrive where data is stored in block of a determined size and a partition table is used for read/find the various block who compose a specific data...

the hardware who control all this is called MMU ( http://en.wikipedia.org/wiki/Memory_management_unit )... in modern processor, all is integrated in the CPU...and remember, since the pentium pro, processor have 36 bits and not 32 bits...

A other method is PSE-36... almost never used but a good idea : http://en.wikipedia.org/wiki/PSE-36

A other method that i have not yet test with sins is called Address Windowing Extension ( http://msdn.microsoft.com/en-us/library/aa366527.aspx )... it is mainly use on huge server, when the amount of physical memory is several time bigger that the address range... by example server 2003 datacenter 32 bit who can use 128 gb ram when the register is only 36 bit ( max 64 gb )... virtually, you have no limit with AWE... the only limit is that table ( like the FAT of a harddisk ) are keep in the first 4gb memory, so the real limit is the size of the table stored in the first 4gb... from http://msdn.microsoft.com/en-us/library/aa366796.aspx : "AWE does not require PAE or 4GT but is often used together with PAE to allocate more than 4 GB of physical memory from a single 32-bit process."

Sure that there is other method who can extend the memory used by a process on windows but since i am not a windows guru, i don't know everything...

Now, with Linux, it is something other... the launcher is a process, the game is a process, playonlinux is a process, wine is a process, etc... and there is fundamental difference with opengl and directx... if you have plenty of graphic memory, opengl will cache the texture in the graphic memory in place of main memory... it free main memory and speed up everything... the catch is that if you don't have enough graphic memory, data is send back to the main memory, who add some latency... there is the fact that the text ( execution code ) is very small when compare to all the data... code work badly with a page system, best to have it remain in the first 4gb... data ( mesh, texture, sound, etc ) can be store in block... don't need to know the address of each byte of data... knowing the start address and the size of the data is enough...

In short, memory management is something complex and i am not sure that i understand everything... anyway, devs don't need to worry too much since it is the CPU who make all the work... along with the compiler and a modern programming language... nobody program directly in assembler language or use direct memory writing/reading like the old poke/peek from the old basic...

Quoting TobiWahn_Kenobi,

reply 18

PS.: Wheres the Vorlon Planetkiller?

Waiting the end of the construction of the B5 station... who itself wait more technical info about Rebellion... maybe the B5 station will become a non mobile titan in Rebellion... i need the 5 weapons type that the titan support in place of the 3 weapons type from the starbase/capitalship... not that the B5 station have a huge amount of weapons but it have more that 3 weapons type... will know more once we have the convert.exe for rebellion who make the entity/mesh possible to read with notepad... by the way, i am really busy now with real life ... hmm ... testing rebellion...

Successfully updated karma reason!

minidump-hellboy,

Try playing all on highest, biggest map possible with 10 players. Generate the map yourself if you have to, get 200 planets in there, go nuts for donuts with it. Try pushing mem limit to 3gb.

Until the devs recommend a method for win7 64bit systems, deal with it or get Linux with Wine or whatever it is that Thoumsin has going.

Successfully updated karma reason!

Quoting SemazRalan,

reply 21

minidump-hellboy,

Try playing all on highest, biggest map possible with 10 players. Generate the map yourself if you have to, get 200 planets in there, go nuts for donuts with it. Try pushing mem limit to 3gb.

Until the devs recommend a method for win7 64bit systems, deal with it or get Linux with Wine or whatever it is that Thoumsin has going.

lol..i'm on a Mass Effect 3 kick right now. I've been playing a lot Sins lately... I just might fire up a custom map and see what happens.

Successfully updated karma reason!

Quoting SemazRalan,

reply 21

Try playing all on highest, biggest map possible with 10 players. Generate the map yourself if you have to, get 200 planets in there, go nuts for donuts with it. Try pushing mem limit to 3gb.

Everything on highest, my own multistar random map ( 10 star, 4 pirates base by star, over 350 planet, 10 player )... so, i have try but after more that 10 hours play, Rebellion is just a little over 1.5 gb... can maybe try more but i think that CPU will go nuclear before reaching the so called 2gb memory limit...

Successfully updated karma reason!

TobiWahn_Kenobi

Join Date 08/2008

+83

Quoting Thoumsin,

reply 20

Waiting the end of the construction of the B5 station... who itself wait more technical info about Rebellion... maybe the B5 station will become a non mobile titan in Rebellion... i need the 5 weapons type that the titan support in place of the 3 weapons type from the starbase/capitalship... not that the B5 station have a huge amount of weapons but it have more that 3 weapons type... will know more once we have the convert.exe for rebellion who make the entity/mesh possible to read with notepad... by the way, i am really busy now with real life ... hmm ... testing rebellion...

Damn real life. It always interferes with fun! A non mobil titan would also ensure that there is only 1 babylon station. Nice idea. I'm so anxious to get the new convertdata into my hand (well, more importantly, the new convertXSI!!! The old one doesn't work anymore with rebellion)

But Yarlen told me that there will be no tool love for beta1... Sad sad life

Quoting Thoumsin,

reply 20

data ( mesh, texture, sound, etc ) can be store in block... don't need to know the address of each byte of data... knowing the start address and the size of the data is enough...

Interesting. From a developers perspective that might be debugging hell if something gows awry.

Successfully updated karma reason!

Quoting TobiWahn_Kenobi,

reply 24

Interesting. From a developers perspective that might be debugging hell if something gows awry.

Well, some history first... early time of computer, something like ramdisk.sys was invented for use more ram that CPU was handle... the extra ram was used as a drive for speed up everything...

Later, with 32 bit OS, there ramdisk was forgotten due to CPU able to handle plenty of ram address range... people was short in ram and virtual memory using harddrive was used in case of shortage of real ram...

Today, you can have plenty of ram... ramdisk trick can become interesting again... in some way, it is how work sins... almost of constant data ( mesh, texture, etc ) are cached in ram for fast access... unfortunaly, the caching increase the size of the main application thread... will not be the case for a real ramdrive because OS handle it, like a real super fast drive... imagine a launcher.exe who initialize a ramdrive in the memory address range between 4gb and 64gb on a 32 ( 36 ) bit computer, copy all the game folder there and launch the sins.exe who have everything set on "loadondemand"... result will be a application running in the lower address range ( below 4gb ) using only a few mb... since all data is allready in memory, on the fly reading will be fast...

Well, today, computer don't use ramdisk but the basic principle of page files is similar... at the begin, a pagefile block was 4 kb... with PAE, 2 mb... usual x64, 4 mb ... some processor are ready for 1gb block ... now, processor use PAT who allow to work with various size block, the PAT thing is very similar the the FAT thing from harddrive...

As today, memory managment is something complex, who is handle by the processor or/and the OS... after all, you don't need to be a mechanical expert for drive a car!!!

Just for fun, try this on your MAC ( a shell script, very similar to a .bat on windows OS ):

#!/bin/sh
NUMSECTORS=128000
mydev=`hdid -nomount ram://$NUMSECTORS`
newfs_hfs $mydev
mkdir /tmp/mymount
mount -t hfs $mydev /tmp/mymount

this create a ramdrive with a capacity of 128000*512 bytes, format it, create a mount point, and mount the drive... result is a new drive who is faster that any other drive, a SSD is slow compared to it... for use it, you don't need to worry where are the data in ram, everything is handle by the system/processor... similar way with the pagefile system... devs don't need to remake a OS and use function already supported by the OS...

Well, it is really off-topic... use google and you will find plenty of book, site, etc related to memory management...

for example, you will find that garbage collector is a great memory management tool... but will discover that a microsoft bug who exist for year ( since 2005 and win xp pro x64 ) in WOW64 make the 32 bits application who use it crash on 64 bits OS ( http://social.msdn.microsoft.com/Forums/en/windowscompatibility/thread/1558e9ca-8180-4633-a349-534e8d51cf3a , http://zachsaw.blogspot.com/2010/11/wow64-bug-getthreadcontext-may-return.html )... bug is know but Microsoft don't plan to correct it because it will break software who use undocumented features of WOW64 and because the bug don't happen with 64 bits code...

+1

Successfully updated karma reason!