You can, but you'd need a mod that reduces the total number of items that need to be loaded....this isn't just the number of assets (like # of entities, etc.) but how many objects are actually in a game...if you halved the maximum number of ships and SC, late game performance would improve substantially...
Some of the particle effects are also grossly efficient....a mod that dealt with those would help immensely...
There are a few people working on such endeavors...of course, their mods only help those who use mods, not the vanilla game itself...