Don't know what version you have, but start getting rid of strike craft, mines, and entire factions like the pirates, followed by one of the other three. Also unpin everything from your empire tree. If you have fully developed defenses on planets that are far from your front lines and never get attacked, reduce them to the absolute minimum. If you're not playing against any TEC with the rebel research done, you can likely get rid of all defenses not on the front lines.
Basically, help your computer out by getting rid of as much as possible. 5.5 hours in, you should be plenty strong enough to start assaulting enemies and wiping them out. The less you all have, the better your PC will run.
