I think the biggest issue in gal civ 2 in regards to fighters, was that you could build you own 'fighters' by using the smallest parts, but there was no point. The combat simulation didn't take ship maneuverability into account. So, a massive battle cruiser with a giant cannon, could blow away a bazillion fighters, despite the fact that in reality, the fighters would move so fast that a large ship would have difficulty hitting them with a gun the size of an asteroid.
This is basically the equivalent of the death star, being able to destroy an x-wing with it's planet cannon. In reality, the larger the gun, the less accurate it is (usually).... galciv 2 never looked at that. Although sins of a solar empire rebellion does somewhat. Hopefully this will be fixed in 3.
Part of this is the modular nature of the weapons. There weren't any Death Star sized ship-to-ship weapons, there were multiple banks of small to medium sized weapons. Think instead how likely the Enterprise was to miss a passing shuttle with its phasor banks at close range.
Instead of the GC2 RPS of guns/missiles/beams and their defenses, we could go to a system where different weapon systems work better on different hull types. I suggested something like this years ago for Twilight, so I doubt it will go very far now, either. For example:
Beams are weak, topping out at well below the power of the other systems - but can track tiny hulls really well. As the target gets bigger, there is no accuracy penalty but the damage becomes insufficient to make a good ship. Maybe 5 damage is the top weapon available. Pretty much ineffective above medium-hull targets.
Missiles are good for targeting medium range hulls. They still top out below guns in terms of power, but are better than beams. Maybe 10 damage at maximum. They suffer small accuracy penalties against small hulls and moderate penalties against tiny hulls. Partially effective against large and huge hulls.
Guns are for busting big ships. Massive damage, maybe 15-20 at the high end, but suffer accuracy penalties the smaller a hull gets. Something like 95% hit rate against large hulls, 80% for medium, 50% for small, 15% for tiny.
No restrictions as to which hulls can mount which type of weapons, but hull size would limit how many weapons of any type you could mount. A tiny hull could mount a single gun and maybe some shields to protect itself, or a single missile launcher and more shields, or a couple beams and even more shields for fighter-on-fighter fights. Medium hulls could mount 2-3 guns for ganging up on bigger ships, or missiles for taking out other mediums, or lots of beams to take on the anti-fighter role. Huge hulls would likely mount mostly guns to take on other huge hulls, but with enough missiles and/or beams to make swarming them difficult.
Defenses would retain the off-type capacity, but would be more specialized to the hull size to be protected rather than the enemy weapon system - the enemy weapons are going to be chosen to reflect your hull size not your defenses anyway. Tiny ships would have shields, since the biggest threat is beams and they couldn't carry enough PD or armor to matter against the likely hits anyway. Medium hulls would carry PD since they are most likely to be taking missile fire; enough laser fire could hurt them but it would be minor hits adding up rather than a single knockout hit. Huge ships would have armor for the same reason. Missiles and beams just can't hand out the hurt fast enough to matter to a battleship, so they'd be protected against the weapons that *could* hurt them quickly.
The usefulness of off-type defense would increase with hull size. Tiny fighters would have no use for armor; no amount they could carry would help much if they got hit anyway. PD might matter a bit more for small hulls, but they are also vulnerable to beams so some balance might be necessary. Mediums have the same problem with not being able to carry a practical amount of armor, but need PD more than shields. Large hulls would need a balance between armor and PD, but don't have to worry much about shields. Huge hulls would be mostly armor with enough PD and maybe even shields to prevent death-by-a-thousand-cuts from M-S-T hulls they aren't really equipped to fight.
You get the same basic RPS gameplay, but with hull sizes as a lesser determinant for weapons. Balanced ships, or fleets, would be required to fight anything but a single-type enemy. If all the enemy has is tiny fighters, a huge hull with good defense and lots of lasers would swat them like flies for little to no damage. That same ship would be suicidally underequipped for dealing with a proper gun-armed huge hull - think trying to sink the Yamato with only anti-aircraft guns. It might do some cosmetic damage, but you'll get massacred while doing that cosmetic damage. Gun armed fighters would work well against improperly equipped and unescorted huge hulls, but would be another massacre if faced with a swarm of beam-armed fighters.