The 2008 Sins of a Solar Empire was the first release of the game. Sins of a Solar Empire Rebellion is basically a stand alone "Gold Edition" of the game, containing the content of the three expansions since then, Entrenchment, Diplomacy and of course Rebellion. While not a fundamentally different game, upgrading to Rebellion from the original game will get you a lot of new content from the three expansions.
Entrenchment was the first expansion that focused on improving planetary defenses, since these were somewhat lackluster in the original release. They did this by adding various researchable upgrades to defense structures, mines, and most importantly Starbases, basically customizeable space fortresses.
Diplomacy attempted to add more interactions with other players, adding things like envoy ships that can give bonuses to your allies while making them like you more, and pacts, in which two Empires agree to focus on an area of technology to give both sides significant bonuses. It also added a new victory condition, diplomatic victory, some new artifacts, and a bunch of other improvements I'm forgetting right now.
Rebellion was the last expansion and much larger than the previous two (they were actually called micro-expansions). Among the new features were two new ship types, Titans (basically mega capitalships you can only have 1 of) and Corvettes, a rework of the graphics, including all new weapons effects and shadows, several new victory conditions including Capital Victory, Research Victory, Occupation (King of the Hill) Victory and Flagship Victory (you start with a special ship, if it is destroyed, you lose) and a new capitalship for each race. Also added were "Subfactions" of each race. Basically for each Race, there are two subfactions, which have a unique titan class ship and several unique new technologies to give them different play styles. Finally, as a sort of side benefit, most people play Rebellion these days so you have access to a large multiplayer and mod community.
Also worth mentioning is that Rebellion has since gotten two mini-DLCs that add new planets, stars, planet bonuses, planet customization options and even random events.
