Red is bad and green is good, yes.

Personally, I find leaving planets unspecialized is the best option most of the time. The cost of specialization is pretty high, and as an economic investment it's passable. Industrial specialization is a superb late-game option on a planet that you've completely filled with trade ports, but it only really starts to shine once you're getting close to saturating your logistics capacity. Logistics upgrades are expensive, and you can get way more trade ports in total if you skimp out on upgrading individual planets. Social specialization has the annoying issue of taking quite some time to build up to its population totals. That really kills it compared to other economic investments, which will pay for themselves much more quickly. The culture spread rate is nice, but usually you're better off just putting up an extra culture center. Unless you're playing on a tiny scenario with sharply limited logistics space, you're best off not specializing the planet for now but leaving the option open for later once you start to fill out your empire's logistics capacity.

As for feudal society in specific, the 5% ship build discount is the biggest benefit on offer. If you put up 3 factories here and start pumping units you can easily produce tens of thousands of credits worth of units, which will add up to a fair chunk of cash savings. However a mere 5% savings is not worth a long journey to the front lines so only do so if you're reasonably close to your enemy.