I think you're missing the fact that culture spread rate is divided by the number of phase lanes a planet has. A planet with one phase lane might generate 10 culture per second on that one phase lane. An identical planet with two phase lanes will generate 5 culture per second on both lanes, for a total of 10, the same as the first. A planet with three would generate 3.333 on the three lanes etc.

So you're probably not gaining or losing culture, the culture just gets spread around differently depending on how well connected a planet is.