I have observed this in single player 1.1 / 1.11
To reproduce:
Select some ships and create a fleet.
Set the rally point of a planet to be that fleet.
Build some ships and allow them to automatically move and join the existing fleet.
Mouseover the fleet icon and notice that the tooltip displayed matches the fleet composition. If you expand the fleet, you will see the full fleet.
When the fleet is collapsed in the empire tree, mouseover the dots beside the fleet icon, and notice that the tooltip does not reflect the new ships. Additionally, the number of dots shown is not updated to reflect the true strength of the fleet.
If this is not clear I can post screenshots.