the ships they sent out didn't merge into a fleet. All of the ships appeared in the same hex, but individually, not as a group. I was able to take advantage of this and destroy their ships, one by one, like ducks in a row.
Yes, seen this multiple times. One of the big AI mistakes that make it lose no matter how much you give it handicap goodies...
Could the odd behavior be a consequence of routing the ships to different destinations? If I send two ships to two different destinations and they happen to end in the same hex while moving, they don't fleet up but remain separate. That would fit the observation.
In the AI discussion thread it was mentioned that the "no fog-of-war" actually causes it to make bad decisions because it sees too much and doesn't really understand why immediate near threats might be more important than stuff happening on the other side of the galaxy. So, maybe it's simply the AI trying to send ships out to do stuff somewhere else and it just can't understand that they will never get there because you will shoot them to bits one by one...
Anyway, the AI should definitely use fleets more. Actually, I can't see a downside for the AI operating only in max fleets of military ships in all its military related goals. That would automatically limit its options to a fewer number which might help it to find the most important tasks while at the same time keeping its forces concentrated. Even in cases where there are more things to do than available fleets it probably still would be better, at least for the AI, to just pick the more important one and sacrifice the other instead of dividing its fleet and having them both be defeated in detail.