Also, in your game I noticed you left the desert planet un-colonized next to your homeworld. In this map I think it's critical to get the 3 adjacent to your homeworld and not let the AI establish a foothold so close. I would also setup some basic trade in a FFA after your inital defenses and small fleet are established.
Actually that was a volcanic planet, and I intentionally did not colonize it until after I had the terran planet, enough fortification research (including enforced loyalty), and full static defenses on the terran planet. It took a while to get all that set up. If I'd tried to colonize it earlier, I would have spent a lot on imperial labs that wouldn't have really paid off for some time.
I was trying to go through the star to grab the terran planets. Two of them had 3 resource asteroids, and the volcanic planet next to me only had 2 metal. As it played out, I just got the static defenses built when the first pirate raid appeared. I started building Imperial labs after that. With Vasari, I like to prioritize Enslaved Labor over trade. It requires less labs and provides discounts on almost everything.
The static defenses still needed cap ship help when the 2nd pirate raid arrived, and at about the same time purple hit me from the other direction, so I had to shore up defenses there. That further delayed the Imperial research. After the 2nd raid was cleared up, I tried an attack on one of the inner terran worlds, but I had to turn right back around and defend against pirates again.
I did research volcanic occupation in there somewhere, but by the time I was able to free up forces to take that planet, Red had already claimed it. I lost my asteroid world to Purple because I didn't have a starbase with loyalty quickly enough.
By then (a little over an hour into the game) I was #1 military and #4 in empire & economy. I could have gone for trade earlier, but I think I might have had to sacrifice a logistics slot on one of my 3 planets for the 4th imperial lab, and not put a trade port there.
It actually turned out better than I had hoped. Eventually I was able to build my fleet up enough that I could push Red back, and Green helped there. The problem with Vasari is that Empire stuff is so damn resource expensive early on, especially on a tight map like that where you have to field fleets right away.
Now to watch your replay.