The community doesn't know whether mitigation applies first or armor but they work separately as follows:
Shield Mitigation is a direct percentage based point for point drop in damage. 30% Mitigation= a literal 30% drop in the damage being applied. Theoretically, IF you could get 100% mitigation, the target would reduce damage by 100 % and no damage would be taken.
Mitigation starts at 15% and goes up 1% per 10 pts damage. It falls 1.25% per second so it requires 12.5 dps to keep mitigation constant.
Armor is trickier:
2 Armor=10% reduction; 5= 25% reduction; 10=50% reduction; 20=100% reduction; 30=150% reduction
Before people go crazy and start screaming "YOU CAN"T HAVE 150% REDUCTION!" Let me explain the fuzzy math that comes into play, like I said, armor calcs are trickier.
The Armor Reduction acts as a dividor vs the original damage in order to cut the damage but never completely stop all damage. It is easier to see in the equation than explain.
Assume 100 damage against 2 armor, the calc flows as follows: 100/1.1= 91 damage after armor; Against 5 Armor, 100/1.25= 80 damage; Against 10 armor, 100/1.5= 67 damage; etc etc.
Even if you could get 100 armor you would still take some damage ie 100/5= 20 damage.
This is the same mechanic at play with the Vasari Super Shield on the Orky, hence why it can say that it reduces damage 200/300%. Yes the damage is GREATLY mitigated, but it never is completely gone. I suppose you could do a mod where you got 3001 armor, this would theoretically round down to zero after fully divided but I don't know if the game always rounds up to at least 1 damage per weapon hit.
Those are the basics. Again it is unknown if the armor applies first or mitigation and there are also unconfirmed reports that some cap ship specials bypass mitigation.
Hope this helps.