I recently bought E:TW as well and yes there are some irritating problems with it. But the company was 'aledgely' put into an impossible situation (or just wanted to rip you off depending on which conspiracy theory your currenty subscribed to
) and released way too early.
Overall i enjoy the game, it's quick, simple and easy (relatively) and an enjoyable distraction from Europa Universalis III, so i'm more than happy to put it on the shelf for a little bit and get it back out in a month or so when the next (in a they promise to be a long running series) set of bug fixes and updates.
But to answer your question, yes if you have fewer troops than walls then the fort is basically pointless, in fact it's a major hindrance. Even with a full army in the largest fort available, you don't have enough troops to man all the walls ..... they'll sort it out no doubt. So basically abandon the fort and set up defensive positions next to it.