My group has tested the FM a bunch and she does not struggle with swarms for the most part. It's very easy to kill a large swarm with a Force Wave + Wall of Fire/Pikes combo. The key to beating the FM is using "mage only" creatures which are immune to Mind Control, Iron Golems if you're an earth Wizard or Warlord, or integrating a Mind Shield or two into most of your books. Also, include ways to gain the Unavoidable trait, because that will force the FM to start using her Forcefield, which while very powerful is also very expensive mana-wise.
I'd really recommend using at least one Mind Shield in your aggro books. It's extremely cost-efficient and protects against the worst offenders, like Sleep, Pacify, Charm and of course, Mind Control.