Make sure you have enough counters to armor. Opponents with 3+ armor and a veterans belt are basically immune to weak attacks. I would add multiple [mwcard=MW1J03]Tooth & Nail[/mwcard] totems in your book to make sure your little creatures can do damage.
In general, mana denial cards will wreck you! Especially [mwcard=MW1J21]Suppression Orb[/mwcard] and [mwcard=MW1J16]Mordok's Obelisk[/mwcard]. Expect your opponent to summon it on the opposite side of the board. You NEED to destroy it very fast or you will have lost. Especially wizards and forcemasters tend to use these cards. So, it's essential in your book to have a very solid counter against these cards.
Don't forget zone attacks. A single [mwcard=MW1A12]Ring of Fire[/mwcard] or even a [mwcard=FWA03]Hail of Stones[/mwcard] can destroy your entire army at once. I have experienced this myself (and used it myself).
Slavorg! Every beastmaster swarm book needs him. I've summoned him for 3 mana once. (with a lair so 1 mana from my mage)
Like Mystery mentioned, you need a couple of rounds to set up. When you'll be rushed, you will have to spend mana and actions on armor and things like that.
Overall, I don't think an opponent will ever let you summon 3 creatures each round. You will be forced to spend actions and mana on other stuff. Every decent opponent has some ways to deal with swarms.