I have had mixed luck against Druids (mostly bad) so take this with all relevant caveats, but...
If I have a mage-only or mage-buddy build, I go after the Druid and her tree. If I go after the plants I will inevitably get outgunned and overwhelmed, so time is against me.
If I have an army of my own, then I will go after the plants, since if I win that war then the druid will have nowhere to go.