ב"ה
Because not everyone plays a rush. And if you get out a GA or a few NVs, you can defend it rather easily.
If you try to go for a long game, the temple of asyra is just better. If you harmonize it instead of harmonizing a crusade banner, you could bring a creature every turn instead of two creatures only once.
Keep in mind that enemy creatures could probably get to your banner a lot before turn 9 even if the opponent didn't rush.
Killing you in 3-5 turns is a rush. Killing you in 10 turns is a slow win...
Even if your deck is uber-slow and wants 15 turns it can still Fireball/Boulder a banner before Turn 10, or it's fundamentally flawed.
I've currently got about 30 books, about 2/3rd of which are slower controlling books. All of them could easily take this out by turn 5, even with a Guardian Angel Guarding... It's worth it for them too...
I've never seen a game that ended on turn 3 or 4. A rush that rolls well could end the game by turn 5, and with avarage rolls on turn 6 (assuming everything goes as planned).
Now, about fireball or hurl boulder on a crusader banner, it has a few drawbacks-
1) A fireball or a hurl rock cost 8 mana and a quick action. The banner cost 7 mana and a quick action. to kill it will cost you more than the cost to create it.
2) A single boulder is not so likely to one shot a crusader banner, and a single fireball might fail in it as well. That would mean you just spent all that mana and all you got was making the opponent use that banner a little sooner.
3) Casting a fireball or hurl boulder require your mage to be within 2 zones of the banner, and that means you just went to the Paladin's face. Assuming the paladin is going for a melee build, going all the way to the paladin just make the game easier for him. Against melee builds it is usually safer to stay away as much as you can.