The expectation damage for Acid Ball against an unarmed Target is 11/3 if I'm doing my math right.
E[Damage] = E[X] + E[Y]
Where X is the damage from attack dice and Y is the damage from the effect die.
E[X] = 2*(1/3)*[0 + 1 + 2] = 2
E[Y] = (1/3)*1 + (2/3)*2 = 5/3
=> E[Damage] = 11/3
Doing slightly less than 4 damage on average for 5 mana against an unarmed target is pretty bad. You're much better off playing Hurl Rock or Flameblast as an attack spell. If you consider other options, Bear Strength is superior if you'll get off at least 2 attacks.
The question of Acid Ball vs Rust depends largely (assuming you don't have a particular reason to value one over the other such as Curse Weaving or Smoldering Curses) on how many attacks you're likely to get in with the armor reduction. Sure, the 2 die attack may help, but against 2 armor, that's only an average of 1.11 damage.
Let's say that you're making 5 dice attacks, since you're focused on creatures that make 5 dice attacks. The relevant question then is how many 5 dice attacks do you need to make for the expectation damage with Rust to be better than the expectation damage with Acid Ball. To simplify the calculation, let's assume the target is at 2 armor before the armor reducing effect is applied.
E[A] will be the expected damage from a single 5 dice attack.
Rust: E[Ar] = 5
Acid Ball: E[Aa] = (4.13)(1/3) + (5)(2/3) = 4.71
If we set nE[Aa] + 1.11 = nE[Ar], and solve for n, we get the number of attacks for which the expected damage output from the two investments is the same.
This gives n = 3.84
That means that you're statistically better off playing Rust if you will make 4 5 dice attacks before the effect is eliminated against a target that had 2 armor before having their armor reduced. Now, again, Rust is easier to remove for most mages, but I still think you're pushing Acid Ball over Rust more than is justified on a statistical basis. You're going to need to make more than 4 attacks at that strength to win.
