Why use the Goblin Builder? Because your Warlord can't be everywhere at once. If nothing else, the Builder gives you more flexibility in the arena for throwing down walls, caltrops, towers, etc., if your Warlord is having mobility issues, or has better things to do that round than conjur a wall.
I view the Builder as cheap insurance in my Warlord spellbook. For only 1 point out of 120, the Builder is kinda like Jello... there's always room for at least one.