Using someone else's spellbook is very challenging, yes. It would be about just as challenging for your opponents. It could be the ultimate test of the players' skill, to be able to figure out how to use someone else's spellbook that you're not necessarily familiar with. It would also be a way to inspire more creativity in deck builders.
Teams do not need to have the exact same number of players. Matches do. So, if you exclude the option, in a match where a team of two is facing a team of four, one of those teams will need to have 2 points to win.
In a match where a team of one is facing a team of two, one of those teams will need to have 1 point to win the match.
In a match where a team of three faces a team of three, one of those teams will need to have 3 points to win the match.
This is a correction from how I originally imagined it: Both game and match results are now calculated as scores. A win is +1, a loss is -1, and a draw is 0.
A match ends when every spellbook has been played in a game once, or when one of the teams has the number of points required to win the match. When the match ends, whichever team has the higher score wins the match, and adds 1 to their total score. The losing team subtracts 1 from their total score.
With the option, teams face against each other in a second match using their opponent's spellbooks. The players get to choose which of their opponent's spellbooks to play (if there's a disagreement, it's decided randomly). All teams would have separate scores for the regular matches and the rotation matches. At the end, the two scores will be added together, and the team with the highest score are the winners!