The Bucks, who were without Giannis Antetokounmpo for the fourth straight game, made 19 of 39 three-pointers.
Fred Van Vleet led the Raptors with 39 points. Gary Trent Jr. added 28, while Pascal Siakam collected 23.
The game came alive, but was halted at 6:40 of the fourth quarter, when the Bucks were leading 113-110, due to an altercation between several players.
Brook Lopez of the Bucks received two technical fouls and was sent off. Technical fouls were also awarded to Raptors players OG Anunoby and Jamaal Magloire. No free kicks were awarded.
Lopez made 15 of 19 points in the third quarter before being ejected.
The Bucks will enter a three-day break after going 2-2 without Antetokounmpo. The two-time Most Valuable Player winner is struggling with left knee pain.
After three close first quarters, the Bucks never lost the lead in the fourth quarter, thanks in part to nine points from Holiday.
The Bucks led by 10 points after the first quarter and the Raptors led by 7 at halftime.
