A source close to the tennis star has revealed that they were "very angry" with the behavior of fans in his semi-final victory over Roberto Bautista Agut.
Djokovic beat his opponent in four sets to set up another Wimbledon final against Roger Federer.
There were several moments in the fight that are said to have rattled the Serbian affiliation however.
Novak Djokovic sat down and invited the spectators to cheer even higher for the underdog when he lost the second set to Bautista Agut.
World No. 1 then puts his finger in his mouth after winning an incredible 45-shot rally to save a break in the third set.
An insider told the Daily Star Sunday: "Novak's box was really very angry and sorry about the fact that some fans cheered their mistakes.
"They just feel he deserves greater respect as a competitor. This doesn't happen to other top players in the game."
Djokovic will again be on the receiving end if he is not the crowd when he takes Federer at Center Court today.
The Swiss is a legend of the sport and loved the All England Club after winning the title an incredible eight times.
Defense match Djokovic, 32, admitted: "Look, it wouldn't be the first time to play against Federer at Center Court.
"I've had that experience more than once. As I said, I know what to expect.
"I'm going to go out there and fight and give it all. It's Wimbledon's finals.
"This is the kind of battle I always dreamed of as a young boy with the tennis racket, dreamed of being part of. This is what I worked for.
"I would be in this position.
"I have a chance to fight for a trophy.
"Whatever is across the web or what's going on around, I'll definitely give it all."
BBC Sport pundit John McEnroe believes that Djokovic can use it for his own benefit.
He said, "Novak can draw on the crowd against him to become more fire."