Hosts Portugal only needed a draw to qualify and looked to have done enough, with Renato Sanches' opener cancelled out by Dusan Tadic's equaliser.
Mitrovic, who started the game on the bench, was sent on at half-time but Portugal were wise to his threat and kept him largely quiet.
However, the home side let their concentration slip in the final seconds and Mitrovic pulled away from his marker to meet Tadic's deep ball and send a downward header into the back of the net.
The goal prompted wild celebrations on the bench and on the pitch from the Serbia players and staff while the Portuguese team stood in disbelief with the home crowd silenced.
There were wild celebrations from the away side after the late winner
Serbia finish top of Group A with 20 points, three more than Portugal, whose hopes of playing in Qatar next year rest on two rounds of play-offs in March.
The defeat also ended Portugal's unbroken winning run in home World Cup qualifiers which had stretched back to October 2013.
It had looked like being a long night for the visitors after they fell behind inside just two minutes when Sanches finished confidently after Bernardo Silva had won possession.
But Serbia did not let their heads drop and deservedly equalised when Rui Patricio failed to keep out Tadic's shot, which crept over the line.
Cristiano Ronaldo, himself familiar with scoring late goals with Manchester United this season, was quiet and came closest with a free-kick in the first half.
Instead it was Mitrovic who provided the drama in the closing stages as his header secured a famous win for Serbia.