Root and Malan recovered England's first innings with a 133-run stand after the tourists had slumped to 95 for three following the loss of Melbourne double-centurion Alastair Cook.
At the close, England were 233 for five with Malan left unbeaten on 55 off 160 balls when Bairstow was caught behind off Josh Hazlewood for five.
"It sort of sums up where we've been in on this tour, we've been on top for so long in games and then make a couple of mistakes and let the Aussies back in," Malan said.
Malan, on 28, cut towards backward point and there was confusion over the run but the throw from wicketkeeper Tim Paine was off target and bowler Nathan Lyon was unable to affect the run out with Malan well out of his ground. He also survived on 34 when Smith dropped a sharp ankle-high chance at slip off Lyon.
Only last week in Melbourne Smith put down two catches in Alastair Cook's unbeaten double century.
Cook, who carried his bat through the England first innings in the fourth Test for an unbeaten 244, looked on his way to another big score until Hazlewood struck in the 33rd over.
Hazlewood's leg before wicket appeal was turned down on-field, but on review, the ball was found to be just pitching in line and Cook was given out for 39 off 104 balls.
It was the third time in the series Cook has fallen lbw and it left him just five runs short of 12,000 career Test runs in his 152nd Test match.
Fellow opener Mark Stoneman earlier hit some nice drives before he nicked a rising Pat Cummins delivery to Tim Paine for a run-a-ball 24 in the 10th over.
James Vince looked set after 82 minutes at the crease before he shaped for a square cut at a Cummins delivery only to be caught behind for 25 in the 29th over.
Australia has already clinched the Ashes with an unassailable 3-0 lead in the five-Test series.