But Gloucester roared back as the sides shared six first-half tries, with England trio Jonny May, Charlie Sharples and Henry Trinder all crossing for the visitors before Exeter hit back with a penalty try shortly before the break.
After the restart Rob Cook and Burns added to the try tally for Gloucester but the nerveless Gareth Steenson kicked Exeter to victory in the 79th minute.
"We have moved forward, we have the chance to double-up on the Heineken Cup now, which is very important," said Baxter.
"The successful teams in the Heineken Cup, they get there not now and again - they get there regularly. For us to be there again is a big boost to how we want to develop as a side.
"You could tell from the players that it was all about winning the game and that's not the worse quality to have. That game was probably about as bad as it gets for both myself and Nigel [Davies], really.
"We're just thrilled with the result if we reviewed the game - which we're not going to do now as it's the end of the season - we'd probably be a little disappointed with a couple of interception passes we threw and a bit of loose play on the floor, because the one thing we did highlight was that Gloucester are one of the best teams on the counter attack and the one thing you don't do is give them loose ball.
"But we managed to do that and we rode the pressure that came back at us and stayed in the game and ultimately won."
With fifth place in the Aviva Premiership sewn up ahead of kick-off but no chance of making the play-offs, Gloucester had less riding on the result that their opponents.
But director of rugby Nigel Davies was impressed with his side's fight back from early setbacks, going nearly 20 points down and losing Sione Kalamafoni to injury after just 15 minutes.
"The one thing that this Gloucester side has done, apart from one game, is show their resilience and come back," said Davies.
"Through that resilience, the ability that we have in the team and the pace to finish off half-chances, we came back in to the game. By the end of the match I thought that we should have won the game."
Exeter Chiefs: Arscott, Nowell, Dollman (Whitten 61), Shoemark, Jess, Steenson, Thomas (Chudley 66), Sturgess (Rimmer 60), Yeandle (Whitehead 61), Tui (Mitchell 64), Mumm, Welch, White (Ewers 52), Scaysbrook, Baxter (Muldowney 75)
Replacements not used: Slade
Gloucester: Cook, Sharples (Monahan 41 (Tindall 73)), Trinder, Twelvetrees, May, Burns, Robson (Lewis 61), Wood (Murphy 58), Britton (Dawidiuk 43), Knight (Chistolini 63), Savage, Lokotui, Kalamafoni (Buxton 15), Qera (Hazell 58), Morgan
Tries: Mumm, Welch, penalty try
Conversions: Steenson 2
Penalties: Steenson 7
Tries: May, Sharples, Trinder, Cook, Burns
Conversions: Burns 4
Penalties: Burns 2
Referee: Greg Garner