But Quins hit back after the break, with Ben Botica adding to Nick Evans' first-half penalty before Smith's late score earned them the narrowest of wins.
The game had started at pace with both sides trading early penalties - Carlisle with the first for the visitors before Evans levelled immediately for Quins after Wasps were pinged for not rolling away.
Wasps were showing the more attacking intent, with Kearnan Myall breaking through a tackle on 19 minutes but Carlisle was unable to keep the scoreboard ticking over dragging his penalty wide after Harlequins were caught offside.
But the breakthrough came on 28 minutes as, after a considerable amount of time being camped on the Quins line, the Wasps pack drove relentlessly towards the line, forcing referee Luke Pearce to award the penalty try.
Carlisle added the extras to put the hosts 10-3 up heading towards the half-hour mark.
Quins were struggling to get out of their own half as the visitors again turned the screw but they were able to launch a rare attack shortly before the break, brought about by a dancing run by Ugo Monye.
Having won the penalty, they opted to kick for the corner instead of the posts but Wasps' defence stood firm to guard their lead heading into the break.
Harlequins had an immediate chance to reduce the deficit following the restart but Botica, on for the injured Evans, sliced his kick wide from in front of the posts.
And Wasps were on the attack immediately, turning over the ball from a line-out but Tom Palmer was unable to release Jake Cooper-Wooley on the overlap before being tackled by Monye.
The tackles were continuing to fly in from Quins with Samoan flanker Maurie Fa'asavalu putting in a big hit on Andrea Masi, who knocked on.
The tackle seemed to have energised both the Quins team and home crowd, with Botica slotting over a 50th minute penalty to take the score to 10-6.
And with time ticking down, they got themselves ahead for the first time in the match as their continual pressure finally told with Smith touching down in the corner on 77 minutes.
Much of it owed to their continued perseverance as, with their route seemingly blocked on the near side, the ball was switched out to the right, with Smith diving for the line.
James Short also came across to make the tackle but with the decision referred to the TMO, Quins were awarded the all important try.
And while Botica missed the conversion it was still enough as Quins completed the season's double over their opponents.
Every match in the Aviva Premiership Rugby Season is crucial, to be there to support your team buy your ticket from premiershiprugby.com/tickets