Two penalties apiece for the respective fly-halves – Duncan Hodge of Leeds and Rotherham's Phil Jones – saw the score at 6-6 as half-time approached.
But a 35th-minute try from Leeds' new South African scrum-half Ryan Walker, on loan from Natal Sharks, turned the tide in favour of the hosts, Hodge converting.
And then 10 minutes into the second half Welsh fullback Matt Cardey extended the lead with their second try, in front of a passionate Yorkshire crowd for the occasion.
The third touch-down followed 14 minutes later when dual-code centre Liam Botham crossed the line – Leeds having been searching for their first win of the season after losses to Bath and London Wasps.
Fullback Jon Benson restored some respectibility to the scoreline when he got Rotherham's one and only try of the match with five minutes to go, converted by Jones, but it was never going to be enough to make a significant dent in the Leeds defence, as they held out for the win.
For Leeds Tykes:Tries: Walker, Cardey, BothamCon: HodgePens: Hodge 2
For Rotherham:Try: BensonCon: JonesPens: Jones 2