Carayol only joined the Whites on loan from Middlesbrough on Friday but made an immediate impression with a stunning strike just before the interval.
United then made sure of progression when Souleymane Doukara punished awful Rotherham defending for a 90th-minute tap in.
The victory gained immediate revenge for November’s 1-0 league loss to Evans’ former side who are now under the care of Neil Redfearn who lost out in a game apparently watched by Whites chairman Massimo Cellino in the crowd.
Steve Evans made four changes to his side with the United chief handing a full debut to Carayol and a first home start to Lewis Coyle with Giuseppe Bellusci and Mirco Antenucci also recalled.
Stuart Dallas, Lewis Cook, Scott Wootton and Sol Bamba were on the bench with Chris Wood, Alex Mowatt and Tom Adeyemi not part of the 18 ahead of Tuesday’s Championship visit to Ipswich Town.
Lee Erwin failed to make the squad because of a back niggle but there was a place among the substitutes for Tommaso Bianchi but not for Jordan Botaka.
Much of the attention was focused on new boy Carayol who offered United’s threat with the winger twice showing good skill and pace on the left flank to produce two decent crosses, the second of which led to a corner which was cleared.
A long ball from Bellusci then almost found Doukara before Millers keeper Lee Camp intercepted, after which Rotherham had their first attempt on goal but Paul Green’s low effort was easily saved by Marco Silvestri.
Antenucci then broke in behind the Millers defence but the attack ended with Camp eventually claiming a cross from Coyle.
But Rotherham were also causing problems with a 17th-minute free-kick ending with a decent effort from Shay Facey that squirmed just wide of Silvestri’s right hand post.
Grant Ward then tried his luck from the edge of the area but his effort was closer to the corner flag then Silvestri’s goal.
A very quiet five minutes then followed before some quick passing from Rotherham almost created an opening for Ward who was taken out by a decent tackle from Liam Cooper.
A foul from Green on Bellusci then threatened to escalate into something silly when the Italian raised his hands to push away his opponent but referee Andy Woolmer saw sense and merely gave both players a talking to.
A decent long ball from Charlie Taylor then led to a Leeds corner but Luke Murphy’s corner sailed straight into Camp’s arms.
The Elland Road crowd were getting frustrated but United then finally produced their first effort on target when Liam Bridcutt’s beautifully flighted cross was headed goalwards by Byram whose effort was well kept out by Camp.
Not for the first time, Leeds were then almost the beneficiaries of an own goal when Murphy’s left flank free-kick was headed narrowly wide by Kirk Broadfoot for another Whites corner which was cleared.
Folk had already started heading for their half-time cuppa ten minutes before the interval which said it all though Leeds did at least end the half attacking with a Doukara effort deflected just wide before Antenucci overcooked a high effort into an empty South Stand.
But those that had headed for half-time early would have then missed the one magical moment of the half as Carayol lit up the game with his 45th-minute stunner.
There appeared little danger when the winger picked up the ball on the edge of the area, after a magnificent cut-back from Taylor, but the 27-year-old launched a stunning drive into the roof of the net that gave Camp absolutely no chance.
The goal was greeted by huge cheers and then a pitch invader who made his way to the centre circle before being taken out by Leeds stewards.
And after a very dull opening 35 minutes but much better last ten, United were taken into the interval with a 1-0 lead thanks to Carayol.
Millers boss Redfearn brought on Jonson Clarke-Harris for Facey after the half-time break with the second period beginning like the first, quietly.
But Leeds so nearly doubled their led in the 51st-minute when Bridcutt showed terrific resilience and strength to power towards the byline before producing a cross for Byram whose effort was scrambled just wide for a corner.
Byram then fired another effort into Camp’s arms moments later before Cooper intervened after Rotherham’s Clarke-Harris had got away from Bellusci before crossing into United’s six-yard box.
Redfearn then made his second change in bringing on Emmanuel Ledesma for Danny Ward but it was Leeds who were looking more threatening with Cooper producing a good header from a 60th-minute corner that Camp saved.
Millers sub Ledesma then looked threatening when racing to the byline but his cross was snuffed out by Silvestri as the game passed the hour mark.
And it was Ledesma again then fired a dangerous effort from range goalwards that Silvestri had to palm away for a corner which was cleared.
The Millers were building a decent spell of pressure and Bellusci presented them with a free-kick on the edge of the area when upending Broadfoot but Ledesma fired over the bar.
Clarke-Harris then hit a low effort straight at Silvestri before Carayol fired wide at the other end which proved the final act for the winger who then left the field to a staning ovation when subbed for Dallas with 16 minutes left.
Dallas was then presented with a 77th-minute corner but kicked his effort straight at a defender infront of him, allowing Rotherham to counter but the move ended with Silvestri comfortably collecting a left flank cross.
Evans then brought on Bamba for Bellusci with ten minutes left with the Italian suffering from cramp before Clarke-Harris finally received the game’s first yellow card for a foul on Taylor with seven minutes left.
The subsequent free-kick from Dallas was headed wide by Bamba as Leeds built up a head of steam with a fine 86th-minute move denied the goal it deserved when Dallas thumped an effort against the crossbar from the edge of the area.
The Whites were finishing the game in the ascendency with Evans then making his final change with two minutes left as Cook replaced Murphy.
And United then ensured their place in the fourth round as Doukara was presented with a 90th-minute tap in after horrendous defending from Rotherham as former Leeds skipper Frazer Richardson & Camp made a mess of a clearance from Silvestri.
The striker helped himself by tapping into an empty net for probably the easiest goal of his career as part of what ended up being an easy passage for Leeds into the fourth round.
Leeds United: Silvestri, Coyle, Bellusci (Bamba 80), Cooper, Taylor, Byram, Bridcutt, Murphy (Cook 88), Carayol (Dallas 74), Antenucci, Doukara. Subs not used: Peacock-Farrell, Wootton, Phillips, Bianchi.
Rotherham United: Camp, Richardson, Broadfoot, Belaid, Mattock, Facey (Clarke-Harris h/t), Green, Smallwood, Newell, G Ward (Derbyshire 78), D Ward (Ledesma 57). Subs not used: Cairns, Collins, Thorpe, Warren.
