The hosts got themselves off to a dream start with a goal inside three minutes.
Carl Baker was tackled as he raced through the middle, but the loose ball fell invitingly for Stockport's skipper Johnny Mullins and he smashed home from an acute angle.
Leeds were stunned and they didn't find a response until the 13th minute.
At that point, Jermaine Beckford swivelled acrobatically but his overhead kick struck the crossbar.
The visitors were not to be denied on 19 minutes. Robert Snodgrass' corner was flicked on by Jonathon Douglas and there was Luciano Becchio who bundled the ball home from close range.
Becchio was denied by a brilliant save by Owain Fon Williams minutes later then, with Stockport now on the back foot, Beckford again tried an overhead kick which this time flew inches over the top.
As half time approached, Stockport's Baker and Dominic Blizzard were both denied by goalline clearances.
Leeds were first to go close in the second half when Snodgrass curled a superb free-kick inches over the crossbar, but Stockport replied with Mullins planting a far post header just wide.
However, Leeds went in front with 17 minutes left thanks to a crazy goal.
Stockport keeper Fon Williams could only find Delph with a regulation clearance and he made no mistake with a first-time finish from 35 yards.
With four minutes left, a long punt from Manuel Rui Marques found Christie and he lashed home a clinical half-volley off a Stockport post.
After the game, Leeds boss Grayson expressed his delight having picked up four points from his opening two games in charge of the Elland Road club.