Kelly, 50, will work alongside head coach Uwe Rosler, who succeeded Neil Redfearn in the role on May 20.
Rosler said of the appointment on Leeds' official website: "I felt that it would be very good to have such an experienced coach like Rob next to me.
"He knows this league in and out and he knows the players in and out. He's good on the training field.
"I was always looking to bring in somebody experienced who has worked at this level for quite some time - I think that Rob is a very good option.
"We have known each other for a while and I think it's a good option for everybody involved.
"He's the second man in command. He will be influential in the day-to-day running of our team in terms of training sessions.
"He will be important in terms of matchdays and he'll be my close ally."
Kelly, forced to retire as a player at the age of 24 due to a back injury, was most recently joint assistant head coach at West Brom but departed The Hawthorns in January a week after Alan Irvine had been sacked.
The former Leicester and Wolves midfielder began his coaching career in the youth set-up at Wolves and following a similar role at Watford became academy director at Blackburn before returning to Leicester.
He had a spell as assistant manager at Leicester and after stepping in as caretaker following Craig Levein's departure was appointed boss in April 2006.
Kelly was sacked a year later and senior coaching roles followed at Preston, Sheffield Wednesday and Nottingham Forest. He had spells as caretaker manager at each of those clubs.