It had looked as if the Gunners were set for a third-round exit for the first time since 1996 after Robert Snodgrass' penalty early in the second half.
However, with just moments remaining, substitute Theo Walcott was tugged back by Ben Parker, and Cesc Fabregas stepped up to blast the spot-kick past Kasper Schmeichel, who had been in superb form.
"Obviously it is tinged with a bit of disappointment because we were minutes away from pulling off a famous result," said Grayson. "I said to the players beforehand to just do themselves and the fans proud and they have walked off the pitch having done that.
"Individuals have done well but any success we have had in the past and will do in the future is about us as a team collectively.
"Arsenal are a fantastic football club and it is testament to our players they have had to bring on Fabregas and Walcott to try to win the game. There are many teams who come here and have been on the end of a hiding. We have done well for ourselves and will approach the replay with the same attitude."