Rocked by a pair of costly blunders from keeper David Raya, Mikel Arteta's side trailed 3-2 in the second half of a pulsating clash at Kenilworth Road.
But Arsenal roared back to consolidate their lead at the top of the table thanks to Kai Havertz's equaliser and Rice's header deep into stoppage-time.
The wild celebrations from Arteta and his players underlined the significance of a gritty escape that will fuel the belief Arsenal can finally win their first title since 2004.
‘We knew it would be a battle’
"I'm so happy. We knew it would be a big battle here, we made it hard for ourselves the way we conceded the goals," Arteta said.
"The spirit of the team, how we fought, showed our quality in the right moments and never gave up. We went for it and got our rewards today.
"Big compliments to Luton, it was a tough night. I've spoken with the team, and they are all jumping.”
Gabriel Martinelli had opened the scoring for Arsenal, with Luton's Gabriel Osho equalising and Gabriel Jesus restoring the visitors' advantage just before half-time.
Elijah Adebayo scored Luton's second equaliser soon after half-time thanks to the first of Raya's mistakes and Ross Barkley benefitted from another error from the Spaniard.
Second-placed Liverpool can close back to within two points of Arsenal if they win at bottom-of-the-table Sheffield United on Wednesday.
Arteta's men have been road warriors since the start of last season - winning 17 away games in that period -- and the Gunners' first trip to Kenilworth Road for 32 years brought their first victory at the venue since 1984.
They struck first in the 20th minute when Bukayo Saka weaved his way down the right flank for a low cross that Martinelli met with a scuffed finish into the bottom corner from 12 yards.
Rice to the rescue
Undone by a well-worked corner routine, Arsenal were left flat-footed five minutes later as Osho found space to thump his header past Raya from 10 yards for his first Premier League goal.
Jesus put Arsenal back ahead on the stroke of half-time with a close-range header from Ben White's looping cross.
Luton's set-piece prowess provided another plot twist in the 49th minute when Adebayo rose highest to head home from close-range after Raya flapped at Alfie Doughty's corner.
Rob Edwards' side lead the Premier League in set-piece goals this season, with over 50 percent coming from free-kicks, corners and penalties.
The Hatters had no need to unleash their dead-ball excellence as they grabbed a third goal thanks to another Raya blunder in the 57th minute.
Former Chelsea midfielder Barkley glided away from White inside the area for a low shot that squirmed under Raya's weak attempted save.
But Arsenal were level again just three minutes later through Havertz, who poked home from close-range after Jesus showed impressive strength to hold off his marker before picking out the German's run.
Showing the fortitude of potential champions, Arsenal kept pushing for the winner and completed their spirited fightback in the seventh minute of stoppage-time.
Arteta dances with delight
Rice met Martin Odegaard's cross with a glancing header that nestled in the far corner as Arteta danced a jig of delight on the touchline.
In Tuesday's other game, Hwang Hee-chan sealed Wolves' 1-0 win against Burnley at Molineux.
Hwang's ninth goal of the season settled the affair when he netted in the 42nd minute, teed up by Matheus Cunha after Burnley surrendered possession too easily.
Burnley remain second bottom after an eighth defeat in their last nine games, while Wolves, who recorded a first clean-sheet since August, climb to 12th place.