The Blades twice had the ball in the back of the net either side of the opening goal as Mikel Arteta's side again struggled to defend set pieces but on both occasions John Lundstram and John Egan's efforts were correctly disallowed for offside.
Failure to deal with a Jack Robinson long throw gave McGoldrick the chance to volley home with three minutes remaining from close range.
The Blades then went hunting for a winner and soon after Billy Sharp was thwarted by Emiliano Martinez. But the ball went down the other end and the on-loan Real Madrid man Ceballos sealed the visitors' path to the last four, striding past Enda Stevens before slotting inside Dean Henderson's near post a minute into added time.
Young Gunners show glimpse of brighter future. It is very difficult to see Arsenal challenging for a European spot in the league and, after Norwich on Wednesday, they'll rightly be outsiders to beat Leicester, Tottenham and Liverpool who should all expose the vulnerabilities at the back. That said, there is hope. Kieran Tiernay put in as good a performance as he has in an Arsenal shirt, impressing with his link play and calm delivery finding players in dangerous areas.
In midfield, Joseph Willock and Ainsley Maitland-Niles showed class going forward as well as desperation and a willingness to work backwards to deny Sheffield United on the counter-attack. Eddie Nketiah also looked a threat after coming off the bench and of course there is their record signing who is likely to be one of the best players in the Premier League over the next year.
Dani Ceballos scores the winner
Image credit: Getty Images
MAN OF THE MATCH
Nicolas Pepe (Arsenal): There were periods of the game where he wasn't sighted but there was no more dangerous player on the pitch. Arsenal's creative force going outside his defender to the by-line to cross into the area or inside onto his left foot from the right flank creating play or striking from long range.
He also calmly and clinically hit the penalty past the outstretched arm of Dean Henderson. Mikel Arteta has a long way to go to create the Arsenal team he wants but if the Gunners are to have a successful end to the season, much will rely on the Ivory Coast forward.
8' - DISALLOWED GOAL! A corner found McBurnie at the far post and he headed back across the goal for Lundstram to head home from close range. But... Arsenal not having a man on the line saves them. Lundstram was beyond all the Gunners players except their keeper. A let off for Arsenal.
23' - PENALTY! They'll check this but Basham has come through the back of Lacazette just inside the penalty area....The contact was slight but definitely in the box. The decision will stay
25' - GOAL! Sheffield United 0-1 Arsenal. Superb penalty into the side-netting to Henderson's left from Pepe.
59' - DISALLOWED AGAIN! The linesman's flag denies United. Great low ball behind the defence from Norwood's set piece. McGoldrick shot at goal was blocked by Martinez and then Egan headed into the net. McGoldrick was in an offside position though.
87' - GOAL! Sheffield United 1-1 Arsenal. There's no denying Sheffield United this goal. Once more horrendous defending from a set piece. Kolasinac hits the ball against Holding trying to clear Robinson's throw and McGoldrick then volleys the ball past Martinez.
90+1' - GOAL! Sheffield United 1-2 Arsenal. Ceballos calmly slots home into the near post after Robinson had dispossessed Pepe on the right hand of the Blades penalty area.
30 - The number of FA Cup semi-finals Arsenal have reached (joint most with Manchester United).