Stephan Lichtsteiner had given the champions a 66th-minute lead but Icardi drew Inter level almost immediately then set up the winner for Ivan Perisic 10 minutes later.
It was a stunning response from Inter after their midweek Europa League defeat to Hapoel Be'er Sheva, and was barely spoiled by a last-minute red card for Ever Banega.
Juve boss Massimiliano Allegri sprung a surprise by dropping star striker Gonzalo Higuain to the bench as he sought to restore momentum after a frustrating goalless draw at home to Sevilla in the Champions League on Wednesday.
Serie A
Mathias Pogba detained over alleged plot to extort money from Paul Pogba
18/09/2022 AT 13:49
Higuain's replacement Mario Mandzukic looked lively in the opening exchanges but both sides struggled to assert any kind of authority and it took 20 minutes for the game to catch alight.
In end-to-end action, Juventus striker Paulo Dybala saw a strong effort blocked in the box by Joao Miranda before a dangerous Eder cross was headed clear by Leonardo Bonucci at the other end.
The increasingly dangerous Eder fired just wide for Inter moments later, but Juve should have taken the lead on 35 minutes when Alex Sandro crossed for Sami Khedira who headed wide of target from 12 yards.
Another immediate counter-attack by Inter saw Icardi out-muscle Giorgio Chiellini only to see his eventual shot creep just wide of Juve keeper Gianluigi Buffon's far post.
Miralem Pjanic shot just wide as the visitors began to look the most likely to break the deadlock, and the dangerous Mandzukic was well shacked by Miranda in the Inter box as he shaped to shoot.
If Inter will have been pleased to reach the interval on level terms after their recent struggles, they clearly began to believe the points were there for the taking in the second period with Eder in particular finding space to press forward.
The Brazilian sent a heavily deflected effort just wide past the beaten Buffon, then Icardi set up Antonio Candreva for a stunning volley that drifted agonisingly across the face of goal for the hosts.
Against the run of play, Juve snatched the opener in the 66th minute when Sandro sent in a brilliant low cross for full-back Lichtsteiner to tap in from close range.
But Inter launched a superb response and were level inside two minutes, the impressive Icardi rising high to power home from a corner and immediately restore parity.
Higuain arrived for the final 15 minutes but it was Inter who had the momentum and they grabbed their second in the 77th minute when Icardi rounded Buffon and crossed for Perisic to nod home.
Higuain almost headed an equaliser with one of his first touches of the game then Pjanic sent a long-range free-kick inches over the bar as Juventus frantically sought to draw level.
And Inter's nerves were not helped when Banega was sent off on the stroke of full-time for pulling back Sandro, forcing the hosts to cling on for what would be a famous victory with 10 men.
Serie A
Monza stun 10-man Juventus to heap more pressure on Allegri
18/09/2022 AT 12:04
Serie A
Juve have last-gasp winner ruled out by VAR; three players, Allegri sent off in injury time
11/09/2022 AT 18:13